No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from SC117/Nex-N2-mini-APEX-GGUF
All models by SC117 β--chat-template-file neededNex-N2-mini's original chat template uses complex vision processing macros that stock llama.cpp's Jinja parser cannot handle correctly. This causes thinking tags to not be injected, breaking --reasoning-format.
The official fix: Use Nex's patched llama.cpp, which works with the unmodified GGUF and unmodified template. Once Nex's upstream patch is merged into stock llama.cpp, this workaround will no longer be needed.
β οΈ Do NOT modify chat_template.jinja. The model was trained strictly on the current template β editing the tags deviates from the training-time format and may degrade output quality. See discussion #3.
These GGUF files are quantized using APEX, a MoE-aware mixed-precision quantization technique that outperforms standard quantization methods while being significantly smaller.
APEX beats Q8_0 perplexity at half the size β and even beats F16.
APEX classifies every tensor by its role β routed expert, shared expert, or attention β and applies a layer-wise precision gradient, giving the most sensitive edge layers higher precision and compressing the redundant middle layers more aggressively.
Download Nex's patched llama.cpp
Binaries: nex-agi/llama.cpp Β |Β Docker: ghcr.io/nex-agi/llama.cpp:server-cuda-nex-b9596-fix-b9598-8c0d5c9
./llama-server \ -m Nex-N2-mini-APEX-Quality.gguf \ -ngl 99 -ncmoe 19 -c 32768 \ --host 0.0.0.0 --port 8081
Replace Nex-N2-mini-APEX-Quality.gguf with your preferred quantization tier. Add --mmproj mmproj-Nex-N2-mini.F16.gguf for vision. Recommended sampling: temperature 0.7, top_p 0.95, top_k 40, min_p 0.
From the original Nex-N2-mini model card (BF16, full precision).
If you cannot use Nex's patched llama.cpp, a template-fixed version is available at SC117/Nex-N2-mini-template-fix-APEX-GGUF. These GGUFs have a modified chat_template.jinja embedded so that --reasoning-format works on stock llama.cpp without --chat-template-file.
β οΈ The Nex team explicitly advises against modifying the chat template β the model was trained strictly on the original template, and deviating from the training-time format may degrade output quality. See discussion #3. Use the template-fixed version only if you have no alternative, and be aware of the potential quality trade-off.
