No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from inclusionAI/Ling-3.0-tiny-UBTree
All models by inclusionAI →Website · Paper · Target model
Ling3-Tiny-UBTree-B16 is a UBTree speculative-decoding draft model for inclusionAI/Ling-3.0-tiny. It combines a five-layer DFlash backbone with a learned bigram selector to propose multiple candidate continuations for target-model verification.
This checkpoint is used alongside the matching target model; it cannot serve as a standalone text-generation model.
[Image blocked: UBTree architecture and tree-native training pipeline]
UBTree pipeline
UBTree pairs a unigram proposer with a bigram selector:
The paper also introduces tree-native training with renormalized KL supervision on high-temperature target traces, encouraging plausible alternative branches. See the paper for the method and experiments.
The parameter count covers tensors distributed in this repository. The draft shares token embeddings and the language-model head with the target; these shared weights are not included here. B16 denotes the draft block size, not the verification tree's node budget.
Use the asahi/ubtree SGLang branch for the commands below.
Upstream integration is in progress in sgl-project/sglang#44050.
All methods use block size 16, TP1, concurrency 1, BF16, and greedy no-thinking decoding. UBTree uses a 49-node tree.
Each cell shows acceptance length / E2E speedup / TPS (tok/s). E2E speedup is relative to target-only autoregressive (AR) decoding.
TPS is total returned output tokens divided by total client HTTP time, including prefill and generation. Acceptance length is returned output tokens divided by target verification calls, including the anchor. Speculative AL and TPS are independent medians over three runs; E2E speedup is speculative TPS divided by the matched AR TPS. AR uses one run per benchmark.
Each Avg. value is the arithmetic mean of the six per-dataset values, computed before display rounding. Average speedup is the mean of per-dataset speedups, rather than the ratio of average TPS.
If you use UBTree in your work, please cite:
The model weights are released under the MIT License. The pipeline figure is from Liang et al., Figure 2, reproduced without modification under CC BY 4.0.
CUDA_VISIBLE_DEVICES=0 \SGLANG_OPT_FA3_VERIFY_METADATA_IN_GRAPH=1 \SGLANG_OPT_UBTREE_FA3_TREE_ONLY_MASK=1 \python -m sglang.launch_server \ --model-path /path/to/ling3-tiny \ --served-model-name ling3-tiny \ --tp-size 1 \ --host 0.0.0.0 \ --port 30000 \ --attention-backend fa3 \ --page-size 1 \ --context-length 4096 \ --max-total-tokens 4096 \ --max-running-requests 1 \ --mem-fraction-static 0.80 \ --max-prefill-tokens 4096 \ --chunked-prefill-size 4096 \ --disable-radix-cache \ --mamba-backend triton \ --linear-attn-backend triton \ --linear-attn-verify-backend triton \ --mamba-ssm-dtype float32 \ --cuda-graph-backend-prefill disabled \ --cuda-graph-backend-decode full \ --cuda-graph-bs-decode 1 \ --disable-cuda-graph-padding \ --tool-call-parser ling3 \ --reasoning-parser ling3 \ --trust-remote-code \ --speculative-algorithm UBTREE \ --speculative-draft-model-path /path/to/ling3-tiny-ubtree \ --speculative-num-draft-tokens 49@article{liang2026ubtree, title = {UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding}, author = {Liang, Chumeng and Wang, Linxuan and Peng, Xinyu and Liu, Huabin and Chen, Yuxin and Liu, Ge and Lin, Guang and Song, Qifan and Li, Jianguo}, journal = {arXiv preprint arXiv:2609.39972}, year = {2026}, url = {https://arxiv.org/abs/2609.39972}}