No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from DedeProGames/Kiyo-135M
All models by DedeProGames →Kiyo-135M is a decoder-only language model pretrained from scratch on 200B tokens drawn from FineWeb-Edu, DCLM-Baseline, FineMath, and Stack-v3-train.
The model follows the SmolLM2-135M architecture: a Llama-style decoder with grouped query attention, RMSNorm, SwiGLU MLPs, and tied input/output embeddings. SmolLM2 was chosen as a base architecture because it is specifically tuned for small-scale pretraining efficiency — narrow hidden size with a comparatively deep layer stack, and a large 49k-token vocabulary that keeps sequence lengths short for a model this size. Kiyo-135M reuses this architecture but is trained independently from a random initialization on its own data mixture, rather than starting from SmolLM2's own weights.
Self-reported results from the official BananaMind Base Bench 1.1 script, all measured with the same runner, dtype (bfloat16) and GPU.
Figures for BananaMind-2-Pro, Rose-Pro, and GPT-2 are as self-reported on their own model cards, all against the same BananaMind Base Bench 1.1 suite.
Detailed Kiyo-135M result
* = passes 1.96σ vs. chance; n=50 per category
By difficulty
Summary
Scores are self-evaluated and may vary with the benchmark revision, Transformers version, dtype, hardware, and generation settings.
This is a base model, not instruction-tuned — it continues text rather than following instructions. At 135M parameters it produces fluent, well-structured text and is strong on language completion and code, but accuracy drops on quantitative and multi-step context-tracking tasks. It can generate incorrect facts and should not be used for high-stakes decisions without verification. Keep a finite generation limit to avoid repetition or drift on long outputs.
Apache 2.0
pip install -U transformers safetensors torchimport torchfrom transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "DedeProGames/Kiyo-135M"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.bfloat16 if torch.cuda.is_available() and torch.cuda.is_bf16_supported() else torch.float16,).cuda().eval()
prompt = "The meaning of life is "input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device)
with torch.no_grad(): output = model.generate( input_ids=input_ids, max_new_tokens=64, do_sample=False, repetition_penalty=1.1, pad_token_id=tokenizer.eos_token_id, eos_token_id=tokenizer.eos_token_id, )
print(tokenizer.decode(output[0], skip_special_tokens=True))