Skip to content
#

decoder-only

Here are 55 public repositories matching this topic...

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool agents, quantization-aware training. Scales: Tiny (25M) to Large (1.3B).

  • Updated Aug 21, 2026
  • Rust

🚀 从零亲手训练大语言模型(LLM)——全流程实战教程!涵盖数据清洗与预处理、自定义Tokenizer训练、模型架构搭建、从头预训练,以及指令微调(SFT)。没有黑盒,不依赖现成API,保姆级手把手带你打造属于自己的LLM,深入理解大模型的每一处细节。

  • Updated Nov 18, 2026
  • Python

Clean-room GPT-2/GPT-3 implementation: tokenizers, architecture blocks, training loop with AdamW + cosine decay, CLI scripts, inference tools, and pytest suite. Covers OpenWebText-10k & WikiText-103 workflows. Designed as an academic reference for understanding and scaling decoder-only transformers

  • Updated Feb 18, 2026
  • Python

Add this topic to your repo

To associate your repository with the decoder-only topic, visit your repo's landing page and select "manage topics."

Learn more