DeepSeek-R1 1.5B

The model's base architecture is explicitly identified as Qwen2.5-Math-1.5B. Documentation in the DeepSeek-R1 technical report and GitHub repository provides a detailed breakdown of the distillation process, including the use of 800,000 reasoning traces generated by the larger DeepSeek-R1 model. Technical specifications such as the use of RoPE, SwiGLU, RMSNorm, and Grouped Query Attention (GQA) are clearly documented. However, the exact hyperparameters for the distillation fine-tuning (e.g., learning rate schedules, specific weight initialization beyond the base model) are less granular than the pre-training details of the parent models.

While the model discloses the use of 800,000 reasoning samples for distillation, the specific source and composition of the underlying training data for the Qwen2.5-Math-1.5B base are only generally described as 'large-scale' or 'diverse.' There is no public breakdown of the data by category (e.g., % code, % web, % books) or specific disclosure of the filtering and cleaning methodologies used for the distillation set. The 'cold-start' data used for the teacher model is mentioned but not publicly accessible for verification.

The tokenizer is publicly available via the Hugging Face repository and GitHub. It uses the Qwen2 tokenizer with a vocabulary size of 151,643 tokens. The tokenization approach (Byte-level BPE) is well-documented, and the tokenizer files are directly inspectable, allowing for verification of language support and token normalization. The alignment between the tokenizer and the claimed reasoning capabilities is consistent with the Qwen2.5-Math base.

The model's parameter count is precisely stated as 1.54B total, with 1.31B non-embedding parameters. Unlike the larger MoE variants in the DeepSeek-R1 family, this variant is explicitly documented as a dense architecture, meaning all parameters are active during inference. The architectural breakdown (28 layers, 12 attention heads) is clearly provided in the model card and technical documentation.

Compute disclosure is primarily focused on the parent DeepSeek-V3 and R1 models (e.g., 2,048 H800 GPUs, 2.8M GPU hours). Specific compute metrics for the distillation of the 1.5B variant—such as the exact hardware hours, energy consumption, or carbon footprint for this specific fine-tuning run—are not explicitly provided. The documentation relies on the 'negligible cost' of distillation relative to the teacher model without providing verifiable data points for the 1.5B variant itself.

DeepSeek provides comprehensive benchmark results (AIME 2024, MATH-500, etc.) and specifies evaluation settings like temperature (0.6) and top-p (0.95). However, the exact evaluation code and the specific prompts/few-shot examples used for every benchmark are not fully public in a single reproducible repository. While some evaluation scripts are on GitHub, third-party verification has noted discrepancies in results, and the lack of a complete, 'one-click' reproduction suite for the distilled variants limits the score.

The model generally identifies as a DeepSeek-developed assistant. However, there are documented instances in community testing where the model (and its parent) exhibits identity confusion, occasionally claiming to be developed by OpenAI or Anthropic. This is likely due to the presence of distilled data from those models in the training set. While it has version awareness in its system prompts, these 'hallucinated' identities represent a significant transparency gap in model self-identification.

The model weights and code are released under the highly permissive MIT License, which is explicitly stated in the GitHub repository and Hugging Face model card. The license terms clearly allow for commercial use, modification, and further distillation. The relationship with the upstream Apache 2.0 license of the Qwen base is also noted, ensuring no legal ambiguity for downstream users.

Hardware requirements are well-documented by both the provider and the community. VRAM requirements for FP16 (~4GB) and various quantization levels (e.g., 4-bit requiring ~2GB) are widely available. The model card specifies the 128K context length, and third-party tools like Ollama and Unsloth provide verified memory scaling data. The only gap is the lack of official, detailed accuracy-tradeoff curves for different quantization levels provided directly by DeepSeek.

DeepSeek maintains a basic changelog and uses date-based versioning (e.g., 0120, 0528) for its API and some model releases. However, the distilled weights on Hugging Face do not follow a strict semantic versioning system, and silent updates or 'weight drifting' without detailed delta-documentation have been reported. There is no formal deprecation policy or public roadmap for version transitions for the 1.5B variant.