The model's base architecture is explicitly identified as Llama-3.2-3B. The DeepSeek-R1 technical report and associated GitHub documentation provide a clear methodology for the distillation process, which involves supervised fine-tuning (SFT) on 800,000 reasoning traces generated by the larger DeepSeek-R1 (671B) teacher model. While the base model is a standard transformer, the specific fine-tuning hyperparameters (learning rate 2e-5, batch size 16, 1 epoch) are publicly documented in the model card and repository.

The training data is described as a curated set of 800,000 samples consisting of reasoning traces (Chain-of-Thought) and non-reasoning data generated by DeepSeek-R1. While the quantity and general nature of the data are disclosed, the specific breakdown of sources (e.g., percentage of math vs. code vs. general text) is not provided in detail. Furthermore, the actual 800k dataset has not been fully released for public inspection, though the methodology for its creation via rejection sampling and verification is documented in the R1 paper.

The model uses the standard Llama 3 tokenizer with a vocabulary size of 128,256 tokens. The tokenizer is publicly accessible via the Hugging Face repository and is fully compatible with existing Llama 3 infrastructure. Documentation specifies the use of custom special tokens (e.g., <think> and </think>) to facilitate the reasoning process, and these are correctly integrated into the provided chat templates.

The model is a dense architecture with 3.21 billion total parameters. Unlike Mixture-of-Experts (MoE) models where active parameters vary, this model's density is consistent and clearly stated. The architectural breakdown follows the standard Llama 3.2-3B configuration, and the impact of the distillation process on these parameters is well-understood within the context of SFT.

While the training compute for the teacher model (DeepSeek-R1) is extensively documented (2.788M H800 GPU hours), the specific compute resources used for the distillation of this 3B variant are not explicitly stated. Estimates suggest the cost is negligible compared to the teacher model, but verifiable hardware hours, carbon footprint, and specific cluster configurations for this variant are missing from official documentation.

DeepSeek provides comprehensive benchmark results (AIME, MATH-500, GPQA) in their technical report and GitHub. However, the evaluation code for the distilled variants is not as thoroughly documented as the main R1 model. While third-party verification on the Open LLM Leaderboard is available, the exact prompts and few-shot examples used for the distilled 3B variant specifically are less transparent than the main model's evaluation suite.

The model consistently identifies itself as a distilled version of DeepSeek-R1 based on the Llama architecture. It adheres to the expected reasoning format using <think> tags and does not exhibit identity confusion with other models like GPT-4. Versioning is clear, and the model's behavior aligns with its stated purpose as a reasoning-focused distilled variant.

The model is governed by the Llama 3.2 Community License, as it is a derivative of Meta's Llama 3.2-3B. While DeepSeek's own code is MIT licensed, the weights are subject to Meta's restrictive terms (e.g., the 700M monthly active user limit). There is some potential for confusion between the MIT license of the R1 repository and the Llama community license of the distilled weights, though the model card explicitly clarifies the latter.

Hardware requirements are well-documented by both the provider and the community. VRAM requirements for FP16 (~7GB) and various quantization levels (4-bit ~2.5GB) are clearly established. Documentation includes guidance on using tools like vLLM and llama.cpp, and the scaling of memory with context length (up to 128k) is publicly known.

The model follows a basic versioning scheme, but a detailed changelog or history of iterations for the 3B variant specifically is lacking. While the main DeepSeek-R1 model has seen updates (e.g., the 0528 version), the distilled variants are often released as static checkpoints without a clear roadmap for tracking or addressing behavioral drift over time.