Phi-4

Microsoft provides a technical report and model cards that explicitly identify Phi-4 as a 14-billion parameter decoder-only Transformer. It is documented as an evolution of the Phi-3 architecture with minimal structural changes but significant enhancements to the attention mechanism and context length (extended from 4K to 16K). The training methodology, including a multi-stage process (pre-training, mid-training for context extension, and post-training alignment), is described. However, specific architectural hyperparameters like the exact number of layers, hidden dimensions, or attention head configurations for the 14B variant are less prominently detailed in the primary technical report compared to the 3.8B 'mini' variant.

The model's reliance on synthetic data is a central theme of its documentation, with a disclosed mixture of 40% synthetic data, 30% web/web-rewrites, 20% code, and 10% acquired academic/book data. While the high-level proportions and generation techniques (multi-agent prompting, instruction reversal) are public, the specific datasets, exact web sources, and the 'acquired' academic books remain proprietary. The 'high-quality' filtering criteria are described conceptually but the actual code or specific classifiers used for data selection are not public.

Phi-4 uses a tiktoken-based tokenizer with a vocabulary size of 100,352 (an upgrade from the 32K Llama-based tokenizer in previous versions). The tokenizer is publicly accessible via the Hugging Face repository, allowing for direct inspection and verification of tokenization behavior. Documentation confirms its design for improved multilingual support. Some community-reported issues regarding EOS/BOS token consistency exist, but the technical specifications and access are well-provided.

The model is clearly defined as a dense 14-billion parameter model. Unlike Mixture-of-Experts (MoE) models where active parameters are often obscured, Phi-4's dense nature means all 14B parameters are active during inference. This is consistently stated across official Microsoft Research blogs, technical reports, and model cards. There is no ambiguity regarding its parameter count versus its active computational footprint.

Information regarding the specific compute resources used to train Phi-4 is conspicuously absent. There is no public disclosure of GPU/TPU hours, specific hardware cluster sizes, or the total energy consumption of the training run. While the training duration is vaguely mentioned as occurring between October and November 2024, no carbon footprint calculations or environmental impact data are provided in the official technical report or model cards.

Microsoft reports performance on standard benchmarks (MMLU, MATH, GPQA) and uses the open-source 'simple-evals' framework for some evaluations, which aids reproducibility. However, they also rely heavily on 'internal benchmarks' and custom evaluation platforms (e.g., Eureka) for which the exact prompts and methodology are not fully public. While they discuss decontamination efforts in an appendix, the lack of full evaluation code and prompt sets for all claimed results limits independent verification.

Phi-4 demonstrates high identity consistency, correctly identifying itself as a Microsoft-developed model in standard deployments. It maintains clear versioning within the Phi family and does not exhibit the identity confusion (e.g., claiming to be GPT-4) seen in some other fine-tuned models. Its limitations regarding instruction following and factual knowledge are openly acknowledged in the model card.

The model weights and code are released under the highly permissive MIT License, which is explicitly stated on the Hugging Face repository and in official announcements. This license allows for unrestricted commercial use, modification, and distribution. The clarity of the licensing terms is exemplary for a model of this capability level, with no conflicting 'open weights' vs 'open source' marketing ambiguity.

Basic VRAM requirements are documented, with official guidance suggesting the model is suitable for latency-sensitive and memory-constrained environments. Third-party documentation (e.g., on Hugging Face and community guides) provides detailed VRAM estimates for FP16 (~28-30GB) and various quantization levels (4-bit AWQ requiring ~8-10GB). While Microsoft's own documentation is slightly more general, the availability of the model on platforms like Ollama and Hugging Face ensures that hardware requirements are well-understood by the community.

Microsoft uses clear naming conventions (Phi-4, Phi-4-mini, Phi-4-multimodal), but a formal, detailed changelog or semantic versioning system for weight updates is not prominently maintained. While the release date and data cutoff are provided, there is limited infrastructure for tracking silent updates or behavioral drift over time, especially as the model is integrated into various Azure services.