Llama 4 Scout

Meta provides a clear architectural name and high-level description for Llama 4 Scout, identifying it as a Mixture-of-Experts (MoE) model with 16 experts and 109B total parameters. Documentation specifies the use of 'early fusion' for native multimodality and 'iRoPE' (interleaved Rotary Positional Embeddings) for length generalization. While the model is described as being distilled from a larger 'Behemoth' teacher model, the full pretraining methodology and specific architectural modifications for the 10M context window are described in blog posts and model cards rather than a formal peer-reviewed technical paper, leaving some technical implementation details opaque.

Disclosure regarding training data is limited to high-level generalities. Meta states the model was trained on ~40 trillion tokens of 'multimodal data from a mix of publicly available, licensed data and information from Meta's products and services,' including posts from Instagram and Facebook. However, there is no public breakdown of dataset percentages (e.g., code vs. web vs. books), no detailed documentation on filtering or cleaning methodologies, and no sample data provided for verification. This falls under the 'minimal information' category with significant gaps.

The tokenizer is publicly accessible via the Hugging Face repository and official GitHub. It uses a combination of BPE and WordPiece with a stated vocabulary size of approximately 128,000 tokens. Documentation includes details on special tokens for multimodal content and control. The tokenizer's support for 12 primary languages is well-documented, though its performance on the broader claimed 200 languages is less verifiable without extensive third-party testing.

Meta is transparent about the MoE structure, explicitly stating the model has 109 billion total parameters with 17 billion active parameters per token. The distribution across 16 experts is clearly defined. While a full architectural breakdown of attention vs. FFN parameter ratios is not provided in the standard model card, the active vs. total parameter distinction is handled with high clarity, avoiding the common pitfall of advertising only the total count.

Information on training compute is sparse. While some third-party reports (e.g., Azure/HPCwire) estimate the training required millions of H100 hours and provide carbon footprint estimates (approx. 1,999 tons CO2e), Meta's official documentation lacks a comprehensive compute report. There is no official disclosure of the exact hardware hours, total energy consumption, or detailed cost breakdown in the primary model documentation.

While Meta provides benchmark scores (e.g., MMLU-Pro, ChartQA) in its model cards, independent researchers have reported difficulty reproducing these results, particularly in coding and long-context tasks. Evaluation code is not fully public in a way that allows for one-click verification of the official scores. The lack of detailed prompting strategies and exact few-shot examples in official documentation further hinders reproducibility.

Llama 4 Scout demonstrates high identity consistency, correctly identifying its version and family in standard interactions. It maintains a clear distinction from its larger sibling, Maverick, and the teacher model, Behemoth. There are no documented cases of the model claiming to be a competitor's product or denying its nature as an AI developed by Meta.

The 'Llama 4 Community License Agreement' is a custom, restrictive license that is not OSI-compliant. While the terms are publicly accessible, they contain significant geographic restrictions (specifically excluding EU-based entities from using multimodal features) and commercial usage caps (requiring a separate license for entities with >700M monthly active users). These 'open-weights' but not 'open-source' terms create a complex legal landscape that is less transparent than standard permissive licenses like Apache 2.0.

Hardware requirements are well-documented for various configurations. Meta and partners (NVIDIA, Unsloth) provide specific VRAM estimates for FP16 (~218GB) and Int4 (~55GB), confirming it can run on a single H100 with quantization. However, the memory scaling for the 10M context window is less transparent; while the theoretical limit is stated, practical VRAM requirements for the KV cache at extreme lengths are only available through third-party estimates rather than official scaling tables.

Meta uses a versioning system (e.g., Llama-4-Scout-17B-16E-Instruct), but the changelog and history of updates are not maintained with the rigor of a software project. While major releases are documented, there is limited transparency regarding 'silent' updates or behavioral drift in the hosted versions of the model. The lack of a public, detailed version history for weight checkpoints reduces the score.