Llama 4 Maverick

Meta provides comprehensive documentation for the Llama 4 Maverick architecture, explicitly identifying it as a Mixture-of-Experts (MoE) model with 128 experts and alternating dense/MoE layers. Technical details include the use of 'early fusion' for native multimodality and 'iRoPE' for context scaling. The model is documented as being pretrained from scratch on a 22-trillion token multimodal dataset, a significant departure from the dense architectures of previous Llama generations.

While Meta discloses the scale of the pretraining data (22 trillion tokens for Maverick) and general categories (publicly available web data, licensed data, and Meta product data like Instagram/Facebook posts), it lacks a granular percentage breakdown of sources. The methodology for filtering and cleaning is mentioned in high-level terms but lacks the detailed documentation found in fully open-source datasets. The inclusion of user interaction data from Meta AI is noted but not quantified.

The tokenizer is publicly accessible via the official Llama GitHub and Hugging Face repositories. It supports a stated vocabulary and 12 primary languages (including English, Hindi, and Thai). Documentation confirms the tokenizer's alignment with the multimodal training data, and its performance is verifiable through standard library integrations like 'transformers'.

Meta is highly transparent regarding the MoE structure of Maverick, clearly distinguishing between the 400 billion total parameters and the 17 billion active parameters engaged per token. The documentation specifies the expert count (128) and the routing mechanism (one shared expert plus one routed expert per layer). This level of detail exceeds industry standards for sparse models.

Meta provides exemplary transparency regarding training compute. Official model cards disclose 2.38 million H100 GPU hours for Maverick, hardware specifications (H100-80GB), and detailed environmental impact metrics, including location-based greenhouse gas emissions (645 tons CO2eq) and a market-based estimate of 0 tons due to renewable energy matching.

Meta reports strong results on standard benchmarks (MMLU-Pro, GPQA, etc.), but independent reproducibility has been inconsistent. While some evaluation metrics are provided, third-party audits have noted significant performance gaps between official claims and public checkpoints. The lack of public evaluation code for the exact 'experimental' variants used in some leaderboard submissions limits full verification.

The model consistently identifies itself as part of the Llama 4 family in standard deployments. However, there have been documented instances of the model exhibiting 'identity confusion' or hallucinating capabilities (e.g., character counting errors) common in large-scale LLMs. Version tracking is clear, with distinct labels for 'Instruct' and 'Experimental' builds.

The Llama 4 Community License Agreement is a custom 'open-weights' license rather than a standard OSI open-source license. It contains significant restrictions, most notably a total ban on use by individuals or companies domiciled in the European Union for multimodal models. It also includes a 700-million monthly active user (MAU) threshold for commercial use, which creates legal ambiguity for large-scale deployments.

Hardware requirements are well-documented for various precisions (FP16, FP8, INT4). Meta provides guidance on VRAM needs for different context lengths, noting that Maverick requires a multi-GPU setup (e.g., an 8xH100 node) for efficient inference. Quantization trade-offs are mentioned, though specific accuracy-loss curves for the 1M token context window are less detailed.

Meta uses versioned releases on Hugging Face and maintains a basic changelog. However, the community has reported 'silent' differences between experimental builds used for benchmarks and the final weights released to the public. While semantic versioning is present, the documentation for behavioral drift between minor updates is not comprehensive.