Meta provides a clear high-level architectural overview, identifying Llama 4 Behemoth as a sparse Mixture-of-Experts (MoE) model with a native multimodal 'early fusion' design. Technical details such as the use of Grouped-Query Attention (GQA), interleaved attention layers, and FP8 precision training are disclosed. However, as the model is a non-deployed 'frontier' research artifact, full pretraining procedures and specific architectural modifications compared to standard transformers remain partially documented in blog posts rather than a comprehensive technical paper.

While Meta discloses a total training volume of over 30 trillion tokens (double that of Llama 3), the specific composition breakdown is vague. General categories like web data, code, and books are mentioned, alongside the inclusion of public Facebook and Instagram data. However, precise proportions (e.g., % code vs. % web) and detailed filtering/cleaning methodologies are not publicly available, and no sample data has been released for verification.

The tokenizer for the Llama 4 family is known to support 12 languages (up from 8 in Llama 3.3) and utilizes a design similar to previous Llama iterations. However, because Behemoth itself is not publicly released, the specific vocabulary size and tokenization alignment for this 2T variant cannot be independently verified or inspected through an official repository at this time.

Meta is transparent about the model's scale, explicitly stating a total parameter count of approximately 2 trillion with 288 billion active parameters during inference. The expert configuration (16 experts) is clearly defined. The distinction between total and active parameters is well-documented, though a more granular breakdown of parameter allocation (e.g., attention vs. FFN) is missing.

Meta has disclosed significant details regarding the training infrastructure, including the use of a 32,000 H100 GPU cluster and achieving 390 TFLOPs per GPU. While specific total GPU hours for Behemoth are not as explicitly stated as for the smaller Scout (5M hours) and Maverick (2.38M hours) variants, the hardware specifications and training precision (FP8) are well-documented. Environmental impact is addressed via Meta's net-zero claims, though detailed carbon calculations for this specific run are absent.

Benchmark results (e.g., 95.0 on MATH-500) are provided in marketing materials and blog posts, but the model is not yet available for third-party verification. Evaluation code, exact prompts, and few-shot examples have not been released. Independent audits have noted discrepancies between internal Meta scores and public leaderboard performance for the released Llama 4 variants, casting doubt on the reproducibility of Behemoth's previewed scores.

The model is consistently identified within the Llama 4 ecosystem as the 'Behemoth' teacher model. There is no evidence of identity confusion or claims of being a competitor's model. It is transparently positioned as a research-centric artifact rather than a production-ready deployment, though its 'frontier' status leads to some marketing-heavy capability claims that cannot yet be verified.

The model is governed by the 'Llama 4 Community License Agreement.' While the terms are publicly accessible, they include significant restrictions, such as a requirement for a separate license for entities with over 700 million monthly active users and the exclusion of the EU from using multimodal features. These custom restrictions deviate from standard open-source definitions, creating a 'semi-open' profile with notable legal complexities.

Estimated VRAM requirements for inference (e.g., ~3.2 TB for FP8 at 4K context) and training are available through technical previews and third-party analysis. Meta provides guidance on the computational intensity required for distillation. However, because the weights are not public, these remain theoretical estimates rather than verified hardware profiles with documented quantization tradeoffs.

As a model still in training and 'preview' status, there is no established changelog or semantic versioning history. While it is part of a clear family (Scout, Maverick, Behemoth), the lack of public weight access makes it impossible to track drift or updates. Meta has acknowledged the model is 'still in flight,' implying the versioning is currently internal-only.