GLM-4-9B-Chat-1M

The model is explicitly identified as a dense transformer architecture with 40 layers and a hidden dimensionality of 4096. It utilizes specific, documented techniques for its 1M context window, including Rotary Position Embeddings (RoPE) combined with YaRN scaling. The pre-training methodology is described in a technical report as involving an autoregressive blank infilling approach, and the model is part of a clearly defined evolutionary lineage (GLM-130B to GLM-4). However, specific details on the exact architectural modifications for the 1M variant versus the 128K base are somewhat high-level.

The training data is described as a multilingual corpus of approximately 10 trillion tokens, primarily in Chinese and English. While general categories like 'books', 'Wikipedia', and 'high-quality web data' are mentioned, there is no specific percentage breakdown or detailed disclosure of data sources. The 1M variant's specific fine-tuning data is noted to include synthetic data generated by the GLM-4-128K model, but the exact composition and filtering methodology remain largely proprietary.

The tokenizer is publicly available via the Hugging Face repository and the official GitHub. It uses a unified vocabulary of 150,000 tokens, which is clearly stated in the technical report. The tokenizer supports 26 languages, and its implementation is verifiable through the provided source code and integration with the 'transformers' library.

The model's parameter count is clearly stated as 9 billion. As a dense architecture, all parameters are active during inference, which is explicitly confirmed in technical documentation to distinguish it from MoE designs. Detailed internal dimensions (40 layers, 4096 hidden size) are provided, allowing for a clear understanding of parameter distribution.

Information regarding the specific compute resources used for training the GLM-4-9B-Chat-1M variant is extremely limited. While the technical report mentions that the GLM-4 family was trained on large-scale clusters, it does not disclose specific GPU/TPU hours, hardware counts, or the carbon footprint for this specific 9B variant. Environmental impact data is entirely absent.

The model provides results on standard benchmarks like MMLU, GSM8K, and LongBench-Chat. However, independent reproduction attempts (e.g., on GitHub) have noted discrepancies between reported and achieved scores, often due to sensitive sampling parameters or chat template applications. While evaluation code for some benchmarks is available in the 'LongAlign' repository, the full suite of prompts and exact settings used for official claims are not comprehensively documented.

The model consistently identifies itself as part of the GLM-4 family developed by Zhipu AI. It maintains clear versioning between the 128K and 1M variants. There are no documented cases of the model claiming to be a competitor's product (e.g., GPT-4) or misrepresenting its fundamental nature as an AI.

The licensing structure is split: the accompanying code is under the Apache 2.0 license, while the model weights are governed by a separate 'GLM-4 Model License'. This community license allows for free use but includes a requirement for commercial entities to apply for a separate agreement if they exceed certain scale thresholds. This dual-license approach is documented but adds complexity compared to pure open-source licenses.

VRAM requirements are well-documented for standard inference (approx. 19-21GB for FP16) and for various quantization levels (Q4, Q5, Q8) through community and official documentation. The impact of the 1M context window on memory scaling is addressed with specific guidance on using vLLM and tensor parallelism to avoid OOM errors, though detailed context-length-to-VRAM scaling tables are mostly community-derived.

While the model has a clear release date and version name (GLM-4-9B-Chat-1M), there is no formal, public-facing changelog or semantic versioning system for weight updates. Users must rely on GitHub commit history or Hugging Face 'last updated' timestamps to track changes. There is no formal policy for documenting or notifying users of behavioral drift or safety alignment updates.