GLM-4

The model's architecture is well-documented as a dense transformer with specific enhancements including Rotary Positional Embeddings (RoPE) and Group Query Attention (GQA). Technical reports and GitHub documentation specify architectural details such as a hidden dimension of 6144, 61 layers, and 48 attention heads. The training methodology is described as a multi-stage process involving pre-training on 15 trillion tokens followed by post-training alignment using human preference data, rejection sampling, and reinforcement learning.

While the total token count (15 trillion) and the inclusion of 'high-quality' and 'synthetic reasoning' data are disclosed, there is no detailed public breakdown of the dataset's composition by source (e.g., specific percentages of web, code, or books). The filtering and cleaning methodologies are mentioned in general terms (deduplication and semantic similarity filtering) but lack the granular documentation required for a higher score.

The tokenizer is publicly accessible via the official GitHub repository and Hugging Face. It uses a byte-level BPE algorithm based on TikToken's CL100k_base with a customized vocabulary of 151,552 tokens. Documentation clearly states the vocabulary size and the tokenizer's alignment with bilingual (Chinese-English) support, which is verifiable through the provided configuration files.

The model is explicitly identified as a dense architecture with 32 billion parameters. Unlike MoE models in the same family, there is no ambiguity regarding active vs. total parameters. Technical specifications provide a clear breakdown of the model's structural components, including layer counts and attention head configurations, which are consistent across official sources.

There is almost no verifiable information regarding the specific compute resources used for training. While the scale of the training (15T tokens) implies massive compute, the actual GPU/TPU hours, hardware specifications, training duration, and carbon footprint are not disclosed in the technical reports or model cards.

The model provides scores for standard benchmarks like IFEval, BFCL-v3, and MMLU. However, while some evaluation code is available in the GitHub repository, the exact prompts and few-shot examples used to achieve the reported scores are not fully disclosed for all benchmarks. Third-party verification is limited, and there is a lack of detailed reproduction instructions for the specific 32B variant.

The model consistently identifies itself as part of the GLM-4 family and correctly distinguishes between its base, chat, and specialized reasoning (Z1) variants. It does not exhibit identity confusion with competitor models and is transparent about its bilingual focus and versioning (e.g., the '0414' suffix indicating the April 14 release).

The model is released under a custom commercial license that allows for free use and commercial activity but includes specific restrictions, such as the requirement to include 'glm-4' in the name of derivative models and prohibitions on military or illegal use. While the terms are stated, the license is not a standard open-source license (like Apache 2.0), leading to some complexity in commercial compliance.

Hardware requirements are well-documented by both the provider and the community. VRAM requirements for various quantization levels (FP16, INT8, INT4) are available, and the model's memory scaling for its 128K context window is discussed in technical documentation. Guidance on using techniques like YaRN for context extrapolation is also provided.

The model uses a date-based suffix ('0414') for versioning, which provides some tracking. However, there is no comprehensive public changelog or detailed documentation of behavior drift between checkpoints. Updates to the model weights and supporting code (e.g., llama.cpp integration) are tracked primarily through community forums and GitHub commits rather than formal semantic versioning.