GLM-4-9B-Chat

The model is documented in a comprehensive technical report ('ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools') and on Hugging Face. It explicitly details the use of a dense transformer architecture with 40 layers, a hidden dimension of 4096, and 9 billion parameters. Key technical modifications such as Grouped Query Attention (GQA), Rotary Position Embeddings (RoPE), and SwiGLU activation functions are clearly stated. The pre-training methodology (autoregressive blank-infilling) and post-training pipeline (SFT, RLHF) are well-described, though specific hyperparameter configurations for the 9B variant are less detailed than the larger family members.

The technical report states the model was trained on 10 trillion tokens of multilingual data (primarily Chinese and English). While it names general categories (webpages, Wikipedia, books, code, papers) and describes a three-stage cleaning pipeline (deduplication, filtering, tokenization), it lacks a precise percentage breakdown of the 10T token corpus. There is no public access to the raw training data or a detailed audit of the specific sources used beyond high-level descriptions.

The tokenizer is publicly available via the Hugging Face repository and the official GitHub. It uses a unified vocabulary of 151,552 tokens (often cited as 150k in reports) and is based on Tiktoken. Documentation confirms support for 26 languages, and the tokenizer files (tokenizer.model, tokenizer_config.json) are accessible for inspection and verification. The alignment between the tokenizer and the claimed multilingual capabilities is verifiable through local testing.

The model is explicitly defined as a dense architecture with 9.0 billion total parameters. Unlike MoE models, all parameters are active during inference, which is clearly stated in the documentation. The architectural breakdown (40 layers, 4096 hidden dim) is provided, allowing for verification of the parameter count. There is no ambiguity regarding 'active' vs 'total' parameters.

The technical report mentions that GLM-4-9B was trained with 'less training compute' than Llama-3-8B, but it fails to provide specific GPU/TPU hours, hardware cluster specifications, or energy consumption data. While the general methodology is public, the lack of concrete compute metrics (e.g., total FLOPs or H100 hours) makes it impossible to verify the environmental impact or exact resource investment.

The model card and technical report provide results for standard benchmarks (MMLU, GSM8K, MATH, AlignBench). However, while some evaluation code is available in the 'LongAlign' and 'ChatGLM' repositories, users have reported difficulties in exactly reproducing reported scores (e.g., on LongBench-Chat) due to missing specific sampling parameters or chat templates in the provided scripts. Third-party verification on leaderboards like Open LLM Leaderboard exists but shows some variance from official claims.

The model consistently identifies itself as GLM-4-9B-Chat and maintains a clear versioning identity within the Zhipu AI / THUDM ecosystem. It does not exhibit the identity confusion common in some fine-tuned models (e.g., claiming to be GPT-4). It is transparent about its 128K context limit (or 1M for the specific variant) and its capabilities regarding tool use.

The model is released under a custom 'GLM-4-9B License'. While it is free for academic research, commercial use requires a separate registration/application process via an official form. This creates a 'semi-open' status that is often marketed as open source but contains significant commercial restrictions and a requirement for 'Built with GLM-4' attribution, which is more restrictive than standard Apache 2.0 or MIT licenses.

VRAM requirements are well-documented for various precisions (FP16, INT8, INT4). Official documentation and community resources (like the Model Memory Utility on Hugging Face) provide clear guidance: ~18-21GB for FP16 inference at 1K context, scaling up for longer contexts. Quantization trade-offs are also discussed in the community and supported by official quantization scripts in the repository.

The model uses a versioning system (e.g., GLM-4-9B-Chat vs GLM-4-9B-Chat-1M), and the GitHub repository maintains a basic changelog. However, updates to the model weights or underlying code (like the August 2024 update requiring transformers>=4.44.0) are sometimes communicated via README updates rather than strict semantic versioning, making it difficult to track subtle behavior drifts over time.