ChatGLM3-6B

ChatGLM3-6B is explicitly documented as a dense Transformer-based model utilizing Multi-Query Attention (MQA), SwiGLU activation functions, and Rotary Positional Embeddings (RoPE). The architecture is a refinement of the General Language Model (GLM) framework, which is well-documented in the 2024 technical report 'ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools'. While the high-level architecture is clear, the specific layer-by-layer configuration and pre-training hyperparameter details are less granular than top-tier open-source models.

The model is stated to be trained on approximately 1 trillion tokens of a diverse bilingual (Chinese-English) corpus. However, the exact composition breakdown (e.g., specific percentages of web, code, and books) is not publicly disclosed. Documentation vaguely refers to 'more diverse training datasets' and 'high-quality and educational sources' without providing a verifiable data provenance map or sample data for inspection.

The tokenizer is publicly accessible via the official GitHub repository and Hugging Face. It uses a unified vocabulary of 150,000 tokens based on the SentencePiece implementation, specifically optimized for Chinese and English. The vocabulary size and tokenization approach (byte-level BPE) are clearly stated and verifiable through the provided 'tokenization_chatglm.py' and 'tokenizer.model' files.

The model's parameter count is consistently stated as 6.2 billion (often rounded to 6B). As a dense architecture, all parameters are active during inference. The architectural breakdown is verifiable through the 'config.json' and 'modeling_chatglm.py' files in the official repository, which specify 28 layers and a hidden size of 4096.

There is virtually no public information regarding the specific training compute resources used for ChatGLM3-6B. While the technical report mentions 'sufficient training steps' and 'optimized training strategies,' it fails to disclose GPU/TPU hours, hardware specifications, training duration, or the associated carbon footprint. This lack of transparency makes environmental and cost impact assessments impossible.

The model provides scores for standard benchmarks like MMLU, GSM8K, and MATH. While the GitHub repository includes some evaluation scripts and mentions few-shot/zero-shot settings, it lacks a comprehensive, one-click reproduction suite with exact prompt templates for all 40+ claimed benchmarks. Third-party verification is available through leaderboards like OpenCompass, but the internal evaluation methodology remains partially opaque.

ChatGLM3-6B generally identifies itself correctly as an AI assistant developed by Zhipu AI and Tsinghua University. It maintains a consistent versioning identity in its system prompts. However, some users have reported minor identity confusion in specific edge cases where it might default to generic 'AI assistant' responses without branding, though it does not falsely claim to be a competitor's model.

The model weights are released under a custom 'ChatGLM3-6B License' which allows for free academic research and free commercial use after a mandatory registration process. While the terms are relatively clear, the requirement for a questionnaire-based registration for commercial use adds a layer of friction and potential restriction that deviates from standard open-source licenses like Apache 2.0 (which is used for the code but not the weights).

Hardware requirements are well-documented. The official documentation provides specific VRAM estimates for FP16 (approx. 13GB) and INT4 quantization (approx. 5GB). It also details support for various acceleration backends including TensorRT-LLM, OpenVINO, and MPS for Mac deployment, providing users with clear guidance on deployment feasibility.

The model uses basic versioning (e.g., v1.0.0), but a detailed, public changelog tracking behavioral drift or specific weight updates over time is lacking. While the GitHub repository shows commit history, it does not provide a high-level semantic versioning map that explains the impact of updates on model performance or safety alignment.