ChatGLM2-6B

The model is explicitly identified as the second generation of the ChatGLM series, built on the General Language Model (GLM) framework. Technical documentation and the official GitHub repository detail significant architectural modifications from the first generation, including the adoption of Multi-Query Attention (MQA) for KV cache efficiency, Rotary Position Embeddings (RoPE), and RMSNorm for stability. While the pre-training objective (hybrid GLM objective) is named, the specific layer-by-layer configuration is primarily accessible through the open-source code rather than a formal peer-reviewed technical paper for this specific version.

The provider discloses that the model was pre-trained on 1.4 trillion bilingual (Chinese and English) tokens. However, there is no detailed breakdown of the dataset composition (e.g., percentages of web, code, or books). The specific sources of the data are not named, and the filtering or cleaning methodologies are described only in vague terms like 'more and better data' or 'high-quality data.' No sample data or specific data collection protocols are publicly available.

The tokenizer is publicly accessible via the official repository and Hugging Face (tokenization_chatglm.py). It uses a SentencePiece-based approach with a documented vocabulary size of 64,793 tokens. The implementation details, including special tokens like <bos>, <eos>, and <pad>, are clearly defined in the source code. It is specifically optimized for bilingual support, though some minor inconsistencies between tokenizer and model config vocabulary sizes have been noted in community issues.

The model is clearly stated to have 6 billion parameters. As a dense transformer architecture, the active parameters equal the total parameters. The architectural choices, such as the use of Multi-Query Attention, provide a clear understanding of how parameters are distributed across the attention mechanism versus feed-forward networks. However, a precise numerical breakdown of parameter counts per component (e.g., exact FFN vs. Attention split) is not explicitly provided in summary documentation, though it can be derived from the code.

There is almost no transparency regarding the training compute. The provider does not disclose the total GPU/TPU hours, the specific hardware cluster used for the 1.4T token training, or the training duration. Environmental impact metrics, such as carbon footprint or energy consumption, are entirely absent from official documentation.

The provider reports scores on several standard benchmarks (MMLU, C-Eval, GSM8K, BBH) and provides a comparison to the previous version. While the benchmarks are named, the exact evaluation prompts, few-shot settings, and specific versions of the datasets used are not fully documented in a reproducible format. Third-party verification is available through public leaderboards, but the lack of an official evaluation script or detailed methodology limits full reproducibility.

The model consistently identifies itself as ChatGLM2-6B or an AI assistant developed by the GLM team. It maintains a clear versioning identity distinct from its predecessor and successor (ChatGLM3). There are no widespread reports of the model claiming to be a competitor's product (like GPT-4) or denying its nature as an AI, though its self-knowledge is limited to what was included in its alignment training.

The model uses a custom 'ChatGLM2-6B License.' While the weights are open for academic research and free commercial use is permitted, it requires users to complete a registration questionnaire for commercial applications. The license includes specific restrictions related to use cases that might 'undermine China's national security,' which introduces some legal ambiguity for international users compared to standard OSI licenses like Apache 2.0.

Hardware requirements are well-documented. The provider explicitly states VRAM needs for different precision levels, noting that 6GB of VRAM is sufficient for INT4 quantization. Documentation includes specific performance gains from MQA and FlashAttention. Quantization impact is acknowledged, and community-driven tools (like the Hugging Face Model Memory Utility) provide further verifiable data on memory scaling.

The model follows a clear generational versioning (ChatGLM -> ChatGLM2 -> ChatGLM3). However, within the ChatGLM2-6B lifecycle, updates to checkpoints and code are often pushed to the main branch of the repository without rigorous semantic versioning or detailed changelogs for minor revisions. This makes tracking silent performance drift or behavioral changes difficult for developers relying on the latest 'main' branch.