ChatGLM-6B

ChatGLM-6B is explicitly built on the General Language Model (GLM) framework, which is well-documented in peer-reviewed research (Du et al., 2022). The architecture is a dense Transformer that uniquely combines autoencoding and autoregressive objectives. While the base model and its 2D positional encoding scheme are clearly defined, specific internal layer configurations and hyperparameters for the 6B variant are primarily found in the model's configuration files rather than a dedicated technical report for this specific version.

The model was trained on approximately 1 trillion tokens of a bilingual (Chinese and English) corpus. While the general categories of data are mentioned (webpages, Wikipedia, books, code, and research papers), there is no specific percentage breakdown or disclosure of the exact datasets used. The filtering and cleaning methodology is described at a high level (deduplication, quality filtering), but the lack of source-specific proportions or access to sample data limits transparency.

The tokenizer is publicly available via the 'icetk' library and the official GitHub repository. It uses a byte-level Byte-Pair Encoding (BPE) algorithm with a clearly stated vocabulary size of 130,528 (often cited as ~150k in later iterations, but 130k for the original 6B). The implementation is open-source, allowing for full inspection of tokenization logic and vocabulary alignment with the claimed bilingual support.

The model is consistently identified as having 6.2 billion parameters. As a dense model, all parameters are active during inference. The architectural breakdown is verifiable through the provided source code (e.g., 28 layers, hidden size of 4096). However, detailed documentation on the specific parameter distribution between attention and feed-forward networks is not explicitly summarized in a model card, requiring manual code inspection.

Information regarding the training compute is minimal. While some third-party sources mention a cluster of 1,000 GPUs, official documentation does not disclose the specific GPU/TPU hours, hardware specifications (e.g., A100 vs. V100), or the total training duration. Environmental impact data and carbon footprint calculations are entirely absent.

The model provides results on standard benchmarks like MMLU and C-Eval. While the repository includes some evaluation scripts and the model is integrated into the 'InstructEval' framework, exact prompts and few-shot examples used for the original 6B release are not comprehensively documented in a centralized location. Third-party verification is possible but often shows variance due to the lack of standardized evaluation parameters in the initial release.

ChatGLM-6B demonstrates high identity consistency, correctly identifying itself as an AI assistant developed by Tsinghua University and Zhipu AI in its default system prompts. It maintains a clear versioning distinction from its successors (ChatGLM2/3) and does not exhibit significant identity confusion with competitor models in standard deployments.

The code is released under the Apache 2.0 license, which is highly transparent. However, the model weights are governed by a separate 'Model License' that requires users to fill out a questionnaire for commercial use. This dual-licensing approach creates some ambiguity for commercial developers, as the 'open-source' claim applies to the code but not fully to the weights without additional registration.

Hardware requirements are exceptionally well-documented. The developers explicitly state VRAM needs for various quantization levels (e.g., 13GB for FP16, 6GB for INT4). They provide clear guidance on local deployment on consumer-grade hardware and document the performance-efficiency trade-offs associated with quantization, making it one of the most transparent models in this category.

The project uses a basic versioning system (e.g., v1.1.0) and maintains a Change Log on GitHub. However, the transition between the original ChatGLM-6B and subsequent versions was marked by significant architectural shifts that were not always clearly mapped for backward compatibility. Silent updates to weights on Hugging Face have been noted by the community, and a formal semantic versioning policy is not strictly followed.