GLM-130B
Parameters
130B
Context Length
2K
Modality
Text
Architecture
Dense
KV Cache Rate
280 KB/tok
License
Apache 2.0
Release Date
4 Aug 2022
Knowledge Cutoff
Jul 2022
API Pricing (per 1M)
Self-hosted only
Hardware Sizing
Precision
Context
Workload Memory: 245.19 GB
FP16 · 2048 context
Base Weights: 242.14 GB
KV Cache: 0.55 GB
Runtime & Graphs: 2.50 GB
Minimum Required Hardware
8x NVIDIA H100 80GB
245.2 / 576.0 GB usable · TP 8
Determine requirements for your specific workload:
Open in VRAM Calculator
Architecture Diagram
Evaluation Benchmarks
No evaluation benchmarks for GLM-130B available.
Rankings
Overall Rank
-
Coding Rank
-
About GLM-130B
GLM-130B is a bidirectional bilingual foundation model developed by Tsinghua KEG and Zhipu AI for English and Chinese language comprehension. Built with DeepNorm and RoPE, it enables efficient large-scale inference and blank-infilling generation.
Technical Specifications
Attention
Attention Structure
Multi-Head Attention
Position Embedding
Absolute Position Embedding
Normalization
Deep Normalization
Activation Function
GELU
Dimensions
Hidden Dimension Size
12,288
Number of Layers
70
About GLM Family
General Language Models from Z.ai