ApX logoApX logo

GLM-4.7

Active Parameters

358B

Context Length

200K

Modality

Text

Architecture

Mixture of Experts

KV Cache Rate

368 KB/tok

License

MIT

Release Date

8 Jan 2026

Knowledge Cutoff

Sep 2024

API Pricing (per 1M)

Input: $0.60 · Output: $2.20

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 731.81 GB

FP16 · 8K context

Base Weights: 726.43 GB

KV Cache: 2.88 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

8x NVIDIA H200 141GB

731.8 / 1015.2 GB usable · TP 8

Determine requirements for your specific workload:

Upgrade to Pro

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 4.54.0

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Architecture Diagram

Input TokensToken EmbeddingPosition: AbsoluteHidden: 5.1k · Context: 200K · Vocab: 151.6kx 92 layersRMSNormPre-AttentionMulti-Head Attention96Q / 8KV headsHead dim: 128+RMSNormPre-FFNSparse MoE FFN (8/160 experts)SwishIntermediate: 1.5k+Final RMSNormOutput Logits

Evaluation Benchmarks

Rank

#88

BenchmarkScoreRank

Professional Knowledge

MMLU Pro

0.843

18

Graduate-Level QA

GPQA

0.857

42

Web Development

WebDev Arena

1435

76

General Text

Text Arena

1441

81

0.45

96

Intelligence Index

Artificial Analysis
auto

0.22

Standard

0.17

151

197

Rankings

Overall Rank

#88

Coding Rank

#100

About GLM-4.7

GLM-4.7 is an open-weights Mixture-of-Experts model by Z.ai engineered for agentic programming, terminal automation, and frontend vibe coding. It incorporates a triple-tier thinking framework to preserve reasoning coherence across long interactions.

Specifications

Attention

Attention Structure

Multi-Head Attention

Attention Heads

96

Key-Value Heads

8

Attention Head Dimension

128

Position Embedding

Absolute Position Embedding

RoPE Theta

1,000,000

Normalization

RMS Normalization

Activation Function

Swish

Dimensions

Base Precision

FP16

Hidden Dimension Size

5,120

Number of Layers

92

FFN Intermediate Size (Dense)

1,536

Multi-Token Prediction Heads

1

Tokenizer

Vocabulary Size

151,552

Mixture of Experts

Total Expert Parameters

32.0B

Number of Experts

160

Active Experts

8

Shared Experts

1

FFN Intermediate Size (per Expert)

1,536

Dense Layers Before MoE

3

About GLM-4

GLM-4 is a series of bilingual (English and Chinese) language models developed by Zhipu AI. The models feature extended context windows, superior coding performance, advanced reasoning capabilities, and strong agent functionalities. GLM-4.6 offers improvements in tool use and search-based agents.


Other GLM-4 Models