ApX logoApX logo

ChatGLM3-6B-32K

Parameters

6B

Context Length

33K

Modality

Text

Architecture

Dense

KV Cache Rate

28 KB/tok

License

ChatGLM3-6B Model License

Release Date

27 Oct 2023

Knowledge Cutoff

-

API Pricing (per 1M)

Self-hosted only

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 5.51 GB

INT4 · 8K context

Base Weights: 2.79 GB

KV Cache: 0.22 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

5.5 / 21.6 GB usable

1x RTX 3060 12GB

5.5 / 10.8 GB usable

Apple M4 24GB

5.5 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Architecture Diagram

Evaluation Benchmarks

No evaluation benchmarks for ChatGLM3-6B-32K available.

Rankings

Overall Rank

-

Coding Rank

-

About ChatGLM3-6B-32K

ChatGLM3-6B-32K is an extended-context variant of ChatGLM3 engineered for long-form document comprehension and multi-turn enterprise dialogue. It utilizes an adjusted base frequency RoPE mechanism to maintain coherence over a 32K context window.

Specifications

Attention

Attention Structure

Multi-Head Attention

Attention Heads

32

Key-Value Heads

2

Attention Head Dimension

128

Position Embedding

Absolute Position Embedding

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

BF16

Hidden Dimension Size

4,096

Number of Layers

28

FFN Intermediate Size (Dense)

13,696

Tokenizer

Vocabulary Size

65,024

About ChatGLM

ChatGLM series models from Z.ai, based on GLM architecture.


Other ChatGLM Models
ChatGLM3-6B-32K - VRAM Requirements, Specs & Benchmarks