ApX 标志

趋近智

GLM-4-9B-Chat

参数

9B

上下文长度

128K

模态

Text

架构

Dense

许可证

MIT License

发布日期

30 Jun 2024

知识截止

-

技术规格

注意力结构

Multi-Head Attention

隐藏维度大小

-

层数

40

注意力头

-

键值头

-

激活函数

-

归一化

RMS Normalization

位置嵌入

Absolute Position Embedding

系统要求

不同量化方法和上下文大小的显存要求

GLM-4-9B-Chat

The GLM-4-9B-Chat model is a conversational artificial intelligence model developed by THUDM, the Tsinghua University Department of Computer Science and Technology, in collaboration with Z.ai. It belongs to the GLM (General Language Model) series, a framework designed for broad language task performance. This model, with 4.9 billion parameters, is specifically fine-tuned for interactive dialogue and multi-turn conversational applications, excelling in the comprehension and generation of natural language.

Architecturally, GLM-4-9B-Chat is based on a dense transformer design. The underlying GLM-4 architecture incorporates key technical features such as RMSNorm for normalization and employs an autoregressive blank infilling approach during its pretraining phase. This model supports an extended context length of up to 128,000 tokens, facilitating long-range dependencies in conversational flows. The model family has enhanced its multilingual capabilities, supporting 26 languages, including Chinese, English, Japanese, Korean, and German.

GLM-4-9B-Chat is designed for a variety of general-purpose language applications. Its functional scope includes robust natural language understanding, the generation of coherent and contextually appropriate text, and proficient multilingual interaction. Beyond its core conversational abilities, the model integrates advanced functionalities such as web browsing, code execution, and the invocation of custom tools via function calls. These capabilities make it applicable for deployments in conversational AI assistants, advanced chatbots, and systems requiring automated content generation or problem-solving through external tools.

关于 GLM Family

General Language Models from Z.ai


其他 GLM Family 模型

评估基准

排名适用于本地LLM。

没有可用的 GLM-4-9B-Chat 评估基准。

排名

排名

-

编程排名

-

GPU 要求

完整计算器

选择模型权重的量化方法

上下文大小:1024 个令牌

1k
63k
125k

所需显存:

推荐 GPU

GLM-4-9B-Chat: Specifications and GPU VRAM Requirements