ApX 标志

趋近智

GLM-4V

参数

9B

上下文长度

128K

模态

Multimodal

架构

Dense

许可证

MIT License

发布日期

15 Jan 2024

知识截止

-

技术规格

注意力结构

Multi-Head Attention

隐藏维度大小

4096

层数

40

注意力头

32

键值头

32

激活函数

-

归一化

-

位置嵌入

Absolute Position Embedding

系统要求

不同量化方法和上下文大小的显存要求

GLM-4V

The GLM-4V model variant, developed by Z.ai, represents a significant advancement in multimodal artificial intelligence. It is a member of the GLM-4 series, designed to process and interpret both high-resolution image and video data alongside textual input. This architecture facilitates a deep integration of visual and linguistic features, enabling the model to perform complex multimodal tasks without degradation in natural language processing capabilities. The design goal is to provide a unified framework for understanding diverse data modalities.

Technically, GLM-4V incorporates a sophisticated architecture that includes a Visual Encoder, an MLP Projector, and a Language Decoder. The Visual Encoder processes visual inputs, including images and videos, often utilizing a modified Vision Transformer (ViT) and handling arbitrary image aspect ratios and resolutions up to 4K pixels. The MLP Projector serves as an intermediary, translating visual features into a format compatible with the language model, and may incorporate techniques like 3D-RoPE for enhanced spatial understanding. The Language Decoder is based on the underlying GLM architecture, responsible for generating coherent textual responses by integrating the processed visual and textual information.

GLM-4V is engineered to support a range of practical applications, including visual question answering, image captioning, and complex object detection. Its capabilities extend to video understanding, where it incorporates temporal information to analyze sequences effectively. The model's design focuses on enabling robust performance in tasks requiring both visual perception and advanced linguistic reasoning, such as interactive tutoring for STEM subjects or generating step-by-step solutions from visual problems.

关于 GLM Family

General Language Models from Z.ai


其他 GLM Family 模型

评估基准

排名适用于本地LLM。

没有可用的 GLM-4V 评估基准。

排名

排名

-

编程排名

-

GPU 要求

完整计算器

选择模型权重的量化方法

上下文大小:1024 个令牌

1k
63k
125k

所需显存:

推荐 GPU