技术规格

系统要求

不同量化方法和上下文大小的显存要求

Gemma 1 7B

Gemma is a family of lightweight, decoder-only language models developed by Google, drawing upon the same research and technology used to create the Gemini models. The 7 billion parameter variant, Gemma 1 7B, is specifically designed for text-to-text generation tasks, including question answering, summarization, and reasoning. This model employs a transformer decoder-only architecture.

Key architectural components include Multi-Head Attention (MHA) for its attention mechanism and Rotary Positional Embeddings (RoPE) for encoding positional information. The activation function utilized is GeGLU, and normalization is performed using RMSNorm. The model's training leveraged Google's fifth-generation Tensor Processing Units (TPUv5e), utilizing JAX and ML Pathways for efficient large-scale training.

Gemma 1 7B was trained on approximately 6 trillion tokens of primarily English-language data, encompassing diverse web documents, mathematical texts, and code. Data preprocessing involved stringent filtering to remove harmful or sensitive content, aligning with responsible AI development practices. The model's relatively compact size allows for deployment across various environments, from personal laptops and workstations to cloud infrastructure.

没有可用的 Gemma 1 7B 评估基准。