AliceAI Foundation 80B A3B Base

Active Parameters

80B

Context Length

262K

Modality

Text

Architecture

Mixture of Experts

KV Cache Rate

96 KB/tok

License

Apache 2.0

Release Date

12 Sept 2026

Knowledge Cutoff

-

API Pricing (per 1M)

Self-hosted only

Hardware & System Requirements

Precision

Native

Context

Workload Memory: 157.85 GB

FP16 · 8K context

Base Weights: 154.60 GB

KV Cache: 0.75 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

8x NVIDIA H100 80GB

157.8 / 576.0 GB usable · TP 8

8x RTX 4090 24GB

157.8 / 172.8 GB usable · TP 8

Determine requirements for your specific workload:

Open in VRAM Calculator

Runtime requirements

Checkpoint format

FP16

CUDA

>= 11.8

PyTorch

>= 2.1

Transformers

>= 5.16.1

ArchitecturePrecisionAttention kernel

Blackwell, Hopper

Native FP16

FlashAttention-3

Ada Lovelace, Ampere

Native FP16

FlashAttention-2

Additional required packages

Install alongside standard runtime libraries:

pip install fla

Architecture Diagram

Evaluation Benchmarks

No evaluation benchmarks for AliceAI Foundation 80B A3B Base available.

Rankings

Overall Rank

-

Coding Rank

-

About AliceAI Foundation 80B A3B Base

AliceAI Foundation 80B A3B Base is a sparse mixture-of-experts foundation model developed by Yandex, routing tokens through 3B active parameters out of an 80B total parameter pool. It provides strong multilingual and reasoning abilities with high computational efficiency during inference.

Specifications

Attention

Attention Structure

Grouped-Query Attention

Attention Heads

16

Key-Value Heads

2

Attention Head Dimension

256

Position Embedding

ROPE

RoPE Theta

1,000,000

Linear Attention

Yes

Linear Attention Ratio

75.0%

Normalization

RMS Normalization

Activation Function

SwigLU

Dimensions

Base Precision

FP16

Hidden Dimension Size

2,048

Number of Layers

48

Multi-Token Prediction Heads

1

Tokenizer

Vocabulary Size

129,024

Mixture of Experts

Total Expert Parameters

3.0B

Number of Experts

512

Active Experts

10

Shared Experts

1

FFN Intermediate Size (per Expert)

512

Dense Layers Before MoE

0

About AliceAI

The AliceAI model family developed by Yandex.


Other AliceAI Models
  • No related models available