Typhoon-2-70B
Parameters
70B
Context Length
128K
Modality
Text
Architecture
Dense
License
Apache-2.0
Release Date
1 Jun 2024
Knowledge Cutoff
-
Technical Specifications
Attention Structure
Multi-Head Attention
Hidden Dimension Size
-
Number of Layers
-
Attention Heads
-
Key-Value Heads
-
Activation Function
-
Normalization
-
Position Embedding
Absolute Position Embedding
System Requirements
VRAM requirements for different quantization methods and context sizes
Typhoon-2-70B
Typhoon-2-70B is a 70 billion parameter Thai language model with enhanced reasoning and context understanding. It is optimized for complex Thai language tasks including legal document analysis, cultural content generation, and multi-turn conversations. The model maintains the 128K context length of its smaller variant. Released under the Apache 2.0 license.
About Typhoon
Typhoon is a Thai language model family developed by SCB 10X. It is specifically optimized for the Thai language, addressing complexities such as the lack of word delimiters and tonal nuances. The models are trained on Thai-centric datasets including legal, cultural, and historical documents to ensure localized context and knowledge.
Other Typhoon Models
Evaluation Benchmarks
No evaluation benchmarks for Typhoon-2-70B available.
Rankings
Overall Rank
-
Coding Rank
-
GPU Requirements
Full Calculator
Choose the quantization method for model weights
Context Size: 1,024 tokens