Typhoon-2-8B

The model is explicitly identified as being built on the Llama 3.1-8B architecture. The technical report ('Typhoon 2: A Family of Open Text and Multimodal Thai Large Language Models') provides a detailed description of the continual pre-training methodology, including the use of Grouped-Query Attention (GQA), Rotary Positional Embeddings (RoPE), and SwiGLU activation. The transition from Typhoon 1.5 (Llama 3) to Typhoon 2 (Llama 3.1) is well-documented, though specific hyperparameter tuning for the continual pre-training phase is only partially disclosed.

The technical report discloses the use of the 'Typhoon2-Corpus,' which includes a mixture of English and Thai data. It mentions specific sources like CommonCrawl and a 'Synthetic Textbook' dataset (5,000 samples) inspired by the Phi/Cosmopedia approach. However, while categories and some methodologies (MinHash/LSH deduplication) are public, the exact percentage breakdown of the full training mixture and the specific proprietary Thai datasets used by SCB 10X are not fully transparent.

The tokenizer is publicly accessible via the Hugging Face repository and is documented as an extension of the Llama 3.1 tokenizer with Thai-specific tokens. Vocabulary size and tokenization behavior (e.g., handling of Thai characters and tone marks) are described in official documentation. The alignment between the tokenizer and the Thai-centric training data is verifiable through the provided code and model files.

The model is clearly stated to be a dense 8 billion parameter model. As a dense architecture, the active parameters match the total parameters. While the architectural breakdown (layers, attention heads) is standard for Llama 3.1, the documentation confirms these specifications remain consistent in the Typhoon variant. Quantization impacts are mentioned but not comprehensively documented for all precision levels.

Documentation mentions the use of H100 GPUs for training and provides some high-level context (e.g., 'academic budget' equivalents for research previews). However, the specific total GPU hours, energy consumption, and carbon footprint for the production Typhoon-2-8B model are not publicly disclosed in detail.

The technical report provides extensive benchmark results across Thai-specific exams (ThaiExam, ONET, TGAT) and standard benchmarks (MT-Bench, IFEval). While the results are detailed, the full evaluation harness and exact prompt templates used for every benchmark are not fully public, making exact third-party reproduction difficult without additional internal information.

The model consistently identifies itself as Typhoon-2 and acknowledges its Llama 3.1 foundation in its naming convention and system prompts. It is transparent about its bilingual nature and its limitations as a base or instruct model. There is no evidence of the model claiming a competitor's identity or misrepresenting its origin.

The model is released under the Llama 3.1 Community License, which is a custom license with specific commercial use restrictions (e.g., for entities with >700M monthly active users). While the weights are 'open,' the license is not a standard OSI-approved open-source license like Apache 2.0, despite some marketing materials occasionally using the term 'open source' loosely.

Basic VRAM requirements are provided in community documentation and Hugging Face model cards, noting that 8B models generally fit on consumer GPUs with 12GB+ VRAM. Quantized versions (Q4_K_M) are available with associated file sizes, but a comprehensive official guide on context-length memory scaling and quantization-accuracy tradeoffs is missing.

The model follows a versioning scheme (v1.5 to v2.0), and deprecation timelines for older versions are communicated via the OpenTyphoon API documentation. However, a detailed, granular changelog for weight updates or specific training data additions between minor versions is not consistently maintained in a public repository.