MaLLaM-3B is explicitly documented as a dense decoder-only transformer model based on the Mistral architecture. Technical details are provided in the official GitHub repository and an arXiv technical report, confirming the use of Grouped-Query Attention (GQA), SwiGLU activation, RMSNorm, and Rotary Position Embeddings (RoPE). The model was trained from scratch rather than being a fine-tuned version of an existing model, which is clearly stated and supported by the training methodology documentation.

The training corpus is described as a 349GB (90 billion token) dataset specifically curated for the Malaysian context. Documentation identifies five primary categories: Dedup text, Extra dedup (research papers), Filtered StarCoder, Instruction data, and MADLAD-400 MS. Specific sources like Lowyat, Cari, and government documents are named. While the general composition is clear, a precise percentage breakdown of each category within the final 90B tokens is not explicitly tabulated in a granular format.

The model uses a custom-trained Byte Pair Encoding (BPE) tokenizer with a vocabulary size of 32,000. The tokenizer is publicly available on Hugging Face and was specifically trained on a multilingual corpus including Malay, English, Mandarin, Tamil, Jawi, and Arabic to capture local linguistic nuances. Documentation provides clear instructions on its development and intended language support.

The model is a dense architecture with 3 billion parameters. Unlike Mixture-of-Experts (MoE) models where active parameters might be obscured, the dense nature makes the parameter count straightforward. The architectural configuration (Mistral-based) is standard and verifiable through the provided configuration files on Hugging Face.

Training was conducted using a distributed cluster of 40 GPUs (NVIDIA A100s) managed via Kubernetes and DeepSpeed Zero3. The use of spot instances and AWS infrastructure is disclosed. However, while the hardware type and cluster size are provided, the exact total GPU-hours and the specific carbon footprint calculations are not detailed in the available technical reports.

The technical report mentions competitive performance against ChatGPT-3.5 and Malaysian Mistral on instruction-following tasks. However, the specific evaluation code, exact prompt templates, and comprehensive results across standard benchmarks like MMLU or GSM8K are less detailed compared to major foundational releases. There is a lack of third-party verification for the reported internal benchmarks.

The model is consistently branded as MaLLaM (Malaysia Large Language Model) across all official documentation, GitHub, and Hugging Face. It distinguishes itself clearly from English-centric models and maintains a coherent identity as a localized foundational model. There are no reports of the model misidentifying itself as a competitor's product.

The model and its weights are released under the Apache 2.0 license, which is a standard, permissive open-source license. This is explicitly stated on the Hugging Face model card and the official GitHub repository, providing clear terms for commercial and non-commercial use.

The model is designed for edge deployment, and its 3B parameter size implies a baseline VRAM requirement of approximately 6GB for FP16. While basic VRAM estimates are available through community calculators and the model's compact nature, official documentation could be more explicit regarding the specific accuracy-performance tradeoffs for various quantization levels (Q4, Q8).

While the model is part of a family (1.1B, 3B, 5B), there is limited evidence of a formal semantic versioning system or a detailed public changelog for weight updates. The repository tracks development, but clear markers for 'v1.0' vs 'v1.1' with associated drift analysis are not prominent.