Rankings
Overall Rank
-
Coding Rank
-
About ILMU 1.0
ILMU is a Malaysian sovereign language model. Based on a fine-tuned DeepSeek-V3 Reasoning Architecture, it is optimized for localized reasoning and the linguistic profile of Bahasa Malaysia. The model uses reinforcement learning to align logic with local professional standards, specifically ensuring an inclusive approach to Malaysia's diverse religions and traditions, with strict respect for cultures, institutions, and royalty (3R). It is calibrated to recognize Malaysian sensitivities and profanity, to maintain proper social etiquette.
Technical Specifications
Architecture specifications are undisclosed for proprietary models.
Attention
Attention Structure
Multi-Head Attention
Attention Heads
128
Key-Value Heads
1
Attention Head Dimension
Undisclosed
Position Embedding
ROPE
RoPE Theta
Undisclosed
Sliding Window Attention
Undisclosed
Sliding Window Size
Undisclosed
Sliding Window Ratio
Undisclosed
Linear Attention
Undisclosed
Linear Attention Ratio
Undisclosed
Normalization
RMS Normalization
Activation Function
SwigLU
Dimensions
Hidden Dimension Size
2,048
Number of Layers
61
FFN Intermediate Size (Dense)
Undisclosed
Multi-Token Prediction Heads
Undisclosed
Tokenizer
Vocabulary Size
Undisclosed
Mixture of Experts
Total Expert Parameters
37.0B
Number of Experts
257
Active Experts
9
Shared Experts
Undisclosed
FFN Intermediate Size (per Expert)
Undisclosed
Dense Layers Before MoE
Undisclosed
Resources
About ILMU
Intelek Luhur Malaysia Untukmu (ILMU) is a language model developed by YTL AI Labs. Trained on YTL AI Cloud infrastructure, the model is designed for Malaysian social norms and linguistic nuances including Bahasa Melayu and Chinese.
Other ILMU Models
- No related models available