The Yi-6B model is well-documented in an official technical report ('Yi: Open Foundation Models') which details its 'modified' Transformer architecture. It explicitly identifies the use of Grouped-Query Attention (GQA), SwiGLU activation, and Rotary Positional Embeddings (RoPE). While it acknowledges being based on the Llama implementation, it clarifies that it was trained from scratch. However, the 'proprietary' nature of the specific training infrastructure and some methodology details are not fully public, preventing a higher score.

01.AI discloses that the model was trained on a 3.1 trillion token multilingual corpus (primarily English and Chinese). While the technical report describes a 'cascaded data deduplication and quality filtering pipeline' involving heuristic and learned filters, it lacks a detailed percentage breakdown of data sources (e.g., specific web crawls, books, or code proportions). The data itself is not public, and the description remains at a high level of 'highly-engineered' data without granular source transparency.

The tokenizer is publicly accessible via the Hugging Face repository and the official GitHub. It uses a SentencePiece BPE implementation with a vocabulary size of 64,000 tokens. Documentation explains the choice to avoid dummy prefixes for better bilingual (English/Chinese) performance and the use of an 'identity tokenizer' for punctuation. The vocabulary is well-aligned with the claimed bilingual support.

The model's parameter count is clearly stated as 6 billion. As a dense model, all parameters are active during inference. The technical report provides a specific architectural breakdown: 32 layers, a hidden size of 4096, 32 query heads, and 4 KV heads. This level of detail is exemplary for a dense architecture.

Information regarding the specific compute resources used for training Yi-6B is extremely limited. While the report mentions a 'scalable super-computing infrastructure,' it does not disclose the total GPU/TPU hours, the specific hardware count used for the 6B variant, the training duration, or the carbon footprint. This is a significant transparency gap.

01.AI provides benchmark results on standard sets like MMLU, C-Eval, and AlpacaEval in their technical report. They mention following Llama 2's evaluation methodology and using greedy decoding. However, the exact evaluation code and full prompt sets used for all internal benchmarks are not fully public in a single reproducible repository, and independent verification has noted sensitivity to prompt formatting.

The model generally identifies as an AI developed by 01.AI in its chat variants. However, research indicates that the base model (Yi-6B) lacks inherent self-identity without fine-tuning, and there have been documented instances of identity confusion in the broader Yi family where models might misidentify their origin or version when prompted in specific languages or contexts.

The model weights and code are released under the Apache 2.0 license, which is highly transparent. However, 01.AI's official website and some documentation include a requirement to 'apply for a commercial license for free' for certain use cases, creating a conflict with the standard 'unrestricted' nature of Apache 2.0. This ambiguity in the commercial terms reduces the score.

Hardware requirements are well-documented on the Hugging Face model card and in the GitHub README. It provides specific VRAM estimates for inference (approx. 12GB for FP16) and training (approx. 45GB with Adam). Furthermore, it details requirements for 4-bit and 8-bit quantized versions (AWQ/GPTQ), making it highly accessible for users to plan deployment.

The model follows a basic versioning scheme (e.g., Yi-6B, Yi-1.5-6B), and 01.AI maintains a changelog on GitHub and Hugging Face. However, the versioning does not strictly follow semantic versioning for weights, and some updates (like the 200K context extension) were released as separate variants rather than versioned iterations of the base, making tracking of 'drift' in the original model difficult.