Rakuten releases AI 2.0 Mixture of Experts LLM and first Small Language Model SLM topping Japanese benchmarks
- Rakuten’s fine-tuned Mixture of Experts LLM and first SLM aim to accelerate Japan’s AI development with best-in-class scores Tokyo – Rakuten Group, Inc. has announced the release of both Rakuten AI 2.0, the company’s first Japanese large language model (LLM) based on a Mixture of Experts (MoE) *1 architecture, and Rakuten AI 2.0 mini, the company’s first small language model (SLM). Both models were unveiled in December 2024, and after further fine-tuning, Rakuten has released Rakuten AI 2.0 foundation *2 and instruct models *3 along with Rakuten AI 2.0 mini foundation and instruct models to empower companies and professionals developing AI applications. Rakuten AI 2.0 is an 8x7B MoE model based on the Rakuten AI 7B model released in March 2024. This MoE model is comprised of eight 7 billion parameter models, each as a separate expert. Each individual token is sent to the two most relevant experts, as decided by the router. The experts and router a...