Computing power: Llama 4 multi-version parameters are eye-catching, 2 trillion multimodal behemoths reappear on the throne

Meta officially announced the open-source first native multimodal Llama 4, which adopts MoE architecture for the first time, supports 12 languages, and the first batch of releases has a total of two: the first is Llama 4Scout, which is smaller, with a total of 109 billion parameters, 17B active parameters, 16 experts, and 10 million contexts; The second is the Llama 4 Maverick, which is larger, with a total of 400 billion parameters, 17B active parameters, 128 experts, and 1 million contexts.

In the LMSYS rankings of large models, Llama 4 Maverick jumped to second place (ELO score 1417), behind the closed-source Gemini 2.5 Pro. The biggest highlight of Llama 4 Scout is that it supports 10 million contexts, which is equivalent to 20+ hours of video, and can run on a single H100 GPU (after Int4 quantization).

In benchmarks, the performance outperformed Gemma 3, Gemini 2.0 Flash-Lite, and Mistral 3.1. The Llama 4 model is one of the first models in the Llama family to feature a Hybrid Expert (MoE) architecture. In the MoE model, a single token activates only a small fraction of the total parameters. Compared to traditional dense models, the MoE architecture is computationally more efficient for training and inference, and is able to generate higher quality results with the same training FLOPs budget.

Llama 4 is a native multimodal model that uses early fusion techniques to seamlessly integrate text and visual tokens into a unified model framework. Early fusion is a big step forward because it can pre-train models with massive amounts of unlabeled text, images, and video data.

Meta has also developed a new training method called MetaP, which allows them to more precisely set key model hyperparameters, such as the learning rate and initialization scale of each layer. These carefully selected hyperparameters work well with different batch sizes, model widths, depths, and training tokens. Llama 4 supports open-source fine-tuning by pre-training on 200 languages, including more than 1 billion tokens in more than 100 languages, and the overall number of multilingual tokens is 10 times more than Llama 3.

AI Applications: Gemini Search Visits +9.62% QoQ, DeepSeek Announces New Paper on Scaling for Inference

　　　 Recently, researchers from DeepSeek and Tsinghua University explored different approaches to the Reward Model (RM) and found that the Points-by-Point Generative Reward Model (GRM) can unify the scoring of single, paired, and multiple responses in pure language representation. Based on this preliminary result, the authors of the paper propose a new learning method, Self-Principle Critical Adjustment (SPCT), to promote effective inference time scalable behavior in GRM.

By leveraging rules-based online RL, SPCT enables GRMs to learn to adaptively formulate principles and criticisms based on input queries and responses, resulting in better outcome rewards in the general domain.

Based on this technology, DeepSeek proposes DeepSeek-GRM-27B, which is based on Gemma-2-27B for post-training with SPCT. For inference time scaling, it scales compute usage with multiple samples. With parallel sampling, DeepSeek-GRM can generate different sets of principles and corresponding criticisms, and then vote for the final reward. With larger sampling, DeepSeek-GRM can more accurately judge principles with higher diversity and output rewards at finer granularity to solve challenges.

In addition to voting for better scaling performance, DeepSeek also trains a meta-RM. From the experimental results, SPCT has significantly improved the quality and scalability of GRM, outperforming existing methods and models in multiple comprehensive RM benchmarks without serious domain bias. The authors also compared the inference time scaling performance of DeepSeek-GRM-27B to a larger model with up to 671B parameters, and found that it can achieve better performance in model size than training time scaling. While current methods present challenges in terms of efficiency and task-specificity, with efforts beyond SPCT, DeepSeek believes that GRM with enhanced scalability and efficiency can serve as a versatile interface to a universal reward system, pushing the frontier of post-LLM training and inference.

AI Financing Trends: Xinghaitu's "Small Steps" Financing, Valuation Has Doubled This Year

On April 3, Xinghaitu announced the completion of a series of A2 and A3 rounds of financing, led by Cathay Capital, with a total financing amount of more than 300 million yuan. This means that since 2025, Xinghaitu has raised nearly 100 million US dollars.

The A2 and A3 rounds of financing of Xinghaitu were led by Cathay Capital, with the participation of industrial capital such as Lenovo Venture Capital and Haier Capital, and followed by old shareholders IDG Capital, Hillhouse Venture Capital, Baidu Venture Capital, Tongge Venture Capital, etc., among which some old shareholders continued to raise their quotas and excess in multiple rounds.

The A1 round of financing was completed in February this year, with a total financing amount of nearly 300 million yuan, led by Ant Group, with additional investment from old shareholders such as Hillhouse Venture Capital, IDG Capital, Beijing Robot Industry Fund, Baidu Venture Capital, and Tongge Venture Capital. It can be seen that the cumulative total financing of the Series A series launched by Xinghaitu in 2025 has reached about 100 million US dollars.

According to Xinghaitu, investors are most concerned about the company's full-stack elements and strong strength. The success of embodied smart products does not only depend on the model, but also on the systematic capabilities of the underlying components, the design and manufacturing of the whole machine, and the ability to understand the scene. The company's founding team has industry-leading model technology strength and industrial landing experience, and its hardware capabilities have also been rapidly completed in the past year. At present, Xinghaitu has become one of the very few embodied intelligence companies in China that has end-to-end AI algorithm capabilities, full-link forward R&D and manufacturing capabilities, and actual commercial verification capabilities. If the valuation of Xinghaitu reaches 5 billion yuan, it will become the "vanguard" of the second echelon in the industry.

Investment advice On

April 8, the White House issued a directive requiring federal agencies to appoint chief artificial intelligence officers and develop strategies to expand the application of artificial intelligence in the government. The memo also directs agencies to "develop an AI strategy within six months to identify and remove barriers to the responsible use of the technology and achieve institution-wide adoption maturity." We remain firmly committed to the fact that AI applications are expected to be some phenomenal this year. It is recommended to pay attention to the successful implementation and verification of clinical AI products (688246. SH), iFLYTEK (002230.SZ), a leading manufacturer with AI as the core, and Cambrian (688256. SH), the high-speed communication connector business or significantly benefit from the GB200 volume of Dingtong Technology (688668.SH), Emdoor Information (001314. SZ), accelerate the expansion of the computing power business precision parts leader Maixinlin (688685. SH), Honglin Power (301439. SZ), the new energy business has increased and supplied global motor giants such as Kollmorgen (301196. SZ) and so on.

Risk Warning:

1) The iteration speed of the underlying AI technology is not as fast as expected. 2) Policy supervision and copyright risks. 3) The implementation effect of AI applications is not as expected. 4) Recommend the risk that the company's performance is less than expected.