(Yicai) July 21 -- The metric for evaluating embodied intelligence needs to shift from "whether the output is realistic" to "how reliably a robot can execute a task," according to executives from six Chinese world model companies.

"It is essentially about giving the area a new yardstick," Tao Dacheng, chief scientist at Ace Robotics, said at a forum hosted by the company during the World Artificial Intelligence Conference, which was held from July 17 through yesterday in Shanghai.

Unlike large language models, which already have widely accepted benchmarks such as massive multitask language understanding, the embodied intelligence industry still lacks an independent, third-party evaluation system.

Physical IQ, a new benchmark platform jointly initiated by the Shanghai AI Industry Association, Shenzhen Hetao College, and the East China branch of the China Academy of Information and Communications Technology, and with the joint participation of over 20 universities and companies, was launched during the forum. However, experts are not unanimous on how to measure reliability.

"I don't think world models need to truly understand physics," said Shen Yujun, chief scientist at Ant Group's embodied-intelligence unit Robbyant.

Some video-generation foundation models the industry relies on are moving toward closed-source, noted Zhu Zheng, co-founder and chief scientist at GigaAI, also known as Jijia Shijie. If the industry does not start training its physical-world foundation models soon, it may face a situation where "no models are available" in the future, Zhu stressed.

During testing, AgiBot found instances where objects disappeared out of thin air during grasping actions, or "stuck" to the robot without any physical contact, said Ren Guanghui, executive director of the startup's Embodied AI Research Center. "This clearly violates the laws of physics."

Behind the discussion lies the same unresolved question of whether embodied intelligence needs to "understand" the physical world, or does it only need to "predict" it well enough?

Price of Reliability

From demonstration to deployment, the experts unanimously raised the issue of cost.

The cost required to push deployment success rates from 0 to 90 percent is roughly on par with the cost needed to advance from 90 to 99 percent, while the investment to refine from 99 percent to 99.9 percent equals that of the 90 to 99 percent phase, noted Wang Hao, chief technology officer of X Square Robot.

On the surface, the closer one gets to perfection, the smaller the numerical gains appear, yet for every order-of-magnitude reduction in the error rate, the resources and costs needed are at equal magnitude, Wang added.

Ace Robotics' experience in real-world scenarios is that making intelligence cheaper and reliable is more challenging than making models smarter, Tao pointed out. The company unveiled its Kairos 3.1 embodied intelligence model, Ambient Capture Engine 2.0, and three commercial solutions at WAIC on July 19.

The Kairos 3.1 is an action-oriented world model that unifies generative, physical, and cognitive intelligence, which can autonomously adjust its strategy and retry after execution failures, according to the firm. The three solutions target retail, hospitality, and quadruped robots, with the instant-retail one having already been deployed in some convenience stores and planning to expand to about 1,000 outlets within a year.

Data is critical, said Wang Xiaogang, chairman of Ace Robotics. To reach embodied intelligence's ChatGPT moment, when robots can handle tasks and environments they have never seen before, "you first need to get enough data," he noted. "When we have ten million hours of data, then we can see the emergence of intelligence for robotics."

Unlike autonomous driving, where vehicles on the road automatically accumulate massive amounts of data, embodied intelligence lacks such a feedback loop, Wang pointed out, adding that the industry has accumulated roughly 100,000 hours of robot manipulation data, but achieving true intelligence emergence may require "one hundred or even one thousand times more."

To try and solve this challenge, Ace Robotics has factory workers and hotel cleaners wear the ACE Ego sensor kit to collect data during their daily work. The system can keep synchronization errors between different sensors within one millisecond, the company said.

The move is Ace Robotics's concrete path to bridging the gap between "100,000 hours" and "ten million hours," Wang stressed. Before data collection workflows mature, companies specializing purely in data services still have some room to survive, but "that window won't last very long," he added.

China's Physical AI Opportunity

Asked whether physical AI might produce a breakthrough like DeepSeek and send a sudden signal to the world that China leads in a given technology, Wang said that "for language models, OpenAI already showed the way, Chinese teams, to some extent, follow what's published and try to catch up.

"For embodied AI, for world models, it's not converged yet," Wang noted. "So even for Chinese companies, they also have the opportunity to try to take some leading role."

Chinese companies' position near the front in the race comes down to three factors: supply chain, application scenarios openness, and a vast talent base, he pointed out. "Robot is a complex system, you need a long supply chain, you need many partners, which can make the cost very low."

Leading firms in China's retail, hospitality, and other sectors "want to try new things and collaborate with embodied intelligence companies," scenarios not only contribute data but also provide real-world feedback, Wang added. "Over the past few years, China has cultivated a large number of young talents, which is why you can see so many embodied intelligence startups."

Regarding why open-source models, such as Kairos, dominate the landscape, Wang said that "in China, some focus on hardware bodies, some on data, some on models. In this situation, open source becomes particularly important.

"If you don't open source, others have no idea what you're doing, and they can't help you improve," he stressed, adding that this is in contrast with the closed-source model more common in the United States.

On whether China will be the first to achieve the ChatGPT moment, Wang noted, "You have the possibility, but for world models, there's no answer. So everyone is trying their own innovation."

Editor: Martin Kadiev