ByteDance's Zhang Rejects Model Distillation as Shortcut in Long-Term AI Push(Yicai) Aug. 6 -- ByteDance founder Zhang Yiming has banned the use of model distillation from open-source models, a technique in which new artificial intelligence models learn from the outputs of more advanced models rather than from human-labeled training data, arguing that the Chinese AI-focused tech firm should be willing to sacrifice short-term gains in pursuit of long-term technological breakthroughs, The Paper reported today.
Zhang told ByteDance's large language model development team during a recent internal meeting that, despite mounting pressure from the rapid progress of open-source AI models developed by Chinese rivals this year, the Beijing-based company would not use model distillation as a shortcut to improve its AI models, the report said.
AI model development should be guided by long-term principles and delayed gratification, rather than relying on the outputs of other models to achieve a temporary ranking on performance benchmarks, Zhang added.
ByteDance should be willing to sacrifice some short-term profits in order to achieve long-term goals, he said. In his view, distillation could interfere with genuine long-term technological breakthroughs. ByteDance has therefore implemented a strict internal ban on distilling open-source models and has strengthened enforcement through measures such as Application Programming Interface-based detection.
Competition among Chinese AI model developers has become extremely fierce this year, with new models being released almost every few days, the report said, citing ByteDance insiders. Whenever a powerful new open-source model is launched, discussions within ByteDance over whether to use distillation tend to intensify.
Zhang's comments come as ByteDance boosts investment in AI. The Beijing-based company's AI capital expenditure is likely to surge six-fold this year from two years ago to USD70 billion, according to public data. ByteDance is the only major AI firm in China whose LLM portfolio remains almost entirely proprietary, while its competitors have focused on open-source models.
Competition in China's AI model sector continues to heat up. Earlier today, DeepSeek said it will hike pricing for its API services in the near future. Industry insiders believe the price adjustment signals the imminent launch of the official version of the Hangzhou-based firm's V4-Pro, which is expected to affect overall computing demand.
Editor: Kim Taylor
