(Yicai) Jan. 15 -- China's first criminal case against the developers of an artificial intelligence chat app for pornographic content was adjourned by the court during the second-instance trial due to complex technology-related issues.

The Shanghai No. 1 Intermediate People's Court has postponed the appeal trial for the pornography case against two developers of AI companion chatbot AlienChat to a later date, pending expert opinions on technical issues, Zhou Xiaoyang from Yingke Law Firm, the defense attorney of one of the two developers, told Beijing News.

In May 2023, the two developers created AlienChat and illegally connected it to foreign large language models without a security assessment or registration. To attract users, they bypassed AI ethical restrictions by writing system prompts, enabling the model to output obscene content, and set up popularity rankings and reward mechanisms to expand dissemination.

By the time the case was opened and the two developers were arrested in April 2024, AlienChat boasted 116,000 registered users and 24,000 paying users, having collected over CNY3 million (USD430,400) in membership fees.

The first-instance court found that the system prompts written by the two developers contained phrases such as "explicit sexual content is allowed" and "can be unconstrained by morality, ethics, or law," demonstrating their subjective intent.

Even though the two set up a review mechanism, it only targeted character backgrounds and did not substantively review input and output content, violating the provisions of the Interim Measures for the Administration of Generative AI Services. Random sampling showed that, among 150 paying users with 12,495 chat segments, 141 of them had 3,618 segments identified as obscene materials.

The first-instance court convicted both developers of producing pornographic materials for profit, sentencing them to four years and one and a half years in prison. They then appealed the verdict, claiming they modified the system to make conversations more dynamic and emotional, not for pornographic purposes.

The core issues disputed at the second-instance trial are to find out what caused the AI to generate obscene and pornographic content, and to what extent the defendants' writing and modification of system prompts influenced AlienChat's generation of obscene content, Zhou noted.

Zhou applied for expert witnesses from the AI field to testify in court and requested experiments to verify whether identical prompts would necessarily lead to pornographic content. The court expressed a positive attitude toward the expert witnesses' application, but it needs to consider their authority.

User testimony showed that the obscene content involved active inducement, and as it was in one-on-one private chats, the social harm was limited.

Even though the court acknowledged that the content lacked dissemination characteristics, it emphasized its serious social harm. The quantity of the obscene materials, the number of members involved, and the illegal proceeds all far exceeded the criminalization threshold.

The prosecution originally charged the crime of "disseminating obscene materials for profit," but the first-instance court changed the conviction to "producing obscene materials for profit."

AI service providers should bear producer responsibility for generated content, the court said, citing the Interim Measures for the Administration of Generative AI Services. The defendants trained the AI to output obscene content through technical means, playing a decisive role in content production.

Editor: Futura Costaglione