Science & Technology / Technology & Society / Tech Ethics & Impact

AI Black & Gray Industry Explodes: Global Risks and the Fierce Crackdown Battle

The rapid development and popularization of generative AI have spawned a rampant black and gray industry chain worldwide, with various illegal acts such as AI-generated pornographic content, fake news, deepfake fraud and rumormongering emerging one after another. xAI, backed by a $20 billion financing, has been investigated by multiple countries including the EU for its Grok allowing the generation of explicit content involving minors, and overseas AI companies have relaxed content restrictions for commercialization, leading to a surge in AI-related violations. In China, the generative AI user scale has hit 515 million, with minors accounting for 33.8% of the total users, making them a key victim group of AI rights infringements, and various AI-related illegal cases have been continuously exposed and punished. In response to the AI abuse chaos, China has continuously deepened regulatory efforts by launching special rectification campaigns and issuing relevant management measures, while the US has enacted the TAKE IT DOWN Act to crack down on AI-generated illegal content, with major overseas AI companies facing lawsuits over juvenile network security issues. The AI black and gray industry has formed a complete industrial chain, with criminal groups carrying out automated and large-scale illegal operations through AI tools, and the cross-border spread of illegal content and the lack of compliance of some small and medium-sized AI products have increased the difficulty of platform governance. Internet platforms are facing severe risk control challenges, as AI has greatly reduced the cost of illegal crimes while increasing the difficulty and cost of platform review. Illegal forces often take advantage of loopholes such as relaxed night review mechanisms to launch automated attacks, as seen in the paralysis of Kuaishou's content review system caused by black hat attacks. Platforms such as Douyin, Kuaishou and Bilibili have stepped up content cleanup, but they still struggle with problems such as low illegal costs and cross-platform jump of criminal acts. To combat the AI black and gray industry, platforms need to increase investment in review capacity, build inter-platform linkage mechanisms, and adopt AI-based security systems for defense, and the whole society is facing a protracted battle to curb AI abuse.