Recently, the Hangzhou Intermediate People's Court in Zhejiang province, east China, published a case study of two individuals who had collected photos of others online. They used AI to generate dynamic videos with these photos featuring actions such as nodding, shaking their heads and blinking. These videos were then used to bypass the facial verification systems of e-commerce platforms and steal personal information from victims' accounts.

With AI-driven "face-swapping" technology capable of bypassing mainstream verification methods, is facial recognition secure? How should we address these risks? Science and Technology Daily spoke with experts to find out.

Why is it so difficult to distinguish a deepfake from a real human face?

"Traditional facial verification relies on 'liveness detection,' using dynamic movements like nodding or shaking one's head to differentiate real users from static photos or videos," said Wei Jiaqi, a senior expert at Tianyi Security Technology Co., Ltd. "However, with the rapid advancement of generative AI, it is now possible to create highly realistic dynamic videos from just a single photograph, even simulating the specific movements required for verification."

Some attacks go further by incorporating techniques such as virtual cameras or video-stream injection to bypass the camera capture process, making it difficult for systems to distinguish between genuine and synthetic faces.

This does not mean that facial recognition technology has become obsolete. Rather, it indicates that traditional approaches, which rely on simple movements for verification, are facing new security challenges. "Facial recognition remains highly valuable for various applications, but it should not be regarded as an infallible method for identity authentication," said Wei.

What new requirements does AI safety governance face?

According to Wei, the ability of AI-generated videos to bypass facial verification indicates a shift in AI safety governance from traditional, static risk prevention to a dynamic, adversarial process — a continuous "arms race" between generative and detection technologies. Consequently, improvements are required across regulatory mechanisms and technical capabilities.

From a regulatory perspective, Wei suggests that the AI safety governance framework should be strengthened. For high-risk scenarios such as identity authentication, financial transactions, and retrieving personal information, authorities should promote establishing tiered and classified safety requirements that align with specific business risks. This ensures that systems do not rely solely on a single biometric identification method. At the same time, mechanisms for labeling deep-synthesized content, tracing the origins of AI-generated material, and conducting safety assessments must be refined.

Additionally, systems should be established for sharing new types of attack samples, issuing risk alerts, and performing dynamic evaluations. This will help ensure that regulatory standards adapt to emerging technologies and risks.

From a technical standpoint, the focus should be on "strengthening defense through offense" by transitioning from passive detection to proactive, adversarial defense. Advanced generative AI models can be used to create highly realistic attack samples, enabling adversarial evaluations of facial verification systems to identify vulnerabilities.

A multi-layered, in-depth defense system should be developed by integrating technologies such as trusted device verification, video-stream injection detection, liveness detection (analyzing physiological traits), multi-factor authentication, and risk-based monitoring of user behavior.

"Safety assessments should not be limited to known attack samples," Wei cautions. "They must also test a system's resilience against the latest generative models, novel forgery techniques, and complex combinations of attacks."

He advocates creating a dynamic, closed-loop process that incorporates real-world cases into safety assessment scenarios. This would cover attack generation, risk identification, defense validation, and continuous optimization, enhancing AI security defenses.