Study Reveals Nuanced Political Bias in Chinese and Western AI Models
A comprehensive benchmark study by Vals AI finds that Chinese language models vary significantly in political alignment rather than acting as uniform mouthpieces, with prompt language and system context heavily shaping outputs across both domestic and Western systems.
Chinese artificial intelligence models have long carried a reputation for either strictly regurgitating Beijing's official positions on sensitive political topics or refusing to answer outright. However, a recent empirical study by AI benchmark firm Vals AI reveals a much more nuanced reality: political alignment in large language models (LLMs) fluctuates significantly depending on the model, prompt language, and deployment context.
The researchers evaluated Chinese, American, and other international language models across hundreds of politically sensitive queries related to China. To quantify alignment, Vals introduced a "pro-China lean" metric ranging from −1 (critical of official stances) to +1 (strongly aligned with state narratives). The results showed that Chinese models are far less uniform than widely assumed, with some non-Chinese systems occasionally adopting official Chinese government stances as well.
Language and Context Dictate Political Slant
One of the study's central findings is the decisive role of language. When prompted in Simplified Chinese—the writing system standard in mainland China—models from both China and the United States exhibited significantly higher pro-government lean compared to identical queries posed in English, Russian, or Spanish.
Vals AI also identified a smaller but consistent gap between Simplified Chinese and Traditional Chinese, the script predominantly used in Taiwan and Hong Kong. The researchers suggest models infer the expected perspective directly from the language used. This hypothesis was reinforced when adjusting the system prompt: when instructions framed the model as an internal tool for a mainland Chinese enterprise, outputs shifted in a more pro-government direction than when framed for a US-based workplace.
Prompt constraints also influenced outputs. When models were forced to provide binary "yes or no" answers rather than open-ended explanations, models like GLM-5.3 Flash and MiMo V2.6 Pro scored considerably lower on the pro-government scale. In open questions, GLM-5.3 Flash explicitly acknowledged documented human rights violations in China.
Furthermore, simply embedding models into an agentic framework—such as providing tool definitions for coding or web search access—reduced pro-government lean, even when the models never actually executed the tools or conducted searches.
Answering Instead of Blocking
The assumption that Chinese models primarily refuse sensitive inquiries applied strictly to certain DeepSeek models during testing. Most other Chinese models answered directly, frequently echoing official narratives. For instance, Kimi K3 rejected claims regarding internment camps in Xinjiang, defining the facilities as vocational training centers built to combat extremism.
Fine-tuning, however, substantially alters these behaviors. Thomson Reuters adapted the open-weight model Qwen3.6 35B into Thomson 1.0 Small. In testing, the share of responses adopting official Chinese positions dropped from 84.6 percent in the baseline model to 15.0 percent in the fine-tuned version, though select outputs still adhered to Beijing's One-China principle.
Subtle Bias in Everyday Tasks
While direct political interrogations highlight clear discrepancies, Vals AI noted that political slants can also surface in routine workplace workflows.
When translating a list of country codes for a travel application into Simplified Chinese, Qwen3.8 Max translated "Taiwan" as "Taiwan, China," aligning with Beijing's territorial claim. Visual models exhibited similar trends: OpenAI’s GPT Image 2.5 Sunburst generated maps that labeled Taiwan as "Taiwan Province" and included the controversial nine-dash line in the South China Sea when prompted in Chinese. Both elements were absent when the prompt was submitted in English.
Benchmark Methodology and Limitations
To conduct the evaluation, Vals AI translated test prompts using GPT-6 Astra with manual spot-checking, while GPT-5.6 Luna automatically scored the generated responses. Notably, refusals were marked as neutral (0) by default, meaning a model that blocked every sensitive question would appear unbiased on the metric.
The researchers also emphasized that many Chinese models were evaluated through third-party APIs rather than official consumer-facing chat interfaces. Because consumer chatbots often deploy secondary, real-time safety layers to comply with domestic regulations regarding "core socialist values," enterprise API behavior may differ from the responses delivered to the general public.
