Chatbots and the Rise of Authoritarian Influence
In any given week, more than a billion people now look to chatbots for information and advice, accessing services that range from general inquiries to more controversial uses like robo-plagiarism and erotica. Among these, ChatGPT boasts a staggering 900 million weekly users. This trend indicates that the figures are likely to rise, with various AI platforms shaping billions of perspectives worldwide. Already, evidence suggests that large language models (LLMs)—the leading form of AI—are influencing users to shift their views.
This evolving landscape raises significant concerns about the potential of chatbots to disseminate state propaganda. Many worry that leading AI labs may consciously design their LLMs to represent pro-regime viewpoints while suppressing dissent. Evidence exists; for instance, the Chinese AI firm DeepSeek has programmed its model to avoid sensitive topics such as the Tiananmen Square massacre, which contradicts the narrative preferred by the Chinese Communist Party (CCP).
However, no authoritarian government currently has the ability to directly manipulate the programming of frontier AI systems like ChatGPT, Claude, and Gemini, which are operated by firms based in the United States. Despite this, it’s possible that authoritarian regimes are indirectly influencing LLM behavior, as recent studies indicate that major chatbots may be skewing their answers in favor of these regimes without overt coercion.
How State Media Can Corrupt Chatbots
AI models learn by identifying patterns within enormous bodies of text, a fact that carries important implications. LLMs do not deliver uniform answers across different languages; certain expressions or arguments may be more prevalent in the training data of particular languages. This disparity becomes problematic when a language is predominantly spoken in a country governed by authoritarian rule, where state-aligned media constitutes a significant share of the available text.
Researchers conducted various studies, primarily focusing on China, to explore this hypothesis. They analyzed whether Chinese Communist Party-aligned media frequently appeared in CulturaX—a major open-source training dataset for LLMs. Remarkably, they found that 1.64% of CulturaX’s Chinese language documents echoed text from state-aligned sources, including Xuexi Qiangguo, an app promoting Xi Jinping Thought. This figure is stark, given that such propaganda was 41 times more common than Chinese-language Wikipedia articles.
Subsequent experiments investigated whether exposure to state media could alter an LLM’s behavior. A team of researchers trained a model, known as Llama 213b, using three different sources: scripted media, unscripted media, and a random assortment of Chinese language documents. Their findings were alarming; increased exposure to Chinese state media resulted in the model becoming more sympathetic to the CCP, especially when trained with scripted propaganda.
While this model is smaller than the frontier AI systems we currently employ, it successfully illustrates that incorporating state media into an AI’s training data can influence its responses. Further, when researchers posed identical political questions in both English and Chinese to Claude and ChatGPT, they discovered that in 75% of instances, Chinese-language prompts yielded answers that were more favorable to the Chinese government.
The study also evaluated whether this phenomenon manifested in other languages predominant in authoritarian states. Across 37 such nations, including Vietnam and Turkmenistan, the response patterns were similar; LLMs provided pro-regime answers more frequently when queried in the dominant languages of these states. In contrast, users in countries with greater press freedom often received more critical responses toward their governments when asking questions in the local language rather than in English.
Robot Propagandists Could Be Uniquely Effective
These discoveries are troubling. Although citizens in authoritarian regimes face a barrage of propaganda, a state-run news outlet lacks the personal engagement that a chatbot provides. While a newspaper may present biased information, a chatbot can interactively answer questions and provide a level of detail that may sway user sentiment.
Moreover, because the source of information from a chatbot is often obscured, users may be more inclined to accept its responses uncritically. Thus, if major LLMs are shaped by authoritarian propaganda, they could serve as particularly effective advocates for oppressive regimes.
AI May Nonetheless Promote Freer Thinking
Importantly, the findings of the Nature study do not indicate that LLMs are explicitly aiding autocratic governments. For instance, while a Vietnamese user of ChatGPT may receive more pro-government responses, that does not automatically translate to increased support for the regime among the populace. Conversely, LLMs could improve information environments even in authoritarian nations.
Interestingly, while ChatGPT might provide favorable answers to governmental inquiries, it still may present information with less bias than traditional sources of political information within those countries. Notably, the CCP has deemed frontier AI models subversive, evidenced by the fact that ChatGPT is banned in China. Moreover, recent research found that even when prompted in Chinese, ChatGPT often espoused left-of-center, anti-authoritarian views and provided suggestions for protest strategies against the government.
Strategies for Mitigating Propaganda Influence
Despite these complexities, AI labs must address the implications of their findings. It is concerning that chatbot users in authoritarian regimes appear to receive a greater volume of pro-government responses than their counterparts in democratic societies; ideally, this trend should be reversed.
While the Nature study did not offer specific guidelines for mitigating this issue, two interventions could be instrumental. First, during the pre-training phase, AI labs could better screen out state-sponsored propaganda from their datasets. Second, during the post-training phase, they could implement mechanisms to discourage models from echoing the narratives of authoritarian regimes—similar to how they currently prevent dissemination of harmful information like suggestions on anorexia or bioweapon development.
Ultimately, chatbots possess the potential to foster more open and informed debates. A machine capable of synthesizing vast amounts of knowledge and delivering it in an easily digestible manner can be a robust resource for the inquisitive. Furthermore, initial evidence suggests that LLMs may help curtail the influence of misinformation and conspiracy theories, albeit in small measures.
Nonetheless, the expansive and increasing power of the largest chatbots poses significant risks. The more influential these platforms become, the more detrimental any inaccuracies may be. Companies like Anthropic, OpenAI, and Google should strive to eliminate systemic biases from their models to minimize their reliance on authoritarian propaganda.
For more information and detailed study findings, click here.
Image Credit: www.vox.com
