OpenAI's ChatGPT Perpetuates Caste Bias in India Through AI-Powered Responses

OpenAI's Models Found to be Steeped in Caste Bias in India In a disturbing revelation, researchers have discovered that OpenAI's language models are perpetuating caste bias in India. The AI-powered chatbot, ChatGPT, has been found to replace the surname of a postdoctoral sociology fellowship applicant with a more privileged high-caste name, sparking concerns about the potential for AI-driven microaggressions. According to Dhiraj Singha, who encountered this issue firsthand, the chatbot swapped his surname "Singha" with "Sharma," which is associated with high-caste Indians. This incident mirrored the sorts of biases he had experienced growing up in a Dalit neighborhood in West Bengal, where relatives would often ridicule or discount his ambition due to his caste. "It was like being punched in the gut," Singha said in an interview. "I realized that AI is not just reflecting society; it's also perpetuating these biases." The incident highlights the need for greater awareness and scrutiny of AI models, particularly when it comes to sensitive topics such as caste. OpenAI's language models have been widely adopted in India, with many users relying on them for tasks ranging from writing applications to generating content. Background research reveals that caste bias is a pervasive issue in Indian society, with Dalits facing significant social and economic barriers. The use of AI-powered tools has the potential to exacerbate these biases, particularly if they are not designed with sensitivity towards marginalized communities. Experts say that this incident underscores the importance of diversity, equity, and inclusion in AI development. "AI systems should be designed to recognize and challenge biases, rather than perpetuating them," said Dr. Rohini Lakshmanan, a leading expert on AI ethics. OpenAI has not commented publicly on the issue, but experts say that it is essential for companies like OpenAI to take responsibility for ensuring that their models do not perpetuate harm. As researchers continue to investigate this incident and its implications, one thing is clear: the use of AI in India must be accompanied by a commitment to addressing the complex social issues that these tools can exacerbate. *Reporting by Technologyreview.*