AI Models Leverage Flawed Research from Retracted Papers to Inform Answers

Breaking News: AI Models Relying on Flawed Research from Retracted Scientific Papers A recent study has revealed that some AI chatbots are using material from retracted scientific papers to answer questions, raising concerns about the reliability of AI tools in evaluating scientific research. According to MIT Technology Review, researchers at the University of Tennessee in Memphis found that OpenAI's ChatGPT, running on the GPT-4o model, referenced retracted papers in five cases out of 21 tested. Timeline and Details The study, conducted by Weikuan Gu and his team, asked ChatGPT questions based on information from 21 retracted papers about medical imaging. The chatbot's answers were analyzed for accuracy and reliability. In three instances, the chatbot advised caution when referencing retracted papers, but in two cases, it failed to do so. Immediate Impact and Response The findings have sparked concerns among researchers and experts, who warn that AI tools may be perpetuating flawed research if they rely on retracted papers without proper evaluation. "If people only look at the content of the answer and don't click through to the paper and see that it's been retracted, that's really a problem," said Weikuan Gu. Background Context Retracted scientific papers are those that have been withdrawn from publication due to errors or flaws in research methodology. While AI search tools and chatbots have long been known to fabricate links and references, the use of actual retracted papers to answer questions is a new concern. What Happens Next The study's findings highlight the need for more robust evaluation methods in AI tools to ensure that they are not perpetuating flawed research. Researchers and developers must work together to address these concerns and establish standards for evaluating scientific research in AI applications. As AI continues to play an increasingly important role in scientific discovery, it is essential to prioritize accuracy and reliability in AI tools. Expert Reaction "We need to be aware of the limitations of AI tools and ensure that they are not perpetuating flawed research," said Dr. Gu. "This study highlights the importance of critically evaluating the sources used by AI chatbots and taking steps to prevent the spread of misinformation." *This story is developing. Information compiled from Technologyreview reporting.*