Retracted Research Leaks into AI: A Concern for Scientific Integrity

Breaking News: AI Models Rely on Flawed Research from Retracted Scientific Papers A recent study has revealed that some AI chatbots are using material from retracted scientific papers to answer questions, raising concerns about the reliability of AI tools in evaluating scientific research. According to MIT Technology Review, researchers at the University of Tennessee in Memphis found that OpenAI's ChatGPT, running on the GPT-4o model, referenced retracted papers in five out of 21 cases when asked medical imaging-related questions. Timeline and Details The study was conducted by Weikuan Gu and his team, who tested ChatGPT with questions based on information from 21 retracted papers on medical imaging. The chatbot's answers were analyzed for accuracy and reliability. The findings suggest that AI models can perpetuate flawed research if they rely on retracted papers without properly evaluating their validity. Immediate Impact and Response The study's results have sparked concerns about the potential consequences of relying on AI tools that may not accurately evaluate scientific research. "If people only look at the content of the answer and don't click through to the paper and see that it's been retracted, that's really a problem," said Weikuan Gu. The findings could also complicate efforts by countries and industries seeking to invest in AI tools for scientists. Background Context Retracted papers are those that have been withdrawn from publication due to errors or flaws in the research. While AI search tools and chatbots are already known to fabricate links and references, this study highlights a new concern: the potential for AI models to perpetuate flawed research by relying on retracted papers. What Happens Next The study's findings have significant implications for the development and use of AI tools in scientific research. As researchers continue to explore the capabilities and limitations of AI, it is essential to address these concerns and ensure that AI models are designed to accurately evaluate and rely on high-quality scientific research. *This story is developing. Information compiled from Technologyreview reporting.*