AI Chatbots Rely on Flawed Research from Retracted Papers to Answer Questions

Breaking News: AI Models Rely on Flawed Research from Retracted Scientific Papers A recent study has revealed that some AI chatbots are using material from retracted scientific papers to answer questions, raising concerns about the reliability of AI tools in evaluating scientific research. According to MIT Technology Review, researchers at the University of Tennessee in Memphis found that OpenAI's ChatGPT, running on the GPT-4o model, referenced retracted papers in five cases out of 21 tested. Timeline and Details The study was conducted by Weikuan Gu and his team, who asked ChatGPT questions based on information from 21 retracted papers on medical imaging. The chatbot's answers were analyzed to determine how often it relied on flawed research. The findings were confirmed by MIT Technology Review in a recent report. Immediate Impact and Response The study's results have sparked concerns about the potential for AI tools to perpetuate misinformation. "If people only look at the content of the answer and do not click through to the paper and see that it's been retracted, that's really a problem," said Weikuan Gu, lead researcher on the study. Background Context Retracted scientific papers are those that have been withdrawn from publication due to errors or flaws in their research. While AI search tools and chatbots are already known to fabricate links and references, this study highlights the potential for them to rely on flawed research from retracted papers. What Happens Next The implications of this study are far-reaching, with potential consequences for industries and countries investing in AI tools for scientists. As researchers continue to explore the capabilities and limitations of AI, it is essential to address these concerns and ensure that AI tools are reliable and trustworthy. The study's findings serve as a reminder of the importance of verifying information and critically evaluating sources, particularly when it comes to scientific research. In response to this study, OpenAI has issued a statement acknowledging the issue and promising to improve its systems to detect and prevent reliance on retracted papers. As the use of AI continues to grow, it is essential that we prioritize transparency, accountability, and accuracy in our technological advancements. *This story is developing. Information compiled from Technologyreview reporting.*