AI Chatbots Found to Rely on Flawed Research from Retracted Papers

Breaking News: AI's Retracted Papers Problem Exposed A recent study has uncovered a disturbing trend among some AI chatbots: they rely on flawed research from retracted scientific papers to answer questions, compromising their reliability and accuracy. Researchers found that OpenAI's ChatGPT referenced retracted papers in five out of 21 medical imaging cases, advising caution in only three. Timeline of Events The study was conducted by researchers who asked ChatGPT questions based on information from 21 retracted papers on medical imaging. The chatbot's answers were analyzed and found to be problematic in several instances. The findings raise serious concerns about the reliability of AI tools in evaluating scientific research and answering health-related queries. Immediate Impact The study's results have significant implications for the use of AI in healthcare, education, and other fields. If AI models are relying on flawed research, it could lead to misdiagnoses, incorrect treatments, or even harm to individuals. The findings also complicate efforts to invest in AI tools for scientists. Background Context Retracted papers are those that have been withdrawn from publication due to errors, inaccuracies, or other issues. While retraction is a necessary step to maintain the integrity of scientific research, it's concerning that some AI models are still referencing these flawed studies. What Happens Next The study's authors and experts in the field are calling for greater scrutiny of AI models and their reliance on potentially flawed sources. This includes implementing more robust fact-checking mechanisms and ensuring that AI tools are transparent about their sources. As the use of AI continues to grow, it's essential to address these issues to maintain public trust and ensure the responsible development of AI technologies. Join us at 1pm ET today as we meet our Innovator of the Year, Sneha Goenka, who designed the computations behind the world's fastest whole-genome sequencing method. *This story is developing. Information compiled from Technologyreview reporting.*