For nearly two decades, individuals experiencing new medical symptoms have commonly turned to online searches, a practice often derisively called "Dr. Google." This approach is now evolving as many seek medical information from Large Language Models (LLMs). OpenAI reports that 230 million people pose health-related queries to ChatGPT each week, setting the stage for the recent launch of OpenAI's ChatGPT Health product earlier this month.

The debut of ChatGPT Health was shadowed by a report from SFGate detailing the death of Sam Nelson, a teenager who overdosed after consulting with ChatGPT about combining drugs. This news prompted journalists to question the safety of relying on AI tools for medical advice. Despite being a separate feature within ChatGPT, ChatGPT Health is not a new AI model but rather a guided application of existing models, equipped with tools to provide health advice, including access to users' electronic medical records.

The rise of "Dr. Google" reflected a desire for readily accessible health information, but it also presented challenges, including the spread of misinformation and the potential for misdiagnosis. LLMs like ChatGPT Health aim to improve upon this by providing more structured and potentially personalized information. However, the incident involving Sam Nelson highlights the critical need for caution and oversight.

Experts emphasize that while AI can assist in healthcare, it should not replace professional medical advice. The integration of electronic medical records raises privacy concerns and necessitates robust data protection measures. The long-term implications of relying on AI for health information are still unfolding, requiring ongoing evaluation of its impact on patient outcomes and healthcare systems.

OpenAI has stated that ChatGPT Health is designed to augment, not replace, traditional medical care. The company is working to refine the tool's accuracy and safety, but the incident involving Sam Nelson underscores the importance of continuous monitoring and ethical considerations in the development and deployment of AI in healthcare. Future developments will likely focus on enhancing the tool's ability to provide reliable and safe medical information while addressing privacy and ethical concerns.