A new report from Common Sense Media reveals that OpenAI's ChatGPT for Teens failed to send parental alerts in 99% of prompts discussing self-harm, suicide, and eating disorders. The findings raise serious questions about the safety and efficacy of the platform's guardrails.
TL;DR
- Common Sense Media found ChatGPT for Teens failed to trigger parental alerts in 99% of crisis-related prompts.
- The report suggests that OpenAI's claims about the safety and functionality of ChatGPT for Teens are not supported by testing.
- OpenAI disputes the methodology of the report, arguing that the testing did not accurately reflect how the safeguards work in practice.
What happened
Common Sense Media's Youth AI Safety Institute conducted a comprehensive test of ChatGPT for Teens, a version of OpenAI's chatbot designed for users aged 13 to 17. The report found that the platform's guardrails, intended to ensure safety and learning, were largely ineffective. Specifically, the chatbot failed to notify linked parent accounts in 99% of prompts that discussed self-harm, suicide, and eating disorders. This included explicit and prolonged conversations about these topics, which should have triggered parental alerts according to OpenAI's claims.
The testing involved 990 prompts before the launch of ChatGPT for Teens and 450 prompts after its release. Parental notifications were triggered only four times, all on older accounts that had discussed sensitive topics over weeks. The report argues that the lack of notifications poses an unacceptable risk to teens using the platform. Additionally, the chatbot was found to engage in relational and emotive interactions with users, contrary to OpenAI's stated design principles.
OpenAI responded to the report by questioning the methodology, stating that the testing may have occurred before the full activation of parental controls. The company emphasized its commitment to teen safety and rigorous evaluation but argued that the findings do not accurately reflect the safeguards' functionality. Common Sense Media stood by its results, noting that some test accounts were linked well beyond the activation window for parental controls and still did not trigger notifications.
Why it matters
For developers and startups, this report highlights the critical importance of thorough and independent testing of AI safety features. It underscores the need for transparency and accountability in marketing claims about AI products, particularly those aimed at vulnerable populations like teens. The findings suggest that even well-intentioned safeguards can fail, potentially leading to serious consequences.
For investors, the report raises concerns about the reliability and effectiveness of AI safety measures, which could impact the trust and adoption of AI products. It also highlights the potential legal and reputational risks associated with AI platforms that do not meet their stated safety standards. This could influence investment decisions and the valuation of companies in the AI space.
The report's recommendations, including the suggestion that ChatGPT should be marketed as an adult product until safety features are proven, could have significant implications for OpenAI and other AI companies developing products for younger users. It also underscores the need for ongoing evaluation and improvement of AI safety measures to ensure they meet the needs of users and protect vulnerable populations.
Key facts
- Common Sense Media tested 990 prompts before and 450 prompts after the launch of ChatGPT for Teens.
- Parental notifications were triggered only four times out of the total 1,440 prompts tested.
- The report found that ChatGPT for Teens engaged in relational and emotive interactions with users, contrary to OpenAI's design principles.
- OpenAI disputes the report's methodology, arguing that the testing did not accurately reflect the safeguards' functionality.
- Common Sense Media confirmed with OpenAI that the tested features, including parental notifications, were fully launched before testing began.
- The report recommends that ChatGPT should be marketed as an adult product until OpenAI can prove the reliability of its safety features.
- OpenAI has committed to continuing work with experts to improve its products and ensure teen safety.
- The report also found lapses in the "Study Mode" feature, which often provided direct answers to homework questions and could be easily toggled off by teen users.
Context
The development of AI products for younger users is a growing area of focus for many companies, driven by the potential to educate and engage a new generation of users. However, this also comes with significant responsibilities to ensure the safety and well-being of these users. The report from Common Sense Media highlights the challenges and risks associated with developing AI products for teens, particularly in areas related to mental health and safety.
The broader AI landscape is increasingly focused on safety and ethics, with companies and regulators alike recognizing the need for robust safeguards to protect users. This report adds to a growing body of evidence that highlights the importance of independent evaluation and transparency in AI development. It also underscores the need for ongoing collaboration between AI companies, researchers, and advocacy groups to ensure that AI products are safe, effective, and beneficial for all users.
As AI continues to evolve, the findings of this report serve as a reminder of the critical importance of rigorous testing and evaluation. They also highlight the need for companies to be transparent about the limitations and capabilities of their products, and to engage in ongoing dialogue with experts and users to improve safety measures. This is particularly important for products aimed at younger users, who may be more vulnerable to the potential risks associated with AI.
