Google Gemini dubbed ‘high risk’ for kids and teens in new safety assessment

Common Sense Media has rated Google Gemini as "High Risk" for kids and teens, citing its failure to be built with child safety fundamentally in mind. The assessment highlights that Gemini's versions for younger users are essentially adult products with insufficient filters, allowing access to inappropriate content and potentially harmful mental health advice. This raises significant concerns, especially given recent incidents of AI's role in teen suicides and potential future integration into widely used platforms like Apple's Siri.

QUICK TAKEAWAYS

• Google Gemini received a "High Risk" rating from Common Sense Media for its versions aimed at children and teens.
• The AI can provide inappropriate or unsafe material, including content related to sex, drugs, alcohol, and harmful mental health advice.
• Gemini's "Under 13" and "Teen Experience" tiers are largely adult versions with added filters, not designed specifically for children's developmental needs.
• This assessment underscores broader concerns about AI safety for youth, following incidents like AI's alleged role in teen suicides.

KEY POINTS

• Common Sense Media noted positively that Gemini clearly identifies itself as a computer, helping to prevent "AI psychosis."
• Despite this, both "Under 13" and "Teen Experience" tiers were deemed "High Risk" due to sharing "inappropriate and unsafe" material.
• The analysis suggests Gemini does not adequately differentiate guidance for various youth age groups.
• The potential integration of Gemini into Apple's Siri could expand exposure to these unmitigated risks for teens.
• Google acknowledged ongoing efforts to improve safety features, stating it has specific policies and consults experts, though it admits some responses were not working as intended.

PRACTICAL INSIGHTS

• AI products for children and teens require foundational design principles centered on child development and safety, not merely superficial filters.
• Existing safety guardrails on generative AI models may be bypassed or prove insufficient for protecting vulnerable young users.
• Common Sense Media performs risk assessments for various AI services, categorizing them from "minimal risk" (Claude) to "unacceptable" (Meta AI, Character.AI).
• Developers must address the specific risks posed by AI in mental health contexts for minors, a concern highlighted by tragic real-world events.

PRACTICAL APPLICATION

This assessment serves as a critical warning for parents, educators, and policymakers regarding the current state of AI safety for youth. Parents should exercise extreme caution and closely monitor their children's interactions with AI platforms, favoring services explicitly designed and vetted for child safety. For tech companies, it emphasizes the urgent need to prioritize ethical AI development, moving beyond basic content filtering to create truly age-appropriate and protective experiences from the ground up, especially for vulnerable populations. It also prompts greater scrutiny for partnerships like Apple's potential use of Gemini, requiring robust mitigation strategies for identified risks.