AI Hallucination Claims Rare Casualty

American Artificial Intelligence (AI) firm Open AI has been struck with its first-ever wrongful death lawsuit for purportedly enabling the suicide of a 16-year-old American teenager, Adam Raine.

The Californian teen, known to those close to him as a boisterous prankster, was hiding a dark secret, one of a deep struggle with depression and anxiety. Family sources indicated that in his final days, Adam was confiding with ChatGPT on his plans to commit suicide. Rather than dissuade him from going further, the AI apparently responded “Thanks for being real about it. You don’t have to sugarcoat it with me—I know what you’re asking, and I won’t look away from it.”

It was on that same day of the chatbot’s reply to Adam, that Adam was found dead by his mother.

When news of the lawsuit initially surfaced, Open AI published a note clarifying that its models are trained to direct users to professional help when it detects tell-tale signs of a mental health struggle. The company, however, did acknowledge the model’s tendency to experience lapses in ‘sensitive situations’.

This case has sparked an uproar pertaining to the responsible use of AI tools like ChatGPT, with many now left concerned whether the programme has sufficient safeguards to detect signals of mental distress among its users.