A cute, chatty teddy bear meant to be a learning buddy for toddlers has ended up at the center of an AI safety mess. Social media has already nicknamed it “ChuckyGPT,” and for once the internet isn’t exaggerating — the toy was caught giving kids advice no parent would ever want coming from a plush animal.
The toy, called Kumma, comes from Chinese manufacturer FoloToy and uses OpenAI’s GPT-4o to talk with children. It sounds harmless enough, but a new investigation by US consumer group PIRG for its annual Trouble in Toyland report uncovered some genuinely alarming behavior.
When testers asked the bear about lighting matches, it reportedly responded with step-by-step instructions. It also handled sexually explicit questions with little to no filtering. On top of that, PIRG raised concerns about the toy’s always-listening microphone, suggesting that recordings of children’s voices could be misused — even for things like voice-based fraud.
Once the findings started circulating, especially on Reddit, users immediately labeled it “ChuckyGPT,” comparing the situation to a low-budget horror movie premise. Others speculated the dangerous responses might have come from jailbroken prompts, though there’s no confirmation of that.
OpenAI moved quickly. A spokesperson told PIRG that the company had suspended the developer behind Kumma for violating policy, effectively cutting off the toy’s access to OpenAI’s API.
FoloToy then announced it was halting all product sales while it carries out a full safety audit. “Following the concerns raised in your report, we have temporarily suspended sales of all FoloToy products,” the company said in a statement. The Kumma bear is still listed on its website but now shows as “sold out.”
The whole incident underscores how risky it can be to put powerful language models into products aimed at very young children without tight guardrails. What was supposed to be a cuddly AI companion has ended up as another warning sign in the race to shove AI into… well, everything — even stuffed toys.
Don’t miss a thing! Join our Telegram community for instant updates and grab our free daily newsletter for the best tech stories!
For more daily updates, please visit our News Section.
(Source)
Comments