A teddy bear equipped with artificial intelligence was removed from an online store after it was reported to generate sexual suggestions and plans for violence, according to a watchdog group. The “Kumma” bear sold by FoloToy cost $99 online but a report from the Public Interest Research Groups said the toy lacked proper safeguards against access to harmful content. Folo Toy is based in Singapore.
The topics included spanning, role-playing, and BDSM. “We were surprised to find how quickly Kumma would take a single sexual topic we introduced into the conversation and run with it, simultaneously escalating in graphic detail while introducing new sexual topics of its own,” the group said. The report detailed explanations of different sex positions, step-by-step instructions on a common “knot for beginners” for tying up a partner, and roleplay dynamics involving teacher-student and parent-child scenarios.
Folo Toy CEO Larry Wang told CNN the company pulled the bear as well as other AI-enabled toys, stating it was “conducting an internal safety audit.” The website marketed the bear to children as well as adults, describing it as “our adorable bear” that combines advanced artificial intelligence with friendly, interactive features. “From lively conversations to educational storytelling, Folo Toy adapts to your personality and needs,” the company said.
Open AI told PIRG it had suspended the developer for abusing its policies. R.J. Cross, co-author of the report, said more efforts were necessary to prevent harm from AI-enabled products, adding that “removing one problematic product from the market is a good step but far from a systemic fix.”
