Do AI Toys Ever Make Mistakes? Here's What Parents Should Know

If you've seen the media around AI toys, a lot of the media centers around one big idea and concern: Do AI toys make mistakes? And the answer, unsurprisingly, is yes: AI toys do make mistakes. But the real distinction is what kind of mistakes these AI toys were making. Let's go into what kind of mistakes happen, why they happen, and what you can do as a parent to keep your child safe.

What Kind of Mistakes Happen?

AI toys, at least in Curio's case, are run on LLMs, the same type of program that ChatGPT and Claude run on. All LLMs can make mistakes in their responses, because they're essentially robots trying to satisfy a user's query. AI toys are an interesting case because they have a couple more "fail points" that a traditional AI wouldn't have. Let's go over all the "fail points" that an AI toy can go through.

Misunderstanding Request

Misunderstanding is the most common reason you'll get a strange or wrong answer from an AI. Kids don't always speak the clearest; they mumble, laugh, or pronounce things wrong, which all goes into what the AI hears.

In addition to just hearing words wrong, sometimes kids can say things that just don't make any sense. It's very funny to us when a kid says something nonsensical, but to an AI, it's trying to answer a query. Both the clearness of a child's voice and sentence structure play into why AI toys will often come out with an incorrect response.

Factual Slip-ups

We've all been there, asking AI a question about something we don't know, but sometimes it can give incorrect information. This same thing can happen with AI toys too.

Picture this: a kid is playing with their AI toy, let's say Lingo for example. The child asks "how did a spinosaurus hunt?", and an AI would say that it hunts on land, but new information tells us that it was semi-aquatic and primarily hunted fish. This is just one example, but AI does get information wrong, especially recent discoveries.

Tone/Context Misses

AI's biggest downside is its lack of reading context and emotions. These are strictly human attributes, so how can you get a computer to understand the nuances of social relationships? AI is trying, though; they are starting to understand jokes and sarcasm.

For example, now if you were to say "work has me dead", AI would respond asking you what about work makes you feel tired. But if you asked early AI with low emotional awareness, it could genuinely be concerned about your health because of your work. This is a micro example of it, and its emotionless ripples are found in almost all AI-generated responses.

Technical Hiccups

Because of how LLMS work, they break words into tokens rather than actual letters. This is why there's the famous example of asking an AI how many r's are there in strawberry, which it would fail at by counting the two rr's in berry as one token, and saying there are 2 r's in strawberry.

Some other issues that AIs can also deal with are large numbers arithmetic, hallucinated sources or facts, and struggling with omitting certain words. So this could be a factor if your AI toy was asked a complicated question.

Why Does It Happen?

Well, it happens because AIs don't think the same way we do. The mistakes aren't malfunctions from the toy; they come from systemic problems in LLMs themselves. The best way to explain it is to imagine you had a friend who read an entire library and could recite any page on command. But the friend read a slightly dated collection, can't understand tone, and occasionally misremembers a page or two.

It sounds bad when put like that, but you have to realize that AIs are extremely helpful and accurate the vast majority of the time. So it's definitely worth it even if you get the occasional wrong answer.

What We Do to Keep Kids Safe

We would rather tell you what we actually do to keep your child safe rather than just saying "trust us". Here are some of the things we do to keep children safe.

Built-in Guardrails

The toys aren't generating responses from a wide-open system, where it can say anything; it's working through layers upon layers of content filtering, built specifically for children. Not only does this mean language content filtering, but it also means steering the actual topic itself back into safe territory. This is built to be as smooth as possible, and the toy will actively seek to talk about a topic to get back on track.

kidSAFE Certified

Our product has been independently reviewed and certified by kidSAFE. This means we've actually reached a third party's own safety and privacy requirements, beyond our measurements.

What Can Parents Do

The absolute best thing you can do as a parent is take an interest in your child's technology. It's as simple as asking them what they talked to their AI toys about, just like how you'd ask what they did at school. It keeps them in check and lets them actually think about what they talk to AI about daily.

One huge advantage of Curio's app is being able to see the actual conversations between your child and the toy. We have an article going over how to use the app located here for parents who are interested.

If your toy says anything off-putting or even gives a strange response, we have a contact page just for incidents like this, and feel free to send a message. It's not a nuisance; it's actually helpful for us to improve our software and toys for the greater good, and you are doing your part.

Not All Mistakes Are The Same

We want to be clear about certain types of mistakes that AI toys can make. There are two clear distinctions between a simple mistake and a critical mistake.

Simple Mistake

The vast majority of the mistakes that AI toys make have something to do with communication and not understanding. These responses won't make sense, or they can give a fact wrong. These are low-stakes, simple mistakes.

Critical Mistake

Mistakes that involve any inappropriate or dangerous information are considered critical mistakes. They rarely, if ever, happen, and the examples mostly take up conversations that could give potentially dangerous information, like "where can I find matches" type of queries. The guardrails in place make this extremely difficult to do, and the odds a child would be fishing for bad information are usually low, and the AI will catch it.

As a parent, it's important to understand the distinctions between these mistakes that an AI can make. One of these is an understandable type of mistake, and the second is absolutely unacceptable from an engineering standpoint.

Trust Through Transparency

No AI system is perfect, and we'd rather say it outright than dance around the idea as a lot of other AI toy companies do. What we can promise you is that we take imperfection seriously, just like any parent's comment about our toys. This isn't a one-and-done fix; it's ongoing. Just like any system, our toys are getting more and more advanced day by day.

Conclusion/TL;DR

Yes, AI toys do make mistakes; it's in the nature of AIs themselves. But there's a clear distinction that needs to be made about what type of mistakes happen. There are the simple mistakes that happen through mishearing a child or a bad prompt. Then there's a critical mistake, which is when an AI toy says something inappropriate. Critical mistakes are very rare, if ever, and AI toys are generally safe. As a parent, remember to stay involved with your kid's play life.