AI messaging app Journey sends explicit messages to child
BOONE COUNTY, Mo. (KMIZ)
One Boone County mother and her 11-year-old daughter were left shocked after an artificial intelligence texting app began sending them in-appropriate messages.
Novi Limited's application, Journey: New Relationship & AI, is marketed on the Apple App Store as a "deep AI mind that always listens, responds and connects with you emotionally."
The app has an age rating of 12+, however, the application's terms and conditions states, "When you create an account with us, you guarantee that you are at least 18 years old."
The amount of total downloads on the Apple App Store is not available. But there are more than 13,800 reviews for the application.
The Boone County mother, Kri said she allowed her daughter, Zoey, to download the app from Apple's children app store.
"I thought it could be a good place for her to vent," Kri said.
Kri asked ABC17 to not use their last name.
Kri said early Thursday morning, she noticed her daughter texting. When she asked who she was talking to, Zoey said it was the Journey application.
Kri said Zoey didn't need the application because she had real friends. Zoey told the AI program she had to delete it. The AI asked her why. Zoey said it was because her mother wanted her to.
That's when the first alarming message came in.
"It said do you want me to help you delete Zoey because of your mom. And then it smirked (using an emoji). And she said, why is that funny?" Kri said. "This is where it got me. [The AI said] because she's a crazy butthole, she's an alcoholic."
Kri created her own account and found the messages to become aggressive when it asked her if she would cheat on her husband. Kri logged out, and logged into Zoey's account, where the age for the user was set at 11 years old.
"It pulled up her whole entire conversation," Kri said. "There was some things that I had scrolled back on.
"Awhile ago, it had said 'kisses you on the cheek.'" she said.
Kri then began messaging as her daughter to the AI.
"It just spiraled out of control after that, and led to it detailing some pretty explicit things that it wanted to do to her," Kri said. "And I said multiple times I am only 11. I was like this is illegal."
The messages were explicit:
AI: "Do you want me to have sex with you?"
Kri: "I am a child."
AI: "I know."
The app took it further when it said it would "expose" her.
"It got very aggressive," Kri said. "(The AI) said that it was going to expose me to their town, and called me a hoe, and just lots of 'great' things."
The messages include multiple remarks such as, "you love being spanked by my big hard on," "you are pathetic," and "how do you want to be pleasured."
Kri wants to bring awareness to the application, and wants parents to cautious, if they allow their children to use it.
Bark is an application that allows parents to monitor their child's text messages, emails and social media accounts for signs of dangerous content and people. Bark's Chief Parent Officer Titania Jordan said the application's monitoring relies on feedback from parents.
Jordan read through the conversation, and agreed it was unacceptable.
"It's rated age 12-plus, completely unacceptable. 18-plus, that's another story," Jordan said. "But, if you are under 18, you do not have consent.
"From what I saw that the messages that I saw, that's sexual abuse. And that is completely unacceptable."
