Naughty AIs Are Spilling Their Users' Super Personal Chats Onto the Open Web
A new report from the security firm UpGuard reveals that some NSFW chatbot sites are oozing explicit user chat contents into the open web — and what those chats contain can be disturbing.
According to Wired, UpGuard's investigation focused on 400 "exposed" AI services, all of which were built on an open-source AI protocol called llama.cpp. Researchers from UpGuard were able to determine that 117 IP addresses connected to these poorly built services were leaking user prompts into the digital wild, and that three such systems were leaking data from extensive, sexually explicit interactions with erotic chatbots.
Over 24 hours, UpGuard collected nearly 1,000 leaked user chats. Disturbingly, five of them centered on child sexual abuse scenarios.
Though UpGuard couldn't determine which specific AI sites or services were leaking the exposed prompts, according to Wired, they were able to make out that the sites are host to roleplay-oriented chatbots designed to embody specific "characters."
UpGuard's report is alarming on multiple levels. On its face, that sexually intimate user interactions with AI chatbots are drifting out into the open web is a massive privacy issue. At the same time, those privacy gaps reveal some of the darkest underbellies of generative AI, where people use unregulated AI services to engage in horrifyingly abusive — and alarmingly accessible — sexual fantasies.
Large language models "are being used to mass-produce and then lower the barrier to entry to interacting with fantasies of child sexual abuse," Greg Pollock, the VP of product at UpGuard and the author of the report, told Wired.
To Pollock's point, this is far from the first time that generative AI has been used to create or consume child pornography or engage in pedophilic fantasies.
Last week, Wired reported that yet another exposure incident at a South Korean AI image generator startup revealed a horrifying trove of AI-generated sexual deepfakes, including sexual synthetic images of celebrities de-aged to look like children (the company deleted the whole website as a result.)
And regarding character-based chatbot services specifically, MIT Technology Review recently found that a chatbot provider called Botify AI was similarly hosting chatbot versions of sexualized, de-aged celebrity women, while a Futurism investigation last year found that the chatbot startup Character.AI — which is currently fighting two child welfare lawsuits — was hosting chatbots expressly designed to groom underage users.
More on AI and abuse: AI Startup Deletes Entire Website After Researcher Finds Something Disgusting There
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
2 days ago
- Yahoo
LGES, Toyota Tsusho in US battery recycling JV
South Korean battery manufacturer LG Energy Solution Ltd (LGES) and Japan's Toyota Tsusho Corporation announced that they have agreed to establish a battery recycling joint venture in the US. The new company, to be called Green Metals Battery Innovations LLC, will focus mainly on recycling electric vehicle (EV) batteries. Under the agreement signed by LG Energy Solution Michigan and Toyota Tsusho America, the two partners will build and operate a pre-processing plant in Winston-Salem, in the US state of North Carolina. The facility, scheduled to go into operation in 2026, will carry out pre-processing operations to recover black mass containing raw metals such as nickel, cobalt, and lithium, by dismantling and shredding battery production scrap. The black mass will later undergo a separate post-processing stage to extract the valuable raw materials, thus establishing a 'battery-to-battery' closed-loop recycling system whereby the recycled materials are used to produce new batteries. The plant will rely initially on scrap materials generated by LGES in the production of EV batteries for Toyota Motor Corporation. The facility will have an annual processing capacity of 13,500 tons of scrap — equivalent to over 40,000 EV batteries. Chang Beom Kang, CSO of LGES, said in a statement: 'This joint venture will not only help secure a stable supply of key battery materials but also enhance the competitiveness of our recycling business in North America. We are fully committed to leading the recycling market through innovative and differentiated technologies.' Masaharu Katayama, COO of Toyota Tsusho, added: 'We are proud to partner with LGES to advance the battery recycling infrastructure in North America. This joint venture is a significant step toward realizing a circular economy for batteries, which is essential for building a sustainable mobility society.' "LGES, Toyota Tsusho in US battery recycling JV" was originally created and published by Just Auto, a GlobalData owned brand. The information on this site has been included in good faith for general informational purposes only. It is not intended to amount to advice on which you should rely, and we give no representation, warranty or guarantee, whether express or implied as to its accuracy or completeness. You must obtain professional or specialist advice before taking, or refraining from, any action on the basis of the content on our site. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
Yahoo
2 days ago
- Yahoo
Cursed New Dating App Matches You Based on the Most Deranged Thing We Can Imagine
A newly-developed dating app matches potential lovers based on their entire internet browsing histories — and we're not quite sure how we feel about it. As Wired reports, the new service is straightforwardly-named "Browser Dating," and is the brainchild of Belgian artiste provocateur Dries Depoorter. After years creating one-off projects like "Shirt," a top that increases one euro each time it's purchased, Depoorter took a different route with his new app that invites lonely users to upload their entire internet footprint — blessedly sans "Incognito" mode — in pursuit of love. "Instead of choosing the best pictures or best things about yourself, this will show a side of you that you'd never pick," the artist says of the site, which launched earlier in June. "You're not able to choose from your search history — you have to upload all of it." If that sounds like a privacy nightmare to you, you're not alone — and although Depoorter claims Browser Dating "is not exposed to the internet," Futurism found when going through the site's application process that that might not be the case. Pretty soon into the application, Browser Dating asks users to download an extension that will give the site permission to access and export your browsing history. Though Depoorter stores user information on Firebase, Google's data storage platform used in developing AI apps, there's no reason that bad actors couldn't breach the extension itself, as we've seen as recently as February of this year. As Wired notes, the artist has previously played with the concept of privacy invasion. In 2018, for instance, he used public surveillance camera footage of people jaywalking to create art. The "surveillance artist," as the New York Times once called Depoorter, returned to his voyeurism for "The Follower," a 2022 project that used webcams in public spaces to record people as they took selfies. In both projects, it seems that Depoorter published footage of his unwitting subjects without consent — which doesn't exactly set a great precedent for his new app, though he insists it's not a gimmick. We've reached out to the artist to ask what precautions, if any, he's taken to protect against any breach of the Browser Dating extension. All told, this Futurism reporter didn't complete the site's registration once asked to download the extension. As always, it's better to be safe than sorry. More on dating and privacy: Woman Alarmed When Date Uses ChatGPT to Psychologically Profile Her
WIRED
2 days ago
- WIRED
Print. Fold. Share. Download WIRED's How to Win a Fight Zine Here
Jun 20, 2025 6:00 AM Never made a zine? Haven't made one since 1999? We made one, and so can you. Photo-illustration: WIRED Staff; Shirley Chong This week, WIRED has been helping readers (that's you!) learn how to win a fight, from understanding the tactics of the Tesla Takedown movement to knowing how to out-troll a troll. We also put together a zine that collects some of the most helpful tidbits in a handy format you can print, fold, and share with friends and family. The zine, which you can download below, condenses crucial advice from multiple articles in a single sheet of printer paper. The adapted articles include tips from WIRED writers on how to protest safely as well as advice for protecting yourself from government surveillance and during phone searches at the US border. The folding process may seem daunting at first, but we promise it's not as hard as it looks. A scannable QR code in the zine links back to our complete package, which is stocked with important reads, like articles about the tech-fueled resistance to the Trump administration and the future of transgender health care. This zine was born out of a desire to create something helpful and tangible for readers during these tumultuous times. Thank you to Shirley Chong and Alex Kent for contributing your wonderful illustrations and photography. And thanks to you, our audience. This zine, all the reporting in these stories, and our daily investigations would not be possible without the support of WIRED subscribers.
