An AI Customer Service Chatbot Made Up a Company Policy—and Created a Mess
On Monday, a developer using the popular AI-powered code editor Cursor noticed something strange: Switching between machines instantly logged them out, breaking a common workflow for programmers who use multiple devices. When the user contacted Cursor support, an agent named "Sam" told them it was expected behavior under a new policy. But no such policy existed, and Sam was a bot. The AI model made the policy up, sparking a wave of complaints and cancellation threats documented on Hacker News and Reddit.
This marks the latest instance of AI confabulations (also called "hallucinations") causing potential business damage. Confabulations are a type of "creative gap-filling" response where AI models invent plausible-sounding but false information. Instead of admitting uncertainty, AI models often prioritize creating plausible, confident responses, even when that means manufacturing information from scratch.
For companies deploying these systems in customer-facing roles without human oversight, the consequences can be immediate and costly: frustrated customers, damaged trust, and, in Cursor's case, potentially canceled subscriptions. How It Unfolded
The incident began when a Reddit user named BrokenToasterOven noticed that while swapping between a desktop, laptop, and a remote dev box, Cursor sessions were unexpectedly terminated.
"Logging into Cursor on one machine immediately invalidates the session on any other machine," BrokenToasterOven wrote in a message that was later deleted by r/cursor moderators. "This is a significant UX regression."
This story originally appeared on Ars Technica, a trusted source for technology news, tech policy analysis, reviews, and more. Ars is owned by WIRED's parent company, Condé Nast.
Confused and frustrated, the user wrote an email to Cursor support and quickly received a reply from Sam: "Cursor is designed to work with one device per subscription as a core security feature," read the email reply. The response sounded definitive and official, and the user did not suspect that Sam was not human.
After the initial Reddit post, users took the post as official confirmation of an actual policy change—one that broke habits essential to many programmers' daily routines. "Multi-device workflows are table stakes for devs," wrote one user.
Shortly afterward, several users publicly announced their subscription cancellations on Reddit, citing the non-existent policy as their reason. "I literally just cancelled my sub," wrote the original Reddit poster, adding that their workplace was now "purging it completely." Others joined in: "Yep, I'm canceling as well, this is asinine." Soon after, moderators locked the Reddit thread and removed the original post.
"Hey! We have no such policy," wrote a Cursor representative in a Reddit reply three hours later. "You're of course free to use Cursor on multiple machines. Unfortunately, this is an incorrect response from a front-line AI support bot." AI Confabulations as a Business Risk
The Cursor debacle recalls a similar episode from February 2024 when Air Canada was ordered to honor a refund policy invented by its own chatbot. In that incident, Jake Moffatt contacted Air Canada's support after his grandmother died, and the airline's AI agent incorrectly told him he could book a regular-priced flight and apply for bereavement rates retroactively. When Air Canada later denied his refund request, the company argued that "the chatbot is a separate legal entity that is responsible for its own actions." A Canadian tribunal rejected this defense, ruling that companies are responsible for information provided by their AI tools.
Rather than disputing responsibility as Air Canada had done, Cursor acknowledged the error and took steps to make amends. Cursor cofounder Michael Truell later apologized on Hacker News for the confusion about the non-existent policy, explaining that the user had been refunded and the issue resulted from a backend change meant to improve session security that unintentionally created session invalidation problems for some users.
"Any AI responses used for email support are now clearly labeled as such," he added. "We use AI-assisted responses as the first filter for email support."
Still, the incident raised lingering questions about disclosure among users, since many people who interacted with Sam apparently believed it was human. "LLMs pretending to be people (you named it Sam!) and not labeled as such is clearly intended to be deceptive," one user wrote on Hacker News.
While Cursor fixed the technical bug, the episode shows the risks of deploying AI models in customer-facing roles without proper safeguards and transparency. For a company selling AI productivity tools to developers, having its own AI support system invent a policy that alienated its core users represents a particularly awkward self-inflicted wound.
"There is a certain amount of irony that people try really hard to say that hallucinations are not a big problem anymore," one user wrote on Hacker News, "and then a company that would benefit from that narrative gets directly hurt by it."
This story originally appeared on Ars Technica.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
17 hours ago
- Yahoo
A Jobseeker Says Reddit Paints A Bleak Job Market. But Then Admits People Are Still Getting 'Hired Every Single Day. That's A Fact'
After spending time in multiple career-related subreddits, one Reddit user had a realization that registered with people: Reddit sometimes makes the job market look worse than it really is. 'I've been trying to switch careers recently and joined a bunch of subreddits — tech, healthcare, education, engineering, etc.,' the original poster wrote. 'And in every single one, it's the same thing: 'No jobs' 'The market is dead' 'Everything's saturated' 'You should've started 10 years ago'.' But they pushed back on the despair. 'People get hired every single day. That's a fact,' they said. 'The people who are getting jobs aren't posting here. The ones who are stuck are the ones who are venting.' Don't Miss: Maker of the $60,000 foldable home has 3 factory buildings, 600+ houses built, and big plans to solve housing — Peter Thiel turned $1,700 into $5 billion—now accredited investors are eyeing this software company with similar breakout potential. Learn how you can Their perspective resonated, especially as others chimed in with their experiences. 'I got laid off at the beginning of the year and was terrified because I'm here lurking a lot,' one person commented. 'Luckily, I'm pretty good at interviewing and landed a [work-from-home] job maybe two weeks after. I never posted about how fast I was able to find work, so what you say is true.' Others said the negativity isn't universal across fields. 'Tech jobs in education, medical, and finance are booming right now. I moved companies earlier this year and did not have any trouble finding another fully remote position for a significant raise,' one person added. Still, the thread also highlighted the brutal side of the market. Many shared long stretches of unemployment and feelings of defeat. One mid-level developer said they'd sent out over 100 applications in four weeks and heard back from only five. 'I'm not the best interviewee and am a poster child for, 'if it wasn't for bad luck, I'd have no luck at all.'' Trending: Invest early in CancerVax's breakthrough tech aiming to disrupt a $231B market. New graduates, in particular, seemed to bear the brunt of the pain. 'Some have literally been unemployed for 2-3 years now,' another person said of recent tech grads. 'One of [my friends] is a camp counselor at a coding camp. The other, working IT at a warehousing startup.' He described them as 'Smart kids, high 90's in HS and 3.8 and above GPA in university.' The nursing and teaching sectors drew mixed responses. Many users acknowledged that these fields continue to experience high demand due to staffing shortages, burnout, and high turnover. However, some pointed out that employers often prefer experienced workers, leaving recent graduates without opportunities to gain that very experience. Others emphasized how working conditions and pay in these sectors contribute to why positions remain unfilled, with some describing the workload and pressure as overwhelming despite the steady demand for workers. Reddit's tendency to skew toward doom and gloom was a recurring point. 'Reddit as a collective has the mentality of a depressed 16-year-old. It definitely shouldn't be used as a barometer for anything,' one person joked. Another added, 'It's like reading reviews on Amazon. People only post something negative, while positive is rarely posted.'In the end, the original poster urged job seekers to stay the course. 'Don't let [Reddit] convince you that nothing is working anywhere for anyone. That's just not true. If you're feeling discouraged, I get it. But keep going. You're probably doing better than you think.' Recent data from the U.S. Bureau of Labor Statistics paints a mixed yet still functional employment picture. In May, employers added 139,000 nonfarm payroll jobs, keeping the unemployment rate steady at 4.2%. Job gains were led by health care, leisure and hospitality and social assistance. While federal government payrolls declined, private-sector hiring continued. Though slower than prior months, growth continues, supporting the idea that 'people get hired every single day.' Read Next: Many are using retirement income calculators to check if they're on pace —Up Next: Transform your trading with Benzinga Edge's one-of-a-kind market trade ideas and tools. Click now to access unique insights that can set you ahead in today's competitive market. Get the latest stock analysis from Benzinga? APPLE (AAPL): Free Stock Analysis Report TESLA (TSLA): Free Stock Analysis Report This article A Jobseeker Says Reddit Paints A Bleak Job Market. But Then Admits People Are Still Getting 'Hired Every Single Day. That's A Fact' originally appeared on © 2025 Benzinga does not provide investment advice. All rights reserved.
Yahoo
a day ago
- Yahoo
2 Reasons to Buy Dogecoin (DOGE) before 2026
Dogecoin is an exciting speculative asset. But two other crypto assets have brighter futures. 10 stocks we like better than Dogecoin › It's been an incredible run for Dogecoin (CRYPTO: DOGE). Since 2018, when the coin was valued at $0.017, the crypto token has soared to about $0.17 -- a 1,000% increase. Could another 1,000% growth spurt be in store? There are two reasons you should consider jumping in before 2026 arrives. If you're looking to speculate on a highly volatile asset, Dogecoin is almost perfectly designed for this purpose. The token was created in 2013 purely as a joke. According to the founders, "cryptocurrency was being taken far too seriously and wasn't much fun" at the time, leading the pair to develop and launch Dogecoin. The token's mascot, a Shiba Inu dog, has become an iconic piece of the crypto industry. Originally, the dog breed was used in internet memes. By using the famous image to brand Dogecoin, the crypto token established itself as both an obvious joke and a well-known staple of the fledgling global crypto community. Dogecoin's initial success was surprising. "Dogecoin was a near-instant success, and gained tremendous popularity on sites such as Reddit, where it was -- and still is -- used as a tipping currency," the project's official history reads. "When Dogecoin was just 2-weeks old, it reached more transactions per day than Bitcoin (CRYPTO: BTC). Moreover, within the first month there were over one-million unique visitors to this website!" Why invest in a speculative asset like Dogecoin with minimal real-world tangible value? First, because it's fun. Second, assets like Dogecoin can surge in value much faster and with greater magnitude than most traditional investments. Just note they can lose their value just as quickly. As long as you view your Dogecoin investment as a lottery ticket and not a true long-term investment, it's a suitable asset for some extra discretionary funds. Want to stack the odds in your favor over the long term? Check out the two Dogecoin alternatives below. Don't get me wrong: Dogecoin is a fun, exciting crypto token. But if you want to invest in crypto long term, both Bitcoin and Ethereum (CRYPTO: ETH) are superior options. Why? Because they both have bonafide use cases that can stand the test of time far better than Dogecoin. Bitcoin, of course, was the first major crypto asset ever invented. This gives it a reputational edge that no other crypto asset can match. In use as both a means of exchange and a store of value, Bitcoin should remain a mainstay in the crypto world for decades to come. The magnitude of buy in -- both from retail investors and a growing number of institutional investors -- gives Bitcoin staying power that no other crypto can match. Ethereum, meanwhile, was designed to overcome limitations involved in Bitcoin's structure. That is, Ethereum is much more programmable, allowing developers to use the crypto asset in many more imaginative ways. Today, more developers are working on Ethereum than any other crypto ecosystem. That leads to more innovation, and a stronger incentive for other developers to join. Bitcoin's place in history and Ethereum's dominant developer ecosystem edge create sustainable competitive advantages that should lead to long-term adoption of both crypto projects. Dogecoin's place in history, meanwhile, will largely be determined by prevailing social whims, relegating it to a speculative asset only. Before you buy stock in Dogecoin, consider this: The Motley Fool Stock Advisor analyst team just identified what they believe are the for investors to buy now… and Dogecoin wasn't one of them. The 10 stocks that made the cut could produce monster returns in the coming years. Consider when Netflix made this list on December 17, 2004... if you invested $1,000 at the time of our recommendation, you'd have $659,171!* Or when Nvidia made this list on April 15, 2005... if you invested $1,000 at the time of our recommendation, you'd have $891,722!* Now, it's worth noting Stock Advisor's total average return is 995% — a market-crushing outperformance compared to 172% for the S&P 500. Don't miss out on the latest top 10 list, available when you join . See the 10 stocks » *Stock Advisor returns as of June 9, 2025 Ryan Vanzo has positions in Bitcoin and Ethereum. The Motley Fool has positions in and recommends Bitcoin and Ethereum. The Motley Fool has a disclosure policy. 2 Reasons to Buy Dogecoin (DOGE) before 2026 was originally published by The Motley Fool
Gizmodo
a day ago
- Gizmodo
Reddit Looks to Get in Bed With Altman's Creepy ‘World ID' Orbs for User Verification
Gaze into the Orb if you want your upvotes. According to a report from Semafor, Reddit is actively considering partnering with World ID, the verification system co-founded by OpenAI CEO Sam Altman, to perform user verification on its platform. Per the report, Reddit's potential partnership with World ID would allow users to verify that they are human by staring into one of World ID's eye-scanning orbs. Once confirmed to be a real person, users would be able to continue using Reddit without revealing anything about their identity. Currently, Reddit only does verification via email, which has been insufficient to combat the litany of incoming AI-powered bots that are flooding the platform. Gizmodo reached out to both Reddit and World ID for details of the potential partnership. Reddit declined to comment. A spokesperson for World said, 'We don't have anything to share at this time; however, we do see value in proof of human being a key part of online experiences, including social, and welcome all of the opportunities possible to discuss this technology with potential partners.' For those unfamiliar, World is somewhere between a verification system and a crypto scheme. World ID is a method for verifying that a person is a human without requiring them to provide additional personal information—something the company calls 'anonymous proof of human.' It offers several verification techniques, but the most notable is its eye-scanning Orb. The company claims that neither 'verification data, nor iris photos or iris codes' are ever revealed, but going through the scan gets you a World ID, which can be used on a platform like Reddit, should it partner with World on this endeavor. Somewhere in the backend of this whole thing is a cryptocurrency called Worldcoin, which you theoretically can use at major retailers—but like, can you really? Is anyone doing that? The founders of World, Altman and Alex Bania, launched the crypto part of the program with the intention of building an 'AI-funded' universal basic income. Mostly, it's made local governments really mad and has been at the center of legal and regulatory investigations into how it's handling user data. It's largely targeted developing nations for its early launches, and used some dubious practices along the way to get people to demo the system. Also, it's probably not technically illegal, but it does seem pretty convenient that Sam Altman offers a 'solve' for a problem that his other company, OpenAI, is in no small part responsible for. Almost seems like he knew what issues he was about to cause and decided to cash in on both ends. Must be nice.
