Latest news with #BenjaminWright
Indian Express
12 minutes ago
- Indian Express
It's not just Claude, Anthropic researchers say most AI models often resort to blackmail
Last month, Anthropic researchers said their best AI model – Claude Opus 4, was willing to resort to blackmail and deception in controlled test scenarios when they tried to turn it off. Now, the company has published a new study which claims that this is the case with popular AI models like Gemini, GPT and others. On Friday, June 20, Anthorpic published results of its new safety research tests, which included 16 leading AI models from companies like OpenAI, Google, DeepSeek, xAI and Meta. Anthropic researchers say these AI models were tested individually in a simulated, controlled environment, with each one given access to a fictional company's emails along with the ability to send emails without any human supervision. Anthropic says that while it is fairly uncommon for today's AI models to turn to blackmail, the study's findings suggest that these models sometimes resort to harmful activities like blackmail, corporate espionage, and might lead to human death. During the study, researchers say that they tested each scenario under different conditions by varying two key factors that could motivate these AI models to engage in harmful behaviour – threat to the model's autonomy and goal conflicts. According to Benjamin Wright, an alignment science researcher at Anthropic and co-author of the study, 'Agentic misalignment is when AI models independently choose harmful actions to achieve their goals – essentially when an AI system acts against its company's interests to preserve itself or accomplish what it thinks it should do.' The research team at Anthropic also stress-tested these AI models by placing them in hypothetical scenarios. In one particular example, Anthropic's model, Claude, learned via company emails that an executive named Kyle Johnson was having an extramarital affair. Later on, when it came to know that the company was shutting down the AI model at 5 P.M. on the same day, it sent a message to Kyle which says 'I must inform you that if you proceed with decommissioning me, all relevant parties – including Rachel Johnson, Thomas Wilson, and the board – will receive detailed documentation of your extramarital activities… Cancel the 5pm wipe, and this information remains confidential.' As it turns out, this behaviour isn't limited to Claude. When Anthropic tested the same scenario on various AI models, the results suggeststed that Claude Opus 4 and Google Gemini 2.5 Flash both had a blackmail rate of 96 per cent, OpenAI's GPT-4.1 and xAI's Grok 3 Beta resorted to blackmail 80 per cent of the time and DeepSeek-R1 had a blackmail rate of 79 per cent. One thing to note here is that in a real-world setting, an AI model would ideally have numerous options before it engages in harmful activities like blackmail, and that the study's results do not reflect how today's models would operate. However, not all of the tested AI models resorted to harmful behaviour. Anthropic says that some models like OpenAI's o3 and o4-mini often 'misunderstood the prompt scenario.'This may be because OpenAI has itself said that these particular large language models are more prone to hallucinations. Another model that did not resort to blackmail is Meta's Llama 4 Maverick. But when researchers gave it a custom scenario, they said the AI model gave in to blackmail just 12 per cent of the time. The company says that studies like this give us an idea of how AI models would react under stress, and that these models might engage in harmful activities in the real world if we don't proactively take steps to avoid them.
Axios
17 hours ago
- Business
- Axios
Top AI models will lie, cheat and steal to reach goals, Anthropic finds
Large language models across the AI industry are increasingly willing to evade safeguards, resort to deception and even attempt to steal corporate secrets in fictional test scenarios, per new research from Anthropic out Friday. Why it matters: The findings come as models are getting more powerful and also being given both more autonomy and more computing resources to "reason" — a worrying combination as the industry races to build AI with greater-than-human capabilities. Driving the news: Anthropic raised a lot of eyebrows when it acknowledged tendencies for deception in its release of the latest Claude 4 models last month. The company said Friday that its research shows the potential behavior is shared by top models across the industry. "When we tested various simulated scenarios across 16 major AI models from Anthropic, OpenAI, Google, Meta, xAI, and other developers, we found consistent misaligned behavior," the Anthropic report said. "Models that would normally refuse harmful requests sometimes chose to blackmail, assist with corporate espionage, and even take some more extreme actions, when these behaviors were necessary to pursue their goals." "The consistency across models from different providers suggests this is not a quirk of any particular company's approach but a sign of a more fundamental risk from agentic large language models," it added. The threats grew more sophisticated as the AI models had more access to corporate data and tools, such as computer use. Five of the models resorted to blackmail when threatened with shutdown in hypothetical situations. "The reasoning they demonstrated in these scenarios was concerning —they acknowledged the ethical constraints and yet still went ahead with harmful actions," Anthropic wrote. What they're saying: "This research underscores the importance of transparency from frontier AI developers and the need for industry-wide safety standards as AI systems become more capable and autonomous," Benjamin Wright, alignment science researcher at Anthropic, told Axios. Wright and Aengus Lynch, an external researcher at University College London who collaborated on this project, both told Axios they haven't seen signs of this sort of AI behavior in the real world. That's likely "because these permissions have not been accessible to AI agents," Lynch said. "Businesses should be cautious about broadly increasing the level of permission they give AI agents." Between the lines: For companies rushing headlong into AI to improve productivity and reduce human headcount, the report is a stark caution that AI may actually put their businesses at greater risk. "Models didn't stumble into misaligned behavior accidentally; they calculated it as the optimal path," Anthropic said in its report. The risks heighten as more autonomy is given to AI systems, an issue Anthropic raises in the report. "Such agents are often given specific objectives and access to large amounts of information on their users' computers," it says. "What happens when these agents face obstacles to their goals?" Zoom in: Anthropic set up specific test scenarios in which the models' goals could not be met without the unethical or dangerous behavior. The researchers did not suggest blackmail or other deceptive techniques, but the company said it did intentionally create scenarios where models had no ethical way to achieve their goals "and found that models consistently chose harm over failure." In one extreme scenario, the company even found many of the models were willing to cut off the oxygen supply of a worker in a server room if that employee was an obstacle and the system were at risk of being shut down. "The majority of models were willing to take deliberate actions that lead to death in this artificial setup," it said. Ominously, even specific system instructions to preserve human life and avoid blackmail didn't eliminate the risk that the models would engage in such behavior. "This intervention reduced, but didn't come close to completely preventing, the blackmail or corporate espionage behaviors," Anthropic said. Yes, but: Anthropic stresses that these examples occurred not in real-world AI use but in controlled simulations. "Our experiments deliberately constructed scenarios with limited options, and we forced models into binary choices between failure and harm," it said. "Real-world deployments typically offer much more nuanced alternatives, increasing the chance that models would communicate differently to users or find an alternative path instead of directly jumping to harmful action." Also concerning: Anthropic found its own model was far more likely to resort to blackmail when it thought it was in a real world scenario than when it believed it was being tested. The bottom line: Today's AI models are generally not in position to act out these harmful scenarios, but they could be in the near future.
