Latest news with #Vox
Vox
4 hours ago
- Vox
AI doesn't have to reason to take your job
is a senior writer at Future Perfect, Vox's effective altruism-inspired section on the world's biggest challenges. She explores wide-ranging topics like climate change, artificial intelligence, vaccine development, and factory farms, and also writes the Future Perfect newsletter. A humanoid robot shakes hands with a visitor at the Zhiyuan Robotics stand at the Shanghai New International Expo Centre in Shanghai, China, on June 18, 2025, during the first day of the Mobile World Conference. Ying Tang/NurPhoto via Getty Images In 2023, one popular perspective on AI went like this: Sure, it can generate lots of impressive text, but it can't truly reason — it's all shallow mimicry, just 'stochastic parrots' squawking. At the time, it was easy to see where this perspective was coming from. Artificial intelligence had moments of being impressive and interesting, but it also consistently failed basic tasks. Tech CEOs said they could just keep making the models bigger and better, but tech CEOs say things like that all the time, including when, behind the scenes, everything is held together with glue, duct tape, and low-wage workers. It's now 2025. I still hear this dismissive perspective a lot, particularly when I'm talking to academics in linguistics and philosophy. Many of the highest profile efforts to pop the AI bubble — like the recent Apple paper purporting to find that AIs can't truly reason — linger on the claim that the models are just bullshit generators that are not getting much better and won't get much better. Future Perfect Explore the big, complicated problems the world faces and the most efficient ways to solve them. Sent twice a week. Email (required) Sign Up By submitting your email, you agree to our Terms and Privacy Notice . This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply. But I increasingly think that repeating those claims is doing our readers a disservice, and that the academic world is failing to step up and grapple with AI's most important implications. I know that's a bold claim. So let me back it up. 'The illusion of thinking's' illusion of relevance The instant the Apple paper was posted online (it hasn't yet been peer reviewed), it took off. Videos explaining it racked up millions of views. People who may not generally read much about AI heard about the Apple paper. And while the paper itself acknowledged that AI performance on 'moderate difficulty' tasks was improving, many summaries of its takeaways focused on the headline claim of 'a fundamental scaling limitation in the thinking capabilities of current reasoning models.' For much of the audience, the paper confirmed something they badly wanted to believe: that generative AI doesn't really work — and that's something that won't change any time soon. The paper looks at the performance of modern, top-tier language models on 'reasoning tasks' — basically, complicated puzzles. Past a certain point, that performance becomes terrible, which the authors say demonstrates the models haven't developed true planning and problem-solving skills. 'These models fail to develop generalizable problem-solving capabilities for planning tasks, with performance collapsing to zero beyond a certain complexity threshold,' as the authors write. That was the topline conclusion many people took from the paper and the wider discussion around it. But if you dig into the details, you'll see that this finding is not surprising, and it doesn't actually say that much about AI. Much of the reason why the models fail at the given problem in the paper is not because they can't solve it, but because they can't express their answers in the specific format the authors chose to require. If you ask them to write a program that outputs the correct answer, they do so effortlessly. By contrast, if you ask them to provide the answer in text, line by line, they eventually reach their limits. That seems like an interesting limitation to current AI models, but it doesn't have a lot to do with 'generalizable problem-solving capabilities' or 'planning tasks.' Imagine someone arguing that humans can't 'really' do 'generalizable' multiplication because while we can calculate 2-digit multiplication problems with no problem, most of us will screw up somewhere along the way if we're trying to do 10-digit multiplication problems in our heads. The issue isn't that we 'aren't general reasoners.' It's that we're not evolved to juggle large numbers in our heads, largely because we never needed to do so. If the reason we care about 'whether AIs reason' is fundamentally philosophical, then exploring at what point problems get too long for them to solve is relevant, as a philosophical argument. But I think that most people care about what AI can and cannot do for far more practical reasons. AI is taking your job, whether it can 'truly reason' or not I fully expect my job to be automated in the next few years. I don't want that to happen, obviously. But I can see the writing on the wall. I regularly ask the AIs to write this newsletter — just to see where the competition is at. It's not there yet, but it's getting better all the time. Employers are doing that too. Entry-level hiring in professions like law, where entry-level tasks are AI-automatable, appears to be already contracting. The job market for recent college graduates looks ugly. The optimistic case around what's happening goes something like this: 'Sure, AI will eliminate a lot of jobs, but it'll create even more new jobs.' That more positive transition might well happen — though I don't want to count on it — but it would still mean a lot of people abruptly finding all of their skills and training suddenly useless, and therefore needing to rapidly develop a completely new skill set. It's this possibility, I think, that looms large for many people in industries like mine, which are already seeing AI replacements creep in. It's precisely because this prospect is so scary that declarations that AIs are just 'stochastic parrots' that can't really think are so appealing. We want to hear that our jobs are safe and the AIs are a nothingburger. But in fact, you can't answer the question of whether AI will take your job with reference to a thought experiment, or with reference to how it performs when asked to write down all the steps of Tower of Hanoi puzzles. The way to answer the question of whether AI will take your job is to invite it to try. And, uh, here's what I got when I asked ChatGPT to write this section of this newsletter: Is it 'truly reasoning'? Maybe not. But it doesn't need to be to render me potentially unemployable. 'Whether or not they are simulating thinking has no bearing on whether or not the machines are capable of rearranging the world for better or worse,' Cambridge professor of AI philosophy and governance Harry Law argued in a recent piece, and I think he's unambiguously right. If Vox hands me a pink slip, I don't think I'll get anywhere if I argue that I shouldn't be replaced because o3, above, can't solve a sufficiently complicated Towers of Hanoi puzzle — which, guess what, I can't do either. Critics are making themselves irrelevant when we need them most In his piece, Law surveys the state of AI criticisms and finds it fairly grim. 'Lots of recent critical writing about AI…read like extremely wishful thinking about what exactly systems can and cannot do.' This is my experience, too. Critics are often trapped in 2023, giving accounts of what AI can and cannot do that haven't been correct for two years. 'Many [academics] dislike AI, so they don't follow it closely,' Law argues. 'They don't follow it closely so they still think that the criticisms of 2023 hold water. They don't. And that's regrettable because academics have important contributions to make.' But of course, for the employment effects of AI — and in the longer run, for the global catastrophic risk concerns they may present — what matters isn't whether AIs can be induced to make silly mistakes, but what they can do when set up for success. I have my own list of 'easy' problems AIs still can't solve — they're pretty bad at chess puzzles — but I don't think that kind of work should be sold to the public as a glimpse of the 'real truth' about AI. And it definitely doesn't debunk the really quite scary future that experts increasingly believe we're headed toward.
Vox
5 hours ago
- Politics
- Vox
The top priority of progressive politics may be slipping out of reach forever
is a senior correspondent at Vox. He covers a wide range of political and policy issues with a special focus on questions that internally divide the American left and right. Before coming to Vox in 2024, he wrote a column on politics and economics for New York Magazine. A protester wearing a Trump paper mâché head stands in front of a barricade and holds a sign that reads, 'Death and taxes' in New York in years ago, America was on the cusp of the largest expansion of its welfare state since the 1960s. Under Joe Biden in 2021, House Democrats passed legislation that would have established a monthly child allowance for most families, an expansion of Medicaid's elder care services, federal child care subsidies, universal prekindergarten, and a paid family leave program, among other new social benefits. But that bill failed — and then, so did Biden's presidency. Now, Republicans are on the brink of enacting the largest cut to public health insurance in American history. And the outlook for future expansions of the safety net looks dimmer than at any time in recent memory. There are two primary reasons why progressives' prospects for growing the welfare state have darkened. This story was first featured in The Rebuild. Sign up here for more stories on the lessons liberals should take away from their election defeat — and a closer look at where they should go next. From senior correspondent Eric Levitz. First (and most straightforwardly), the Democrats are not well-positioned to win full control of the federal government anytime soon. To win a Senate majority in 2026, the party would need to win multiple states that Trump carried by double digits last year. And the 2028 map isn't that much better. The basic problem is that Democrats have built a coalition that's heavily concentrated on the coasts and thus, systematically underrepresented in the Senate. To win the robust congressional majorities typically necessary for enacting large social programs, Democrats would likely need to transform their party's brand. Second, although Democrats developed grander ambitions for social spending over the past decade, they simultaneously grew more averse to raising taxes on anyone but the super-rich. In the 2010s, when inflation and interest rates were persistently low, the party could paper over this tension with deficit spending. But Biden-era inflation revealed the limits of this strategy. And if Congress passes President Donald Trump's tax cut plan, then interest rates and inflationary risk are likely to remain elevated for years, while the cost of servicing America's debts will soar. Add to this the impending exhaustion of Social Security's trust fund, and space for new welfare programs is likely to be scant, unless Democrats find a way to enact broad-based tax increases. Liberals could respond to all this by paring back their ambitions for the welfare state, while seeking to advance progressive goals through regulatory policy. It is perhaps not a coincidence that the two most prominent policy movements in Democratic circles today — the anti-monopoly and 'abundance' crusades — are both principally concerned with reforms that require no new tax revenue (antitrust enforcement in the former case, zoning liberalization in the latter). But expanding America's safety net remains a moral imperative. In the long-term, Democrats must therefore strive to build the electoral power and political will necessary for raising taxes on the middle-class (or at least, on its upper reaches). Related The US government has to start paying for things again Democrats like social welfare programs. But they like low taxes on the upper middle-class even more. Over the course of the 2010s, the Democratic leadership's appetite for new social spending grew. Bernie Sanders's insurgent campaigns in 2016 and 2020 put Medicare-for-All at the center of the party's discourse, and moved its consensus on the welfare state sharply leftward. In the latter primary, even the Democrats' most moderate contender — Joe Biden — vowed to establish a public option for health insurance and tuition-free community colleges, among other social programs. Biden's agenda only grew more ambitious upon taking office. No president since Lyndon B. Johnson had proposed a more sweeping expansion of social welfare than the Build Back Better Act. And yet, while Democrats' aspirations for social spending had become historically bold, the party's position on taxes had grown exceptionally timid. In 2016, Hillary Clinton had promised not to raise taxes on any American family earning less than $250,000. Four years later, Biden vowed to spare all households earning less than $400,000 – despite the fact that tax rates on upper middle-class families had fallen during Trump's first term. Meanwhile, the Democrats' congressional leadership was actually pushing to cut taxes on rich blue state homeowners by increasing the state and local income tax deduction. In other words: In 2021, Democrats were promising to establish an unprecedentedly large welfare state, while keeping taxes on 98 percent of households historically low. Officially, the party believed that it could square this circle by soaking the super-rich. After all, America's highest-earning 1 percent had commandeered more than 20 percent of the nation's annual income. The government could therefore extract a lot of revenue by merely shaking down the upper class. In reality, though, Biden's vision was also premised on the assumption that America could deficit-finance new spending with little risk of sparking inflation or high interest rates. The Build Back Better Act did not actually raise taxes on the rich by enough to offset its social spending. Instead, Democrats leaned on budget gimmicks to 'pay for' its agenda: Although the party intended the law's new programs to be permanent, it scheduled many of them to expire after just a few years, so as to make the policies look cheaper over a decade-long budget window. Absent these arbitrary expiration dates, the bill would have added $2.8 trillion to the deficit over a decade. Even as written, the law would have increased deficits by $749 billion in its first five years. More fundamentally, Biden's basic fiscal objective — to establish wide-ranging social benefits through taxes on the super rich alone — only made sense in a world of low inflation. Western Europe's robust welfare states are all funded through broad-based taxation. This is partly because administering a large safety net requires managing economic demand. When the government expands its provision of elder care, social housing, child care, and pre-K, it increases overall demand for workers and resources in the economy. And if the supply of labor and materials doesn't rise in line with this new demand, then inflation can ensue. Taxes effectively 'pay for' new spending by freeing up such resources. When households see their post-tax income decline, they're often forced to make fewer discretionary purchases. Raise taxes on an upper middle-class family and it might need to postpone its dreams of a lake house. That in turn frees up labor for public programs: The fewer construction workers needed to build vacation homes, the more that will be available to build affordable housing. But soaking the extremely rich does less to dampen demand than taxing the upper middle-class does. Even if you increase Elon Musk's tax rate by 50 percent, he won't actually need to reduce his consumption at all — the billionaire will still have more money than he can spend in a lifetime. The same general principle applies to multimillionaires, albeit to a lesser extent: Raise their taxes, and they're liable to save less money, but won't necessarily consume fewer resources. And if they do not curb their consumption in response to a tax hike, then that tax hike will not actually free up resources. In 2021, Democrats felt no obligation to sweat these details. For nearly a decade after the Great Recession, economic demand had been too low. Workers and materials had stood idle on the economy's sidelines, as there wasn't enough spending to catalyze their employment. In that context, unfunded welfare benefits can boost growth without generating inflation. But as Democrats moved Build Back Better through Congress, the macroeconomic terrain shifted beneath their feet. Biden likely would have struggled to get his social agenda through the Senate (where Democrats held only 50 votes) even in the absence of 2022's inflation. But that surge in prices all but guaranteed the legislation's defeat: Suddenly, it became clear that the government could not increase economic demand without pushing up inflation and interest rates. America had returned to a world of fiscal constraints. Unfortunately, those constraints could prove lasting, especially if Donald Trump's tax agenda makes it into law. Related The reconciliation bill is Republicans doing what they do best Building a comprehensive welfare state is about to get harder The most lamentable aspect of Trump's 'Big Beautiful Bill' are its cuts to healthcare and food assistance for the poor. Yet even as it takes health insurance from 10 million Americans and reduces food assistance to low-income families by about $100 a month, the legislation would add $2.4 trillion to the debt over the coming decade, according to the Congressional Budget Office. Yet the actual cost of the GOP's fiscal vision is even larger. To reduce their bill's price tag, Republicans' set some of their tax cuts to arbitrarily expire. Were these tax cuts made permanent, the bill would add roughly $5 trillion to the deficit over the next 10 years. This is likely to render the US economy more vulnerable to inflation and high interest rates in the future. Thus, the next Democratic government probably won't have much freedom to deficit spend without increasing Americans' borrowing costs or bills. Meanwhile, if that administration holds power after 2032, it will also need to find a ton of new revenue, just to maintain America's existing welfare state. Social Security currently pays out more in benefits than it takes in through payroll taxes. For now, the program's dedicated trust fund fills in the gap. But in 2033, that fund will likely be exhausted, according to government projections. At that point, the government will need to find upward of $414.5 billion in new revenue, each year, to maintain existing Social Security benefits without increasing the deficit. Given Democrats' current stance on taxes, the imperative to keep Social Security funded would likely crowd out the rest of the party's social welfare agenda. Indeed, merely sustaining Americans' existing retirement benefits would almost certainly require raising taxes on households earning less than $400,000. Maintaining such benefits while also creating new welfare programs — in a context of structurally high deficits and interest rates — would plausibly entail large, broad-based tax increases, the likes of which today's Democrats scarcely dare to contemplate. Granted, the robots could solve all this To be sure, it is possible that technological progress could render this entire analysis obsolete. Some analysts expect artificial intelligence to radically increase productivity over the next decade, while devaluing white-collar labor. This could slow the pace of wage and price growth, while turbo-charging income inequality. In a world where robots can instantly perform work that presently requires millions of humans, America could plausibly finance a vast social welfare state solely through taxes on capital. But until AI actually yields a discernible leap in productivity, I don't think it is safe to take an impending robo-utopia as a given. Democrats eventually need to sell Americans on higher taxes Democrats probably can't escape the tension between their commitments on taxation and social spending. But they can seek to mitigate it in a few different ways. One is to scale down the party's ambitions for the welfare state, while seeking to advance progressive economic goals through other means. Such a retreat would be understandable. The party's fear of raising taxes is not baseless. In a 2021 Gallup poll, only 19 percent of Americans said they would like to have more government services in exchange for higher taxes, while 50 percent said they'd prefer lower taxes in exchange for fewer services. Meanwhile, Democrats have grown increasingly reliant on the support of upper middle-class voters. In 2024, the highest-earning 5 percent of white voters were more than 10 percentage points more Democratic than America as a whole. The lowest earning two-thirds of whites, by contrast, were more Republican than the nation writ large. In this political environment, calling for large middle-class tax hikes could well ensure perpetual Republican rule. In the short term then, Democrats might therefore be wise to narrow their agenda for social welfare, focusing on modest programs that can be funded exclusively with taxes on the rich. At the same time, the party could seek to better working people's lot through regulatory policy. You don't need to raise middle-class taxes to expand collective bargaining rights, guarantee worker representation on corporate boards, or raise the minimum wage. And the same can be said of relaxing regulatory barriers to housing construction and energy infrastructure. (Of course, achieving any of these goals federally would require Democrats to win a robust Senate majority — one sufficiently large and progressive enough to abolish the legislative filibuster, which currently establishes a 60-vote threshold for enacting new, non-budgetary legislation.) In the long run though, Democrats must not forfeit the pursuit of a comprehensive welfare state. America lets more of its children suffer poverty — and more of its adults go without health insurance — than similarly rich countries. These deprivations are largely attributable to our nation's comparatively threadbare safety net. And they can only be fully eliminated through redistributive policy. A higher minimum wage will not ensure that children with unemployed parents never go hungry, or that every worker with cancer can afford treatment. Furthermore, as technological progress threatens to rapidly disemploy large segments of the public, robust unemployment insurance is as important as ever. And as the population ages, increasing investment in eldercare will be increasingly imperative. Democrats should seek to make incremental progress on all these fronts as soon as possible. Even if the party is only willing to tax the rich, it can still finance targeted anti-poverty spending. But absent an AI-induced productivity revolution, building a holistic welfare state will require persuading the middle-class to accept higher taxes. How this can be done is not clear. But part of the solution is surely to demonstrate that Democratic governments can spend taxpayer funds efficiently and effectively. So long as blue areas struggle to build a single public toilet for less than $1.7 million — or a high-speed rail line in less than 17 years — it will be hard to persuade ordinary Americans to forfeit a larger chunk of their paychecks to Uncle Sam. All this said, Democrats have plenty of time to debate the future of fiscal policy. In the immediate term, the party's task is plain: to do everything in its power to prevent Trump's cuts to Medicaid and food assistance from becoming law. The path to a comprehensive welfare state won't be easy to traverse. Better then not to begin the journey toward it by taking several steps backward.
Irish Daily Star
18 hours ago
- Politics
- Irish Daily Star
Donald Trump moans about not playing golf for 3 weeks while making Iran threat
President Donald Trump admitted that he hasn't been able to play golf for several weeks after swiftly dismissing questions surrounding the United States' involvement in Iran Donald Trump conceded that he hasn't been able to play golf in about a month after the President navigated around several questions regarding the United States' involvement in Iran. On Tuesday, Trump made a swift exit from the G7 Summit post-dinner and implored residents of Tehran to "immediately evacuate" as the conflict between Iran and Israel continues to escalate. Given the current events, Trump, who was recently accused of using his immigration policy to violate and abuse, admitted that he hasn't been able to hit the greens and fairways for several weeks. Trump also recently hit the headlines after soccer bosses seemingly ditched their anti-racism policy ahead of next year's World Cup. '35 club championships, you all know that,' he told reporters who gathered at the White House on Wednesday to see the two 'beautiful' flagpoles that were installed. '35 club championships. I haven't hit a ball in 3 weeks, 4 weeks.' When one correspondent joked: 'It's time to get out there,' Trump replied: 'Been a little busy. Wouldn't you rather have me doing what I'm doing in the end?' This is hardly the first time that Trump has shed light on his golfing prowess, yet some have pushed back on his championship claims. After Trump shared that he'd won at least 20 club championships at the 14 golf courses he owns, sportswriter Rick Reilly — who authored the 2019 book 'Commander in Cheat' — poured cold water on the bold assertion. 'Trump's going around telling people he has won 20. But that's 100 percent a lie,' Reilly told Vox. 'I actually played with him once, and he told me how he does it: Whenever he opens a new golf course, because he owns 14 and operates another five, he plays the first club champion by himself and declares that the club championship and puts his name on the wall.' As Wednesday's conversation switched gears to the rising tensions in the Middle East, Trump swiftly stopped short of confirming the United States' role in the conflict. "I can't say that," he said. "I may do it, I may not do it. Nobody knows what I want to do. But I can say this: Iran's got a lot of trouble and wants to negotiate." Though Trump appeared uninterested in discussing Iran, the 79-year-old was more than happy to talk about the nearly 100-foot-tall flagpoles that were being installed on the north and south lawns of the White House. 'It is my Great Honor to announce that I will be putting up two beautiful Flag Poles on both sides of the White House, North and South Lawns. It is a GIFT from me of something which was always missing from this magnificent place,' Trump wrote on Truth Social. 'They are tall, tapered, rust proof, rope inside the pole, and of the highest quality. Hopefully, they will proudly stand at both sides of the White House for many years to come!'
Euronews
a day ago
- Politics
- Euronews
Spain's parliament turns chaotic as Sánchez and opposition trade barbs
The first parliamentary session in Spain since a top aide of Prime Minister Pedro Sánchez was implicated in a corruption scandal quickly turned chaotic, with various lawmakers calling on Sánchez to resign as political pressure mounts against him. Santiago Abascal, leader of the far-right Vox party, first left the chamber without listening to Sánchez, staring at him with contempt as he passed by. 'You are indecent. And not even your supporters have any doubt about that. All of Spain knows it. You are corrupt and a traitor,' said Abascal before leaving the chamber. Then, People's Party (PP) lawmakers banged their seats during the session on Wednesday, shouting "Resignation, resignation". The Spanish premier chose to deploy an offensive tactic, referencing corruption cases linked to other parties. Sánchez spoke of the Gürtel case, which implicated hundreds of PP officers, some of whom subsequently resigned, with corruption, including bribery, money laundering and tax evasion. The PP parliamentary caucus erupted, and the situation quickly turned chaotic, with Parliament Speaker Francina Armengol struggling to control the session. 'You are a president deeply trapped in a corruption scheme. No matter how much you disguise it, you are not the victim. The victims are the Spanish people,' said Alberto Núñez Feijóo, PP party president. 'You came to say you won't call elections because you would lose them. You don't have to save the Spanish people from themselves; the Spanish people have to save themselves from you, and they await your resignation letter,' he added. Sánchez then said that the only thing he's going to address is the PP corruption cases, which are set to be tried in the coming months. Sánchez has completely changed his tone, moving from last week's apology to a coordinated offensive against the opposition PP and Vox. The Spanish leader believes they lack the legitimacy to speak about corruption, given their severe graft cases. The difference, as Sánchez notes, is that the Spanish Socialist Workers' Party (PSOE) acts as soon as there are signs, while the PP and Vox cover up corruption. Sánchez attempted to steer the session away from the PSOE's corruption case, but to no avail. The last few days have been very tense since audio recordings were released by Spanish police last week. The tapes confirmed that the third-highest-ranking PSOE official, who has since resigned, Santos Cerdán, was involved in an illegal scheme that saw him take kickbacks in return for awarding public work contracts. Cerdan has denied any wrongdoing.
Vox
a day ago
- Vox
He's the godfather of AI. Now, he has a bold new plan to keep us safe from it.
is a senior reporter for Vox's Future Perfect and co-host of the Future Perfect podcast. She writes primarily about the future of consciousness, tracking advances in artificial intelligence and neuroscience and their staggering ethical implications. Before joining Vox, Sigal was the religion editor at the Atlantic. The science fiction author Isaac Asimov once came up with a set of laws that we humans should program into our robots. In addition to a first, second, and third law, he also introduced a 'zeroth law,' which is so important that it precedes all the others: 'A robot may not injure a human being or, through inaction, allow a human being to come to harm.' This month, the computer scientist Yoshua Bengio — known as the 'godfather of AI' because of his pioneering work in the field — launched a new organization called LawZero. As you can probably guess, its core mission is to make sure AI won't harm humanity. Even though he helped lay the foundation for today's advanced AI, Bengio is increasingly worried about the technology over the past few years. In 2023, he signed an open letter urging AI companies to press pause on state-of-the-art AI development. Both because of AI's present harms (like bias against marginalized groups) and AI's future risks (like engineered bioweapons), there are very strong reasons to think that slowing down would have been a good thing. But companies are companies. They did not slow down. In fact, they created autonomous AIs known as AI agents, which can view your computer screen, select buttons, and perform tasks — just like you can. Whereas ChatGPT needs to be prompted by a human every step of the way, an agent can accomplish multistep goals with very minimal prompting, similar to a personal assistant. Right now, those goals are simple — create a website, say — and the agents don't work that well yet. But Bengio worries that giving AIs agency is an inherently risky move: Eventually, they could escape human control and go 'rogue.' So now, Bengio is pivoting to a backup plan. If he can't get companies to stop trying to build AI that matches human smarts (artificial general intelligence, or AGI) or even surpasses human smarts (artificial superintelligence, or ASI), then he wants to build something that will block those AIs from harming humanity. He calls it 'Scientist AI.' Scientist AI won't be like an AI agent — it'll have no autonomy and no goals of its own. Instead, its main job will be to calculate the probability that some other AI's action would cause harm — and, if the action is too risky, block it. AI companies could overlay Scientist AI onto their models to stop them from doing something dangerous, akin to how we put guardrails along highways to stop cars from veering off course. I talked to Bengio about why he's so disturbed by today's AI systems, whether he regrets doing the research that led to their creation, and whether he thinks throwing yet more AI at the problem will be enough to solve it. A transcript of our unusually candid conversation, edited for length and clarity, follows. Sigal Samuel When people express worry about AI, they often express it as a worry about artificial general intelligence or superintelligence. Do you think that's the wrong thing to be worrying about? Should we only worry about AGI or ASI insofar as it includes agency? Yoshua Bengio Yes. You could have a superintelligent AI that doesn't 'want' anything, and it's totally not dangerous because it doesn't have its own goals. It's just like a very smart encyclopedia. Sigal Samuel Researchers have been warning for years about the risks of AI systems, especially systems with their own goals and general intelligence. Can you explain what's making the situation increasingly scary to you now? Yoshua Bengio In the last six months, we've gotten evidence of AIs that are so misaligned that they would go against our moral instructions. They would plan and do these bad things — lying, cheating, trying to persuade us with deceptions, and — worst of all — trying to escape our control and not wanting to be shut down, and doing anything [to avoid shutdown], including blackmail. These are not an immediate danger because they're all controlled we don't know how to really deal with this. Sigal Samuel And these bad behaviors increase the more agency the AI system has? Yoshua Bengio Yes. The systems we had last year, before we got into reasoning models, were much less prone to this. It's just getting worse and worse. That makes sense because we see that their planning ability is improving exponentially. And [the AIs] need good planning to strategize about things like 'How am I going to convince these people to do what I want?' or 'How do I escape their control?' So if we don't fix these problems quickly, we may end up with, initially, funny accidents, and later, not-funny accidents. That's motivating what we're trying to do at LawZero. We're trying to think about how we design AI more precisely, so that, by construction, it's not even going to have any incentive or reason to do such things. In fact, it's not going to want anything. Sigal Samuel Tell me about how Scientist AI could be used as a guardrail against the bad actions of an AI agent. I'm imagining Scientist AI as the babysitter of the agentic AI, double-checking what it's doing. Yoshua Bengio So, in order to do the job of a guardrail, you don't need to be an agent yourself. The only thing you need to do is make a good prediction. And the prediction is this: Is this action that my agent wants to do acceptable, morally speaking? Does it satisfy the safety specifications that humans have provided? Or is it going to harm somebody? And if the answer is yes, with some probability that's not very small, then the guardrail says: No, this is a bad action. And the agent has to [try a different] action. Sigal Samuel But even if we build Scientist AI, the domain of 'What is moral or immoral?' is famously contentious. There's just no consensus. So how would Scientist AI learn what to classify as a bad action? Yoshua Bengio It's not for any kind of AI to decide what is right or wrong. We should establish that using democracy. Law should be about trying to be clear about what is acceptable or not. Now, of course, there could be ambiguity in the law. Hence you can get a corporate lawyer who is able to find loopholes in the law. But there's a way around this: Scientist AI is planned so that it will see the ambiguity. It will see that there are different interpretations, say, of a particular rule. And then it can be conservative about the interpretation — as in, if any of the plausible interpretations would judge this action as really bad, then the action is rejected. Sigal Samuel I think a problem there would be that almost any moral choice arguably has ambiguity. We've got some of the most contentious moral issues — think about gun control or abortion in the US — where, even democratically, you might get a significant proportion of the population that says they're opposed. How do you propose to deal with that? Yoshua Bengio I don't. Except by having the strongest possible honesty and rationality in the answers, which, in my opinion, would already be a big gain compared to the sort of democratic discussions that are happening. One of the features of the Scientist AI, like a good human scientist, is that you can ask: Why are you saying this? And he would come up with — not 'he,' sorry! — it would come up with a justification. The AI would be involved in the dialogue to try to help us rationalize what are the pros and cons and so on. So I actually think that these sorts of machines could be turned into tools to help democratic debates. It's a little bit more than fact-checking — it's also like reasoning-checking. Sigal Samuel This idea of developing Scientist AI stems from your disillusionment with the AI we've been developing so far. And your research was very foundational in laying the groundwork for that kind of AI. On a personal level, do you feel some sense of inner conflict or regret about having done the research that laid that groundwork? Yoshua Bengio I should have thought of this 10 years ago. In fact, I could have, because I read some of the early works in AI safety. But I think there are very strong psychological defenses that I had, and that most of the AI researchers have. You want to feel good about your work, and you want to feel like you're the good guy, not doing something that could cause in the future lots of harm and death. So we kind of look the other way. And for myself, I was thinking: This is so far into the future! Before we get to the science-fiction-sounding things, we're going to have AI that can help us with medicine and climate and education, and it's going to be great. So let's worry about these things when we get there. But that was before ChatGPT came. When ChatGPT came, I couldn't continue living with this internal lie, because, well, we are getting very close to human-level. Sigal Samuel The reason I ask this is because it struck me when reading your plan for Scientist AI that you say it's modeled after the platonic idea of a scientist — a selfless, ideal person who's just trying to understand the world. I thought: Are you in some way trying to build the ideal version of yourself, this 'he' that you mentioned, the ideal scientist? Is it like what you wish you could have been? Yoshua Bengio You should do psychotherapy instead of journalism! Yeah, you're pretty close to the mark. In a way, it's an ideal that I have been looking toward for myself. I think that's an ideal that scientists should be looking toward as a model. Because, for the most part in science, we need to step back from our emotions so that we avoid biases and preconceived ideas and ego. Sigal Samuel A couple of years ago you were one of the signatories of the letter urging AI companies to pause cutting-edge work. Obviously, the pause did not happen. For me, one of the takeaways from that moment was that we're at a point where this is not predominantly a technological problem. It's political. It's really about power and who gets the power to shape the incentive structure. We know the incentives in the AI industry are horribly misaligned. There's massive commercial pressure to build cutting-edge AI. To do that, you need a ton of compute so you need billions of dollars, so you're practically forced to get in bed with a Microsoft or an Amazon. How do you propose to avoid that fate? Yoshua Bengio That's why we're doing this as a nonprofit. We want to avoid the market pressure that would force us into the capability race and, instead, focus on the scientific aspects of safety. I think we could do a lot of good without having to train frontier models ourselves. If we come up with a methodology for training AI that is convincingly safer, at least on some aspects like loss of control, and we hand it over almost for free to companies that are building AI — well, no one in these companies actually wants to see a rogue AI. It's just that they don't have the incentive to do the work! So I think just knowing how to fix the problem would reduce the risks considerably. I also think that governments will hopefully take these questions more and more seriously. I know right now it doesn't look like it, but when we start seeing more evidence of the kind we've seen in the last six months, but stronger and more scary, public opinion might push sufficiently that we'll see regulation or some way to incentivize companies to behave better. It might even happen just for market reasons — like, [AI companies] could be sued. So, at some point, they might reason that they should be willing to pay some money to reduce the risks of accidents. Sigal Samuel I was happy to see that LawZero isn't only talking about reducing the risks of accidents but is also talking about 'protecting human joy and endeavor.' A lot of people fear that if AI gets better than them at things, well, what is the meaning of their life? How would you advise people to think about the meaning of their human life if we enter an era where machines have both agency and extreme intelligence? Yoshua Bengio I understand it would be easy to be discouraged and to feel powerless. But the decisions that human beings are going to make in the coming years as AI becomes more powerful — these decisions are incredibly consequential. So there's a sense in which it's hard to get more meaning than that! If you want to do something about it, be part of the thinking, be part of the democratic debate. I would advise us all to remind ourselves that we have agency. And we have an amazing task in front of us: to shape the future.
