AI Risk Forecasts Worsen Each Year As Public Stays Unconsulted

Over four years, the world’s clearest signal of how experts view artificial intelligence has shifted in one direction: worse. An aggregated review of 34 separate risk forecasts from 2022 through 2026 shows that estimated probabilities of severe AI outcomes increased in every consecutive year reviewed. The same analysis notes that no national government has ever asked its citizens directly whether they accept that level of risk.

Rising Risk Estimates Across Four Years

The review, compiled from public forecasts by 34 organizations, academic groups, and individual researchers, found a steady upward movement in the central estimate of catastrophic risk from AI systems. In 2022, the median forecast across the sources put the chance of a civilization-level AI outcome in the single digits. By 2023, that figure had climbed into the low teens. In 2024, as large language models became embedded in daily infrastructure, the median estimate moved past 20 percent. By 2025, several major forecasters had placed the chance above 30 percent. The 2026 data, though still early, has not reversed the trend.

The movement is not limited to a single school of thought. Probability estimates for loss of control, catastrophic misuse, and irreversible societal disruption all moved upward. Even forecasters who began the period skeptical of existential AI ended it with noticeably higher numbers. The review’s authors note that forecasting panels often anchor to base rates and historical precedent; the fact that estimates rose despite that anchoring suggests that new evidence—from system capabilities, unexplained behavior, and corporate deployment speed—was overwhelming prior assumptions.

The Sources Behind the Numbers

The 34 sources are not one uniform survey. They range from small academic workshops and industry safety teams to large public forecasting platforms that ask ordinary experts for long-horizon estimates. Each source independently produced a probability for some form of severe AI outcome, such as loss of human control over a powerful system or irreversible damage to global institutions. Aggregating methods across such a diverse set is methodologically messy, but the review found that the directional pattern is consistent regardless of how the sources are weighted.

Several of the largest jumps appeared in 2023 and 2024, when generative AI tools entered public use at an unprecedented scale. Products once considered research demonstrations became the default interface for millions of users. Companies repeatedly released models with capabilities that surpassed their own safety evaluations. The gap between optimistic internal projections and public risk estimates widened, and forecasters appear to have updated quickly.

Why the Public Was Never Asked

Despite these figures, no government, international body, or major technology company has run a formal public consultation on acceptable AI risk. In democratic countries, citizens are regularly consulted on war, infrastructure spending, and health policy. For AI, the most consequential decisions—what to build, how fast to build it, and what failure would look like—are being made inside laboratories, boardrooms, and unelected agencies.

Some decisions have reached courts and parliaments, but always after the fact. Users have been able to express preferences through terms of service, opt-out buttons, and occasional ballot boxes, but never through a structured process designed to ask the population what level of risk it is willing to tolerate. The title of the review, which stresses that “you were never asked,” is deliberately blunt. It refers to the absence of a citizens’ assembly, a national referendum, or even a standard survey on the subject. Civil society groups have held side events at AI summits, but those are invitations, not mandates. The people who will bear the costs are the people who had no chance to vote on the trade-off.

Governance scholars point out a deeper problem. If existential risk from AI is anywhere near the double-digit percentages that many forecasts suggest, then AI development is a public safety issue as urgent as climate or pandemic preparedness. Public safety issues usually require public consent. With AI, there has been little, partly because the risk is abstract, partly because commercial incentives reward speed, and partly because governments have struggled to understand the technology themselves.

Expert Concern and Divided Views

The new analysis lands during an unusually public disagreement among AI researchers. Some of the most prominent voices in the field have moved toward alarm. The Center for AI Safety’s 2023 open letter, signed by hundreds of researchers and industry figures, stated that “mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” Geoffrey Hinton, who left Google in 2023 to speak freely about the technology, has warned that it may be difficult to keep systems under control and has compared the pace of development to an arms race.

Others are equally forceful in the opposite direction. Meta’s chief AI scientist Yann LeCun has argued that large language models are never going to reach human-level intelligence simply by scaling up, and that the fear of AI takeover is based on a false analogy between intelligence and dominance. Skeptical forecasters point out that every previous decade of AI research produced its own wave of hype, and that high confidence in doom has repeatedly been wrong. The review does not attempt to resolve the disagreement. What it documents is a measurable change in the balance of opinion. Even the skeptics in the 34-source set moved their numbers upward, which is why the aggregate severity grew.

That matters because expert opinion shapes regulation. Central banks watch inflation forecasts; military planners watch threat assessments; public health agencies watch pandemic models. For AI, the forecasts are now available, and they point in a direction that demands a response. Yet no elected official has been held accountable for accepting or rejecting them.

Implications for Governance and Policy

If the worsening trend is accurate, the implications go far beyond model safety teams. Air traffic control, power grids, hospital scheduling, financial clearing systems, and military logistics are already beginning to rely on AI. An AI catastrophe does not require a science-fiction machine breaking free; it could take the form of a system confidently making a wrong decision inside critical infrastructure, then repeating that error faster than any human can intervene.

Public institutions are poorly equipped for that scenario. Regulatory proposals in the European Union and the United States have focused on transparency, documentation, and pre-deployment testing. Some bills require model makers to report dangerous capabilities. Almost none require them to obtain public approval before proceeding. The public is treated as a consumer of the technology, not as the principal on whose behalf the technology is being developed.

Private companies, meanwhile, cannot be expected to answer for risks that accrue to everyone. A company’s shareholders may accept a 30 percent risk of extreme downside if the expected gain is large enough. A society cannot make that calculation the same way, because it cannot diversify away its own collapse. That difference is at the root of the governance gap. The review’s authors argue that the risk level has become too large to remain a private matter. They call for new mechanisms, including public risk-benefit hearings, independent evaluation before deployment, and—if necessary—treating advanced AI research like other technologies that require a social license.

Some countries are beginning to listen. The United Kingdom held AI safety summits in 2023 and 2024, but they were mostly diplomatic gatherings rather than consultations. The European Union’s AI Act is the most ambitious attempt to regulate by risk category, but it was negotiated by institutions, not by citizens. Japan and Canada have published guidance documents that emphasize human dignity and safety, but those documents stop well short of asking the population what chance of catastrophe it finds acceptable.

What Happens Next

The next few years will be a test of whether democratic institutions can absorb a technical risk as abstract and fast-moving as AI. Forecasts will be updated again, and if the pattern continues, the numbers will keep rising. Governments may then be forced to move from voluntary principles to binding rules based on explicit risk thresholds. The first country to adopt such thresholds may set a global norm, for better or worse.

There are also serious questions about who has the right to make this decision. If members of the public are not asked, the decision is being made by industrialists and engineers. If they are asked, the question will have to be framed honestly: what chance of losing control of AI is acceptable, and in exchange for what benefits? Framing such a question is difficult, but deferring it is also a choice.

For now, the historical record is clear. From 2022 to 2026, the people who study AI risk became more worried every single year. The public was neither informed in a way designed to solicit their judgment, nor invited to take part in the decision. That may change as pressure grows, but until it does, the most important forecast of the decade remains a warning without an audience.