As AI takes over the workflows that underpin society, we are increasingly delegating decisions to systems whose behavior we cannot yet reliably evaluate. We want these systems to conduct research, allocate resources, coordinate operations, and act on our behalf.
Yet these systems must operate in a world that is inherently uncertain. Where to allocate capital, which technologies to pursue, how competitors will respond, and whether a public policy will achieve its goals all depend on relationships spanning economics, technology, institutions, and human behavior. These relationships evolve as people act, incentives change, and new information becomes available.
An AI entrusted with consequential decisions needs a grounded understanding of that world, an explicit account of what remains uncertain, and a way to evaluate its expectations against what actually happens.
The ability to act must be accompanied by the ability to reason about consequences.
Much of the value of intelligence lies in recognizing what is emerging while there is still time to influence the outcome. Large language models can generate compelling explanations, but those explanations may offer little guidance about what happens next. For decisions whose consequences have yet to unfold, systems must state what they expect to happen and what evidence would change their judgment. A model’s explanation cannot simply be taken at face value: in controlled experiments, Anthropic researchers found that models could follow hints that influenced their answers without acknowledging those hints in their reasoning.
Consider a company preparing to enter a new market. Months before an announcement, it recruits a regulatory specialist, changes the language in a patent filing, and signs an agreement with a distributor. Each development has several plausible explanations. Together, they may indicate an emerging strategy.
A competitor deciding where to invest cannot necessarily wait for confirmation. By then, useful partners may be committed and the opportunity more expensive. The decision must be made while the evidence remains incomplete.
An AI investigating the expansion should develop competing explanations and identify what would distinguish them. Does entering the market require a regulatory application? Is the hiring unusual relative to the company’s normal behavior? What should become observable if the expansion hypothesis is correct?
This is a form of abductive reasoning: developing plausible explanations for incomplete observations. A hypothesis should guide investigation, generate expectations, and lose credibility when the evidence fails to support it.
The system must also account for people responding to one another. A rival may abandon an expansion after an incumbent cuts prices. A subsidy may attract enough new capacity to undermine the economics that justified it. A warning may prompt an intervention that prevents the predicted event.
Reasoning about the future therefore requires a working understanding of actors, incentives, constraints, and relationships. Who can act? What do they want? What might cause them to change course? How would one institution’s decision alter the choices available to another?
Forecasting and simulation can help investigate these questions, provided their assumptions remain explicit and their conclusions are tested against observed outcomes. Repeated reports may originate from one source. Simulated actors may share the same mistaken assumptions. Confidence must reflect the strength and independence of the evidence supporting it. There is progress to build on: Bridgewater AIA Labs researchers reported a system that matched human superforecasters on one benchmark while underperforming market consensus on another — a reminder that capabilities must be established for particular settings.
AI needs an understanding of the world that persists as the world changes.
A system should retain what it previously believed, the evidence supporting that belief, and the conditions under which it expected an outcome. As new observations arrive and earlier forecasts resolve, it should examine contradictions and update its understanding.
Without this continuity, a system can repeatedly produce convincing analyses without learning where its reasoning succeeds or fails. It may explain each new development while losing sight of what it expected before the development occurred.
A persistent model of the world creates the possibility of cumulative learning. It can maintain an evolving understanding of companies, markets, technologies, governments, and people, preserving both the relationships among them and the uncertainty surrounding its judgments.
But forecasting known questions is only part of the problem.
A government might ask how many households will use a proposed housing subsidy. A more consequential question may be whether constrained supply will cause the subsidy to raise prices, absorbing the benefit intended for buyers. Accurately forecasting participation could still leave the policy’s central assumption untested.
A prompt presupposes that someone has recognized a need for intelligence. The decision-maker may not yet know what to ask.
AI should help discover these questions. It should identify when assumptions begin to fail, investigate emerging consequences, and bring them to decision-makers’ attention while there is still time to act. Doing this well requires judgment about relevance: intelligence must arrive early enough to influence a decision and justify the attention it consumes.
As AI assumes greater responsibility, its judgments should be traceable to evidence, its uncertainty visible, and its recommendations subject to human governance. We need records that allow people to examine what a system knew, what it expected, and why its judgment changed.
The long-term ambition should extend beyond anticipating events to understanding the consequences of our decisions.
Returning to the housing subsidy: did homeownership increase, for whom, and at what cost? How much of any price increase was attributable to the policy? Did it improve access for the households it was intended to serve? What evidence should prompt a change in course?
Answering these questions requires sustained observation, credible causal analysis, and a willingness to revise earlier conclusions. Favorable outcomes do not automatically validate the decisions that preceded them. AI should help institutions examine whether their actions produced the intended effects and where their understanding fell short.
We should build AI that helps make the decisions shaping society more transparent, accountable, and responsive to the people they affect. Objectives, acceptable tradeoffs, and responsibility for action must remain with people and institutions that can be held accountable.
That means building systems that continuously study a changing world, develop and test explanations, anticipate possible outcomes, and learn when reality contradicts their expectations. We must evaluate their judgment with the same rigor we bring to expanding their capabilities.
The decisions that shape human development are made before their consequences are known. If AI is to help us make them, reasoning about the future must become one of its central responsibilities.
— Iman