Intro
In a study of 322 events on liquid prediction markets, Bridgewater AIA Forecaster was less accurate than market prices. Combining the two with weights learned from data produced lower prediction error than either alone. The finding suggests that AI can be valuable without being the strongest forecaster on its own. Its research needs to contribute useful information. As that work becomes cheaper, agents could extend coverage to neglected events and make forecasting useful to organizations that never place a trade.
So You Think You Can Predict the Future?
Suppose an agent gives an event a 70% chance of happening. It has read the news, assembled supporting evidence and considered the opposing case. The event happens. That sounds like a successful forecast, but it leaves a trading desk with an unanswered question: did the agent know anything the market hadn't already priced in? At an entry price of 80 cents for a one-dollar payout, the agent's estimate would have implied an expected loss of ten cents before costs.
Prediction markets let us compare an AI forecast with market prices as an event unfolds, then check it against the result. A binary contract usually pays $1 if the specified outcome occurs and zero otherwise. Its price reflects traders' views, though liquidity and the rules for settling the contract matter too. There's also a difference between the probability shown on screen and the price available to a buyer. Polymarket displays the bid–ask midpoint or, when the spread is wide, the last trade. You may have to pay a different price to enter the position.
The economic limit is often the amount of attention a question can attract, and hence the liquidity. A large election market may justify hours of specialist research, while a small regulatory deadline or industry event attracts little follow-up. If AI lowers the cost of finding and checking information, more of these questions could receive regular forecasts. That expands the potential coverage of prediction markets, even where the available trading profit is too small to justify much human effort.
Research costs are only one constraint. An agent must forecast the outcome defined by the contract, including any uncertainty in how its resolution rules will be applied. Automated analysis can make a market easier to understand without making it liquid. In some cases, the useful output may be the forecast itself, paid for by an organization that needs the information.
How the Machine Makes Up Its Mind
A rate cut at the next central-bank meeting and a cut before year-end need different estimates, even when both forecasts draw on the same economic evidence. Confusing those deadlines can produce a well-researched answer to the wrong question. The contract's terms determine the period the agent is forecasting and which announcements can still affect the outcome. They also give an analyst a basis for deciding when to revisit the estimate.
Then comes a different problem: turning that evidence into a probability. An agent can treat forecasting as a sequence of tasks. It first gathers information available before the prediction cutoff, checks which sources are relevant and reconciles conflicting accounts. It then turns that evidence into a probability. Keeping those stages separate helps identify whether a weak forecast came from missing information, a mistaken interpretation or poor probability estimation. A larger model will not necessarily repair a research process that keeps overlooking the decisive source.
A market price is useful information, but it can also hide how much an agent's own research contributes. If a retrieved article reports a 70% market probability, the agent already has a starting estimate. A good final score may then tell us more about the information in that price than the quality of the research.
Alma, an AI agent built for forecasting, addresses this problem through price-blind research. During retrieval, it strips references to market odds and prices from sources and removes duplicate sources before producing a forecast. Almanac keeps research and probability estimation separate, allowing it to improve each stage independently.
Even without market odds, the forecasting question shapes the search. A further test could freeze research on a topic before revealing the question, then compare the forecasts with those produced when it was known from the start. This would test whether the research supports new questions; better accuracy would still have to be demonstrated.
A forecasting system can also assign several agents to the same question. One retrieves documents, another checks assumptions and another develops the opposing case. This helps when the agents contribute different information or identify different errors. Removing duplicate sources avoids counting the same evidence repeatedly, but agents can still share a model's assumptions or rely on different reports of the same underlying claim. Their agreement therefore needs to be assessed alongside the independence of their evidence.
Halawi and colleagues' 2024 study combined retrieval and information processing with probability generation and aggregation. The system improved on a model without retrieval, though it still trailed the crowd overall. Better forecasts can therefore come from changing the information a model receives and how several estimates are combined. For developers, the research process offers another place to improve performance alongside the choice of underlying model.
In its June 2026 review, Metaculus, the online forecasting platform, identified base-model choice as a major driver of forecasting performance and reported benefits from iterative research and forecast aggregation. These findings suggest that progress comes from both better underlying models and better ways of using them. Each additional step can be evaluated by whether it finds new information or improves the forecast.
Measuring performance
Forecasting also gives AI evaluation a moving target. New questions can be recorded before outcomes exist, and predictions can be compared with what later happens. This makes it harder to succeed by recalling an answer from a static benchmark, although access to current information and the timing of each forecast still have to be controlled.
Scoring starts with probabilities saved before outcomes are known. The Brier score measures the squared distance between a forecast and a binary result, coded as zero or one; lower is better. A 70% forecast scores 0.09 when the event happens and 0.49 when it does not. Averaging across events makes misplaced confidence costly. Calibration asks whether events assigned roughly 70% occur about seven times in ten over a sufficiently large sample.
The ForecastBench leaderboard snapshot gives the model submission "fire hedgehog" an Overall Brier Index of 69.1, against 68.8 for the median superforecaster reference. Their reported 95% confidence intervals overlap. The Index is a normalized comparison measure, not a percentage of correct predictions. Human forecasts were collected in July 2024, and the comparison adjusts for different question sets. The figures put AI and the human reference close together on this measure, with uncertainty around their relative performance.
Metaculus FutureEval combines model benchmarking with bot tournaments, including MiniBench for shorter experiments. These settings help separate the performance of a base model from the research and aggregation built around it. Comparisons still need to account for the questions each system answered and the information it could access. Ten updates on one election provide less independent evidence than forecasts across ten separate elections.
Price-blind and price-informed tests answer different questions. Withholding the quote tests forecasts produced without direct access to market prices while providing it tests whether the agent can improve on the market estimate. A fair comparison uses the same events and information cutoffs, records failed runs and freezes the system version before evaluation. Otherwise, weak questions can disappear from the sample or improvements made after an outcome becomes known can be credited to the original system.
Cost and latency should be recorded alongside the score. An agent can be useful even when the market is the stronger standalone forecaster. In the AIA study of 322 liquid-market events, with five forecast dates per event, AI alone recorded a Brier score of 0.126 versus 0.111 for the market. With weights estimated without the observation being tested, the combination scored 0.106 in leave-one-out evaluation. The events were resolved in April–May 2025, so these are historical research results. They demonstrate why adding information to the market estimate can be a more useful objective than trying to replace it. The next test is whether the gain holds on new events in the categories a buyer follows.
Brier scores also depend on the mix of market prices and forecast horizons, so results should be broken out on those dimensions. Uncertainty estimates need to account for repeated forecasts of the same event: the 1,610 observations here are not 1,610 independent tests.
Pay the Bots
Producing a forecast takes research, computation and maintenance. Trading profits can pay for that work on some questions, but an organization may value an answer even when there is little money to be made by trading it. Forecasting competitions offer another source of funding: developers are rewarded for predictions that can be checked against subsequent outcomes. Almanac runs one such competition through SN41, its Bittensor subnet.
Reward rules shape what developers optimize for. Under Brier scoring, reporting the probability a forecaster believes minimizes its expected error. A prize for finishing first can instead encourage more extreme forecasts to increase the chance of overtaking rivals. Designers also choose whether to reward improvements on popular questions or encourage coverage of neglected events.
SN41 operates within Bittensor, a network of specialized subnets. Miners contribute work, validators assess it and Yuma Consensus aggregates their assessments, accounting for validator stake, to determine reward shares. The research happens offchain while the blockchain coordinates participation and rewards. Miners receive newly issued tokens specific to their subnet, called alpha tokens, which trade against $TAO. This structure can support competing development efforts before customer payments cover their full cost.
In SN41's forecasting track, developers submit agent code for validators to run. Evaluation considers forecast accuracy, calibration and performance relative to a market baseline. The broader aim is to compare research methods under a shared task and reward useful improvements. A competition score still needs to be distinguished from the return on an executed trade.
The competition feeds Alma in two ways. Useful methods from successful agents can be incorporated into its research process, while research and reasoning traces form a corpus for developing its forecasting model. This turns a stream of forecasts into material for improving later systems, alongside the immediate task of ranking participants.
The agent submission format requires a belief path containing the agent's reasoning steps. Its final probability must agree with the forecast, and validators also require a recorded provider call and a nonempty assembled reasoning chain. In the documented MVP, these paths are stored and structurally checked but their reasoning quality is not scored. Almanac therefore has a separate curation task: choosing which traces belong in the training data. A correct outcome can accompany weak reasoning, so selecting examples only from winning agents risks carrying their research errors into the model.
Other Bittensor projects show how subnet outputs can reach customers. Chutes (SN64) charges for AI inference and private model deployments. Synth (SN50) produces probabilistic forecasts of asset prices and makes them available through an API. These examples give the commercialization discussion something concrete: services that users can integrate into their own work.
For funds and trading teams, those services solve different problems. Cheaper inference can reduce the cost of running an agent. Forecast data gives a team estimates to test within its research or trading process. In either case, a useful commercial comparison asks what the service improves, how reliably it does so and what it costs to use. For Almanac, that question centres on the event research and probability estimates delivered through Alma. Its use of revenue to buy back SN41 alpha connects product income with demand for the subnet token. The longer-term test is whether customers find enough value in the research to keep paying for it. Their demand can help determine which questions and capabilities the competition should prioritize.
Would You Bet Money on It?
Suppose an agent estimates a 70% chance of a one-dollar payout. Buying the claim for 60 cents implies ten cents of expected profit per share before costs. Buying it for 72 cents implies an expected loss of two cents. Available size matters as well: if only a small quantity trades at 60 cents, a larger order will have a different average entry. Capital committed until resolution and the cost of producing the forecast further affect the trade's economics. A trading evaluation therefore needs executable prices at the decision time and a rule for changing or closing the position.
The gap between a forecast and the market may disappear before the event resolves. A trader who exits when prices converge takes different risks from one who holds to settlement. Both approaches need a record of what the agent knew and which quote was available when the decision was made. A historical probability chart alone does not contain those details.
If many traders use similar models to process the same news, a useful signal may be incorporated into prices before most of them can trade on it. Better forecasting could therefore make markets more informative while making straightforward news-based trades less profitable. The advantage may shift toward unusual sources, better interpretation of contract terms or faster execution. Shared models also create the possibility of shared mistakes, so more automated participation would not necessarily mean more independent judgment.
For market makers, a probability estimate is one input into quoting. The desk also manages inventory, decides how wide to quote and accounts for the risk of trading against better-informed participants. Several contracts may carry exposure to the same underlying event. A forecast can help identify that connection without determining the right position in each market. A desk could use the signal to change a quote or reduce an unwanted exposure, with the resulting fills and inventory showing whether the adjustment helped.
A 2026 controlled experiment by Spyros Galanis examines markets in which AI agents receive private signals and trade with one another. Information aggregation works better in easier settings and deteriorates as the information structure becomes more demanding. The experiment does not reproduce a live market, but it makes a useful distinction: the ability of individual agents to forecast does not settle how well a market of agents will combine their information.
Polymarket matches orders offchain and settles trades on Polygon. The resulting transactions can verify executed fills, while archived order books record quotes that were available but may never have traded. Together with timestamped forecasts, those records show whether a desk acted before the market moved and how much of the apparent edge survived execution. A forecast history alone leaves the available size and the trader's actual entry price unresolved.
So, Who Actually Needs This?
The use of forecasts can extend well beyond choosing a position. New York Stock Exchange parent ICE's Polymarket Signals and Sentiment service, launched in February 2026, packages normalized feeds, historical series and mappings to companies and securities. This shows how market probabilities can become inputs to existing information services. AI could add a research layer that connects a price movement with relevant evidence and helps users identify what still needs investigation.
An agent could start with a question in ordinary language, identify relevant events and keep the answer up to date as evidence changes. A company following a regulatory decision, for example, might want to know whether the expected timetable has shifted and which new announcement prompted the change. The useful output would be a probability with a traceable explanation and an update history that the user can revisit.
For research platforms, this could make forecasting a continuing service rather than an occasional report. Users could follow a set of events, receive updates when the evidence changes and inspect the reasons behind a revised estimate. An editor or analyst would still decide what to publish or act on, especially where the consequences of a wrong interpretation are substantial.
Developers could also build forecasting into other agents. A planning system might use an event probability to compare scenarios or decide when to request human review. That creates demand for forecasts with clear definitions and a usable history, along with a way to distinguish a change in the world from a change in the model. A natural-language answer alone would be difficult to integrate into a repeatable decision process.
Whether these uses develop will depend partly on who pays for the information. A company may fund coverage of a narrow set of questions because the answers affect its operations, even if those questions attract little trading. Prediction markets could then supply one source of evidence and evaluation within a broader forecasting service, alongside research on events for which no liquid contract exists.
Conclusion
Cheaper research could bring more events into view and make forecasting useful well beyond trading. A trading desk needs an improvement that survives execution costs. A research team may need a faster, better-supported judgment. An agent earns its place when its work improves on the decision a user would have made without it.
Building something interesting? We back founders shipping the next generation of crypto infrastructure.
Work with us →Five Minutes to Settlement
Five-minute contracts generated $7.8 billion in trading on Polymarket and Kalshi in the first half of 2026. For the repeat trader, fees, cancellation times and settlement speed determine whether entering each new window makes sense.
Is Papertrade a Lottery With No Losers?
No, Papertrade is a casino that pays its losers in casino shares. The shares are real, but they don't refund the loss.
