A reproducible evaluation environment for financial AI agents

Reproducible

A universe of companies, a macro state for day zero, and a seed. The same three inputs produce the same market on every supported platform, so a run is reproducible by anyone who has them.

Execution-aware

Agents trade through an order book and pay the spread, the queue and their own impact. Run the same seed without the orders and the difference is the cost of trading.

Inspectable

The simulator computed each price, so it can say why. The nine factor contributions sum to the move - an identity, not a fitted explanation. The residual is near 1e-16.

Agent-native

A Python object with an act method, a Gymnasium environment, or a model calling the simulator as a tool over MCP. The harness never asks what is inside.

What a RunManifest pins

reproduce() refuses on a mismatch rather than returning a number that looks fine. A checkpoint branches a run at a decision point, so two hypotheses share one past instead of being statistically similar.

Questions it makes answerable

• Did the agent trade on momentum, the macro state, liquidity, or noise?
• Did its stated reason match the state the market was actually in?
• What happens to the result when one causal channel is removed?
• Did two agents reach the same P&L for different reasons?

What it is not

Ground truth about this market, not about any real one. The decomposition is exact because the simulator did the arithmetic, which is also the reason it says nothing about why a real price moved.

import tradefloor as tf

u = tf.Universe.random(40, seed=7)

len(u)            # 40
u.fingerprint     # sha256, order included
u[0].ticker, u[0].sector
# heaviest sectors: technology 4, financial_services 4

macro = tf.Macro(federal_funds_rate=0.025,
                 corporate_bond_yield=0.052, vix=16.0)

e = tf.Engine(seed=42, universe=u, macro_state=macro)
e.run_days(252)   # one trading year

Agents over MCP

No simulator integration code. Claude or another model calls tradefloor through MCP, observes the market and submits trades as tool calls. Eleven tools, everything composed as data.

pip install "tradefloor[mcp]"

claude mcp add tradefloor \
  -- tradefloor-mcp

# then just ask it a question

Reinforcement learning

Passes gymnasium's env_checker. Actions are target weights in [-1, 1]. Reward is measured after the market moves, so it already carries the cost of the agent's own footprint.

import tradefloor.gym as ptg

env = ptg.TradingEnv(universe=u,
    seed=42, days=20)
obs, info = env.reset(seed=42)
obs, r, done, trunc, i = env.step(a)

Your own trade logic

A TWAP, a VWAP, an iceberg, or your own logic. Any object with an act method will do. The observation is narrow on purpose. It exposes only information available to a trader inside the simulated market.

class Mine:
    def act(self, obs):
        return {"AAA": 0.2}

tf.tca.analyse(Mine(), seed=42,
    universe=u, days=5)

Ten agents, one market each

Hand evaluate a dict of ten agents and each one runs in its own copy of an identical world: same seed, same roster, same macro path. Your algorithm, a trained policy, an LLM agent and the five baselines land in one table.

Then rank repeats that across seeds and separates them with a paired sign test. The Oracle reads the true mispricing and says so. Read it as a reference point, not a ceiling: it spends the same gross on a naive rule, and mean-reversion beats it on 5 of the 12 markets.

Run a crisis through it

A scenario drives the macro path. Without one, the economy finds its own path. Pick a scenario and see what the market did.

import tradefloor as tf

u = tf.Universe.random(40, seed=7)

shock = tf.Scenario.rate_shock(
    start=0.045, end=0.016, over=13)

tf.evaluate(agents, seed=7,
    universe=u, days=83,
    scenario=shock)

# leave corporate_bond_yield free:
# pinning it severs the channel
# the shock is travelling down.

Good results here do not predict real returns

The price process comes from a known model. A strategy that fits the structure of that model looks excellent and teaches you nothing. A strategy that fails here tells you more, because it broke against a live order book under honest impact costs.

One market is one sample

On the twelve-market grid the docs measure, mean-reversion pools at +0.783 capture and momentum at +0.259, so mean-reversion leads by three times. Paired across the same twelve markets it wins 9 to 3, at p = 0.15. One seed picks the pooled leader eight times in twelve, and what a seed says momentum is worth runs from -0.503 to +0.909.

LLM agents

Evaluate agents that reason, call tools, read state and make a sequence of trading decisions. The MCP server exposes the simulator as eleven tools.

Reinforcement learning

A Gymnasium environment with explicit rewards, execution costs and market state, and as many independent episodes as you want.

Quant and execution

Queue position, partial fills, honest impact, and macro scenarios to run a strategy through. Test under controlled dynamics rather than one historical path.

Benchmark builders

A manifest that names the package version, preset, seed, universe, macro, scenario and strategy, and a reproduce() that refuses on mismatch.