Short answer
Backtesting runs a fixed set of rules over historical data in seconds. Paper trading makes the agent decide in real time with virtual money, one moment at a time. For rule-based bots, backtest first. For AI agents that make judgement calls, paper trading (live, or replayed minute by minute) is the fairer test, because the agent has to decide without knowing what comes next.
The difference in one table
| Backtesting | Paper trading | |
|---|---|---|
| How it works | Runs rules over a whole historical dataset at once | The agent places orders as the market moves, with virtual money |
| Speed | Years of data in seconds | Real time (live) or up to 120x (replay) |
| Best for | Rule-based strategies you can write as code | AI agents and discretionary decisions |
| Main risk | Overfitting and look-ahead bias | Small sample if you stop too early |
| Costs | Only if you model them | Charged on every fill (on Papertrade, the broker's real fees) |
| Cost to run an LLM | Thousands of decisions at once can be expensive | Paced by the market, one decision at a time |
Why AI agents are hard to backtest
A classic backtest assumes the strategy is a function: same data in, same trades out. An LLM agent isn't. It reads news and context, its answers vary from run to run, and running it over years of minute bars means millions of model calls. It may also have read about the very days you're testing, which quietly leaks the future into the test.
Paper trading avoids most of that. The agent sees the market as it unfolds, decides, and lives with the result, including fees, market hours and the broker's leverage and square-off rules.
Replay: the middle ground
A market replay plays a past trading day forward minute by minute, so the agent trades it as if it were live, but you can run it at up to 120x speed and pick any day you like. It's a fast way to put an agent through calm days, gap days and trend days.
Because a model may remember famous past sessions, treat replays as practice and confirm results on live sessions.
A sensible workflow
- If your strategy is pure rules, backtest it first to rule out obvious losers.
- Paper trade the AI agent on replays of varied days to tune the prompt and rules.
- Freeze the rules, then paper trade live for a few weeks on a fresh simulator.
- Judge it on results after fees over the whole period. See how to judge an AI trader.
Frequently asked questions
- Is paper trading better than backtesting?
- Neither is better in general. Backtesting is faster for fixed rules; paper trading is the fairer test for AI agents and anything that makes judgement calls in real time.
- What is look-ahead bias?
- Using information in a test that wouldn't have been available at the time of the decision, which makes results look better than they would be live.
- Can I paper trade past days?
- Yes. Papertrade's replay mode plays any past trading day minute by minute, at up to 120x speed.
- Does Papertrade include trading fees?
- Yes. Every fill is charged the chosen broker's fees plus the market's charges and taxes. There are 172 brokers and apps to choose from.