ALGO TRADING

Python backtesting engines, five strategies, one exact answer

Seven engines, five strategies: a moving-average crossover, a breakout with stop and target, a hedged pair, a 20-asset rotation and limit orders. Every result is checked against a reference simulator before anything is timed.

– WHY THIS STUDY

The engine is part of every backtest result

Most comparisons time one moving-average crossover, which says little about the strategies people actually run: stops and targets, a hedged pair, a portfolio rebalanced across twenty assets, limit orders resting under the market. And a fast engine that answers a slightly different question is not faster at the same job.

So this study writes five strategies down to the last edge case, checks every engine against a simulator built from that text alone, and only then times them, in fresh processes on a 10-core CPU. Every figure comes from one campaign, run on 2026-11-17.

– THE STRATEGIES

One simple, four realistic

Fee 10 bps per fill, 1,000,000 USD of capital, 1-minute bars with real opening gaps.

One simple, four realistic
StrategyRuleWhat it exercisesData
S1Moving-average crossoverLong 100 units while SMA(20) > SMA(100), flat otherwiseThe baseline every engine can run500,000 bars
S2Breakout with stop and targetBuy 100 units on a close above the 50-bar high; stop at −1%, target at +2%Intrabar exits on high and low, gaps through a level, re-entry on the exit bar500,000 bars
S3Pairs statistical arbitrageTrade the log spread of two assets at |z| > 2 over 500 bars, exit at |z| < 0.5Two assets, long and short, fixed dollar legs2 × 250,000 bars
S4Cross-sectional momentumEvery day, hold the 5 best 5-day performers of 20 assets, 50,000 USD eachA 20-asset portfolio, shared cash, rebalancing20 × 50,000 bars
S5Limit-order mean reversionRest a buy limit 0.2% under each close for one bar; once filled, exit at +0.2% or a stop at −0.4%Resting limit orders, gaps through a limit, a stop and a target in one bar100,000 bars

– METHOD

How the measurement was built

Correctness first, then speed, with the same rules for every engine.

01

A reference with no engine in it

A short simulator written from the specification alone, with every edge case decided: a gap through a stop, a stop and a target inside one bar, a re-entry on the exit bar. Every engine is compared with it, never with another engine.

02

Each engine at its best

Every strategy is written the way an expert user of that engine would write it for speed: its own indicators, its vectorised or compiled paths, its batch APIs, its normal configuration. Where an engine cannot express a rule, the cell says so instead of faking it.

03

10 pinned cores

A 10-core CPU. Every measurement runs in a fresh process pinned to the same 10 cores, with thread pools capped at 10, so no engine inherits another one's imports or caches.

04

Statistics

Three rounds in shuffled order, the first call of each process set aside as the cold start, medians rather than means, and 95% intervals from a hierarchical bootstrap (processes, then repetitions).

– RESULTS

Time and agreement, engine by strategy

Median wall time of one backtest, from data loaded in the engine to the final equity read back. Whether each engine got the right answer is in the next section.

Median time of one backtest, per strategy (shared log scale)

S1 · Moving-average crossover

10 ms0.1 s1 s10 smanifoldbtmanifoldbt, S1: 9.6 ms (95% interval 9.4 ms – 9.7 ms)9.6 msraptorbtraptorbt, S1: 39 ms (95% interval 39 ms – 40 ms)39 msvectorbtvectorbt, S1: 937 ms (95% interval 926 ms – 949 ms)937 msbacktesting.pybacktesting.py, S1: 1.42 s (95% interval 1.42 s – 1.44 s)1.42 sbtbt, S1: 5.22 s (95% interval 5.20 s – 5.26 s)5.22 sbacktraderbacktrader, S1: 19.0 s (95% interval 18.9 s – 19.1 s)19.0 sNautilusTraderNautilusTrader, S1: 25.0 s (95% interval 24.8 s – 26.1 s)25.0 s

S2 · Breakout with stop and target

10 ms0.1 s1 s10 smanifoldbtmanifoldbt, S2: 19 ms (95% interval 19 ms – 19 ms)19 msraptorbtraptorbt, S2: 205 ms (95% interval 204 ms – 206 ms)205 msvectorbtvectorbt, S2: 717 ms (95% interval 697 ms – 723 ms)717 msbacktesting.pybacktesting.py, S2: 1.37 s (95% interval 1.36 s – 1.38 s)1.37 sbtnot expressiblebacktraderbacktrader, S2: 20.7 s (95% interval 20.1 s – 20.8 s)20.7 sNautilusTraderNautilusTrader, S2: 12.3 s (95% interval 12.2 s – 13.0 s)12.3 s

S3 · Pairs statistical arbitrage

10 ms0.1 s1 s10 smanifoldbtmanifoldbt, S3: 52 ms (95% interval 52 ms – 53 ms)52 msraptorbtraptorbt, S3: 3.82 s (95% interval 3.80 s – 3.90 s)3.82 svectorbtvectorbt, S3: 489 ms (95% interval 485 ms – 492 ms)489 msbacktesting.pynot expressiblebtbt, S3: 2.91 s (95% interval 2.89 s – 2.93 s)2.91 sbacktraderbacktrader, S3: 20.4 s (95% interval 19.5 s – 20.5 s)20.4 sNautilusTraderNautilusTrader, S3: 24.6 s (95% interval 24.5 s – 25.4 s)24.6 s

S4 · Cross-sectional momentum

10 ms0.1 s1 s10 smanifoldbtmanifoldbt, S4: 58 ms (95% interval 57 ms – 59 ms)58 msraptorbtraptorbt, S4: 489 ms (95% interval 484 ms – 511 ms)489 msvectorbtvectorbt, S4: 192 ms (95% interval 189 ms – 194 ms)192 msbacktesting.pynot expressiblebtbt, S4: 1.11 s (95% interval 1.09 s – 1.12 s)1.11 sbacktraderbacktrader, S4: 28.6 s (95% interval 28.2 s – 28.9 s)28.6 sNautilusTraderNautilusTrader, S4: 14.4 s (95% interval 14.4 s – 14.5 s)14.4 s

S5 · Limit-order mean reversion

10 ms0.1 s1 s10 smanifoldbtmanifoldbt, S5: 12 ms (95% interval 12 ms – 12 ms)12 msraptorbtraptorbt, S5: 945 ms (95% interval 941 ms – 948 ms)945 msvectorbtnot expressiblebacktesting.pybacktesting.py, S5: 629 ms (95% interval 624 ms – 634 ms)629 msbtnot expressiblebacktraderbacktrader, S5: 8.93 s (95% interval 8.56 s – 9.26 s)8.93 sNautilusTraderNautilusTrader, S5: 17.7 s (95% interval 17.7 s – 18.4 s)17.7 s
One marker shape per strategy, the same in every figure. Same engine order in all five panels, fastest to slowest on S1. Hover a bar for its 95% interval; the full table is below.
Distance to the fastest engine, strategy by strategy
  • S1 · Moving-average crossover
  • S2 · Breakout with stop and target
  • S3 · Pairs statistical arbitrage
  • S4 · Cross-sectional momentum
  • S5 · Limit-order mean reversion
1×10×100×1,000×manifoldbtmanifoldbt, S1: 1.0× (9.6 ms, 95% interval 9.4 ms – 9.7 ms)manifoldbt, S2: 1.0× (19 ms, 95% interval 19 ms – 19 ms)manifoldbt, S3: 1.0× (52 ms, 95% interval 52 ms – 53 ms)manifoldbt, S4: 1.0× (58 ms, 95% interval 57 ms – 59 ms)manifoldbt, S5: 1.0× (12 ms, 95% interval 12 ms – 12 ms)fastest on all five strategiesraptorbtraptorbt, S1: 4.1× (39 ms, 95% interval 39 ms – 40 ms)raptorbt, S2: 11× (205 ms, 95% interval 204 ms – 206 ms)raptorbt, S3: 73× (3.82 s, 95% interval 3.80 s – 3.90 s)raptorbt, S4: 8.4× (489 ms, 95% interval 484 ms – 511 ms)raptorbt, S5: 79× (945 ms, 95% interval 941 ms – 948 ms)vectorbtvectorbt, S1: 98× (937 ms, 95% interval 926 ms – 949 ms)vectorbt, S2: 38× (717 ms, 95% interval 697 ms – 723 ms)vectorbt, S3: 9.4× (489 ms, 95% interval 485 ms – 492 ms)vectorbt, S4: 3.3× (192 ms, 95% interval 189 ms – 194 ms)backtesting.pybacktesting.py, S1: 148× (1.42 s, 95% interval 1.42 s – 1.44 s)backtesting.py, S2: 73× (1.37 s, 95% interval 1.36 s – 1.38 s)backtesting.py, S5: 52× (629 ms, 95% interval 624 ms – 634 ms)btbt, S1: 544× (5.22 s, 95% interval 5.20 s – 5.26 s)bt, S3: 56× (2.91 s, 95% interval 2.89 s – 2.93 s)bt, S4: 19× (1.11 s, 95% interval 1.09 s – 1.12 s)backtraderbacktrader, S1: 1,974× (19.0 s, 95% interval 18.9 s – 19.1 s)backtrader, S2: 1,112× (20.7 s, 95% interval 20.1 s – 20.8 s)backtrader, S3: 391× (20.4 s, 95% interval 19.5 s – 20.5 s)backtrader, S4: 489× (28.6 s, 95% interval 28.2 s – 28.9 s)backtrader, S5: 744× (8.93 s, 95% interval 8.56 s – 9.26 s)NautilusTraderNautilusTrader, S1: 2,600× (25.0 s, 95% interval 24.8 s – 26.1 s)NautilusTrader, S2: 659× (12.3 s, 95% interval 12.2 s – 13.0 s)NautilusTrader, S3: 471× (24.6 s, 95% interval 24.5 s – 25.4 s)NautilusTrader, S4: 247× (14.4 s, 95% interval 14.4 s – 14.5 s)NautilusTrader, S5: 1,477× (17.7 s, 95% interval 17.7 s – 18.4 s)
Each marker is one engine on one strategy, relative to that strategy's best time (1× = fastest). The line joins an engine's five strategies: the longer it is, the more the engine's rank depends on what you ask of it.

The spread depends on the strategy. On the crossover the fastest and the slowest engines are 2,600× apart; on the 20-asset rotation the vectorised engines close in, vectorbt at 3.3× the fastest time, because twenty columns in one array pass cost it little.

raptorbt shows what the calling path costs. Through its array runner it sits at 4.1× the fastest time on the crossover. On the pair it falls to 73×, because the only way it trades two legs is its stepping session, called from Python on every bar, and S5's limit orders go through that same session: 79×, behind backtesting.py's plain Python loop at 52×.

vectorbt pays about 3.68 s of Numba compilation on the first call of every process. manifoldbt and raptorbt have nothing to compile, and their first call comes within a few tens of milliseconds of the next ones.

Show the numbers as a table
Time and agreement, engine by strategy
EngineS1 · Moving-average crossoverS2 · Breakout with stop and targetS3 · Pairs statistical arbitrageS4 · Cross-sectional momentumS5 · Limit-order mean reversion
manifoldbt0.35.1 · Rust core, declarative expression DSL9.6 ms ✓1×19 ms ✓†1×52 ms ✓1×58 ms ✓1×12 ms ✓†1×
raptorbt0.13.4 · Rust core, array runners and a stepping session39 ms ✓4.1×205 ms ✓†11×3.82 s ✓73×489 ms ≠8.4×945 ms ✓†79×
vectorbt1.2.1 · NumPy/Numba, vectorised arrays937 ms ✓98×717 ms ≠38×489 ms ✓9.4×192 ms ✓3.3×not expressible
backtesting.py0.6.6 · Python event loop over NumPy arrays1.42 s ✓148×1.37 s ✓73×not expressiblenot expressible629 ms ✓52×
bt1.4.1 · Python tree of algos over pandas5.22 s ✓544×not expressible2.91 s ✓56×1.11 s ✓19×not expressible
backtrader1.9.78.123 · Python event loop, line objects19.0 s ✓1,974×20.7 s ✓1,112×20.4 s ✓391×28.6 s ✓489×8.93 s ✓744×
NautilusTrader1.231.0 · Rust/Cython core, event-driven25.0 s ¢2,600×12.3 s ✓†659×24.6 s ¢471×14.4 s ¢247×17.7 s ✓†1,477×
  • ✓exact to a millionth of a dollar, same fills
  • ✓†exact under its own convention for a level the bar opens beyond, confirmed by replay
  • ¢same fills, amounts rounded to the cent
  • ≠differs, cause below
  • not expressiblecannot be expressed natively

– WHERE THEY DISAGREE

Every gap, traced to its rule

Each disagreement was followed down to the bar and the rule that causes it, then confirmed by replaying the reference with only that rule changed.

Agreement with the reference, engine by strategy
EngineS1 · Moving-average crossoverS2 · Breakout with stop and targetS3 · Pairs statistical arbitrageS4 · Cross-sectional momentumS5 · Limit-order mean reversion
manifoldbtexactexact under its conventionexactexactexact under its convention
raptorbtexactexact under its conventionexactgap$316exact under its convention
vectorbtexactgap$1,416exactexactnot expressible
backtesting.pyexactexactnot expressiblenot expressibleexact
btexactnot expressibleexactexactnot expressible
backtraderexactexactexactexactexact
NautilusTraderto the cent$0.21exact under its conventionto the cent$0.50to the cent$0.05exact under its convention
Gap in final equity on one million dollars of capital. Hover a cell for the exact gap and the fill count against the reference.
manifoldbt, raptorbt, NautilusTrader (S2, S5): where a gapped level fills
When a bar opens beyond a resting level, a target above the market or a buy limit below it, the reference fills at the open: the better price, the one a resting limit order gets at a real venue, and what backtrader and backtesting.py do. raptorbt and NautilusTrader fill at the level on every such bar. manifoldbt 0.35.1 fills at the level when the bar trades back to it, at the open when it never does. Neither choice is a bug; filling at the level gives up a little profit and never invents any. On S2 the rule touches the target of 17 of 1,344 trades for raptorbt ($73) and 16 for manifoldbt ($71). On S5 it touches 17 limit entries and 40 targets of 1,799 trades for raptorbt ($457), 17 entries and 36 targets for manifoldbt ($441). Replayed with each engine's rule, the reference lands on that engine's equity, to the cent for manifoldbt and raptorbt and within its cent rounding for NautilusTrader.
vectorbt (S2)
One order per bar: after a stop or a target exits on a bar, it cannot enter again on that same bar, which the specification allows. 82 fewer fills and $1,416.
raptorbt (S4)
One position per instrument: it rejects any buy on an asset already held long, so a rebalance cannot top a position up. 46 orders rejected, $316.
NautilusTrader
Account amounts are booked in cents, so on S1, S3 and S4 its equity is never exact to the millionth, and never off by more than $0.50 here. On S2 and S5 each bar price offers a quarter of the bar's volume, so every order fills in two executions.
backtesting.py
It first calls the strategy one bar after its indicators are ready, so a signal already true on the first valid bar is missed. Invisible at these sizes, it shows in the sweep: 27 of 64 combinations differ, by up to $24.
backtrader
An order decided on the final bar never fills, because it would execute while processing a next bar that does not exist. 3 of the 64 sweep combinations trade on their last bar and miss that fill's fee, $9.00.
Not expressible
backtesting.py runs one instrument per backtest, so no pair (S3) and no portfolio (S4). bt transacts at one price per bar with no order types, so no intrabar stop or target (S2) and no limit order (S5). The open-source vectorbt has no limit order type either (S5).

– PARAMETER SWEEP

64 combinations on 10 cores

S1 with fast windows 5 to 40 and slow windows 50 to 250 on 100,000 bars, every combination checked against the reference. Each engine gets two chances, its own sweep mechanism and a generic pool of 10 worker processes, and keeps the faster.

64-combination sweep on 100,000 bars, 10 cores (log scale)
  • own sweep path
  • generic pool
10 ms0.1 s1 s10 smanifoldbtmanifoldbt, own sweep path: 5.2 ms, 64/645.2 msmanifoldbt, generic pool: 49 ms, 64/6449 msraptorbtraptorbt, own sweep path: 98 ms, 64/6498 msraptorbt, generic pool: 84 ms, 64/6484 msvectorbtvectorbt, own sweep path: 1.18 s, 64/641.18 svectorbt, generic pool: 1.72 s, 64/641.72 sbacktesting.pybacktesting.py, own sweep path: 4.74 s, 37/644.74 sbacktesting.py, generic pool: 2.79 s, 37/642.79 sbtown sweep path: –bt, generic pool: 10.5 s, 64/6410.5 sbacktraderbacktrader, own sweep path: 22.8 s, 61/6422.8 sbacktrader, generic pool: 37.3 s, 61/6437.3 sNautilusTraderown sweep path: –NautilusTrader, generic pool: 33.7 s, 0/6433.7 s
The engine's own sweep path and the generic pool of 10 processes, side by side. A dash: the engine has no sweep mechanism.

The own paths: manifoldbt's run_sweep_lite (Rust threads, metrics only, no trade log), raptorbt's batch_single_backtest, vectorbt's column broadcasting (one thread in the open-source version), backtrader's optstrategy and backtesting.py's optimize with a process pool. bt and NautilusTrader have none. backtrader's optimiser beats the pool because it loads the data once; backtesting.py's loses to it because every call starts a new pool.

Show the numbers as a table
64 combinations on 10 cores
EngineOwn sweep pathGeneric poolBestExact combinations
manifoldbt5.2 ms49 msown path64 / 64
raptorbt98 ms84 mspool64 / 64
vectorbt1.18 s1.72 sown path64 / 64
backtesting.py4.74 s2.79 spool37 / 64
btnone10.5 spool64 / 64
backtrader22.8 s37.3 sown path61 / 64
NautilusTradernone33.7 spool0 / 64

– LIMITS

What this does not tell you

Read the numbers with these in mind.

  • Five strategies on bars. An event-driven engine such as NautilusTrader is built for order books, latency and venue behaviour, which bar strategies barely use. A market-making strategy was specified too and dropped: four of the seven engines cannot express resting quotes with partial fills and funding on bars.
  • Synthetic data, with an opening gap on every bar and a larger one every 1,440 bars, but no missing bars, no session breaks and no time zones. Real data adds those, and with them more places where engines can part ways.
  • One shared machine with other applications open. Each engine's three processes agree within a few percent, and a dedicated machine would give other absolute numbers. A first campaign on 2026-11-17 ran the Python engines up to twice as slow, because Windows power management throttled the processes; it was discarded, and the campaign shown here ran with that throttling switched off.
  • Versions move. Every number belongs to the versions shown in the results table, measured on 2026-11-17. vectorbt is the open-source package, not vectorbt PRO. raptorbt's array runner has no fixed-quantity sizing, so S1's 100 units are expressed as one lot of 100.

– FOR A TRACK RECORD

A backtest stays a simulation

Even with every rule written out, the engines did not all print the same number. The differences came from choices nobody thinks to state: where a gapped target or limit fills, whether an exit bar can re-enter, whether a rebalance can add to a position. A backtest figure carries its engine with it.

A live account has none of these degrees of freedom. The fills happened at the venue, at the prices the venue reports, which makes it the half of the comparison anyone can check without picking an engine.

Method references: A. Georges, D. Buytaert and L. Eeckhout, Statistically rigorous Java performance evaluation, OOPSLA 2007. T. Kalibera and R. Jones, Rigorous benchmarking in reasonable time, ISMM 2013. T. Hoefler and R. Belli, Scientific benchmarking of parallel computing systems, SC15.

FAQ

Frequently asked questions

THE LIVE SIDE

Prove the half no engine can change

Connect your exchange or broker. AuditZK builds a verified track record from the venue's own data, without seeing your strategy.