ALGO TRADING
Seven engines, five strategies: a moving-average crossover, a breakout with stop and target, a hedged pair, a 20-asset rotation and limit orders. Every result is checked against a reference simulator before anything is timed.
– WHY THIS STUDY
Most comparisons time one moving-average crossover, which says little about the strategies people actually run: stops and targets, a hedged pair, a portfolio rebalanced across twenty assets, limit orders resting under the market. And a fast engine that answers a slightly different question is not faster at the same job.
So this study writes five strategies down to the last edge case, checks every engine against a simulator built from that text alone, and only then times them, in fresh processes on a 10-core CPU. Every figure comes from one campaign, run on 2026-11-17.
– THE STRATEGIES
Fee 10 bps per fill, 1,000,000 USD of capital, 1-minute bars with real opening gaps.
| Strategy | Rule | What it exercises | Data | |
|---|---|---|---|---|
| S1 | Moving-average crossover | Long 100 units while SMA(20) > SMA(100), flat otherwise | The baseline every engine can run | 500,000 bars |
| S2 | Breakout with stop and target | Buy 100 units on a close above the 50-bar high; stop at −1%, target at +2% | Intrabar exits on high and low, gaps through a level, re-entry on the exit bar | 500,000 bars |
| S3 | Pairs statistical arbitrage | Trade the log spread of two assets at |z| > 2 over 500 bars, exit at |z| < 0.5 | Two assets, long and short, fixed dollar legs | 2 × 250,000 bars |
| S4 | Cross-sectional momentum | Every day, hold the 5 best 5-day performers of 20 assets, 50,000 USD each | A 20-asset portfolio, shared cash, rebalancing | 20 × 50,000 bars |
| S5 | Limit-order mean reversion | Rest a buy limit 0.2% under each close for one bar; once filled, exit at +0.2% or a stop at −0.4% | Resting limit orders, gaps through a limit, a stop and a target in one bar | 100,000 bars |
– METHOD
Correctness first, then speed, with the same rules for every engine.
A short simulator written from the specification alone, with every edge case decided: a gap through a stop, a stop and a target inside one bar, a re-entry on the exit bar. Every engine is compared with it, never with another engine.
Every strategy is written the way an expert user of that engine would write it for speed: its own indicators, its vectorised or compiled paths, its batch APIs, its normal configuration. Where an engine cannot express a rule, the cell says so instead of faking it.
A 10-core CPU. Every measurement runs in a fresh process pinned to the same 10 cores, with thread pools capped at 10, so no engine inherits another one's imports or caches.
Three rounds in shuffled order, the first call of each process set aside as the cold start, medians rather than means, and 95% intervals from a hierarchical bootstrap (processes, then repetitions).
– RESULTS
Median wall time of one backtest, from data loaded in the engine to the final equity read back. Whether each engine got the right answer is in the next section.
S1 · Moving-average crossover
S2 · Breakout with stop and target
S3 · Pairs statistical arbitrage
S4 · Cross-sectional momentum
S5 · Limit-order mean reversion
The spread depends on the strategy. On the crossover the fastest and the slowest engines are 2,600× apart; on the 20-asset rotation the vectorised engines close in, vectorbt at 3.3× the fastest time, because twenty columns in one array pass cost it little.
raptorbt shows what the calling path costs. Through its array runner it sits at 4.1× the fastest time on the crossover. On the pair it falls to 73×, because the only way it trades two legs is its stepping session, called from Python on every bar, and S5's limit orders go through that same session: 79×, behind backtesting.py's plain Python loop at 52×.
vectorbt pays about 3.68 s of Numba compilation on the first call of every process. manifoldbt and raptorbt have nothing to compile, and their first call comes within a few tens of milliseconds of the next ones.
| Engine | S1 · Moving-average crossover | S2 · Breakout with stop and target | S3 · Pairs statistical arbitrage | S4 · Cross-sectional momentum | S5 · Limit-order mean reversion |
|---|---|---|---|---|---|
| manifoldbt0.35.1 · Rust core, declarative expression DSL | 9.6 ms ✓1× | 19 ms ✓†1× | 52 ms ✓1× | 58 ms ✓1× | 12 ms ✓†1× |
| raptorbt0.13.4 · Rust core, array runners and a stepping session | 39 ms ✓4.1× | 205 ms ✓†11× | 3.82 s ✓73× | 489 ms ≠8.4× | 945 ms ✓†79× |
| vectorbt1.2.1 · NumPy/Numba, vectorised arrays | 937 ms ✓98× | 717 ms ≠38× | 489 ms ✓9.4× | 192 ms ✓3.3× | not expressible |
| backtesting.py0.6.6 · Python event loop over NumPy arrays | 1.42 s ✓148× | 1.37 s ✓73× | not expressible | not expressible | 629 ms ✓52× |
| bt1.4.1 · Python tree of algos over pandas | 5.22 s ✓544× | not expressible | 2.91 s ✓56× | 1.11 s ✓19× | not expressible |
| backtrader1.9.78.123 · Python event loop, line objects | 19.0 s ✓1,974× | 20.7 s ✓1,112× | 20.4 s ✓391× | 28.6 s ✓489× | 8.93 s ✓744× |
| NautilusTrader1.231.0 · Rust/Cython core, event-driven | 25.0 s ¢2,600× | 12.3 s ✓†659× | 24.6 s ¢471× | 14.4 s ¢247× | 17.7 s ✓†1,477× |
– WHERE THEY DISAGREE
Each disagreement was followed down to the bar and the rule that causes it, then confirmed by replaying the reference with only that rule changed.
| Engine | S1 · Moving-average crossover | S2 · Breakout with stop and target | S3 · Pairs statistical arbitrage | S4 · Cross-sectional momentum | S5 · Limit-order mean reversion |
|---|---|---|---|---|---|
| manifoldbt | exact | exact under its convention | exact | exact | exact under its convention |
| raptorbt | exact | exact under its convention | exact | gap$316 | exact under its convention |
| vectorbt | exact | gap$1,416 | exact | exact | not expressible |
| backtesting.py | exact | exact | not expressible | not expressible | exact |
| bt | exact | not expressible | exact | exact | not expressible |
| backtrader | exact | exact | exact | exact | exact |
| NautilusTrader | to the cent$0.21 | exact under its convention | to the cent$0.50 | to the cent$0.05 | exact under its convention |
– PARAMETER SWEEP
S1 with fast windows 5 to 40 and slow windows 50 to 250 on 100,000 bars, every combination checked against the reference. Each engine gets two chances, its own sweep mechanism and a generic pool of 10 worker processes, and keeps the faster.
The own paths: manifoldbt's run_sweep_lite (Rust threads, metrics only, no trade log), raptorbt's batch_single_backtest, vectorbt's column broadcasting (one thread in the open-source version), backtrader's optstrategy and backtesting.py's optimize with a process pool. bt and NautilusTrader have none. backtrader's optimiser beats the pool because it loads the data once; backtesting.py's loses to it because every call starts a new pool.
| Engine | Own sweep path | Generic pool | Best | Exact combinations |
|---|---|---|---|---|
| manifoldbt | 5.2 ms | 49 ms | own path | 64 / 64 |
| raptorbt | 98 ms | 84 ms | pool | 64 / 64 |
| vectorbt | 1.18 s | 1.72 s | own path | 64 / 64 |
| backtesting.py | 4.74 s | 2.79 s | pool | 37 / 64 |
| bt | none | 10.5 s | pool | 64 / 64 |
| backtrader | 22.8 s | 37.3 s | own path | 61 / 64 |
| NautilusTrader | none | 33.7 s | pool | 0 / 64 |
– LIMITS
Read the numbers with these in mind.
– FOR A TRACK RECORD
Even with every rule written out, the engines did not all print the same number. The differences came from choices nobody thinks to state: where a gapped target or limit fills, whether an exit bar can re-enter, whether a rebalance can add to a position. A backtest figure carries its engine with it.
A live account has none of these degrees of freedom. The fills happened at the venue, at the prices the venue reports, which makes it the half of the comparison anyone can check without picking an engine.
Method references: A. Georges, D. Buytaert and L. Eeckhout, Statistically rigorous Java performance evaluation, OOPSLA 2007. T. Kalibera and R. Jones, Rigorous benchmarking in reasonable time, ISMM 2013. T. Hoefler and R. Belli, Scientific benchmarking of parallel computing systems, SC15.
FAQ
Where to go next
Selection bias, cost modelling and execution: three causes, three tests.
Count the trials, deflate the winner, validate out of sample, confirm live.
Estimate the probability that the best configuration of a sweep was luck.
THE LIVE SIDE
Connect your exchange or broker. AuditZK builds a verified track record from the venue's own data, without seeing your strategy.