Baseline Testing vs. Optimization
These are two distinct stages in the research process that serve different purposes. Understanding when to use each, and what to expect from each, is essential for sound strategy development.
Baseline Testing
A baseline test is a single backtest with default or reasonable parameter values. Its purpose is to answer one question: does this strategy idea have any merit at all?
What Baseline Testing Shows
- Whether the strategy generates trades
- Whether the direction of results is positive
- Whether the equity curve has any structure (not random)
- Whether the idea is worth spending time optimizing
What Baseline Testing Does NOT Show
- The best possible performance (parameters are not optimized)
- Whether the strategy is robust (only one configuration was tested)
- Whether it will work in live trading (no out-of-sample validation)
Optimization
Optimization systematically searches the parameter space to find regions that produce good, stable results.
What Optimization Shows
- Which parameter regions produce consistent results
- How sensitive the strategy is to parameter changes
- Whether stable plateaus exist (robustness indicator)
- The upper bound of realistic performance
What Optimization Does NOT Show
- Whether those parameters will work on future data (use WFA for that)
- Whether the strategy has a genuine edge or is just overfitted
When to Use Each
| Stage | When | Purpose |
|---|---|---|
| Baseline | First test of a new idea | Quick sanity check |
| Optimization | After baseline shows promise | Systematic parameter exploration |
| WFA | After optimization finds stable regions | Out-of-sample validation |
The Critical Difference
Baseline testing is cheap and fast. If the idea clearly fails at baseline, skip optimization — you will save significant time.
Optimization is more expensive. It explores hundreds or thousands of parameter combinations. Only invest this time on ideas that pass baseline testing.
Anti-Pattern: Skipping Baseline
A common mistake is jumping straight to optimization. This wastes time and increases the risk of finding a false positive — the optimizer may find one narrow parameter set that looks profitable purely by chance.
Always start with a baseline.
Related
- Running a Backtest — How to run a baseline test
- Running Optimization — How to optimize parameters
- The Research Process — The complete workflow