A bad backtest will produce a historical performance that is better than what would have been obtained if we actually deployed the algorithm at the time.
Look-Ahead Bias
The first pitfall we will explore is look-ahead bias. This occurs when you trade on data that would have been made available ahead of time. For example, you short when the stock is at 95% of the day’s high. You wouldn’t know the day’s high until after the day is over. To avoid this, we should use lagged historical data. Lagging data means that you calculate your indicators up to the close of the previous trading day. For example, on December 12th your SMA is calculated using December 11th data.
Luckily, Blankly reduces look-ahead bias by imposing an event driven architecture for model backtesting so that if you are training machine learning algorithms, or optimizing across backtests, look-ahead bias is mitigated.
Data-Snoopping Bias
The second pitfall we will discuss is data-snooping bias. This occurs when you over optimize the parameters of your model based on historical data and will lead to backtest results greater than what you will receive in the future. For example, from the data you have, you identify that the price always goes up after the RSI hits 33, the 30 day moving average crosses the 60 day moving average, the MAC-D indicates a buy, the price is at the bottom most bollinger band, the volume is at 90% of its high, and the moon is full. The best way to avoid this is to keep your model simple with few parameters.
Survivorship Bias
The third pitfall we will discuss is survivorship bias. Many free databases only include stocks that are available today, and not stocks that were delisted. Therefore, your strategy is given a boost because it was not traded on securities that went to zero. This can be especially dangerous if your strategy tries to buy stocks that are “cheap”. These stocks are often cheap for a reason. For example, imminent bankruptcy, existing in a bad political climate, recent illegal activity, the list goes on... It is worth buying data sets that include securities that are not extinct.
Happy backtesting!
Want to learn more?
If you're interested in learning more about technology that can streamline your backtesting, just request a demo of Blankly here. We'd love to chat! We're constantly in our Discord too!