By Vibhu Singh
Backtesting is considered to be an important tool in a Financial trader’s toolbox, without which they wouldn’t even think of diving into the markets. Think about it, before you buy anything, be it a mobile phone or a car, you would want to check the history of the brand, its features etc., and check if it is worth your money. The same principle applies to trading.
This article is divided into the following parts:
- What is Backtesting?
- Key Decisions for Backtesting a Trading Strategy
- Backtesting Platforms
- Choice of Programming Language
- Trading Specific Libraries in Python
- Types of Backtesting
- How to Perform Backtesting?
- Backtesting Evaluation Parameters
- Biases of Backtesting
- Frequently Asked Questions
What is Backtesting?
Backtesting a trading strategy is the process of testing a trading hypothesis/strategy on the historical data.
For example, you want to test a trading hypothesis that if a stock price is higher than its 52-day week high price, then its price will increase in the future too.
By using historical data, you can backtest and see whether your hypothesis is true or not. It helps to assess the feasibility of a trading strategy, by discovering how it performs on the historical data, before implementing it on the live market.
Key Decisions for Backtesting Trading Strategy
Choose the right market/asset segment
There are various factors that you can look at, to make a decision to which market or assets will be best for the kind of trading you are looking to conduct. The factors can be risks you are willing to take, the profits you are looking to earn, and the time for which you will be investing, whether long-term or short-term.
For example, trading in cryptocurrencies might be riskier than others, but can give higher returns and vice versa. Hence, it is a crucial decision to select the right market and asset class to trade in.
Once you have decided on the market segment you would want to invest in, you would try to find some information about them, for which we move to the next segment of this article.
Data to cover the variety of market conditions
The prices in a market are vulnerable to many factors, and hence keep fluctuating depending on the kind of situation going on. These factors may include major announcements like monetary policies, the release of the annual report of a company, inflation rates etc.
The key point to consider here is the fact that the market will not always behave in a similar way, and this is the reason why we need to test the trading strategies on various market conditions, so that we know how the strategy will perform in those conditions.
Incorporate trading costs
It is important to incorporate all kinds of commissions, taxes and slippages while backtesting. It is highly probable that the strategy performs well without these costs, but it drastically affects the appearance of a strategy’s profitability after the inclusion of these costs.
There are platforms available which provide the functionality to perform backtesting on historical data.
The important points to consider before selecting a backtesting platform are:
- knowing which asset classes does the platform support,
- knowing about the sources of the market data feeds it supports, and
- figuring out which programming languages can be used to code the trading strategy which is to be tested.
We will go through the various platforms in detail later in this article.
Retail Backtesting Platforms
TradeStation provides electronic order execution across multiple asset classes. It allows trading from charts and live P&L portfolio management. TradeStation uses ‘EasyLanguage’ to create charts and develop algorithmic trading strategies. This language, as the name suggests, is easy to learn as it is very similar to English, and hence be great for someone who is a beginner in coding.
Their main offering is a FOREX Trading platform. These are custom scripts written in a proprietary language that can be used for automated trading.
NinjaTrader, a free software, uses the very widely used and exquisitely documented C# programming language and the DotNet Framework. It can be used for stock, futures and forex markets for advanced charting, strategy backtesting and trade simulation.
Blueshift is a free and comprehensive trading and strategy development platform and enables backtesting too. It helps one to focus more on strategy development rather than coding, and provides integrated high-quality minute-level data. Its cloud-based backtesting engine enables one to develop, test and analyse trading strategies in a Python programming environment.
Amibroker is a trading analysis software which allows portfolio backtesting and optimisation, and has a good range of technical indicators to analyse the strategy. It uses ‘AmiBroker Formula Language (AFL)’ to develop and implement trading strategies and indicators.
Supports coding in multiple languages. Currently provides access to US Equities and FOREX tick data, and new libraries are being added. It supports high-speed backtesting as it uses hundreds of servers in parallel.
Deltix supports equities, options, futures, currencies, baskets and custom synthetic instruments. Provides an open and flexible architecture which allows seamless and robust integration with multiple data feeds (e.g. Bloomberg, Thomson/Reuters etc.) Widely used by quant funds, proprietary trading firms etc. Not often utilized by retail traders as the software licenses are out of their budget.
Like Deltix, Quanthouse is also mostly used by institutions due to high licensing costs. Provides an all-in-one solution for data collection, strategy development, historical backtesting and live execution across instruments and portfolios.
It’s a Swiss-based firm that offers both an open-source and a commercial license for their system along with a web-based front end. Supports forex, options, futures, stocks, ETF’s, commodities, synthetic instruments and custom derivative spreads etc., allows full historical backtesting and complex event processing.
Choice of Programming Language
It plays an important role while developing a backtesting platform. Different languages have different pros and cons which when weighed carefully as per needs, can boost the system’s performance to a great extent.
We’ve briefly discussed some of the most commonly used languages below:
Python is a free open-source and cross-platform language. It has a rich library for almost every task imaginable and a specialised research environment. The execution speed is more than sufficient for intraday traders, trading on the time scale of minutes and above.
For ultimate execution speed, it offers the most flexibility for managing memory and optimising execution speed, but can lead to subtle bugs and is difficult to learn.
C# and Java
They perform automatic Garbage Collection which leads to performance overhead but more rapid development. Both are good choices for developing a backtester as they have native GUI capabilities, numerical analysis libraries and offer fast execution speed.
MATLAB is a commercial IDE with multiple numerical libraries for scientific computation. It boasts high execution speed, but is still less appealing to retails trades as it is quite expensive.
R is a dedicated statistics scripting environment which is free, open-source, cross-platform and contains a wealth of freely-available statistical packages for carrying out extremely advanced analysis, but lacks execution speed unless operations are vectorised.
Trading Specific Libraries in Python
Zipline is an event driven library that supports both backtesting and live trading. Zipline is free, well-documented, has a great community and supports Pandas integration. It also supports TA-Lib (Technical Analysis Library) integration.
An event-driven library which focuses on backtesting, and supports paper-trading and live trading. PyAlgoTrade supports access to data from Yahoo Finance, Ninja Trader, and any type of time series data in CSV. The documentation is good and it supports TA-Lib integration (Technical Analysis Library). However, its biggest drawback is that it doesn’t support Pandas-object and pandas modules.
It is a vectorised backtesting framework designed to make your backtesting simple and fast. It allows the user to specify trading strategies using the full power of pandas while hiding all manual calculations for trades, equity, performance statistics and creating visualisations.
It is a vectorised system. A Python project for real-time financial data collection, analysing and backtesting trading strategies. Supports access to data from Yahoo Finance, other web sources and Excel. You must be wondering what is event driven and vectorised backtesting. Let’s try to find out.
Types of Backtesting
Backtesting a model can be approached in two ways: Vectorised or Event Driven. Let’s understand both approaches with the help of an example.
Vectorised method is a crude way to quickly test a strategy. You pass the historical data in a tabular format and calculate the signal vector and returns vector. And you multiply the signal vector with the returns vector to get the equity curve.
On the other hand, an Event Driven approach is identical to how you will implement the trading strategy in the live market. It consists of fetching real-time data as input, generating trading signals and placing the orders to brokers.
Consider a scenario, where you need to place orders after 1 minute of market open, based on the logic that if today’s open price is higher than yesterday's close price, you will buy, otherwise sell.
In event driven format, first, you will check if it is 1-minute after the market opens. If it is, you can think of this as an event. You will fetch the real-time data, check if today’s open price is higher or lower than the previous day’s closing price. Based on the result, you will place buy or sell orders to the brokers.
A crude way of testing a strategy idea
Emulates real-time scenario
Less efficient than the vectorised approach
More prone to look-ahead bias
Avoid look-ahead bias
Can’t incorporate transaction cost in real-time
Incorporate transaction cost, slippage in real-time
How to Perform Backtesting?
Vectorised Backtesting in Python
In this part of this article, we will demonstrate how to backtest a trading strategy based on moving averages in Python.
We will use a simple moving average, which is calculated by adding the prices of the last n number of days and then dividing by the number of days. To create and backtest a trading strategy in Python, you can check out this course on Quantra.
The trading rule is very simple. When the short-term moving average (50-day moving average) crosses above the long-term moving average (200-day crossover), we buy the security. This is also called a golden crossover.
And when the short-term moving average crosses below the long-term average, we sell. This is called the death cross.
The steps to backtest this strategy are:
- Fetch Price Data
- Calculate Moving Averages
- Generate Trading Signals
- Plot Equity Curve
We will fetch the daily stock price data from Yahoo finance.
Calculate Moving Averages
We will calculate the moving 50-day and 200-day moving average of closing price. We will use pandas rolling and mean methods to calculate moving average.
Generate Trading Signals
As discussed earlier, we will buy when the 50-day moving average is greater than the 200-day moving average, and short when the 50-day moving average is below the 50-day average.
Plot Equity Curve
We will calculate and plot the cumulative strategy returns.
Event Driven Backtesting on Blueshift
Blueshift is a platform to research and trade systematic investment strategies in Python. It is built on the Zipline library.
We will import some of the Zipline libraries required to backtest the moving average strategy on Blueshift and follow the below steps:
- Import Necessary Libraries
- Schedule Strategy Logic
- Define Strategy Logic
- Fetch Price Data
- Calculate Moving Averages
- Place Orders
Backtesting Evaluation Parameters
We performed backtesting, now it’s time to gauge its performance based on certain parameters.
Let’s try to understand some of the metrics.
It is the return that an investment has earned over one year.
Annualised returns = ((1 + cumulative returns) ^ (252/ number of trading days)) – 1
Cumulative returns are total returns on the investment.
Cumulative returns = ((final value – initial value)/ initial value) *100
Volatility is a statistical measure of the dispersion of returns for a given security, market index or portfolio.
Annualised volatility = daily volatility * square root of 252
The Sharpe ratio is the excess returns per unit of volatility. Excess return is total returns minus the risk-free rate of return. Generally, risk-free return is the return on the risk-free assets such as government bonds.
Sharpe ratio = Excess returns / volatility
Higher Sharpe ratio is preferred as the returns are higher per unit of risk.
It is the maximum value a portfolio lost from its peak. It is the maximum loss the strategy can make. Higher the value of the drawdown, higher would be the losses.
Maximum Drawdown = (Trough value - Peak value) / Peak value
Biases of Backtesting
You backtested a simple trading strategy and also analysed the performance of the same. But wait, a good backtester should be aware of certain drawbacks/biases which might drastically change your backtesting results.
Let’s understand a few of them now.
- Past is the measure of future prediction
Backtesting works on the assumption that the past performance is the measure of future performance. For example, say, you want to test a strategy based on the notion that Internet IPOs outperform the overall market.
If you were to test this strategy during the dotcom boom years in the late 90s, the strategy would outperform the market significantly. However, trying the same strategy after the bubble burst would result in dismal returns.
The maxim, ‘past performance does not necessarily guarantee future returns’, has to be kept into consideration while backtesting a trading strategy.
- Overfitting/Optimisation Bias
Backtesting like any other model is prone to overfitting. While testing the model on historical data, you inadvertently try to fit the parameters to get the best results. The problem here is you get the best result on the historical dataset but when you deploy the same model on the unseen dataset, it might fail to give the same result.
How to avoid overfitting while backtesting?
The best way to avoid overfitting is to divide the dataset into train and test dataset (similar to machine learning). You backtest your trading strategy on the train dataset. And run your strategy on the test dataset with the same parameters that you used on the train dataset, to ensure the effectiveness of the strategy.
- Look-ahead Bias
Look-ahead bias is the use of information in the analysis before the time it would have actually occurred. While devising a strategy, you have access to the entire data, and thus there might be situations where you include future data that was not able in the time period being tested.
A seemingly insignificant oversight such as assuming that the earning report being available one day prior, can lead to skewed results during the backtesting. You need to make sure you are not using data that will only be available in the future to avoid look-ahead bias.
- Survivorship Bias
During backtesting the strategy, you often tend to backtest a strategy on the current stock universe rather than historical stock universe. That is, you use the universe that has survived until today to backtest.
There is a famous example which is used to illustrate the survivorship bias. If you were to use stocks of technology companies to formulate a strategy, but took the data after the dot com bubble burst, then it would present a starkly different scenario than if you had included the data prior to the bubble burst.
It's a simple fact, after the year 2000, the companies which survived did well because their fundamentals were strong, and hence your strategy would not be including the whole universe, and thus your backtesting result might not be able to give the full picture.
How to avoid survivorship bias while backtesting?
One of the approaches is to buy expensive survivorship bias-free data. Another approach is to use only recent data not more than 5 years (not very robust approach)
Frequently Asked Questions
Q. How to interpret backtesting results?
There are various performance parameters that can be used to evaluate and interpret the backtesting results. Some of the performance metrics are Sharpe ratio, drawdown, annualised returns and volatility. These metrics are discussed in this blog in the section “Backtesting Evaluation Parameters”. You can read more about the performance metrics in this article.
Q. How to differentiate between good and bad backtesting results?
To know whether the backtesting results are good or bad, you need to test it. One of the ways is to divide the dataset into training and test dataset. In the training dataset, you can create and optimise the strategy parameters. And on the test dataset, you can analyse the performance.
If the strategy performs well on the test data, it is an indication that the strategy is good. If the performance is sub-optimal, it is possible that you have overfitted the parameters and you need to re-evaluate the model.
Q. What should be the time period for backtesting?
The time period for backtesting depends on the average holding period of your position. If you are trading a strategy with a holding period of more than a month, then it is better to use a long time period, preferably 15 years.
If you are creating an intraday strategy, then 10 years is a good amount of time. However, it is possible that the strategy that worked well for 10 years, no longer works when you start trading. Therefore, it is important to test your strategy on out of sample dataset.
Q. How many stocks to backtest with?
There is no fixed answer to this question. But the strategy includes a diversified set of stocks that belong to different sectors. This is because if you only keep stocks from a particular sector, say technology. Then in scenarios like the Dot-com bubble, your strategy will be doomed. But such situations can be avoided, if you have a diversified portfolio.
Backtesting proves to be one of the biggest advantages of Algorithmic Trading due to the fact that it allows us to test our strategies, before actually implementing them in the live market.
In this blog, we have covered the basic topics one needs to know before starting backtesting. To learn on the subject, you can watch the webinar on Backtesting a Trading Using Python.
Disclaimer: All investments and trading in the stock market involve risk. Any decisions to place trades in the financial markets, including trading in stock or options or other financial instruments is a personal decision that should only be made after thorough research, including a personal risk and financial assessment and the engagement of professional assistance to the extent you believe necessary. The trading strategies or related information mentioned in this article is for informational purposes only.