By Vibhu Singh
Backtesting is considered to be an important tool in a trader’s toolbox. Without backtesting, traders wouldn’t even think of risking money into the financial markets.
Think about it, before you buy anything, be it a mobile phone or a car, you would want to check the history of the brand, its features etc. You check if it is worth your money. The same principle applies to trading, and backtesting helps you with it.
We will cover the following topics in this article.
- Why is backtesting important?
- What is backtesting?
- Prerequisites for backtesting
- How to do backtesting with Python?
- Analyzing the performance of the trading strategy
- Interpreting and analysing backtesting results
- Backtesting vs Walk forward trading testing
- Paper trading & Live trading
- Common mistakes in backtesting
- Backtesting software
Why is backtesting important?
Do you know the majority of the traders in the market lose money?
They lose money not because they lack understanding of the market. But simply because their trading decisions are not based on sound research and tested trading methods.
They make decisions based on emotions, suggestions from friends and take excessive risks in the hope to get rich quickly. If they remove emotions and instincts from the trading and backtest the ideas before trading, then the chance to trade profitability in the market is increased.
What is backtesting?
Backtesting a trading strategy is the process of testing a trading hypothesis/strategy on the historical data.
Let’s say you formed a hypothesis. This hypothsesis states that securities that have positive returns over the past one year are likely to give positive returns over the next one month.
- How would you test this hypothesis?
- How would you know whether the strategy will work in the market or not?
By using historical data, you can backtest and see whether your hypothesis is true or not. It helps assess the feasibility of a trading strategy by discovering how it performs on the historical data.
If you backtest your strategy on the historical data and it gives good returns, you will be confident to trade using it. If the strategy is performing poorly on the historical data, you will discard or re-evaluate the hypothesis.
Prerequisites for backtesting
Before you start backtesting a trading strategy, you need to consider some of the factors:
- Trading logic
- Market segment
- Data
- Programming language
Let us look at each of these factors in detail.
Trading logic/hypothesis for backtesting
You decided to backtest a trading strategy, but before you backtest, you need to have a clear picture in your mind of what you are going to backtest. That is what is the trading logic or hypothesis of this backtest.
In the above example, you calculate the past one year returns of securities and check whether the returns are positive or negative.
- If positive, then you check for the future 1 month returns of the stocks.
- If negative, you will do nothing.
If you are clear with the trading logic, then only you can backtest the trading strategy, and therefore this is the most crucial step in backtesting.
Choosing the right market or asset segment for backtesting
There are various factors that you can look at to decide which market or assets will be best for the kind of trading you are looking to conduct.
The factors can be risks you are willing to take, the profits you are looking to earn, and the time you will be investing, whether long-term or short-term.
For example, trading in cryptocurrencies might be riskier than other asset classes but can give higher returns and vice versa. Hence, it is a crucial decision to select the right market and asset class to trade-in.
Data for backtesting
Once you have shortlisted the assets, you would want to backtest your trading strategy. The next step is to choose historical data of the asset. You can get the data from the data vendor or from your broker.
It is important to select high-quality data, that is, data without any errors. If you choose poor quality data, then the output analysis from backtesting will be incorrect and misleading.
You can check out this free course on Quantra for getting the market data for different asset classes.
Choosing the programming language for backtesting
You were clear with the trading logic, selected the right asset for the trading and got the required data of the asset.
The final step is to decide the programming language which you will use to backtest a trading strategy. Actually, it is a matter of personal choice and the language you are comfortable with. Every programming language has its pros and cons.
Python - Python is a free open-source and cross-platform language. It has a rich library for almost every task imaginable and a specialised research environment. It is more suitable for medium to low frequency trading that is trading on a time scale of minutes and above. However, Python is not suitable for high-frequency trading.
C++ - C++, on the other hand, is suitable for high-frequency trading. It has ultimate execution speed. It offers the most flexibility for managing memory and optimising execution speed but can lead to subtle bugs and is difficult to learn.
MATLAB - MATLAB is another programming language with multiple numerical libraries for scientific computation. It boasts high execution speed but is still less appealing to retail trades as it is quite expensive.
R - R is a dedicated statistics scripting environment that is free, open-source, cross-platform, and contains a wealth of freely available statistical packages for extremely advanced analysis but lacks execution speed unless operations are vectorised.
Note: It is important to note that if you are not comfortable with any programming languages for backtesting, that’s not an issue. It doesn’t hinder you from backtesting your trading strategy. You can also start with Microsoft Excel.
For illustration, we will demonstrate how to backtest a trading strategy in Python in the next part of this article.
How to do backtesting with Python?
To learn how to use Python for backtesting a trading strategy, check out this highly recommended video on How to use Python for Trading and Investment. It introduces you to the basics of Python programming from a financial markets’ point of view.
Example of backtesting trading strategies
The strategy that we are going to backtest is based on the concept of moving average. Moving average is the average of the specified data field such as the price for a given set of consecutive periods.
As new data becomes available, the average of the data is computed by dropping the oldest value and adding the latest one.
The trading logic is very simple.
- When the short-term moving average (50-day moving average) crosses above the long-term moving average (200-day crossover), we buy the security. This is also called a golden crossover.
- When the short-term moving average crosses below the long-term average, we sell. This is called the death cross.
We will follow the below steps to backtest the above trading strategy.
Getting the price data for backtesting
We will do the backtesting on the Microsoft stock. To do that, you need to get the price data of Microsoft stock. We will use Yahoo! Finance to fetch the data.
Calculating the moving averages
We will calculate the moving 50-day and 200-day moving average of the closing price. We will use pandas rolling and mean methods to calculate a moving average.
Generating trading signals
As discussed earlier, we will buy when the 50-day moving average is greater than the 200-day moving average and short when the 50-day moving average is below the 50-day average.
Plotting the equity curve
We will calculate and plot the cumulative strategy returns.
Before we move and analyse the strategy's performance, let's answer two questions that must come to your mind.
- Since we backtested the trading strategy only for six years, what would be the ideal backtesting period?
- How many stocks should be used for backtesting a trading strategy?
What should be the time period for backtesting a trading strategy?
The time period for backtesting depends on the average holding period of your position.
- If you are trading a strategy with a holding period of more than a month, it is better to use a long time period, preferably 15 years.
- If you are creating an intraday strategy, then ten years is a reasonable amount of time.
How many stocks should be used for backtesting a trading strategy?
There is no fixed answer to this question. But the strategy includes a diversified set of stocks that belong to different sectors.
This is because if you only keep stocks from a particular sector, say technology. Then in scenarios like the Dot-com bubble, your strategy will be doomed. Such situations can be avoided if you have a diversified portfolio.
Analyzing the performance of the trading strategy
Before interpreting and analysing the backtesting results, let’s understand some of the common performance metrics used to evaluate the strategy performance.
Cumulative returns
A cumulative return or absolute return is the total amount of money that an investment has gained or lost over time, independent of the time involved. It is expressed as a percentage and is given by a formula.
$$ Cumulative~returns = \frac{Final~value~of~investment - Initial~value~of~ investment}{Initial~value~of~investment}$$Example: Suppose you invested in a company with an initial capital of $10,000. After three years, the investment grows to $18,000. The cumulative return of your investment is 80%.
Cumulative returns = (18000 - 10000) / 10000 * 100
ie. Cumulative returns = 80%
Annualised returns
The annualized return is the geometric average amount of money earned by an investment each year over a given time period. It shows what strategy would earn over a period of time if the annual return was compounded. It is calculated using the below formula.
$$ Annualised~returns = (1+Cumulative~returns^\frac{365}{no.~of~days}) - 1 $$Example: Consider the above example where the cumulative returns on your investment is 80% over three years. The number of days is 365*3. The annualised returns are:
Annualised returns = (1 + 0.80)^ (365 / 365*3) - 1
ie. Annualised returns = 21.64%
Annualised volatility
Volatility is the measure of risk. It is defined as the standard deviation of the returns of the investment. Annualised volatility can be calculated by multiplying the daily volatility with the square root of the number of trading days in a year.
Sharpe ratio
The Sharpe ratio is the excess return calculated as portfolio returns less the risk-free rate of return per unit of the standard deviation. Generally, a risk-free return is a return on risk-free assets such as government bonds.
The Sharpe Ratio can be used to compare the portfolio with the benchmark to get to know how your strategy is repaying for the risk taken on the investment.
$$ Sharpe~ratio = \frac{Portfolio~returns-Risk~free~returns}{Standard~ deviation~of~the~portfolio~returns}$$Example: Consider a portfolio with annualised returns of 10%, and the standard deviation of the portfolio is 4%. An annual return of a risk-free bond is 4%. Then the Sharpe ratio for the strategy is 1.5.
Sharpe ratio = (10 - 4) / 4
ie. Sharpe Ratio = 1.5
A higher Sharpe ratio is always preferable over the lower ones. A trading strategy with Sharpe ratio greater than 1 is considered a satisfactory strategy, while a strategy with Sharpe greater than 2 is a good strategy.
Sortino ratio
The Sortino ratio is the variation of the Sharpe ratio, where the total standard deviation is replaced with the downside deviation. The downside deviation is the standard deviation of negative asset return.
It differentiates the harmful volatility from the total volatility by using the standard deviation of negative returns only. Since an investor is concerned only about the downside volatility, the Sortino ratio is a good measure to assess the returns per risk.
$$ Sortino~ratio = \frac{Portfolio~returns-Risk~free~returns}{Standard~ deviation~of~the~negative~portfolio~returns}$$Beta
Beta is used to capture the relationship between portfolio volatility with respect to market volatility. It tells if the market is moved by x percentage how much a portfolio is expected to increase or decrease.
- If the portfolio moves less than the market, the portfolio beta is less than 1,
- If the portfolio moves more than the market, then the portfolio's beta is greater than 1
- A portfolio with beta 1 means the portfolio has the same volatility as the market.
Example: If the market is moved by 10%, a portfolio with a beta of 1.5 is expected to move by 15%. Similarly, a portfolio with a beta of 0.5 is expected to move by 5%. The formula gives beta of a portfolio:
$$ Beta = \frac{Covariance(Portfolio~returns, Market~returns)}{Variance (Market~returns)}$$Maximum drawdown
Maximum drawdown measures the maximum loss from peak to trough of a portfolio during a specific period. It is measured as the price difference at the trough and at the peak divided by the price at the peak. It is calculated in percentage terms.
$$ Maximum~drawdown = \frac{Price~at~trough - Price~at~peak}{Price~at~peak}$$Example: Suppose you invested $10,000 in a portfolio. In the first year, the portfolio value increases to $12,000 before falling to $7,000. At the end of the 2nd year, the investment rebounds to $8000. The maximum drawdown of the portfolio in this case is
Maximum drawdown = (7000 - 12000)/12000
ie. Maximum drawdown = -41.6%
Interpreting and analysing backtesting results
Now you understand the common metrics used in evaluating the strategy's performance, it's time to use some of the metrics to evaluate our moving average crossover strategy.
The annualised return of the strategy is 23.41, which means that over the period of backtesting, the strategy generates a return of around 23% each year. The Sharpe ratio of the strategy is below 1.
Therefore we can say that the strategy is sub-optimal, and there is a lot of scope for improvement.
There are lots of performance and risk indicators that can be used for evaluation purposes.
But among them which one should you choose? So the next question is:
How should you define risk metrics for yourself?
Volatility and maximum drawdown are the standard measures of risk. If you are concerned about the maximum loss a strategy can incur over a period of time. Then you can use maximum drawdown.
If you want to invest in a less risky strategy, Beta is the most suitable risk metric. You can calculate the Beta of the strategy to compare it with the market volatility.
Generally, traders use the Sharpe ratio as it provides information about the returns per unit risk. So, it is using both factors, risk and returns.
Backtesting vs Walk forward trading testing
Backtesting a strategy gives you a good understanding of what happened in the past, but it's not a predictor of the future. Walk forward testing is a better approach which to some extent, can tell the future.
In the walk forward testing method, we divide the historical data in the training (in-sample) and testing (out-of-sample) dataset. On the training dataset, we optimise the trading parameters and check the performance of the strategy on the testing datasets.
Consider our strategy on moving average crossover where you need to optimise the moving averages periods. That is for which moving average period, the strategy performs the best.
Suppose you have ten years of data.
- You take the first three years of data and calculated moving averages.
- You found that 50 and 200 moving days were the most optimal moving averages in that period.
- You then validate this rule by assessing its performance for the 4th year using various performance metrics.
- Next, you repeat the optimisation using data from years 2–4, and validate using month 5.
- You keep repeating this process until you've reached the end of the data.
- You collate the performances of all the out-of-sample data from year 4 to 10, which is your out-of-sample performance.
Paper trading & Live trading
You created the strategy and analysed the performance of the strategy.
Can you directly start a paper or live trading?
When should you consider your strategy for paper trading or live trading?
If you are satisfied with the backtesting strategy performance, then you can start paper trading. If not, you should tweak the strategy until the performance is acceptable to you. And once the paper trading results are satisfactory, you can start live trading.
Process of Paper trading and Live trading
How many backtests should I do before taking a strategy live?
There is no fixed number. You can take your strategy live after backtesting once or it can be after multiple backtesting. As we mentioned in the previous question, once you are satisfied with the backtesting results, you can consider your trading strategy for paper trading and live trading.
Common mistakes in backtesting
A good backtester should be aware of certain drawbacks/biases which might drastically change your backtesting results.
- Overfitting
- Look ahead bias
- Survivorship bias
- Ignoring trading costs
Let's understand a few of them now.
Overfitting/Optimisation bias
Backtesting, like any other model, is prone to overfitting. While testing the model on historical data, you inadvertently try to fit the parameters to get the best results. You get the best result on the historical dataset, but when you deploy the same model on the unseen dataset, it might fail to give the same result.
The best way to avoid overfitting is:
- To divide the dataset into train and test datasets (similar to machine learning).
- You backtest your trading strategy on the training dataset.
- And run your strategy on the test dataset with the same parameters that you used on the training dataset to ensure the effectiveness of the strategy.
Look-ahead bias
Look-ahead bias is the use of information in the analysis before the time it would have actually occurred. While devising a strategy, you have access to the entire data. Thus, there might be situations where you include future data that was not able in the time period being tested.
A seemingly insignificant oversight, such as assuming that the earning report being available one day prior, can lead to skewed results during the backtesting. You need to make sure you are not using data that will only be available in the future to avoid look-ahead bias.
Survivorship bias
During backtesting trading strategies, you often tend to backtest a strategy on the current stock universe rather than the historical stock universe. That is, you use the universe that has survived until today to backtest.
There is a famous example that is used to illustrate the survivorship bias.
If you were to use stocks of technology companies to formulate a strategy, but took the data after the dot com bubble burst, it would present a starkly different scenario than if you had included it before the bubble burst.
It's a simple fact, after the year 2000, the companies which survived did well because their fundamentals were strong, and hence your strategy would not be including the whole universe. Thus your backtesting result might not be able to give the whole picture.
Ignoring trading costs
It is crucial to incorporate all kinds of commissions, taxes and slippages while backtesting. It is highly probable that the strategy performs well without these costs, but it drastically affects the appearance of a strategy's profitability after the inclusion of these costs.
Backtesting software
There are platforms available that provide the functionality to perform backtesting on historical data. The important points to consider before selecting a backtesting platform are:
- knowing which asset classes does the platform support,
- knowing about the sources of the market data feeds it supports, and
- figuring out which programming languages can be used to code the trading strategy which is to be tested.
Some of the common backtesting software and live trading software are:
- Blueshift
- MetaTrader
- Amibroker
- QuantConnect
- Quanthouse, etc.
Recommended
A complete overview of working with data, formulating and backtesting a trading strategy can be seen in this video that explains all about working with data, formulating and backtesting a trading strategy.
Conclusion
Backtesting proves to be one of the biggest advantages of Algorithmic Trading because it allows us to test our trading strategies before actually implementing them in the live market. In this blog, we have covered all the topics that one needs to be aware of before starting backtesting.
You can improve your likelihood of success in trading by backtesting your trading rules on historical data. This course on backtesting trading strategies by Quantra is just what you need to get the best out of your trading. Learn everything from the basic steps, data, rules, risk management and more. Enroll now!
Disclaimer: All data and information provided in this article are for informational purposes only. QuantInsti® makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information in this article and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.
