Backtesting is considered to be an important tool in a Financial trader’s toolbox, without which they wouldn’t even think of diving into the markets. Think about it, before you buy anything, be it a mobile phone or a car, you would want to check the history of the brand, its features etc. and check if it is worth your money. The same principle applies to trading. But what is backtesting?
In simple words, backtesting a trading strategy is the process of testing a trading hypothesis/strategy on prior time periods. Instead of applying a strategy for the time period forward (to judge performance), which could take years, a trader can simulate his or her trading strategy on relevant past data.
For example, say, a trader wants to test a strategy based on the notion that Internet IPOs outperform the overall market. If you were to test this strategy during the dotcom boom years in the late 90s, the strategy would outperform the market significantly. However, trying the same strategy after the bubble burst would result in dismal returns. The maxim ‘past performance does not necessarily guarantee future returns’ has to be kept into consideration while backtesting a trading strategy.
In this article, we will cover the following topics:
- Automated backtesting and Manual Backtesting
- Key Decisions for Backtesting Trading Strategy
- Typical Backtesting Parameters to Evaluate a Trading System
- Process of Backtesting
- Platforms Used for Backtesting
- Be Aware of Bias
Automated Backtesting and Manual Backtesting
Just like we have manual trading and automated trading, backtesting, too, runs on similar lines. Simply speaking, automated backtesting works on a code which is developed by the user where the trades are automatically placed according to his strategy whereas manual backtesting requires one to study the charts and conditions manually and place the trades according to the rules set by him. If one is good at coding, then automated trading would be of great benefit. However, one needs to keep in mind the current market conditions and tune his strategy and code accordingly to fit these conditions or it may give inaccurate results due to the changing market conditions. We will go through a few concepts in the next section.
Key Decisions for Backtesting Trading Strategy
Choose the right market/asset segment
Taking into consideration various factors such as the risks you are willing to take, the profits you are looking to earn, the time for which you will be investing, whether long-term or short-term, you can make a decision as to which market or assets will be best for the kind of trading you are looking to conduct. Some options such as trading in cryptocurrencies might be riskier than others but can give higher returns and vice versa. Hence, it is a crucial decision to select the right market and asset class to trade in. Once you have decided on the market segment you would want to invest in, you would try to find some information about them, for which we move to the next segment of this article.
Data to cover the variety of market conditions
The prices in a market are vulnerable to many factors and hence keep fluctuating depending on the kind of situation going on. These factors may include major announcements like monetary policies, the release of the annual report of a company, inflation rates, etc. The key point to consider here is the fact that the market will not always behave in a similar way and this is the reason why we need to test the trading strategies on various market conditions so that we know how the strategy will perform in those conditions.
Platform to code and backtest a trading strategy
There are platforms available which provide the functionality to perform backtesting on historical data. The important points to consider before selecting a backtesting platform are knowing which asset classes does the platform support, knowing about the sources of the market data feeds it supports and figuring out which programming languages and be used to code the trading strategy which is to be tested. We will go through the various platforms in detail later in this article.
Evaluate the system on benchmark parameters
We perform backtesting to understand how a trading strategy will work on future data by measuring its performance on the historical data. And we gauge its performance based on certain parameters such as dollar P/L, success ratio, Sharpe ratio etc. Let’s discuss them now.
Typical Backtesting Parameters to Evaluate a Trading System
Total Profit or Loss will help us determine whether the trading strategy actually benefited us or not. We can understand how much overall profit or loss can be incurred through this strategy in similar scenarios as the historical data it was tested on.
Average profit or loss will denote the amount of profit or loss which we can incur in one unit of time (days, minutes, hours) over a specific time period.
The success ratio is the number of trades we won or profited from to the number of trades we lost or incurred a loss on. This is an important indicator to understand how well our trading strategy is working and how much we need to update or optimise it in order to reap maximum benefits.
Maximum Drawdown can be used as a measurement of risk. It denotes the maximum fall in the value of the asset from a peak value. This helps us assess the risk involved and the amount of loss that we could incur from our trading strategy, thus helping us decide the amount of risk we are willing to take.
Two strategies may give us equal returns, in this case, the strategy with a lower risk will be considered better than the other. The measure of this is called the risk-adjusted return and can be calculated using the Sharpe Ratio.
Choice of Programming Language
It plays an important role while developing a backtesting platform. Different languages have different pros and cons which when weighed carefully as per needs can boost the system’s performance to a great extent. We’ve briefly discussed some of the most commonly used languages below:
For ultimate execution speed, it offers the most flexibility for managing memory and optimising execution speed but can lead to subtle bugs and is difficult to learn.
C# and Java
Perform automatic Garbage Collection which leads to performance overhead but more rapid development. Both are good choices for developing a backtester as they have native GUI capabilities, numerical analysis libraries and offer fast execution speed.
MATLAB and R
MATLAB is a commercial IDE with multiple numerical libraries for scientific computation. It boasts high execution speed but is still less appealing to retails trades as it is quite expensive. R is a dedicated statistics scripting environment which is free, open-source, cross-platform and contains a wealth of freely-available statistical packages for carrying out extremely advanced analysis but lacks execution speed unless operations are vectorized.
Python is another free open-source and cross-platform language which has a rich library for almost every task imaginable and a specialized research environment. Execution speed is more than sufficient for intraday traders trading on the time scale of minutes and above.
Process of Backtesting
After finalizing the decisions mentioned above, we can move ahead and create a trading strategy to be tested on historical data. For Backtesting, we can use various methods available including using platforms and simulators to test their strategy.
One can also build a model using Excel VBA and test it later with Python. Apart from this, testing on a simulator can give insight into the problems faced during the execution of a strategy. Simulator behaves like an exchange which can be configured for various market conditions. For simulator testing, the implementation of the testing system would require additional knowledge for Python/C++/java.
We have a large number of vendor-developed backtesting platforms available in the market which can be very efficient in backtesting automated strategies, but to decide which ones will suit your requirements, needs some research.
Types of Backtesters
Ideally, custom development of a backtesting environment within a first-class programming language provides the most flexibility and third-party platforms might make a number of assumptions. Despite this, the choice of available programming languages is large and diverse, which can often be overwhelming. When automating a strategy into systematic rules; the trader must be confident that its future performance will be reflective of its past performance. There are broadly two forms of backtesting system that are utilised to test this hypothesis; research back testers and event-driven back testers. Let’s go through them in detail now.
These tools do not fully simulate all aspects of market interaction but make approximations to provide a rapid determination of potential strategy performance. While these tools are frequently used for backtesting and execution, they are not suitable for strategies that approach intraday trading at higher frequencies. They are widely used within the professional quantitative trading industry to get the “first draft” for all strategy ideas before going for a rigorous backtest in a more realistic environment.
In event-driven backtesting, the automated trading strategy is connected to a real-time market feed and a broker, such that the system receives new market information which will be sent to a system which triggers an event to generate a new trading signal. These systems run in a continuous loop and can have sub-components such as historic data handler and brokerage simulator; allowing backtesting very similar to live execution. The only drawback is that these systems have a complicated design and are more prone to bugs.
Platforms Used for Backtesting
A quick backtesting of trading strategy for certain kind of strategies (for mainly technical trading) can be done using special platforms such as AmiBroker, Tradestation and Ninja Trader.
Let us now discuss the top backtesting platforms available in the market under different categories:
Retail Backtesting Platforms
TradeStation provides electronic order execution across multiple asset classes. It allows trading from charts and live P&L portfolio management. TradeStation uses ‘EasyLanguage’ to create charts and develop algorithmic trading strategies. This language, as the name suggests, is easy to learn as it is very similar to English and hence be great for someone who is a beginner in coding.
Their main offering is a FOREX Trading platform. These are custom scripts written in a proprietary language that can be used for automated trading.
NinjaTrader, a free software, uses the very widely used and exquisitely documented C# programming language and the DotNet Framework. It can be used for stock, futures and forex markets for advanced charting, strategy backtesting and trade simulation.
Amibroker is a trading analysis software which allows portfolio backtesting and optimization and has a good range of technical indicators to analyse the strategy. It uses ‘AmiBroker Formula Language (AFL)’ to develop and implement trading strategies and indicators.
Supports coding in multiple languages. Currently provide access to US Equities and FOREX tick data, new libraries are being added. It supports high-speed backtesting as it uses hundreds of servers in parallel.
Quantopian is actually a Hedge Fund which provides this web-based Algo Trading platform which can be used for coding, backtesting, paper trading and live trading your algorithm. In addition, it provides an amazing Research Platform with flexible data access and custom plotting in IPython notebook. Currently provides US equities data.
Blueshift is a free and comprehensive trading and strategy development platform and enables backtesting too. It helps one to focus more on strategy development rather than coding and provides integrated high-quality minute-level data. Its cloud-based backtesting engine enables one to develop, test and analyse trading strategies in a Python programming environment.
Institutional Backtesting Software
Detflix supports equities, options, futures, currencies, baskets and custom synthetic instruments. Provides an open and flexible architecture which allows seamless and robust integration with multiple data feeds (e.g. Bloomberg, Thomson/Reuters etc.) Widely used by quant funds, proprietary trading firms etc. Not often utilized by retail traders as the software licenses are out of their budget.
Like Deltix, Quanthouse is also mostly used by institutions due to high licensing costs. Provides an all-in-one solution for data collection, strategy development, historical backtesting and live execution across instruments and portfolios.
It’s a Swiss-based firm that offers both an open-source and a commercial license for their system along with a web-based front end. Supports forex, options, futures, stocks, ETF’s, Commodities, synthetic instruments and custom derivative spreads etc. allows full historical backtesting and complex event processing.
Be Aware of Bias
Great! You have taken care of everything and are on your way to successfully backtest your trading strategy. But wait, a good backtester should be aware of certain biases which might drastically change your backtesting results. Let’s understand a few of them now:
One of the most common types of bias in backtesting is when we work on the sample data for so long that we create a strategy which fits the data perfectly. In simple words, given the end of day data, we would try to frame an equation which closely matches the curve produced by the data. The downside of this bias is that it never performs on the same level when it comes to out of sample data.
While devising a strategy, you have access to the entire data and thus, there might be situations where you include future data that was not able in the time period being tested. A seemingly insignificant oversight such as assuming that the earning report being available one day prior can lead to skewed results during the backtesting.
There is a famous example which is used to illustrate the survivorship bias. If you were to use stocks of technology companies to formulate a strategy but took the data after the dot com bubble burst, then it would present a starkly different scenario than if you had included the data prior to the bubble burst. It's a simple fact, after 2000, the companies which survived did well because their fundamentals were strong and hence your strategy would not be including the whole universe and thus, your backtesting result might not be able to give us the full picture.
Backtesting proves to be one of the biggest advantages of Algorithmic Trading due to the fact that it allows us to test our strategies before actually implementing them in the live market. In this blog, we have covered the basic topics one needs to know before starting backtesting. You can learn to develop and implement more than 15 trading strategies in the course below.
Disclaimer: All data and information provided in this article are for informational purposes only. QuantInsti® makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information in this article and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.