Automated Trading Systems: Architecture, Protocols, Types of Latency

The rise of automated trading has significantly transformed the structure of trading systems over the past decade and continues to do so. For firms, especially those involved in high-frequency trading, staying competitive in algorithmic trading now requires constant technological innovation.

In this post, we will break down the architecture behind automated trading systems, compare the new system designs with traditional ones, and explore the key components that make these systems work.

• What is automated trading?
• Difference between automated trading and algorithmic trading
• Evolution of trading systems
• What is the need for an automated trading system?
• Automated trading architecture
• Automated trading system protocols
• Shift to the low latency architectures
• Levels of sophistication in automated trading
• How to build automated trading system?
• Advantages of using automated trading systems
• Disadvantages of using automated trading systems

What is automated trading?

An automated trading system (or fully automated trading) is a subset of algorithmic trading wherein computers are used to generate trading signals and manage the flow of orders in the markets without human intervention. ⁽¹⁾

Automated trading also includes quantitative modelling and risk monitoring.

Automated trading systems are used by an increasingly large percentage of market participants including trading firms, banks, hedge funds, asset managers, and pension funds. They may develop their own systems or use systems provided by third parties. The degree of automation varies from system to system and other factors such as regulatory environment, stock exchanges, and cultural differences.

Recommended read:

What is Automated Trading?

This video below introduces essential concepts of automated trading, including its significance and the process. It also demonstrates a rule-based strategy's automation, offering valuable insights for those seeking to enhance their trading skills.

Moving to learning more about automated trading, let us now discuss how it is different from algorithmic trading.

Difference between automated trading and algorithmic trading

Below you can see the clear distinction between automated and algorithmic trading with each aspect mentioned.

Recommended read:

Algorithmic Trading book

Next, we will check the evolution of trading systems over a period of time.

Evolution of trading systems

The traditional trading system was a mere interaction between the broker and the exchange for these three operations:

• Receiving market data
• Sending order requests
• Receiving replies from the exchange

In order to buy or sell the stocks and securities, the trader had to approach a broker. This broker could be an individual or a firm. This broker used to be hired to trade on behalf of the trader who would do the buying and selling of financial instruments manually.

This traditional form of trading would be time-consuming and would involve making trading decisions based on emotions such as fear, greed, etc. Moreover, traditional trading lacked analytical accuracy since it was done manually. Going by the famous saying “to err is human”, the traditional or manual form of trading needed to be evolved.

With the advent of automated trading, the trading scenario changed or rather evolved into a more accurate, time-saving, and faster approach. In its basic form, we can portray the transfer of data from the exchange and the automated trading system as follows:

The market data, that is received, typically informs the automated trading system of the latest order book. It may also contain some additional information like the volume traded so far, the last traded price, and the quantity of the trading order.

However, to make a decision based on the data, the automated system looks at the old values or derives certain patterns from the history. Based on this historical analysis of data and patterns, the system goes forward to create a trading strategy and execute the same.

Last but not least, a GUI interface is needed for the trader to view all this information on a screen.

Next to discuss is the need for an automated trading system.

What is the need for an automated trading system?

The traditional architecture lacked the facilities that the automated trading system with Direct Market Access (DMA) provided. The latency between the receiving of market data to the order generation went beyond the dimension of human ability and entered the realms of milliseconds and microseconds with the automated trading system.

With time, it was realised that order management also needed to be more robust and capable of handling many more orders per second.

Since the time frame of order execution with algorithmic trading is less compared to manual order execution, risk management measures were also needed. For such practices as fast order execution and simultaneous risk management, an automated system was needed.

For example, even if the reaction time for an order is 1 millisecond, the system is still capable of making 1000 trading decisions in a single second. Thus, each of these 1000 trading decisions needs to go through risk management measures within the same second to avoid losses. You could say that when it comes to automated trading systems, this is just a problem of complexity.

Next to discuss is the automated trading architecture in which we will find out all about the working or the flow of automated trading.

Automated trading architecture

Taking all the points above into consideration, the traditional architecture of the entire automated trading system is broken down as follows -

• The exchange(s) – the external world
• The server
• Receives market data
• Stores the market data
• Store orders generated by the user
• Application
• Takes inputs from the user initially for decisions such as stop loss, limits, the preferred financial instrument for trading etc.
• Acts as an interface for viewing the information including the data and orders
• Acts as an order manager sending orders to the exchange

Although the application layer is primarily a view, some of the risk checks can be offloaded to the application layer, especially those that have to do with the sanity of user inputs like fat finger errors.

The rest of the risk checks in automated trading systems are now performed by a separate Risk Management System (RMS) within the Order Manager (OM), just before releasing an order. The problem of scale also means that where earlier 100 different traders were managing their risk, there is now only one RMS system to manage risk across all logical units/strategies.

However, some risk checks may be particular to certain strategies, and some might need to be done across all strategies. Hence the RMS itself involves strategy-level RMS (SLRMS) and global RMS (GRMS). It might also involve a UI to view the SLRMS and GRMS.

Now let us understand the server components in more detail.

Market Adapter

Exchange or any market data vendor sends data in their own format. Your algorithmic trading system may or may not understand that language. Exchange provides you with an API or an Application Program Interface which allows you to program and create your own adapter which can convert the format of the data into a format that your system can understand. ⁽²⁾

Complex Event Processing Engine

This part is the brain of your strategy. Once you have the data, you would need to work with it as per your strategy, which involves doing various statistical calculations, comparisons with historical data and decision-making for order generation. The type of order and order quantity are prepared in this block.

What do you call a trading system which is actually a CEP System?

A complex event is a set of other events that together imply an occurrence of something of significance. These include stock trends, market movements, news etc. Complex event processing is performing computational operations on complex events in a short time.

CEP systems process events in real-time, thus the faster the processing of events, the better a CEP system is. For example, if an automated trading system is designed to detect a good opportunity for the returns for the next 1 second, but the time taken by the CEP system exceeds this threshold, then the trading system won’t be able to make any good returns.

The CEP system comprises of four parts:

• CEP engine
• CEP rules
• CEP WS
• CEP result interface

The two primary components of any CEP system are the CEP engine and the set of CEP rules. The CEP engine processes incoming events based on CEP rules. These rules and the events that go as input to the CEP engine are mentioned in the trading system (trading strategy).

You can see in the image below where the CEP engine's role fits in the process of trade order generation starting from the application and reaching the exchange.

For a quant, the majority of his or her work is concentrated in this CEP system block. A quant will spend most of the time formulating trading strategies and performing rigorous backtesting, optimisation, and position-sizing among other things.

This is done to ensure the viability of the trading strategy in real markets. No single strategy can guarantee everlasting good returns. Hence, quants are required to come up with new strategies on a regular basis to maintain an edge in the markets.

There are a number of popular automated trading systems that are widely used in current markets. These are classified as different strategies namely momentum strategies, statistical arbitrage, market making, etc.

Let us see some protocols for automated trading systems.

Automated trading system protocols

Since the new architecture is capable of scaling many strategies per server, the need to connect to multiple destinations from a single server has emerged. So the order manager hosts several adapters to send orders to multiple destinations and receive data from multiple exchanges.

Each adaptor acts as an interpreter between the protocol that is understood by the exchange and the protocol of communication within the system. Multiple exchanges, thus, require multiple adapters.

However, to add a new exchange to the automated trading system, a new adapter has to be designed and plugged into the architecture. The need for a new adapter arises because each exchange follows its protocol that is optimised for the features that the exchange provides.

To avoid this hassle of adapter addition, standard protocols have been designed. The most prominent among them is the FIX trading protocol. This not only makes it manageable to connect to different destinations but also drastically reduces the go-to-market time when it comes to connecting with a new destination.

The presence of standard protocols makes it easy for the automated trading system to integrate with third-party vendors for analytics or market data feeds. As a result, the market becomes very efficient as integrating with a new destination/vendor is no more a constraint.

In addition, simulation becomes very easy as receiving data from the real market and sending orders to a simulator is just a matter of using the FIX protocol to connect to a simulator.

The simulator itself can be built in-house or procured from a third-party vendor. Similarly, recorded data can be replayed with the adaptors being agnostic as to whether the data is being received from the live market or from a recorded data set.

Shift to the low latency architectures

With the building blocks of an automated trading system in place, the strategies now have the ability to process huge amounts of data in real-time and make quick trading decisions.

Today, with the advent of standard communication protocols like FIX, the technology entry barrier to setting up an algorithmic trading desk or an automated trading system, has become lower. ⁽³⁾

Consequently, the automated trading domain has become more competitive.

As servers got more memory and higher clock frequencies, the focus shifted towards reducing the latency for decision-making.

Over time, reducing latency has become a necessity for many reasons, such as:

• The strategy makes sense only in a low-latency environment.
• Survival of the fittest – competitors pick you off if you are not fast enough.

The problem, however, is that latency is an overarching term that encompasses several different delays. Although latency is very easily understood, it is quite difficult to quantify. It, therefore, becomes increasingly important as to how the problem of reducing latency is approached.

If we look at the basic life cycle in an automated trading system, the steps are as follows:

1. A market data packet is published by the exchange.
2. The packet travels over the wire.
3. The packet arrives at a router on the server side.
4. The router forwards the packet over the network on the server side.
5. The packet arrives on the Ethernet port of the server.
6. Depending on whether this is UDP/TCP, processing takes place, and the packet makes its way to the memory of the adapter.
7. The adaptor then parses the packet and converts it into a format internal to the algorithmic trading platform.
8. This packet now travels through the several modules of the system – CEP, tick store, etc.
9. The CEP analyses and sends an order request.
10. The order request again goes in reverse from here as the market data packet.

In an automated trading system design, high latency at any of these steps ensures a high latency for the entire cycle. Hence latency optimisation usually starts with the first step in this cycle that is in our control i.e., “the packet travels over the wire”. The easiest thing to do here would be to shorten the distance to the destination as much as possible. The best way to do this is via colocation.

Colocation is the facility provided by exchanges to host the trading server near the exchange.

Recommended course:

Millisecond trading

Next is the levels of sophistication in automated trading.

Levels of sophistication in automated trading

The world of high-frequency algorithmic trading has entered an era of intense competition. With each participant adopting new methods of ousting the competition, technology has progressed by leaps and bounds.

Modern-day algorithmic trading architectures are quite complex compared to their early-stage counterparts. Accordingly, advanced automated trading systems are more expensive to build both in terms of time and money.

Below is the table covering the different network cards that are used for automated trading and the various aspects of the same.

Building an entire automated trading system can be beyond the scope of an individual retail trader. Traders who want to explore the algorithmic way of trading can opt for automated trading systems that are available in the markets on a subscription basis.

A trader can subscribe to these automated systems and use the algorithmic trading strategies that are made available to the users of these systems. Traders, who know programming can formulate and backtest their strategies in programming platforms like Python and C.

Let us see the steps to build an automated trading system now.

How to build automated trading system?

Beginner traders can learn to build their own automated trading system with the algorithms to trade in the markets. For building your own automated trading system, you will need to code the strategy in a programming language, backtest the strategy on historical data to find out its performance, paper trade and then live trade.

Let us see these steps in detail below that can serve as a rough guideline for building an algorithmic trading strategy:

Step 1: Ideation or trading plan

Come up with a trading idea or a strategy for trading in live markets. This ideated strategy can be based on your market observations or can be borrowed from trading books, research papers, trading blogs, trading forums or any other source.

Step 2: Creating the system

This step requires you to create an automated system to identify the trading opportunities in the market in accordance with your preferred financial instruments. Also, you will need to feed the automated trading system with the information regarding how to perform once it finds the opportunities.

In addition, you will also need to give the inputs regarding risk management by entering the parameters such as stop loss, limit order etc.

This entire process of creating the automated system will require you to have the knowledge of a programming language such as Python, C, etc. With this programming language, you will be able to code the system with all the necessary abovementioned preferences.

Step 3: Test and refine your system

The next step is to test and refine the automated trading system so as to improvise the system in case of any malfunctions. After making any necessary changes or improvements, you can be sure that the system is set for taking your trades to the live market.

Step 4: Take your automated system live!

Once you’re sure of your automated system, you can take your trade live with your trading idea or strategy.

Now that you have successfully built an automated trading system, here is a tutorial to help you through the process of downloading historical stock data using Python. If you are interested in automated trading system and want to analyse historical stock data for developing your trading strategies, this video can serve the purpose.

Moving forward, we will discuss the advantages of automated trading systems.

Advantages of using automated trading systems

The advantages of using automated trading systems are as follows:

• Ease of use: Orders can be placed quickly with the help of the software navigating you through the different parameters. Moreover, your own automated system will make it even easier for you to place orders since it is always at your disposal.
• Live portfolio review and market updates: With the automated trading system, you can view market data of financial instruments such as shares, commodities, stock indices, etc anytime. You can also keep track of your portfolio, its underlying assets and performance with the software.
• Notification feature: One of the most advantageous features of the automated trading system is the notification feature. The automated system will send you alerts to notify you about the latest events in your portfolio as well as broker recommendations.
• Related news updates:  Users can also track live updates on developments related to a particular stock or segment with the help of automated systems. The system helps you to be updated so as to modify your trade-related inputs such as the selection of financial instruments, updating of the stop loss value, limit value etc.
• Analysis and charts: Another feature is the provision of historical charts and analysis that allows you to look at previous indices data, and stock prices and provides analytical tools for the same.

Ahead, we will discuss some disadvantages of using automated trading systems.

Disadvantages of using automated trading systems

The following are the disadvantages.

• Expenses to use the automated system: The automated trading system comes with an expense and that may be a disadvantage for some who do not wish to invest in the same.
• Connectivity: A generic disadvantage of using the automated trading system is connectivity. In some remote regions, connectivity disturbances are common, and this may lead to losses during order placements.

Conclusion

This was a detailed post on automated trading system architecture to give you an in-depth and insightful knowledge of the components involved. Also, there are various challenges that the architecture developers need to handle or overcome in order to build a robust automated trading system. So what are you waiting for? Go Algo!!

If you want to learn various aspects of algorithmic trading and automated trading systems, then check out our course on “Algorithmic Trading for Beginners!”. This course builds a foundation in algorithmic trading and is perfect for those who want to get a complete picture of the domain. It includes the what, how and why of algorithmic trading. Different algorithmic trading strategies and regulations for setting up an algorithmic trading business are also included.

Author: Chainika Thakar

Note: The original post has been revamped on 11^th September 2024 for recentness, and accuracy.

Disclaimer: All data and information provided in this article are for informational purposes only. QuantInsti® makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information in this article and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis..