NonLinearity, Artificial Intelligence & Genetic Algorithms
Mon, 04 Oct 2010 16:57:00 GMT
Interview with Adam Afshar, Hyde Park Global
In the latest in our series of interviews with High Frequency Trading Leaders, Mike O’Hara speaks withAdam Afshar, President and CEO of Hyde Park Global Investments.
Adam has over 2 decades of financial industry experience including 12 years at Bear Stearns where he was a Managing Director, overseeing long/short multi asset portfolios for both onshore and offshore clients. Hyde Park Global Investments is a 100% Robotic investment and trading firm based on Artificial Intelligence (AI). The system is built primarily on Genetic Algorithms (GA) and other Evolutionary models to identify mispricings, arbitrage and patterns in electronic financial markets. Additionally, Hyde Park Global Investments has developed programs applying natural language processing and sentiment analytics to trade equities based on machine readable news. Hyde Park Global employs no analysts, portfolio managers or traders, ONLY scientists and engineers.
Adam Afshar has served on the finance committee of the board of trusties of Wofford College and is currently member of the Board of Directors of Ariana Pharma in Paris, France. Ariana is a leading Artificial Intelligence company, focused on transforming data into information to accelerate the discovery, development process and safety of pharmaceutical drugs. Ariana is located in central Paris, and is a spinoff of the Institut Pasteur. Adam Afshar has a BA in Economics from Wofford College and received his MBA from the University of Chicago, Booth School of Business.
High Frequency Trading Review: Adam, what’s your definition of high-frequency trading?
Adam Afshar: It varies. It could mean anything from a sub-second holding period to one day. Internally at Hyde Park Global, we prefer to speak more in terms of holding period and its variance.
We find the term “high frequency” is only slightly more descriptive than terms such as “short-term investment”. It’s important to note here that whatever the definition, high-frequency trading is a means to implement the strategy and never the strategy itself.
HFTR: That’s interesting. That’s the first time somebody has said that.
Adam: Let’s look at who uses high-frequency trading: the first three groups that come to mind are statistical arbitragers, algorithmic execution and electronic market makers. Even at a superficial level it becomes evident that what these three groups have in common is a high level of engineering in their approach.
Engineers have a very difficult time with nonlinearity; engineering students learn very early on that they should try to work the non linearity out of their problem. A linear example is the volume knob on your radio. As you turn the volume knob, the radio becomes louder; non linearity is where you turn the knob a little and the volume goes up a 100 fold; you turn the knob even more and the volume goes down. So, basically high-frequency and low latency trading allows us to implement strategies which have significant amounts of engineering to capture short term and small inefficiencies in the market. In other word, high-frequency and low latency trading attempts to cut timeslots into small enough chunks so that in a particular time period, the data we are dealing with becomes linear enough for our model.
Professor Luboš Pástor of University of Chicago through Bayesian analysis has shown that uncertainty compounds as the time horizon grows longer. Perhaps a simple example would be the degradation of the quality of information such as news. If you have news that is driving your decision and this news is a week old, then it is very possible that it will not be very useful in your trading.
This same time degradation applies to the latency of price and volume quotes. It is easy to imagine that if there is significant latency in your quote delivery system, then you will have an ineffective market making operation.
HFTR: So where does artificial intelligence fit into the high-frequency trading space?
Adam: A prefatory note to disambiguate important concepts; in terms of what we at Hyde Park Global mean by artificial intelligence (AI). What we do not mean is a system that is trying to simulate human behaviour or human reasoning. I have seen several articles written in the financial press about a number of very large banks using AI in an attempt to replicate the behaviour of a human trader. At Hyde Park Global we find no reason to simulate human behaviour or reasoning, when we don’t even argue that the logic behind that reasoning is rational.
Now let us explain what are we do mean by AI at Hyde Park Global. To make better sense of the answer, let us first look at the issues that we face which compel us to use the combination of AI and a robotic platform in the first place. How do you make millions of calculations every second to monitor prices, volume and news on 3,000+ stocks every second with multiple and varying objective functions? Most people can appreciate that only a machine can look at and assess 3,000+ stocks every second and it is obvious to most people that no human trader, analyst or portfolio manager or even legions of traders can do that. If you have 1000 traders instead of one trader, you can do 1000x more calculations but you cannot do your calculations 1000x faster! You can read 1000x more news but you cannot read the news 1000x faster. So the next question is: how does the machine know what to do? The machine is assessing and analysing the news, price, and volume, but to what end? Here, scientists have to program the machine with an objective function or fitness function.
For example, we can ask the machine to give us the highest return (which, in our experience at Hyde Park Global is rarely an optimal objective) or, preferably, we can ask for the highest risk-adjusted return and define risk as volatility, Sharpe Ratio or Ulcer ratio or whatever you may find as the most appropriate fitness function.
To achieve this goal, our AI platform is given a set (or library) of analytic variables, from which it identifies relevancy, and a means of constructing automated trade decision logic from these relevant variables. The platform then attempts to maximize the objective function by tuning the logic and any unknown parameters in the model.
We who have been in finance for a reasonably long time know too well that the same formula does not work all the time. So let’s discuss how the computer learns and how we can verify that the learned models can be applied to as yet unseen (future) data. The computer learns from historical data and using sophisticated genetic algorithms (and I’ll explain what they are in a moment), it finds potentially the best trading method. In other words, the computer finds an optimized solution by learning from a set of data. Then the computer tests its ability to learn on a different set of data that it has not seen its learning set.
To clarify, if we determine that what the machine has learned in period 1 is also applicable to period 2, we can say that the trading model is robust. Then we have the machine analyse and learn this time from the combined data from both period 1 and period 2. And we test the model again this time on new unseen data which will be period 3.The system continues to learn and adapt in this way.
HFTR: Right, so it’s basically an iterative learning process?
Adam: Yes, an iterative learning process in that it learns, it tests its learning, it applies that learning, it recalibrates, and then it goes on to the next set of time series.
At Hyde Park Global, we extensively use Evolutionary Computing algorithms such as genetic algorithms (and related genetic programming) in our machine learning platform. The evidence suggests that for many strategies the shorter the learning period and the higher the frequency the more robust the adaptive system. I should mention here that a huge amount of data is required for learning and testing, and that the sub second analysis of this data itself is a herculean task and a fantastic challenge. Expertise in high speed data storage and retrieval is in our opinion the linchpin of successful robotic adaptive HFT.
HFTR: Can you tell us a little more about the genetic algorithm?
Adam: Sure. It is an evolutionary optimization model, which has four elements within its core: selection, mutation, crossover and fitness scaling.
We use genetic algorithms because we think that our trading models may have many local (sub-optimal) solutions, but we are interested in finding the global optimum.
A simple analogy is the following. Imagine that you are trying to go to the top of the mountain and it’s dark. You can’t see in front of you. You take a step. If you go up, that was a good step. If you go down, that was a bad step.
If you’re taking the right steps, you may be moving up and you will eventually reach a point where every step that you take would be a step back, because every step that you take, will take you a step down.
The key question is whether or not you reached the mountain top? And, the answer is maybe you have, but maybe not. In other words, you have certainly reached a local optimum, but not necessarily the global optimum. So what the genetic algorithm allows you to do is to have a better chance to find the top of the mountain by going and testing a completely different area, or multiple different areas, and learning from the results.
This is a very simple analogy but the point is that the genetic algorithm allows you to have a better chance to find the global optimum, which in our case is the trading model that maximizes our defined objective function.
HFTR: Excellent, you’ve described it in a way that even I can understand! What about news flow algorithms? I understand you use those to statistically analyze news that’s coming into the market. Where do they fit in?
Adam: News flow algorithms parse machine-readable news items and assign numeric values to individual words, phrases and their placement in a story. There are over 100,000 news items in the US per day. Machine-readable news gives us a better chance of predicting whether a story is positive, negative, or neutral, and its potential effect on the price of a security. The machine knows if the story is positive or not and if there is a trend in the story. For example the machine can identify repeated positive or negative stories and the dynamic price elasticity vis-à-vis positive or negative news in respect to a particular security, sector or market.
Given the extremely high volume of news items and the speed at which news can impact the market, this, in our opinion, is a fantastic area for machine trading because it’s obviously very difficult for any human being to look at a hundred thousand news items every day, read them, understand them and act on them with the same accuracy and speed as machines.
HFTR: So if you have a system that is analyzing news flow and converting news items into numerical data, then can those news flow algorithms help your robotic trading systems take into account - and deal with – completely unexpected and statistically unpredictable events?
Adam: Let me re-emphasise that, at Hyde Park Global, when we are talking about artificial intelligence and robotic trading, we mean a trading platform that does not allow any human intervention. It’s very important to stress this point because if the system allows human discretion at any level (idea generation, portfolio management, or trading), and your machine does not have the human discretionary elements correctly modelled in its learning algorithm (which we claim is not possible at this time), what you are left with is simply a quantitative trader that uses certain calculations to assist their trading. It becomes difficult or even impossible to assess if the success or failure was due to the calculations, formula or algorithms.
Though we can argue on the pros and cons of humans as traders, we have to agree that this method is not and cannot be scientific. It is not scientific because it is not possible to back test a model that allows any discretionary human intervention. For example, if you have computers that are generating trades but the execution is done by humans then we would argue that you cannot determine whether the success or failure of the system was due to its robust artificial intelligence or to a very good trader, and there is no way of testing and duplicating the results. Therefore we would argue that any back testing becomes essentially void.
At Hyde Park Global, we do not allow any discretionary human intervention for market events. If we lose electricity in our office, then we will get involved because this is not a market event, it’s an event that is idiosyncratic to us and has nothing to do with the markets. But if the market has a flash crash or there is a problem with a certain exchange or ECN we do not allow any human involvement because, again, it’s a market event.
HFTR: So in that case, how can robotic trading systems, even adaptive ones, take into account such unexpected events?
Adam: First, let me answer by clearly stating that claiming or assuming that all unexpected and statistically unpredictable events can be predicted by artificial intelligence is fatuous. We are just proposing, and we think that we have considerable evidence to back it up, that our artificial intelligence system is significantly more robust and handles these rare events measurably better than any human traders that we have known.
The notion that statistics is the key that will open all doors is wrong, in fact, ridiculous. Similarly, the idea that we are living in a Gaussian world where distributions are always normal is also spurious. In finance, particularly in financial economics, we see considerable evidence of non-Gaussian distributions. So, the question is centred on how you deal with non normal distributions and tail events. What steps and what techniques can you use to better handle these rare but important events? The statistics, though not an answer to everything, is an answer to many things. But you need to supplement your statistical models with rule based models. There are companies, for example in the drug discovery and drug development fields, that have integrated statistical models and rule based models to help them manage events that are very rare and unexpected.
HFTR: So are there any specific developments in these other industries, pharmaceutical or biological for example, that can be adapted and applied to high-frequency trading?
Adam: Yes indeed. Ariana Pharma, a company in Paris France, which is pioneering the development of very sophisticated artificial intelligence technology, in this case for detecting rare anomalies. Its software is licensed I believe by every major pharmaceutical company, and by the US Food and Drug Administration (FDA). In the interest of full disclosure, I am on the board of Ariana.
HFTR: And can some of that research be translated into the financial industry?
Adam: At Hyde Park Global we are convinced that the answer is yes. We use artificial intelligence systems similar to those developed by Ariana Pharma to detect rare events and react appropriately. There are some major differences in the sort of data set that a hedge fund like us is mining versus a pharmaceutical lab. For example the size of the data set in pharmaceutical research is often very small and incomplete. When they test a drug, they don’t test it on 50 million people; they test it on a few hundred or a few thousand people, so they are typically dealing with a sample size that is statistically weak. This is very different than from the size of the data set we use to develop trading programs at Hyde Park Global, where we have access to over a billion data points for pattern recognition and over 40 million individual machine readable news items to learn from.
That notwithstanding, the other area that we see parallels with the investment field is biological research. Biology today, is a convergence of three fields. To do significant research in biology, you have to know biology, but one must also know mathematics and computer science (programming). In my discussions with biologists, I understand that biology has evolved away from just focusing on the mechanical relationships of the different part of the cell. Instead, biologists are looking at the cell itself as a network of relationships. In other words, different parts of a cell affecting other parts of a cell at different times in different ways. And this is continuously evolving.
I see a similar parallel in financial economics. To do significant research in financial economics today, you need to know mathematics, computer science and programming as well as finance. The decision making process has become noticeably more sophisticated in our industry in the last 10 years. More and more of our industry colleagues are making their decisions based on evidence, mathematical and other, rather than what I would call a good story.
HFTR: In conclusion, how does high-frequency and low latency trading affect the market and what do you see as its significance?
Adam: In respect to the equity markets, the effect of low latency data feeds and high-frequency trading is a much researched question and the weight of the evidence suggests that the markets are becoming more efficient due to technology. At Hyde Park Global we are convinced that technology has reduced the large hedge funds and big Wall Street banks’ first mover advantages or what some would call their leapfrogging advantages.
For example, the ability to get low latency (sub-millisecond) data is almost a pure technology issue with numerous vendors offering a plethora of services. Having an algorithmic execution system that allows you to trade or send 3500 messages to the exchanges and ECNs per second is also a pure technology and very much a non intellectual issue. What will separate the winners from the also-rans will, in our opinion, be almost purely the intellectual and knowledge aspects of how you monetize your access to this powerful and more readily available technology.
Today, we can see that small investment firms with intellectual, technological and computational savoir-faire can get the same data just as fast - in fact sometimes faster - than the biggest Wall Street banks and large hedge funds. This development in the markets is levelling the playing field and creates more efficiency. It is not a stretch to see how it also helps our capitalistic system which observably has had a profound impact in the increase of the standard of living of people for the last 100 years. Efficient and strong financial markets are not just good for the Hedge Fund managers, but our entire society including the scientists who are working on a cure for cancer that have never read the Wall Street Journal.
HFTR: A good point to end on. Thank you Adam

