By Igor Tulchinsky and Jeffrey Satinover

The future of trading is quite clear: markets will see a sharp differentiation with a few sophisticated and technologically well-equipped players able to demonstrate an ever-adaptive, unbounded capacity to discover new alphas plus a small number of investors who still rely on traditional research to drive their investment decisions.

Diving deep into the financial histories and management practices of a relatively small number of companies is good as long as such research is able to derive unique insight/an advantage. Some best practitioners of this rigorous and time-intensive discipline will continue to earn above-market returns.

But by some estimates, seventy percent of total trade volume is now driven and executed by computer code. Algorithmic activity encompasses all high-frequency trading worldwide. Analysis and detection of patterns in trillions of bits of data in milliseconds is key. The more data an algorithm can make use of, the greater its effectiveness.

There are three observations to consider.

First, data has indeed become “big data”—in finance it is naturally very big. It requires retrieval and storage capacities if it is to be used. Estimates of the size and growth rate of the digital data universe vary widely. One respected source* projects that from 2005 to 2020, data will have grown by a factor of 300, from 130 exabytes to 40,000 exabytes, or 40 trillion gigabytes. From 2013 until 2020, the amount of data will roughly double every two years. Putting this last fact in mathematical terms, the growth of Big Data is exponential – (Ö2) (time-in-years).

Second, big data requires processing capacity. Moore’s “Law” that processing capacity doubles roughly every one and a half to three years is slowing of late. Unpredictable changes are looming as components grow tinier and quantum effects begin to interfere. Thus the growth of processing capacity is also exponential, and at about the same rate as data growth:  (Ö1.5 to Ö3.0) (time-in-years).

Third, in finance big and growing data is recursive. In other domains, like in medical research, new hypotheses about both illness and treatment are found in previously invisible patterns among widely disparate biological variables. But however large medically pertinent data grows, the underlying universe of all such possible facts doesn’t change by being used. In finance this is not so. The simplest example of this is large-trade impact on the bid-ask spread. But a much larger effect is due to the impact of all trades.

It follows that alphas —, trading rules that yield superior performance—grow without out bound: Bigger data and computational capacity enlarges the ocean of alphas, it does not exhaust it.

Crucially, this ocean is enlarging at a much faster rate than either data or processing capacity. Candidate alphas represent operations on permutations and combinations of alphas. So alphas to hunt and discover are for practical purposes infinite.

This is not a mere guess: Formal studies of agent-based market models show this explicitly**. In even the very simplest binary prediction models of just a single time series, “strategy space” (the ocean of all possible alphas) grows with m, the number of prior data points used, as 22^m, i.e., superexponentially.

This is not all. The SEC recently decided to allow companies to release material information via Twitter. Social chatter of any kind can influence the market in milliseconds. The amount of information that moves prices is virtually endless, limited only by the amount of total activity in the economy, which itself is an increasing number. The result is a premium to those who digest the data faster, but it is not possible to catch up—to exhaust the search space. There will always be room for more alphas.

Extracting signals from an ever-expanding ocean of noise is a growing challenge. The solution space is non-convex, discontinuous, and dynamic: good signals often arise where least expected.  How does one extract such signals? By limiting the search space, using methods previously used by treasure hunters: Search in the vicinity of previous discoveries; Conserve resources to avoid digging too deep; Use validated cues to improve the probability of a find. Yet always allocate some processing power to test wild ideas.

Will exponentially growing data lead to ever diminishing returns? For each individual alpha, the answer is, “yes.”  For alphas in the aggregate, the answer is “no”.  

The successful treasure hunter is the one with the most alphas, who harnesses the latest machines. The complexity and dimensionality of the alpha search game will keep increasing. The proportion of easy-to-find alphas will keep decreasing. The advantage / upper hand /benefit / gain will increasingly be had by the trader who sees the whole picture, who can combine millions and billions of ever-fainter and subtler signals.

• http://www.emc.com/leadership/digital-universe/iview/executive-summary-a-universe-of.htm 

** See the “Minority Game” developed at the Santa Fe Institute and a central object of study in the burgeoning domain of Econophysics–http://www3.unifr.ch/econophysics/.

* Igor Tulchinsky, Financial Entrepreneur, founder and CEO of WorldQuant (founded in 2007). He was  previously Managing Director at Millennium Partners, and has been a leading figure in quantitative trading for 18 years. Mr. Tulchinsky began his career as a computer game programmer.

* Jeffrey Satinover, MD, PhD, is a senior quantitative researcher at WorldQuant and external scientific collaborator at the Swiss Federal Institute of Technology. A physicist and psychiatrist, he is the author of numerous research publications in complex systems theory applied to finance, and of the book, The Quantum Brain

The opinions and writing contained in this article are of the authors alone and do not necessarily represent those of HFTReview.com.