J Doyne Farmer

Early life

Though born in Houston, Texas, Farmer grew up in Silver City, New Mexico. He was strongly influenced by Tom Ingerson, a young physicist and Boy Scout leader who inspired his interest in science and adventure. Scout activities included searching for an abandoned Spanish goldmine to fund a mission to Mars, a road trip to the Northwest Territories and backcountry camping in the Barranca del Cobre. Farmer graduated from Stanford University in 1973 with a BS in Physics and went to graduate school at the University of California, Santa Cruz, where he studied physical cosmology under George Blumenthal.

Beating roulette

While still in graduate school, Farmer and his childhood friend Norman Packard formed a group called Eudaemonic Enterprises. Their goal was to beat the game of roulette and use the proceeds to form a science commune. The word eudaemonia comes from Aristotle and refers to a state of enlightenment derived from a life lived in accordance with reason.

They bought a roulette wheel and did an extensive experimental and theoretical study of its physics. To execute their system, they built the first wearable digital computer, at roughly the same time as the first Apple desktop computer. Farmer hand-coded the three-kilobyte program for the computer in machine language. The program included a floating-point package, a sequencer to perform the calculation, and an operating system that functioned with toe inputs and vibrating outputs. The earliest version of the computer was hidden under the armpits, but a later version was concealed in a shoe.

Their scheme took advantage of the fact that typically more than ten seconds elapse from the time the croupier releases the ball until bets are closed. During this time one person measured the position and velocity of the ball and rotor using his big toe to click a switch in his shoe. The computer used this information to predict the likely landing position of the ball. A signal was relayed to a second person, who quickly placed the bets. They made over eleven trips to Las Vegas, Reno and Tahoe, and achieved a 20% advantage over the house, but suffered persistent hardware problems. This combined with their fear of violence at the hands of the casinos, so that they never played for high stakes and failed to realize the large sums they originally dreamed of.

Chaos and the Dynamical Systems Collective

After the roulette project Farmer switched his dissertation topic to chaotic dynamics and joined together with James P. Crutchfield, Norman Packard, and Robert Shaw to found the Dynamical Systems Collective (subsequently known by others as the Chaos Cabal). Although they had the blessing of faculty members William L. Burke and Ralph Abraham, they essentially co-advised each other's PhD theses. Their most important contribution was a method for state space reconstruction, that made it possible to visualize and study chaotic attractors based only on a single time series. This has now been used to identify chaotic attractors and study their properties in a wide variety of physical systems. In his PhD thesis in 1981 Farmer showed how varying a parameter of an infinite dimensional system could give rise to a sequence of successively more complicated chaotic attractors, resembling the transition to turbulence. He later developed a method for nonlinear time series forecasting that has been used for exploiting low dimensional chaos to make better short term forecasts. Other work included an improved method for state space reconstruction, and a derivation of the fundamental limits in which this becomes impossible, so that the dynamics become inherently random. He and colleagues also developed a method for determining when chaos can be distinguished from the null hypothesis of a correlated linear random process.

The Los Alamos Complex Systems Group

After finishing his doctorate in 1981, Farmer took a post-doctoral appointment at the Center for Nonlinear Studies at Los Alamos National Laboratory and received an Oppenheimer Fellowship in 1983. He developed an interest in what is now called complex systems and co-organized several seminal conferences in this area. In 1988 he founded the Complex Systems Group in the Theoretical Division and recruited a group of postdoctoral fellows who subsequently became leaders in the field, including Kunihiko Kaneko, Chris Langton, Walter Fontana, Steen Rasmussen, David Wolpert, James Theiler and Seth Lloyd.

Farmer and Norman Packard developed the concept of metadynamics, i.e. co-evolving networks and dynamical systems. For example, the nodes of the network might represent chemical species and the edges their possible reactions, whose kinetics give rise to a system of differential equations. As new species are produced the set of reactions changes and the kinetics are in turn altered. This concept was used to model the immune system and the origin of life. Joint work with Richard Bagley produced a simulation of an autocatalytic set of polymers in which a few species are maintained at high concentration, with many of the properties of a metabolism; the autocatalytic set evolved through time in a manner resembling the evolution of living systems, but without a genetic code.

James Keeler and Farmer demonstrated that a system of coupled logistic maps could produce fluctuations with a 1/f power spectrum. They showed that this occurred because the system continually tunes itself to stay near a critical point, a property that was later dubbed self-organized criticality by Per Bak.

Prediction Company

In 1991 Farmer gave up his position at Los Alamos, reunited with Norman Packard and graduate school classmate James McGill, and co-founded the Prediction Company. The prevailing view at the time was that markets were perfectly efficient, so that it was not possible to make consistent profits without inside information. Farmer and Packard were motivated by their desire to prove this wrong. The trading strategy that was developed was an early version of statistical arbitrage, and made use of a variety of different signals derived from processing essentially all quantitative inputs related to the US stock market. It also included a high frequency forecasting model as an overlay to reduce transaction costs. From 1996 onward trading was completely automated. Farmer was one of the chief architects of the trading system as it existed in 1999. Prediction Company was sold to UBS in 2006 and in 2013 was re-sold to Millenium Management, where it is their second largest fund.

Market ecology

Farmer left Prediction Company in 1999 for the Santa Fe Institute, where he did interdisciplinary research at the interface of economics and complex systems, developed a theory of market ecology and was one of the founders of econophysics.

Market ecology is based on the observation that financial firms engage in specialized strategies and can be sorted into groups, analogous to species in biology. Market impact limits the size of any particular strategy. Farmer showed how to a construct a market food web, that describes the way in which trading strategies influence each other's profits and size. The market food web is supported by fundamental economic activities, such as demand for liquidity, lending to the real economy and risk diversification. These create patterns in prices that are exploited by trading firms, who are analogous to predators in biology. Some strategies are stabilizing while others are destabilizing, and shifts in the market ecology can give rise to financial instabilities, e.g. booms and busts. These ideas had an important influence on the adaptive markets hypothesis.

Econophysics and market microstructure

Farmer is considered one of the founders of the field of "econophysics". This is distinguished from economics by a more data-driven approach to building fundamental models, breaking away from the standard theoretical template used in economics of utility maximization and equilibrium. Together with Michael Dempster of Cambridge, Farmer started a new journal called Quantitative Finance and served as the co-editor-in-chief for several years.

His contributions to market microstructure include the identification of several striking empirical regularities in financial markets, such as the extraordinary persistence of order flow. Fabrizio Lillo and Farmer observed that there are long periods where the orders flowing into the market are much more likely to be to buy than to sell, and vice versa, with correlations decaying very slowly as a power law. He and his collaborators developed a zero intelligence model for the continuous double auction that was shown to predict the spread between bid and ask prices. A variety of different empirical studies documented the law of market impact, which states that the average change in price due to an order entering the market is proportional to the square root of the order size. This law is remarkable as it is universal, in the sense that the functional form of market impact remains the same as long as markets are operating under "normal" conditions. The work of him and his colleagues set the foundation that was eventually developed by the group of Jean-Philippe Bouchaud.

Leverage cycles and financial stability

The crisis of 2008 is widely believed to have been an example of a leverage cycle, in which lending first becomes too loose and then becomes too tight. An agent-based model for leveraged value investors shows how the use of leverage can explain the fat tails and clustered volatility observed in financial markets. Similarly, the use of Value at Risk, as embodied in Basel II, can lead to a cycle in which leverage and prices slowly rise while volatility falls, followed by a crash in which prices and leverage plummet while volatility spikes upward, resembling the Great Moderation and subsequent crisis.

Predicting technological progress

Although innovation might seem by its very nature to be unpredictable, in fact there are several empirical regularities that suggest the opposite. Together with several colleagues Farmer developed a theory for explaining Wright's law, which states that costs drop as a power law function of cumulative production. By gathering data on many different technologies, this can be shown to be closely related to Moore's law, which can be used to make reliable forecasts for technological progress under business as usual scenarios. He is currently try to develop an ecology framework for growth economics.

Other interests

Farmer has written about science and adventure and is an avid sailor and backpacker. Doyne is listed as member of BMLL Technologies Ltd, a Cambridge University spin-off working in the field of limit order book data and analytics. He continues his entrepreneurial activities as Executive Chairman of Scientific Investments.

In popular culture

Farmer and Packard's work on roulette, along with their adventures in the casinos of Nevada, has been featured in the 2004 Breaking Vegas documentary series, "Beat the Wheel".