Interview by Joe Kolman

In the early 1980s a group of five graduate students from the University of California at Santa Cruz forced the scientific world to write another chapter in the physics textbooks. Their efforts to discover the structure underlying random events became known as chaos theory. Subsequent experiments proved that chaos exists in a wide variety of natural systems.

Their efforts were chronicled in Chaos: Making A New Science, a bestseller by James Gleick, that introduced their discoveries to a wide audience. In 1991 two of the scientists, Norman Packard and his colleague Doyne Farmer, left conventional physics jobs at the University of Illinois and Los Alamos National Laboratory to start Prediction Company, an enterprise dedicated to financial market trading based on a variety of high-tech stratagies derived from their research. Their efforts are being supported exclusively by Swiss Bank Corp., which provides financing and operational trading expertise. Research director Norman Packard talked with editor Joe Kolman at the company's offices in Santa Fe, New Mexico.

Derivatives Strategy: How did you and your colleagues move from the world of physics into the world of derivatives?

Norman Packard: Well, I first got into the business of data analysis with my partner at Prediction Company, Doyne Farmer, through analyzing physical data, which is to say the data from physics experiments. This was back in the early 1980s. The goal of our analysis was to try to detect a certain kind of structure in noisy-looking data.

DS: What's noisy-looking data? That's an unfamiliar term to most financial people.

NP: Noisy-looking data can first appear random and unstructured but sometimes reveals evidence of what we call chaotic dynamical systems-systems that are predictable for a short period of time but unpredictable in the long run. These techniques were first used on data from chemical reactions. Gradually, the arena widened to encompass nonphysical data, like biological data, data from ecological experiments and data from epidemiological experiments. And ultimately it led to data from economic systems and markets.

DS: And the financial market seemed to you to be a particularly noisy system or set of data.

NP: Well, they certainly are noisy. They were attractive in part because there was a certain body of theory that planned to explain what structure or lack of structure there was in this data. And so there was the possibility of finding structure and being able to make some strong statements about what theories might be correct or possibly which theories should be rejected.

DS: Did you find the structure in the financial markets you were looking for?

NP: When we first started looking at this data, we did not see the strong evidence of chaos that we saw in certain kinds of physical or chemical data.

DS: What would the strong evidence of chaos be?

NP: Finding a strong, significant indication of chaos would mean that the data could look very noisy, but in fact, it would have a certain kind of structure that allows you to predict for a limited time. And then beyond that time, you couldn't predict at all.

DS: But you were never able to find that in the data you looked at. Were you disappointed?

NP: Well, a little bit. But we had actually already been disappointed many times in the past looking at different kinds of data. (Laughter.) So it wasn't particularly surprising. Even though our aims are often associated with the existence of chaos and techniques for finding chaos, we don't really think that the structure that we're finding in financial markets is a result of chaos. Other people may begin with that premise and try to build models using that premise. And it's possible that they could find something. But so far, that premise hasn't been very powerful for us.

DS: But you did discover some kind of predictive structure in the financial markets ...

NP: We thought at the time that we saw structure that might indicate we could make money. At the time, we didn't even know really what the transaction costs were, or how strong the structure needed to be for it to be tradable. But we saw some structure and thought it was worth pursuing. This was basically around 1989 when we first started looking at the data seriously. And then in the fall of 1991, we ended up starting Prediction Company.

DS: What happened when you pitched your ideas to the various investment banks? What was their reaction?

NP: We had kind of a variety of reactions. One of the parts of the vision that we were pushing was that these techniques aren't necessarily for just one kind of market, but could potentially be applied to a wide spectrum of markets. And in fact, they need to be applied to a wide spectrum of markets in order to get the benefits of diversity. But the organization of investment banks typically tended to be compartmentalized. So we might have some interest on the part of the equities desk, but the fixed income and the forex desk maybe weren't interested.

Then we had some institutions that had people who were interested, but the institution as a whole was having some kind of problems. Salomon Brothers was a good example of that. They just had their bond scandal. And so fall of 1991 was not a time to start up new projects there.

DS: It sounds like you have a pretty good fit at Swiss Bank.

NP: Yeah, but the original fit was not really with Swiss Bank. It was with O'Connor and Associates.

DS: Was O'Connor doing anything similar that fit in with what you were doing?

NP: No. They thought that there might be structure in the markets to take directional bets. But all of their technology involved taking non-directional bets using a kind of a statistical arbitrage used to price options and other derivatives.

DS: That's been traditionally the way people have made money trading derivatives-based on analysis of option models. But that's not your direction at all.

NP: Exactly. I distinguish between option pricing approaches to making money, which are based on non-directional bets on volatility, and directional bets, the kind of bets that we make.

In 1991 we did see evidence of some kind of structure that might be tradable, or so it appeared to us at the time. Right now Prediction Company is based on the premise that there is actually a predictive structure in the markets, but we are finessing the question of where that structure comes from.

DS: What constitutes a tradable structure?

NP: Anybody who's worked with financial data knows that you have to distinguish between the existence of structure and the existence of tradable structure. It might be that you see some structure, but that if you try to trade on that structure, the transaction costs don't allow you to make money.

DS: Can you give me an example?

NP: The people who first ran into it probably were "pairs" traders. You try to pick two stocks that generally move together, like GM and Ford. When they go apart, you basically make a bet that they're going to go together again. And so you short one and go long the other.

DS: That's a fairly standard technical trade.

NP: That's right. But you can see the problem of transaction costs. If the stocks go apart a little ways and you bet that they're going to go back together, it may be that to make the trades, that it costs you more than you're going to make when they go back together. (Laughter). If you're looking for tradable structure, the structure has to be large enough and strong enough that when you make the trades, you can still make money on them after the cost of the trade.

DS: Let's distinguish the kind of trading you're doing from what technical traders are doing. Your approach is quite a bit different.

NP: Right. The traditional realm in which directional betting has taken place is the realm of technical traders. But actually, there's another realm where directional trading takes place. And that is the realm of fundamental traders on equities. Typically, fundamental investors and technical traders act on different time scales, with fundamental traders being quite a bit longer on time scale, usually. But both of them are aiming at taking directional bets.

DS: What are the main limitations of such directional betting?

NP: Our approach basically uses elements from both of these areas, technical and fundamental. But we're sig- nificantly different from the traditional technical and fundamental trading paradigms. First of all, the technical trader usually looks at just the price stream of the instrument that he's trading. He tries to extract patterns from the price stream, which takes the form of charts. There are various ways of analyzing charts and making diagrams derived from the charts, and extrapolations from the diagrams about where the price is going to go.

DS: Candlesticks or-

NP:-or support levels or flying wedges or Fibonacci sequences or Elliott waves. All of these things are basically tools that allow a technical trader to try to extract some kind of structure from the price stream to tell where the price is going to go in the future. Now we actually look at the price stream ourselves. We're not above looking at just about all of the technical variables. We call them technical transformations. And we find that some of them do have predictive value and some of them don't. But very few have enough predictive value on their own. But when combined with many, many others, we can sometimes build a model.

But in addition to these technical transformations, we generally have other kinds of information that we use as inputs for a model that would come under the heading of fundamental information.

DS: So an example of the fundamental input might be...

NP: Well, for a model on the yen exchange rate, for example, we might have not just inputs regarding the price of the yen contract, but also short-term interest rates in the US, short-term interest rates in Japan, long-term interest rates here, long-term interest rates in Japan. Euroyen inputs. And possibly transformations based on the Nikkei.

DS: So you take these fundamental inputs, and you take the technical inputs, and you put them all together somehow into a multi-faceted multi-variable model? Is that how would you describe what you do with all these inputs?

NP: Well, we're reaching the boundary now of proprietary information. But basically, we take these different raw data inputs that are the price stream itself plus all these auxiliary inputs, which we feel might be relevant, and then we create transformations of the data. Some of those transformations are purely technical, by which I mean transformations on the price stream itself-the same sort that technical traders might look at.

But other transformations are used based on fundamental data or on economic intuitions. For example, what might be the effect of an interest rate move on a forex rate. Or a stock move on the forex rate. Or combinations of interest rate moves and a stock move on a forex rate.

And then we feed those transformations into a model-building process that takes those transformations as data and turns them into predictions about whether the price is going to go up or down. And that model-building process can actually teach us which transformations are most relevant and which are least relevant. So we may have constructed a certain transformation from our economic intuition, but the model-building process tells us afterward that our economic intuition was full of shit and we might as well get rid of that one.

DS: So it all goes through a series of generations where the most successful ideas are kept and the other ones are discarded?

NP: Well, this stage could involve many different kinds of model-building processes. If it's a genetic algorithm model-building process, then it's natural to think of generations. There there are actually quite a few model-building processes where you go through iterations that are something like a generation, where you refine and further refine both the model and the inputs used in the model. But the modeling process could be any other kind of regression or a combination of different model techniques.

DS: A lot of people are experimenting with these types of techniques right now. Do you think there are some approaches that are clearly wrongheaded?

NP: I think the most common kind of wrongheaded approach doesn't involve any particular model-building tool so much as it involves the way that tool is used. One has to be very careful about the conclusions one draws from model building. In particular, one has to be careful about how much statistical validity there is in a given conclusion, and how much one expects a given result to generalize to unseen data.

The basic problem has many different names in statistics literature. But probably the most descriptive one is "overfitting." You can take a given set of data and try out lots of different models on it and lots of different predictive rules. And in point of fact, it shouldn't be surprising to you that a few of them seem to work really well. The more you try them out, the better ones you'll find. After you've done all of this trying out, though, and you have a particular model that gives you some really good results on that data, there still remains the question of how well you expect that model to work on new data.

DS: Right.

NP: And then you face an important question: Are the good results that you've gotten looking at this given set of data due to particularities in that data, or are they structural patterns that will persist beyond that particular set of data?

DS: So the obvious solution is to try it out on new sets of data and see how it succeeds.

NP: That's right. That's exactly what you want to do. But you quickly run into the same problem. Suppose that you are given one set of data, and you try out 100 different models. And you find ten of them that look pretty damn good. You decide to try out those ten on a new set of data. And then two of those look good. Well now you want to try those two on more new data to make sure that they still look good. But unfortunately, you often quickly run out of data.

DS: That's the problem? You don't have enough data?

NP: That's certainly one of the problems. Suppose I'm trying to build a daily model for the Japanese government bond, for example. That futures market started around 1989. And so if I want to build a model on daily data, 1989 through 1995, that gives me six years of data. And so what is that? 250 trading days per year for six years gives you 1500 data points. And so if you want to divide that in two to build a model and then test it on the second half, you have only 750 points. And that's starting to be a very dicey amount of data to build a model on. The key place where novice model builders get killed is burning their data too much: using it too much and making conclusions that aren't statistically valid, because they've looked at their data too much.

DS: You could run into the same problem with option models as well, couldn't you?

NP: Absolutely.

DS: Do you think that's what's happening?

NP: I don't know the field well enough to say. I think there are instances of that happening, but I think that the technology is well enough developed so that most of the people in the business are aware of these limitations and not fooling themselves.

DS: So you're doing all this model building in a variety of different markets. But some markets are more appealing to you than others. You've focused on the futures markets, for instance ...

NP: As I've said, one thing that is absolutely necessary for us is to be able to have historical data for the market we're trying to build a model for. And that's certainly a limiting factor for certain markets. Another thing that's a big factor for us is liquidity. We want to be able to make predictions and then execute a trade with as close to the price that we're making our prediction on as possible. And so if a market is at all illiquid, the price actually moves as we're executing.

DS: You get the wrong fill.

NP: And we get a variety of fills, some of which are much worse than others. And some of which may be sufficiently bad that we're not going to make any money on our transactions.

DS: And you like the futures markets because there's a lot of data? And because there's good liquidity?

NP: Right. And of course that second statement only holds for certain kinds of futures markets. Some futures markets are obviously more liquid than others. We have tended to start out with markets that are quite liquid, like the T-bond market, the S&P market, the forex markets.

Another thing that we look for though, beyond liquidity, is volatility. Take the futures market on the Eurodollar, for example. There's an enormous amount of liquidity in terms of contracts per day traded and dollars per day traded. But the movement of that market is very small. So there's not very much to it in volatility.

DS: So you're like any kind of trader. You want things to move.

NP: That's right. And obviously, if things don't move at all, we can make money taking a directional bet.

DS: Do you think that the kind of directional trading that you're doing is where everyone will be in the future? Do you think that inevitably you'll have scores of other people climbing in after you?

NP: We know of a few other groups that are putting serious money behind predictive models, but right now, only a few. The entry cost is high-too high for most financial institutions before the technology is proven. We anticipate the entry of other major players and the consequent reduction of predictive structure in the markets. It remains an open research question, however, how much structure is available, and what its diversity is, and how rapidly our efforts will feel pinched.