Many people try to use statistical methods to crack historical price data and make a price prediction model. To be honest, this is not an especially difficult task, with all that fancy machine learning libraries available. Then they try to apply their models on future data expecting tons of cash. I wouldn't be so optimistic about that. The model explaining past data is good only for one thing - for explaining past data.
Let's imagine a datasource producing a lot of data. When you're building a model about this data and trying to use this model on live data, you're making a serious assumption. You're expecting future data to be produced by essentially the same datasource with the same statistical properties. The data may be noisy, but the signal properties are supposed to be the same. There is not much sense in applying a model, describing one datasource, to a completely different datasource, and expecting tons of cash as a result.
Well, when you're trying to apply a model, describing past stock market data, to future stock market data, you're doing exactly that. The core problem is - the datasource, producing a price stream is never the same! It is constantly changing! Every time the price behavior is a result of some different disposition of forces. Trying to use old data properties for predicting future data properties, without making additional assumptions - doesn't have much sense!
Try to compare 2005-2006 SPY data with 2008 SPY data - you won't find anything in common. All distribution moments, all correlations - everything was different. It was essentially different markets, and the fact that those markets are glued together into a single price chart should not mislead.
Initially, you have a "zero hypothesis" stating that past and future returns are datasets with completely different properties. Only after disproving this hypothesis you can go further. But is there anyone who honestly does this? I never saw. Everyone just closes their eyes and take a leap of faith assuming that a datasource won't change anytime soon. This can work, but this is a lottery. Any moment you may find yourself trading a completely different market where your models don't work.
I am not trying to say that all systematic trading is meaningless. Maybe not all of it, I don't know. What I’m trying to do is to point out an additional source of risk, widely overlooked. Everyone's so got used to the "past performance do not guaranty future results" phrase that people have stopped to understand the whole amount of unpleasant truth behind it.
Let's imagine a datasource producing a lot of data. When you're building a model about this data and trying to use this model on live data, you're making a serious assumption. You're expecting future data to be produced by essentially the same datasource with the same statistical properties. The data may be noisy, but the signal properties are supposed to be the same. There is not much sense in applying a model, describing one datasource, to a completely different datasource, and expecting tons of cash as a result.
Well, when you're trying to apply a model, describing past stock market data, to future stock market data, you're doing exactly that. The core problem is - the datasource, producing a price stream is never the same! It is constantly changing! Every time the price behavior is a result of some different disposition of forces. Trying to use old data properties for predicting future data properties, without making additional assumptions - doesn't have much sense!
Try to compare 2005-2006 SPY data with 2008 SPY data - you won't find anything in common. All distribution moments, all correlations - everything was different. It was essentially different markets, and the fact that those markets are glued together into a single price chart should not mislead.
Initially, you have a "zero hypothesis" stating that past and future returns are datasets with completely different properties. Only after disproving this hypothesis you can go further. But is there anyone who honestly does this? I never saw. Everyone just closes their eyes and take a leap of faith assuming that a datasource won't change anytime soon. This can work, but this is a lottery. Any moment you may find yourself trading a completely different market where your models don't work.
I am not trying to say that all systematic trading is meaningless. Maybe not all of it, I don't know. What I’m trying to do is to point out an additional source of risk, widely overlooked. Everyone's so got used to the "past performance do not guaranty future results" phrase that people have stopped to understand the whole amount of unpleasant truth behind it.
This comment has been removed by the author.
ReplyDelete