The paradox of predicting the unknown

Demitry Sevastianov of Multi-Criteria blog posted a comment about one of my posts on methods within Finance. It was a good one highlighting some loose terminology on my part...

So this is kind of a corrective to that, addressing both his comment but also trying to take that debate forwards as I like some of his ideas about fuzzy logic sets. So when I used the phrase efficient system I was being a but naughty - what I meant was the implied efficiency of systems that fit the central limit theorem.

There are many systems that exhibit randomness and complexity (ironically ordered systems are the ones which don't exhibit complexity in a mathematical sense) and these are, by definition, extremely hard to model predictively.

Part of my thinking is that efficient market theories are lacking - in two respects:
1. They fail completely to account for systemic risk - i.e. they only work within their own definition. Now that's a truism, of course they only work within their own frameworks, but when one is truly trying to capture the behaviour of a market it is very well worth emphasising that because when an entire market models itself on a methodology that fails to capture its behaviour reminders that the methodologies are inadequate can never come too often.
2. They fail completely to account for the complexity that agents introduce to the system. It's a central tenet of these theories that agents have no impact, that they're reduced to noise within the system but this isn't the case. Whilst there are emergent structures within complex systems they are led by individual actions (even if once they've commenced their overall shape is decided by axiomatic foundations rather than further individual decisions.

The market uses monte carlo and even simpler methods (although the mind boggles at that!) to manage its risk. I'm continuously surprised at the schoolboy simplicity of the models deployed - and normally on the premise that that's what everyone else does! In my experience, when pressed, people don't actually know the issues and even if they do they aren't capable of distinguishing between the severity of different flaws in different models.

A sea change is coming - at least I'm hopeful it is - where people will recognise that it's inappropriate to use methods for modelling risk on the basis that they're what has always been done. In some senses this level of blind orthodoxy belongs only in fundamentalist religion.

My own view, and the area I think is probably most productive in future, is that agent based modelling is the way forwards. We're never going to be able to model the world 1:1. It's a nonsense anyway, the whole point of this is to gain an operator that produces key features of the real world without having to model the whole world.

Axiomatic agent based models have within their very foundation the rules for complexity and for appropriate levels of predictiveness. The issue with complex systems is that they can't be used for prediction. At least not in the fashion that finance professionals would like - exact results.

One of the key unintended effects of the current crisis is the refocussing of what constitute acceptable ideas of risk. Previously it was in the heart of investing that investments can increase as well as decrease in value. Now we're being pressured to think of ways of insuring our investments can only ever increase in value. Now this ain't never goin' ta happen cowboy. The revelation that most people don't have a clue about the meaning of the word risk is quite scary but the revelation that we're allowing these numpties to guide our legislation and future shape of the market is downright hallucination inducingly moronic.

I like fuzzy logic, I think it's a useful bridge between pseudo-random models and reality, but I'm not yet convinced it's the way forward. Like I say I think agent based modelling, still in its infancy, is the most likely discipline to give us insight into capturing systemic and individualistis factors into market risk. Having said that I'd be as foolish as those people I'm criticising if I was to say that because I've a hunch about which approach may be best at solving the problems ahead of us I wouldn't consider alternatives. I will be looking at Demitry's blog in the future to keep progress of what may very well be quite an exciting approach.

Ultimately though, the rules about arbitrage are model independent. If one exists, people will take advantage of it until it's closed down and people will always be naturally poor judges of long term risk.