Sunday, June 1, 2008

The Accuracy of Financial Models

Lawrence Summers, writing in the Financial Times, outlines six principles for improving the regulatory regime. There's not much to quarrel with, as his principles are pretty broad. His second principle, though, deserves special emphasis:

Second, it should be recognised that to a substantial extent self-regulation is deregulation. Allowing institutions to determine capital levels based on risk models of their own design is tantamount to letting them set their own capital levels. We have seen institutions hurt again and again by events to which their models implied probabilities of less than one in a million. Where it is desired to impose capital requirements, this should be done in a way that can be monitored by supervisors on the basis of balance sheet data.
The standard argument for basing regulations on banks' own models is the kind of argument normally used by only extreme free marketers: Banks are better at designing models than any government regulator (who is presumably stupid), because banks have an incentive to design good models, and employ people who know much more about particular financial markets than government regulators. This line of reasoning is as stupid and naïve as it sounds. I've always liked Emanuel Derman's explanation of why private financial models are so unreliable (from his excellent book, My Life as a Quant):
The majority of options dealers make their living by manufacturing the products their clients need as efficiently as they can -- that is, by providing a service for a fee. For them, a simple, easy-to-understand model is more useful than a better, complicated one. Too much preoccupation with details that you cannot get right can be a hindrance when you have a large profit margin and you want to complete as many deals as possible.
Just because banks have "an incentive" to do something doesn't mean they'll actually do it. Banks have other, often competing, incentives too. As long as the incentive structure on Wall Street leads banks to favor short-run profits at the expense of long-run profitability, then their financial models will incomplete and ultimately inaccurate.