Thanks to my friend John Turner, I just watched the video of a
presentation about FreeRisk at
O'Reilly's ETech conference.
The presentation listed a few problems in our current system of rating credit and investing in debt.
- grade inflation in debt rating as NRSROs attract business with generous ratings
- funds forced by law to invest according to ratings from only a few agencies
- meaninglessness of the rating itself
- opacity of the rating model
- lack of diversity in modeling
They propose FreeRisk as a solution to the problems, which they have flipped into a set of requirements.
- accessible
- open
- diverse
- transparent
Cool, but how does that actually solve all of those problems identified earlier?
- Grade inflation - FreeRisk models are still opaque, so we don't know if a company has bought special preference or not.
- Forced acceptance - FreeRisk needs legislation to eliminate the privileged position of established NRSROs, and/or privilege some part of its own leaderboard as an NRSRO.
- Meaninglessness - Models are still opaque, so scores are still meaningless.
- Opacity - Continues.
- Diversity - With continued opacity, we have no clue whether models are diverse or not.
So on my analysis, FreeRisk isn't doing to well at solving the problems it wants to solve, mainly due to model opacity. Solving model opacity isn't easy.
The first problem is that the market will pay handsomely for good models, so there is a clear incentive to keeping them opaque. FreeRisk must be assuming the OSS model will transfer to finance, a big assumption.
The second problem is knowing where to stop in the desire for transparency. Let's take a simple model such as the Altman Z-Score as an example. I'm not happy just getting a real number in a certain range for an answer. I ask for the model. I get a formula using certain financial accounts and weights. Where did the weights come from? Why those accounts and not others? I need to know the design methodology of the model, and the database used to derive the parameters.
FreeRisk currently supports an API that allows for scoring based on a single period's data. Even the Piotroski Score needs two periods of data to work. More troubling, the simple Piotroski and Altman Z methods touted during the presentation are methods for evaluating the corporate entity, not the debt instrument. The debt instrument has to be evaluated for its terms and conditions. We are still very far away from having public T&C databases available in XBRL and Common Logic.
More troubling still is the belief that all credit scoring is model based. This is certainly not true today. It won't be true even when T&Cs are in XBRL and CL. There is still an element of judgement, of interviewing senior management, of reading the news on a company, that comes into play.
There is a lot of free floating moral outrage powering the FreeRisk presentation. But we should step back from that and think dispassionately about changes to our financial system. Should NRSROs really be expected to be aware of bankruptcy before it is announced? To whom do NRSROs owe a fiduciary obligation to force companies into bankruptcy by downgrading their debt? Do we really want debt market volatility similar to equity market volatility based on quicksilver changes in ratings?
Here are some of the things I took away, even if they were unsaid.
- we need finer grained ratings than the current scales provided
- we need to take named NRSROs out of legislation
- some funds should create softer cutoffs for investing
- the buy side should fund the NRSROs, not the issuers
The last is the most important. It removes one of the largest reasons for distortion of ratings. It aligns the interests of the NRSROs with the capital markets.
It would be great if the buy side funded the public company financial statement audit as well!