The presentation listed a few problems in our current system of rating credit and investing in debt.
- grade inflation in debt rating as NRSROs attract business with generous ratings
- funds forced by law to invest according to ratings from only a few agencies
- meaninglessness of the rating itself
- opacity of the rating model
- lack of diversity in modeling
They propose FreeRisk as a solution to the problems, which they have flipped into a set of requirements.
- accessible
- open
- diverse
- transparent
Cool, but how does that actually solve all of those problems identified earlier?
- Grade inflation - FreeRisk models are still opaque, so we don't know if a company has bought special preference or not.
- Forced acceptance - FreeRisk needs legislation to eliminate the privileged position of established NRSROs, and/or privilege some part of its own leaderboard as an NRSRO.
- Meaninglessness - Models are still opaque, so scores are still meaningless.
- Opacity - Continues.
- Diversity - With continued opacity, we have no clue whether models are diverse or not.
So on my analysis, FreeRisk isn't doing to well at solving the problems it wants to solve, mainly due to model opacity. Solving model opacity isn't easy.
The first problem is that the market will pay handsomely for good models, so there is a clear incentive to keeping them opaque. FreeRisk must be assuming the OSS model will transfer to finance, a big assumption.
The second problem is knowing where to stop in the desire for transparency. Let's take a simple model such as the Altman Z-Score as an example. I'm not happy just getting a real number in a certain range for an answer. I ask for the model. I get a formula using certain financial accounts and weights. Where did the weights come from? Why those accounts and not others? I need to know the design methodology of the model, and the database used to derive the parameters.
FreeRisk currently supports an API that allows for scoring based on a single period's data. Even the Piotroski Score needs two periods of data to work. More troubling, the simple Piotroski and Altman Z methods touted during the presentation are methods for evaluating the corporate entity, not the debt instrument. The debt instrument has to be evaluated for its terms and conditions. We are still very far away from having public T&C databases available in XBRL and Common Logic.
More troubling still is the belief that all credit scoring is model based. This is certainly not true today. It won't be true even when T&Cs are in XBRL and CL. There is still an element of judgement, of interviewing senior management, of reading the news on a company, that comes into play.
There is a lot of free floating moral outrage powering the FreeRisk presentation. But we should step back from that and think dispassionately about changes to our financial system. Should NRSROs really be expected to be aware of bankruptcy before it is announced? To whom do NRSROs owe a fiduciary obligation to force companies into bankruptcy by downgrading their debt? Do we really want debt market volatility similar to equity market volatility based on quicksilver changes in ratings?
Here are some of the things I took away, even if they were unsaid.
- we need finer grained ratings than the current scales provided
- we need to take named NRSROs out of legislation
- some funds should create softer cutoffs for investing
- the buy side should fund the NRSROs, not the issuers
The last is the most important. It removes one of the largest reasons for distortion of ratings. It aligns the interests of the NRSROs with the capital markets.
It would be great if the buy side funded the public company financial statement audit as well!
5 comments:
Thanks for the posting. This was a great presentation.
I understand your concern if the "freerisk" approach still produces models that are opaque.
A couple of ideas here:
1) One could argue that the approach used to get the ratings COULD be a "black box" & still be beneficial. If companies were actually competing on the strength of their rating accuracy, that could actually solve many of the problems, even if the ratings remained "opaque".
In fact, if incentives were aligned to reward companies with the best rating accuracy, it might be necessary to allow people to keep the ratings approach "opaque", to improve the incentives for companies to be able to improve approaches to compete better. (If I know I have to divulge my approach during the competition, that reduces my incentive to innovate.)
2) If you don't buy my argument in #1 above, then it does seem to me that it would be fairly straightforward to enhance the "freerisk" approach to make the ratings approach transparent.
They are using the concept of the "netflix leader board", and I believe netflix did require that the winner of the netflix prize would have to divulge the methods/approaches they used. This is a system that had a financial incentive for the competitors, but also allowed "openness" and sharing of the approach used to compete.
David (& everyone else),
I believe you would be an invaluable contributor to a new wiki site called 'riski'
http://freerisk.org/wiki/index.php/Credit_rating_agencies
While it is housed on the freerisk site you wrote about, it alone, stands to inform and facilitate transparancy in rating practices.
I have noticed on the hill that Congressional staff does not have a central place to access information related to financial markets and regulation.
And as a consequence lobbyists, like Moody's and JP Morgan, end up
being the information providers on
matters related to financial markets legislation. 'Riski' helps to diminish this shortcoming.
'Riski' is an open source project so others can come behind you and
edit for "point of view". And it is this "distillation" process that
hopefully will create the best information source on these topics.
I would encourage you to add content and edit on all parts of the site. Your knowledge, shared in this way, could be very valuable to members of Congress.
I believe 'Riski' is in line with your movement to make markets more
transparent.
Please email me with any thoughts. (mike.s.quinn@gmail.com)
Best, Michael
David (& readers),
Your knowledge and expertice would be invaluable to build a new financial wiki called "riski".
http://freerisk.org/wiki/index.php/Credit_rating_agencies
While 'riski' is housed on freerisk, I think you will come to agree that 'riski' is a very helpful tool.
I have noticed that Congressional staffers do not have a central place to access information related to financial markets and regulation.
And as a consequence lobbyists, like Moody's and JP Morgan, end up
being the information providers on
matters related to financial markets legislation.
To solve this shortcoming we are building a financial markets
regulation wikipedia... for example, you can see the page for raters:
http://freerisk.org/wiki/index.php/Credit_rating_agencies
I would encourage you and your readers to add content and edit all parts of the site. Your knowledge, shared in this way, will be very valuable.
Furthermore, I believe the purpose of riski is in line with your interest in market transparency.
Please contact me with any questions.
Sincerely,
Michael Quinn (mike.s.quinn@gmail.com)
Thanks for your critiques and taking the time to watch our presentation.
There appear to be a number of misunderstandings. Freerisk is an attempt to build, a priori, a new system for evaluating the risk of credit investment - it is a long term project.
There are a number of concerns, I'll go through a few of them.
Opacity: Foremost, Freerisk calculators are open-source - not opaque at all. Currently they exist behind a self-hosted URLs, making it impossible for us to evaluate if the calculator reflects the open-source code; however this is a stop-gap to start the development community. Ultimately Freerisk will host the calculation itself, thus authenticating that the calculator does in fact reflect the source code. This is an engineering constraint - we didn't want to slow down the growth of the community by waiting for the ideal solution.
Transparency: Freerisk is transparent. At all times, it is explicitly clear what decisions have been made and why; however Freerisk models are not transparent, they are open. Expertise will be required to evaluate what a calculator does and if it is reasonable. We hope to make it clear which models are worth evaluating through a leaderboard, but that will not replace a strong understanding of the financial implications.
Incentive problems: All open-source solutions must confront this critique. So far this gap has been bridged successfully in many industries. We agree the finance world is different; however, there are already open-source components of a financial ecosystem that point to the potential in this space. This remains the largest question in the Freerisk model - we'll have to wait and see.
Single-period API: The period parameter is the API is to request a score for a given time period - a calculator may request as much historical information as it needs. As for bond detail information, that will come. We are focused on proving the methodology over the ability of a company to bear its aggregate debt first; followed later by instrument level data. There is a bond detail information gap - from the sound of it, new disclosure regulations will make our lives easier - but if not, there are ways for getting this information - including asking the public at large to supply it directly.
Buy-side Funding: This isn't a practical solution to the problem. Credit ratings form regulatory covenants between investors and money-managers. Without the ability to audit (for free) the behavior of the money-manager, this covenant is toothless.
Charges of populism: We hope we didn't leave that impression. We don't believe this is a story of corruption as much as market failure, but with regard to your specific questions:
Should NRSROs really be expected to be aware of bankruptcy before it is announced?
This depends on the credit rating in question. At low ratings, certainly not. But a very high rating, given the complete lack of a sample size to justify that rating, must be backed by some structural belief in the stability of the firm. When this structural defense changes that should be known.
To whom do NRSROs owe a fiduciary obligation to force companies into bankruptcy by downgrading their debt?
No one at all - this should be a factual decision. The fact that it comes down to a negotiation is, in fact, part of the problem.
Do we really want debt market volatility similar to equity market volatility based on quicksilver changes in ratings?
The question we should ask is "do we want information to flow as fast in the equities market as the bond market?" Bonds will remain more stable than equities by virtue of their senior claims not slow moving information.
Hopefully, as Freerisk evolves we'll be able to convince you that we've addressed your concerns. Don't hesitate to contact me directly at jesper dot andersen at jesperandersen dot net if you'd like to discuss this further.
Thanks,
Jesper Andersen
Co-Founder, Freerisk
Jesper,
Yes, there was a strong populist tone to your presentation. Yes, you said people have a right to be mad. Yes, you talked about Enron, a corruption case, not a market failure. Yes, you lampooned the NRSROs for not downgrading Enron debt and others until after the public announcements.
Since NRSROs are subject to Rule 10b-5, they cannot act on material, non-public information. Anything serious enough to warrant a downgrade would be material.
As you note with Enron, the downgrade killed the white knight deal. In whose best interest was that? (Dynegy shareholders, perhaps, but why was Moody's working for them?)
You argue that rate shopping has caused rate inflation. Do you have a study to support that assertion? It is certainly possible, but on the corporate entity side, fewer and fewer corporates are AAA, so the trend is the opposite of what you imply. Yes, you use words like bribery, and imply there is no firewall between sales and ratings.
NRSROs do have an obligation to explain their methods, contrary to your assertion. Have you looked at Form NRSRO from the SEC, or looked at the Moody's Form NRSRO filing? (Their latest was made 3 weeks after your presentation, but their previous one was 12 months prior to your presentation!)
You say that "AAA" means nothing, but NRSROs have to publish their performance metrics on their web site - the relative quality of their ratings.
I will be following FreeRisk. Maybe I will even be able to participate in it. As one of the creators of XBRL, I am happy to see data used more widely. FreeRisk may grow incrementally. I hope you can outgrow some of the rhetoric of this presentation, and check your facts better.
Post a Comment