Business — Banking — Management — Marketing & Sales

The easiest way to visualize accuracy is through a misclassification matrix. The matrix simply cross-tabulates the actual classification versus the predicted classification. The simplest classification uses two categories only, defaulters and non-defaulters. When modelling ratings, there are as many classes as there are different ratings. It is a square matrix. Correct classifications are along the diagonal, since this is where both predicted and actual classes coincide. Errors are above or below the diagonal.

Another standard technique for visualizing and quantifying scoring accuracy is the use of power curves. These apply to any credit risk quality measure, whether from scoring or more involved modelling such as the KMV expected default frequency Edf ©, or agency ratings predictive ability. The technique measures misclassifications as a single ratio.

Matrices and power curves apply to all types of modelling techniques, multivariate regression, discriminant analyses, multinomial logit or probit, ordered probit and logit, or NN models.

Misclassification Matrices

When modelling ordinal or numerical variables, the further away values are from the diagonal, the larger is the error. Table 37.1 is a simple misclassification matrix for six rating classes. In this case, there are 100 firms, spread over six rating classes. The total of rows shows the total number of firms in each class in this sample. Correct classifications are along the diagonal, and represent 47% of all observations. Since it is interesting to know the number of small errors, say within ±1 notch, the summary table also provides this total, or 75. Larger errors, within ±2 notches, are 87 — 75 = 12. The count of larger errors is 13.

The matrix applies to all models used in classification mode, for classifying in categories, or in ordinal regression mode, for modelling ranks such as ratings. For default models, there are only two cases, default or non-default, and the matrix collapses to 2rows   2 columns.

Power Curves

Power curves also visualize errors. Unlike misclassification matrices that provide all details, they summarize the accuracy of a model by a single percentage, which reaches

100% in the extreme case, where the model provides only correct classifications and no errors. These apply to any credit risk quality measure, whether from scoring or more involved modelling such as the KMV Edf, or agency ratings predictive ability.

Plotting a power curve requires ordering obligors by risk score, from riskiest to safest. For example, the entire population has a small fraction of defaulters, for instance 10%. A model performs well if it classifies all defaulters to the default class. Nevertheless, this default class represents only a small fraction, 10%, of the entire population. Two extreme situations illustrate the shape of the power curve.

Lets assume that we sample 10% of the population among the riskiest scores. Since the fraction of defaulters is 10%, a perfect model would find 100% of all defaulters, or 10% of the total population. We plot on the horizontal axis the percentage of sampling from the total population and on the vertical axis the percentage of the total population of defaulters, which is only 10% of the total population. A perfect model would result in a point with 10% of the sample population on the horizontal axis and 100% of defaulters on the vertical axis. This point is at the top of the steepest line starting from the origin and reaching the 10% of population/100% of defaulters population.

No model is perfect. Lets assume that the score does not provide any information on defaulters. We would sample 10% randomly across the population. The average fraction of defaulters would be 10% as well, or 10% x 10% = 1% of the total population. The point with 10% of total population and 10% of total subpopulation of defaulters is on the diagonal of the diagram. Therefore, the worst performing model (without bias) would have all points on the diagonal and not above.

If the model provides information on who defaults and who does not, we move above the diagonal. In order to understand the shape of the curve when we progressively increase the sampling from the total population, we remark that when we sample the entire population, we necessarily obtain all defaulters, or 100% of them, equivalent to 10% of the total population. Hence, the curve passes through the 100%/100% point. It is between the diagonal and the polygon made of the steep straight line and the horizontal line positioned at 100% of the population. In Figure 37.5, we show a sample of 40% of the population. If the model samples defaulters randomly, we get 40% of them, or 4% of the total population. This point is on the diagonal. If we get all defaulters, we are on the top horizontal line because we have in the sample all defaulters, 100% of them with 10% of the population. If the model discriminates defaulters, we are in between. In this example, we reach around 80% of the defaulters, or 80% x 10% = 8% of the total population.

This rationale underlies using the power curve to measure the accuracy of a model. If the curve is close to the diagonal, the predicting power is close to zero. If it is close to the steepest line, the predicting power is high. Constructing the power curve allows us to see how the model behaves across the entire range of scores. A predictive model has a power curve steeper than the diagonal, but less than the steep straight line. A common way to measure the level of the power curve above the diagonal is the ratio of the area under the power curve and above the diagonal to the area under the steep straight line and above the diagonal. The higher is the ratio, the higher is the accuracy of the model. In the figure, we see that we reach an overall accuracy ratio of around 50% to 60%, using as reference the upper area between the diagonal and the polygon.

The Option Approach to Defaults and Migrations

The option theoretic approach to default modelling follows the simple principles set up by R. Merton in his seminal paper of 1974. In short, the option theoretic approach views default as an economic event triggered by a market value of assets lower than the debt value. This approach views a firms equity as a put option, held by equity holders, to sell assets to lenders with a strike price equal to debt. If equity holders cannot repay the debt, they exercise their option, and the debt holders get the assets. The firms debt has a value lower than a straight debt by the value of the put option, which increases when the default probability increases. The mirror image of the default option on equity is that the equity holders have a call on the upside of asset value beyond the debt value. The call becomes out-of-the-money when the asset value is lower than the debt, which is when the put payoff becomes positive.

The chances of economic default depend on the asset value, the debt value and the asset volatility. The asset value is that of the future cash flows adjusted for risk.

The option framework values the credit risk as the put option of equity holders on the underlying asset of the firm, making the debt value lower by the value of the put option. The put value is the gap between the values of a risk-free debt and a risky debt. It is equivalent to the credit spread over the risk-free rate, bringing the risky debt value in line with the value of the risk-free debt minus the put value. Any gain of the put value is a loss for the lender. Finally, because there is a link between equity value and asset value, it is conceptually feasible to hedge a borrowers default risk by selling short its equity.

KMV Corporations Credit Monitor model makes the Merton model instrumental. Credit Monitor uses public equity prices to infer unobservable asset values and their volatility, and traces back the implied default probability embedded in equity prices and debt. The model looks forward because it relies on market data, which embeds market expectation. KMV uses the distance to default as an intermediate step, measured as the gap between the expected asset value and the threshold value of debt triggering default. The distance to default varies inversely with the default probability. The output is the well-known expected default frequency (Edf ©). The Edf is a closed-form expression of the asset value, asset volatility and debt book value. Edfs contrast with historical default rates because they look forward, and with empirical statistical findings because the Edfs rely on conceptual foundations.

Modelling default probabilities also serves to model migrations from the original risk class to any risk class. A migration is simply an upward or a downward move of the asset value relative to debt. The option model applies to migrations because it models default probabilities and because default probabilities map with risk classes.

The first section explains the basic framework of the option theoretic view of risky debt. The second section explains how to infer, unobservable asset values and volatility from equity prices. The third section describes KMV Corporations Credit Monitor model, which provides the Edf of public companies. The section details: the calculation of Edf; the valuation of the put option value held by equity holders on the assets of the firm; the determination of the equivalent credit spread; the theoretical implications for hedging credit risk by trading stock options. The fourth section summarizes prior and posterior variations of the Merton model. The fifth section introduces a simplified approach for modelling default using a normally distributed asset value plus a predefined default probability. The last section extends the option theoretic framework to migration modelling.