The Stabalized Normal Probability Plot

The Stabilized Normal Probability Plot

Applies to: 3D/Studio surface fitting. The 2D curve-fitting SNP is the same plot and the same numbers.

Every fit statistic that carries a confidence or prediction interval assumes the errors are normally distributed. If they are not, those intervals are wrong -- and usually wrong in the flattering direction, reporting a precision the data does not support. The SNP is the graphical test of that assumption, and it is the one TableCurve has offered since the early 1990s.

Opening It

In the 3D Surface Fit Review, open the Residuals window and choose the SNP format. The pane swaps from the residual surface to the probability plot; choosing any of the other three residual formats brings the surface back.

The plot needs at least eight active data points to mean anything. Below that the option simply does nothing.

How to Read It

The plot shows the fit's residuals, ordered and transformed so that normally distributed errors produce a flat trace running along zero. Four pairs of limit lines are drawn at the 90%, 95%, 99% and 99.9% levels, and the points are coloured by the band they fall in.

· The trace wanders inside the 90% band. Nothing to report: the residuals are consistent with normal error, and the fit's confidence and prediction intervals mean what they say.

· The trace crosses the 95% or 99% limits. Worth attention. Something in the error structure is not normal -- an outlier, a skew, a heavier tail than the model assumes.

· The trace passes the 99.9% limit. The assumption has failed. Any confidence interval the fit reports is dishonestly tight, and the size of the departure is visible rather than inferred.

The limits are the diagnostic, not the eye. A trace can look untidy and stay well inside the 90% band, and it can look mild and still cross 99.9% -- which is exactly why the bands are drawn rather than left to judgment.

What the Limits Depend On

The critical values are a function of the number of points alone. They narrow as the data set grows -- with many points, a small departure from normality is detectable; with few, only a gross one is. Nothing else enters: not the equation, not the number of parameters, and not the dimensionality.

Why a Surface Fit Uses the Same Plot as a Curve Fit

A surface fit predicts a single value at each point, z = f(x,y), so each residual is a single number exactly as in curve fitting. The surface is two-dimensional; the errors are not. That is why the SNP transfers unchanged from 2D to 3D -- the residuals are pooled across the whole surface and read as one sample, and the limits above apply to them without modification.

In TableCurve Studio this is not merely the same method but the same computation: the 3D SNP feeds the identical routine the 2D side uses. The two can never disagree about a set of residuals, because there is only one implementation.

What It Does Not Tell You

The SNP tests whether the residuals are normally distributed. It does not test whether they are independent of position -- a surface fit can have beautifully normal residuals that are systematically positive in one corner and negative in another, which the SNP cannot see because it discards the x,y locations when it pools. For that, look at the residual surface itself: structure in the residuals is structure the model has missed, and the two diagnostics answer different questions.

Provenance [NEEDS RON'S OK BEFORE THIS SECTION SHIPS]

The stabilized normal probability plot is due to John R. Michael (Biometrika, 1983). It reached TableCurve through Dr Lloyd S. Nelson -- chief statistician at General Electric, later at Nashua Corporation, editor of the Journal of Quality Technology, and author of the Nelson rules for control charts -- who considered it the definitive graphical answer to the question of error normality and championed its inclusion. The method might well have passed into obscurity without him.