P-Spline


The P-Spline algorithm is part of the Estimate Gridded Data option in the Non-Parametric menu. Like Whittaker, it is a smoothing procedure rather than an exact interpolator: it balances closeness to the data against overall smoothness, with the trade-off controlled by a single Lambda value.

Description

A P-Spline (penalized B-spline) fits the data grid with the same cubic basis as the interpolating B-Spline algorithm, but adds a roughness penalty. Rather than requiring the surface to pass exactly through every value, it minimizes the sum of squared misfits to the data plus Lambda times the total roughness of the surface, where roughness is measured from the second differences of the spline's own coefficients. A Lambda of 0 removes the penalty entirely and reproduces the interpolating B-Spline exactly. As Lambda increases, the surface is allowed to depart further from noisy values in exchange for a smoother result; at very large Lambda values the surface approaches a plane. Because Lambda trades data units against coefficient roughness, it has no fixed meaning from one data set to the next — it should be re-chosen whenever the data change, which is what the GCV and rGCV buttons are for.

Choosing Lambda — GCV and rGCV

The GCV button selects a Lambda automatically using Generalized Cross-Validation, which estimates how well the surface would predict each grid value if that value were withheld, without needing to refit the surface once per point. TableCurve Studio searches 45 logarithmically spaced Lambda values between 1e-4 and 1e7 and keeps the one that minimizes the GCV score. The rGCV button applies a more robust form of the same criterion, biased toward additional smoothing; it is the better choice if a plain GCV result still looks noisy. Either button gives an excellent starting value rather than a final answer — after using one, check the partial-derivative surfaces as well as the surface itself, since residual noise often shows up in the derivatives before it is visible in the surface.

Order and Knots

Unlike the other algorithms in this dialog, P-Spline does not use the Order X/Y or Knots X/Y fields. Its basis is always cubic, with one coefficient for every grid line in each direction, so the flexibility of the surface is controlled entirely by Lambda rather than by a knot count.

P-Spline or Whittaker?

P-Spline and Whittaker answer the same question with the same single Lambda parameter, and in ordinary use they produce similar surfaces; the difference is how each gets there. Whittaker smooths the grid values first, then runs an ordinary interpolating cubic B-spline through the cleaned grid — two steps. P-Spline fits and smooths in a single step, applying the penalty directly to the spline's own coefficients rather than to an intermediate grid. Both produce a smooth cubic surface with continuous first and second partial derivatives, and both report SSE and rē against the original data. There is no strong reason to prefer one over the other; try both on a given data set and compare.

Statistics

Because P-Spline is a smoothing rather than an exact-interpolation algorithm, the dialog reports a squared sum of residuals (SSE) and a coefficient of determination (r2) so the fit can be compared against parametric results. With Lambda set to 0, the surface passes exactly through the data and SSE will be 0.

Extrapolation

P-Spline does not offer reliable extrapolation. Like any spline, it says nothing dependable about the surface outside the region the data cover — use it only within the bounds of the input grid.

Considerations

P-Spline is best suited to measured grids that contain noise, where a smooth surface and trustworthy derivatives matter more than passing through every value exactly — particularly when the interpolating B-Spline produces a visibly rippled result. It is not intended for scattered data (use the thin-plate spline, Akima, Renka, or Preusser instead), and it should not be used where every value must be honoured exactly — set Lambda to 0, or use B-Spline, in that case.

Author

Paul H. C. Eilers and Brian D. Marx, who joined Carl de Boor's B-spline basis to E. T. Whittaker's difference penalty.

References

Eilers, P. H. C. & Marx, B. D. (1996). "Flexible smoothing with B-splines and penalties." Statistical Science, 11(2), 89–121. de Boor, C. (1978). A Practical Guide to Splines. Springer. Whittaker, E. T. (1923). "On a new method of graduation." Proceedings of the Edinburgh Mathematical Society, 41, 63–75. Craven, P. & Wahba, G. (1979). "Smoothing noisy data with spline functions." Numerische Mathematik, 31, 377–403.