Estimation and Prediction Q&A

When should I use non-parametric estimation?

If your data have a very unusual profile, it is possible that none of TableCurve 2D’s built-in equations will fully describe the features within the data. In these cases, a non-parametric fit may be successful.

If you need only a few simple interpolations, a non-parametric procedure offers near immediate processing and the same evaluation procedure as that of the parametric fitting.

If data are non-uniformly spaced or if you have been forced to disable one or more points containing artifacts, the data can be restored to an equal X-spacing.

Why are there five different non-parametric estimation options?

Each of the procedures has its particular strength.

The Spline Estimationoption offers three spline procedures for interpolation and five spline algorithms for smoothing. First and second derivatives are also available. This procedure is useful for creating uniformly spaced data sets since uniform data are not required. An option offering user-defined knots can be used to finely tailor a least-squares spline fit.

The Local Regression Spline Estimation option combines an adjustable order Loess-type (locally-weighted least-squares) smoothing algorithm with a B-spline interpolant. This procedure is also useful for creating uniformly spaced data sets since uniform data are not required. Although this procedure is much slower than a smoothing spline, the local weighting may result in a more accurate estimation.

The Fourier Estimation option offers interpolation and smoothed estimations based upon the frequency spectrum. Although very slow with large data sets, this option may be more useful for oscillatory data.

The Smoothed Data Spline Estimation procedure combines automated smoothing and B-spline estimation. This is not a smoothing spline, but is rather the fitting of an interpolating B-spline to data that have been pre-smoothed.

The Savitzky-Golay Spline Estimation procedure combines combines the full capability of the Savitzky-Golay smoothing with a constrained spline interpolant. This procedure has been especially tailored for estimating derivatives.

The Spline Estimation option is a very good place to start if only interpolation or modest smoothing is needed. The Local Regression procedure offers smoothing and possibly a greater accuracy as a consequence of the locally-weighted fitting. The Fourier Estimation is useful for estimating data with oscillatory components. It also offers Fourier filtering and estimation in a single step. When significant smoothing is needed, the Smoothed Data Spline Estimation is recommended. When an accurate smoothed derivative is needed, you will probably want to use the Savitzky-Golay Spline Estimation procedure.

How do I predict or forecast data?

The AR Modeling and Prediction procedure offers effective autoregressive forecasting and extrapolation. The AR algorithms include SVD (singular value decomposition) procedures for in-situ noise removal. This option can be used directly for forecasts or to generate predicted data that are then fit to parametric models.

Can I forecast in both directions?

The AR Modeling and Prediction procedure offers three algorithms for predicting ahead, three for predicting earlier values in time, and three that predict in both forward and backward directions.

How do I optimizing the AR Fit?

For SVD-based algorithms, Signal-Noise thresholding is graphical and often straightforward. Once the signal space has been properly specified, model order becomes less important. The higher the model order fitted, however, the less the noise will influence the AR fit.

For non-SVD algorithms, multiple orders can be simultaneously plotted to assist in optimum model order selection. Model order selection criteria are also available as is an option to plot complex roots. Stabilizations are available for roots that lie outside the unit circle.

Can I apply the AR Filter to other data sets?

An AR coefficient set (filter) can be saved to disk and imported and applied to subsequent data sets. You can import as many saved filter sets as you choose and all will be added to the algorithm list using the file name.

The AR filter used for the prediction can also be saved to language code.