The individual predictions and their labels can be toggled on and off with the added reference buttons
in the graph's
toolbar.AR Modeling and Prediction
·
The AR Modeling and Prediction procedure in the Estimate menu or the Process toolbar offers effective forecasting and extrapolation using autoregressive modeling. The AR model is either forward (subsequent points are predicted), backward (prior points are predicted), or forward-backward (both subsequent and prior points are predicted). The algorithms include SVD (singular value decomposition) methods for in-situ (in place) noise removal. The two standard stabilizations are available for roots that lie outside the unit circle.
Uniformly-Spaced Data
This procedure does not automatically generate the uniformly-spaced data required by the autoregressive algorithms. If needed, the Spline Estimation or Local Regression Spline Estimation option can be used to create an interpolated data set with constant X-spacing.
Discrete Output
An autoregressive model produces discrete output; it does not offer interpolation. One of the key uses for AR modeling is to generate a series of extrapolated points that can then be used for extending a parametric fit. The parametric fit of the original data may not offer a valid extrapolation, but a parametric fit of data containing these predicted points may well do so.
Multiple Orders
Multiple AR model order predictions can be simultaneously plotted and an average curve can be used for the prediction. Error bars simplify the task of assessing the impact of model order on the predicted values.
Assessing the Validity of Predictions
The points that are to be processed can be specified, allowing predictions based on a data segment to be compared with actual subsequent data. The extent of the prediction is variable and white noise can be added to test the algorithm's robustness.
Graph
The input data and the AR model fit and prediction for each selected order are plotted in a TableCurve 2D Graph.
The individual predictions and their labels can be toggled on and off with the added reference buttons
in the graph's
toolbar.
The graph’s
toolbar also has a button that offers the full selection
of error bars, as well as an error bar
toggle.
Algorithm
The AR algorithm list offers nine effective AR least-squares procedures. The initial three listed are forward prediction algorithms (Fwd). These are the most commonly used algorithms since they are used to predict future values. The next three are backward prediction algorithms (Bwd). These are used to predict the data existing prior to the start of a data sequence. The final three are forward-backward prediction algorithms (FB) which predict both prior and subsequent time series values.
You must select a Fwd algorithm to predict future values, a Bwd algorithm to predict past values, and an FB algorithm to predict in both directions.
The Data algorithms process a full forward, backward, or forward-backward prediction data matrix. This is the most accurate matrix processing procedure. The Nrml algorithms process the normal equations, a square matrix that can offer significant performance benefits with very large data sets. An AR model fit is a linear non-iterative matrix method, and the Data method should offer sufficient performance except perhaps for the very largest data sets. Typically, the Data algorithms, both with and without SVD, produce stable estimates where all roots lie on or within the unit circle. This is not assured, though.
The SVD algorithms offer in-situ noise removal. With an effective signal-noise threshold selection, the SVD procedures will most certainly offer the most accurate predictions. The predictions are based only on the signal components in the data and are minimally impacted by noise. SVD procedures are up to an order of magnitude slower than their non-SVD counterparts. Apart from processing time, there are few drawbacks to using SVD and a great many benefits.
To be consistent, the statistical results reported for all AR fits are based on a forward prediction filter with partial backward prediction. This will not exactly match any of the linear AR procedures. The AR filter is defined using backward prediction from the model order down to the initial data element, and forward prediction from the model order up to the final data element. This simplifies the goodness of fit statistics since a single estimate is made for each of the input data elements. These statistics should be generally valid for the Fwd models, but they should be used very cautiously as a basis for selecting Bwd prediction algorithms or orders.
Model Order Selection
If the Single Order option is checked, the order field is used to enter the desired AR model order. If this box is not checked, the order minimum (min), maximum (max), and increment (inc) are specified. The Data Fwd, Data Bwd, and Data FB least-squares methods compute all lesser orders when computing a given order, so these algorithms will be very fast in this procedure, regardless of the count of orders plotted. The Nrml algorithms and all of the SVD procedures compute only the target order, so the number of individual orders specified can significantly impact processing time. When multiple orders are fitted, an average is computed, plotted, and used for output of the predicted values.
Since the Data
least-squares methods compute all lesser orders in the process of computing the target order, the Plot
Selection Criteria option will include all orders up the
maximum order specified. Since the Nrml
algorithms and all of the SVD
procedures compute only the evaluated order, the Plot
Selection Criteria option will include only those orders actually
specified by the minimum, maximum, and increment values.
When data consist of harmonic signals in the absence of noise, the minimum order needed will be twice the number of sinusoidal components. An AR model can have both real and complex roots. The real roots, usually at -0.5, 0, or 0.5 normalized frequencies, are not processed since they represent singularities at the bounds. The complex roots produce finite spectral power, and for real data, the positive and negative frequency roots mirror one another. Although TableCurve 2D reports only the positive roots, both sides of the spectrum must be taken into account. This is why the minimum order needed must be twice the number of component sinusoids.
In practice, there is usually some level of noise present in the data and a higher order model is needed. It is often the case that the minimum order needed to resolve all components is significantly higher. You may want to be generous when setting the maximum order value if it is your aim to determine this minimum order needed. Also, signals are often composed of near-harmonic and anharmonic oscillations which require higher model orders.
To achieve a reasonable signal-noise separation with SVD, it is necessary to fit a high enough order so that the primary singular vectors (eigenvectors) span only signal space. Provided the order is sufficiently high to produce an effective partitioning of the signal and noise, the actual order of the fit is not critical. The quality of the fit for the noise components is not a consideration, since these eigenvectors are discarded in the SVD processing. All that is needed is an accurate specification of the signal space.
Signal Subspace Selection
The Graphically
Select Signal and Noise Sub-Spaces option is available only
when an SVD
algorithm is being used. Even when a harmonic component count is known, you should use this option to
insure that a high enough order is being used to achieve the desired signal-noise separation. The eigendecomposition
for only the first of the orders fitted (the minimum order) is available for graphical signal space selection.
Again, to accommodate both positive and negative frequencies, the signal space value must be twice the
number of components.
When there is sufficient signal-noise separation in the eigenmodes, the singular value plot reveals one or more sharp transitions between the signal subspace and the noise subspace floor. The last eigenmode before the long sloping noise floor represents the last element of signal space. Assuming a high-enough AR model order is used, this signal-noise space separation does not become difficult until the noise level approaches that of the signal. At this point, the sharp characteristic transition disappears. An earlier diminishing of this transition occurs when the noise is red.
A full signal space SVD fit, one where the signal space equals the model order, produces the same results as the non-SVD algorithms.
Stabilization
A stable AR filter is one with roots that lie on or within the unit circle. While the least-squares AR procedures often produce fully stable coefficients, this is not assured.
The roots of the AR model can be inspected with the Plot
Roots option. Roots consisting of signal will rest on or
close to the unit circle while those corresponding with noise tend to be found in the interior of the
unit circle.
In those cases where roots are outside the unit circle, they may be attributable to either signal or noise. The Reflect Out option will reflect the outlier root across the unit circle into the interior. Those very close to the unit circle will remain so, and will generally be processed as signal. Those well away from the unit circle will be reflected significantly into the interior and will be processed as noise. The Unit Circle Out will reflect all outlier roots to the unit circle, assuming that they represent valid sinusoidal components.
Data Processed
To facilitate prediction tests, it is possible to specify that only a portion of the data set be processed. The x start and x end values specify the starting and ending x values within the data stream that will be used to create the parametric model. Subsequent (or earlier) data can then be compared with the predicted values. The default values will specify the actual range of the input data.
Note that the order will be reduced automatically when it is required by the size of the subset being processed.
Predicted Points
For the output, the number of predicted elements n is specified. These will be appended to the end of the output data stream if a Fwd algorithm is used, to the beginning of the output data stream if a Bwd algorithm is selected, or to both ends of the data stream for an FB algorithm. By default, n will be one-quarter the number of data elements.
The output data stream will consist of the predicted AR model values for all x-values in the data series and for this number of predicted values following, preceding, or both following and preceding. If multiple orders are used, these values will consist of an arithmetic average of all orders specified.
Although the n predicted points will use exclusively forward or backward prediction, the output data stream values corresponding with the x-values in the input data will be based upon an AR filter that is defined using backward prediction from the model order down to the initial data element, and forward prediction from the model order up to the final data element.
Add Noise
It may be instructive to see where a given procedure starts to break down as a consequence of temporarily adding white observation noise to the input data. The zero noise level is S/N=300dB (fractional noise=1E-15, the IEEE double precision threshold for addition). At this value, no noise is added to the data. A value of 280 would add noise in the 14th significant figure, 260 in the 13th, 240 in the 12th and so on. This option assumes that the current data set is entirely signal, and adds noise accordingly. Typical test values are 40dB(1% noise), 20(10%), 10(31.6%), 6(50.1%), 3(70.8%), and 0(100%).
The SVD procedures have the greatest noise resistance. This noise option is also helpful in ascertaining at what level the SVD procedures can no longer evidence the eigenmode signal to noise transition.
List
The List Data
option lists the index, time, and output signal in a three column table. The listing uses the TableCurve
2D text viewer facility.
Copy
The Copy Data to Clipboard option copies the time and output
signal values to the clipboard. Formats include full precision binary (for spreadsheets such as Excel)
and ASCII (for pasting into text editors).
Save
The Save Data to Disk
option writes the time and output values to a supported file format. These formats include ASCII, Excel,
SPSS, or Systat.
Production Facility
The Automation facility allows unattended processing of large numbers of data sets. The data sets can come from a text/CSV file or an Excel workbook. The graphs can be exported to an RTF file or streamed to Microsoft Word, while the processed data can be exported to an Excel (xlsx) workbook.
The Reset button restores the data to its state when first entering the procedure.
Residuals
The Residuals button opens a TableCurve 2D Graph containing the residuals from the fit. The residuals are the difference between the data and AR model fit. A good model should produce normally distributed residuals. The SNP plot is particularly useful for assessing if the errors in the fit have a Gaussian distribution. Note that if multiple orders are processed, the residuals for only the first of the orders fitted, the minimum order, are displayed.
Plot Selection Criteria
The Plot Selection Criteria
option will include all orders up to the maximum order specified for the Data
least-squares methods. These algorithms compute all lesser orders in the process of computing the target
order. The MDL
(minimum description length) is the most widely accepted AR order selection criterion. This option will
be grayed for the all of the SVD
procedures since these compute only the target order. The MDL
for the current model order is displayed in the dialog.
Plot Roots
The complex roots of the AR model can be inspected with the Plot
Roots option. Roots consisting of signal will rest on or
close to the unit circle while those corresponding with noise tend to be found in the interior of the
unit circle.
MS Word/RTF Export
The MS Word/RTF File Export
option is used to save the current graphs and reports to either a MS Word file or a portable RTF (Rich
Text Format) file. The graphs are inserted into the file as Windows metafiles.
Numeric Summary
The Numeric Summary
displays a basic summary of the analysis and optionally the predicted values, the AR coefficients, the
fit statistics, and singular values. The content is set using the Numeric Summary's Format
menu.
The Numeric Summary's File menu has a Save Predicted Values as XY File option to save only the forecast points to file. There is also a Save Coefficients as Filter File option which allows filters to be saved to disk as later imported as algorithms to be applied to a different data set.
Code Generation
The Code Generation option writes the AR filter to disk as program code, in your choice of twelve languages: C (64-bit or 80-bit), Fortran 77, Fortran 90, Python, Pascal, C++, R, MATLAB, Maple, Java, and VBA for Excel. Since an AR filter must process a discrete data stream, the current data are also saved alongside the code, in an ASCII series (SER) file: the first line holds the count of data values, followed by one value per line.
The Order Notice
If Single Order is unchecked and the order range spans more than one order, the plot shows a family of AR models: over the data, the fitted curve is the average of all the models; in the prediction band, each model's forecast is drawn separately, labeled with its order.
Code Generation exports only one model: the first, lowest order in the range. Before writing any files, a notice explains this and gives you the chance to cancel. To export a specific AR model instead of the lowest order, check Single Order and select that order before pressing Code Generation. With Single Order checked, no notice appears and the exported model is exactly the one on screen.
The Filter Code Generation Dialog
Select a language from the radio-button list, a Type (Full Test Code or Function Code Only), and a Filter Function Name. The dialog opens with the last language used, a choice shared with the 2D Review's own Code Generation.
Full Test Code writes two files: the source file (the filter and prediction functions plus a small test program), and a series file next to it with the same name and an SER extension. The test program reads the SER file and writes ARTEST.TXT, containing the root-mean-square error, the estimated series (index, data, estimate, residual), and the forward and backward predicted values. These are the same formulas TableCurve Studio uses on screen.
Function Code Only writes just the coefficients and the filter and prediction functions, with no test program. VBA is always Function Code Only, since its output is three Excel worksheet functions rather than a standalone program.
Filter Function Name defaults to Filter. For VBA the default becomes AR_Filter, because FILTER is a built-in Excel 365 worksheet function and a user-entered =Filter() would never reach the generated one. A VBA name that is also a cell reference, such as AB12, is refused with an explanation, since Excel could not call it.
What Each Language Generates
C, C++, Fortran, and Pascal write the filter function, the forward and backward prediction routines, and the coefficient table.
Python — Filter(a) fills a with the coefficients and returns the order; ar_filter(n, data, ip, a) returns the mean-square error and the estimated series; ar_predict(n, data, ip, a, npred, fwd) forecasts forward or backward.
R — Filter() returns list(order=, coef=); ar_filter(...) and ar_predict(...) work as in Python. The test program runs under Rscript.
MATLAB — one call, [a,ip,est,res,fwd,bwd] = Filter(ser,npred), returns the coefficients and order, the estimated series and residuals, and npred forward and backward predictions. The whole-series filter uses MATLAB's built-in filter(). MATLAB code also runs in GNU Octave.
Maple — Filter() returns the coefficient list; ar_filter(ser, a) returns the estimated series; ar_predict(ser, a, npred, fwd) forecasts.
Java — class Filter has Order() and Coef() for the order and coefficients, ArFilter(x, a, y) to fill the estimated series and return the mean-square error, and ArPredict(x, a, npred, y, fwd) to forecast. Full Test Code adds a class FilterTest with main(); compile with javac Filter.java, then run java FilterTest.
VBA (Excel) — three worksheet functions: =AR_Filter(B2:B41), the one-step-ahead forecast from the last ip cells of the range, oldest first; =AR_FilterCoef(k), coefficient k; and =AR_FilterOrder(), the order. Import the generated code with File > Import in the Excel VBA editor.
Every language carries up to 500 coefficients, the AR order limit.
Import Filters
The Import Filter from Disk
is used to import filters that have been previously saved for processing the current data set. You may
simultaneously select as many filter (FLT) files as you like. After import, all of the filters will be
available in the algorithm list. They are listed by file name, so it is important to save filters with
meaningful file names. Imported filters are always treated as both forward and backward predictors, regardless
of how they were originally generated.
Save Filter to Disk
The Save Filter to Disk
option saves the current filter coefficients to a filter (FLT) file. This is a binary file that can only
be created within TableCurve 2D.
r² Value
Because the AR spectrum is a fitting procedure, the r² correlation coefficient, F-statistic, and SE (standard error) are shown in informational fields. An r² of 1 represents a perfect fit whereas an r² of 0 represents no fit at all. A high r² (0.95+) is needed for good prediction accuracy. The r² value is based on a forward prediction filter with partial backward prediction. This AR filter is defined using backward prediction from the model order down to the initial data element, and forward prediction from the model order up to the final data element. Although this simplifies the goodness of fit statistics (a single estimate is made for each of the input data elements), the value will be less meaningful for backward predictions.
Automatic Update
The List, Selection Criteria, Complex Roots, Residuals, and Numeric Summary windows are automatically updated when any change is made to the algorithm's settings.
Updating the Data Table
When exiting this procedure with the OK
button, an option will be presented to update TableCurve 2D's main data table with the output data. If
multiple orders were processed, this will be the average.
If Background Thread Fitting is active, the fitting will be initiated as soon as the predicted data are accepted and rescanned.