What's New in v5¶

1. State-of-the-Art Numeric Prediction
1. Automation
1. MS Excel and MS Word Enhancements
1. Derivative Enhancements
1. Denoising and Smoothing Enhancements
1. Reconstruction/Isolation of Signal Components
1. Improved Interpolation and Non-Parametric Estimation
1. Improved Evaluation Procedure
1. Multiple Curve-Fit References in a Single Graph
1. Sort by Maximum Residual
1. RTF Windows
1. Mark and Refit
1. Data Table Reset
1. Improved Residuals
1. New Connect Options in Graphs
1. Differencing, Cumulatives, Normalizations
1. Generating Data
1. Importing Equation References as Data Sources
1. More UDF Fn#s
1. XY Sectioning
1. Preferences
1. Additional Mouse Functionality
1. Automatic Window Placement in Review
1. Revisions to Toolbars and Menus
1. On-Line Documentation
1. Key Features and Enhancements Added to v4

1. State-of-the-Art Numeric Prediction¶

Unless a theoretically appropriate model is being fitted, extrapolations or predictions using TC2D model equations are often of limited utility. TC2D v5 offers state of the art AR (Autoregressive) procedures that offer the means to effectively forward or backward predict many data sets, including those with appreciable levels of noise.

The AR Modeling and Prediction procedure in the Estimate menu offers effective autoregressive forecasting and extrapolation. The AR algorithms include SVD (singular value decomposition) procedures for in-place noise removal. Multiple orders can be simultaneously plotted and stabilizations are available for roots that lie outside the unit circle. The points that are to be processed can be specified, allowing predictions based on a data segment to be compared with subsequent data. The extent of the prediction is variable and white noise can be added to test the robustness of an algorithm's prediction.

One of the weaknesses of traditional AR prediction methods is their susceptibility to noise. Generally, even a small measure of random noise is sufficient to render predictions unreliable or meaningless. TC2D v5 solves this problem by offering nine different procedures (three to predict ahead, three to predict earlier data, and three that predict in both directions). Of the three major least-squares algorithms included in v5, two offer in-place noise removal using advanced SVD and eigendecomposition methods.

TC2D’s Autoregression option offers the means to view complex roots, order selection criteria, and singular values in order to optimize the AR model order and other algorithm parameters. Residuals, numeric summaries, and both numeric and filter export are available. TC2D’s AR procedure is also unique in that multiple AR orders can be simultaneously graphed and displayed.

AR filter code generation is supported in all of TC2D’s supported languages. You can also save a filter to disk that is treated as a separate AR algorithm. It can be imported and applied to any data set desired.

2. Automation¶

While TC2D is designed to be interactive, there are those instances where a large number of data sets need to be processed using only one or two different models.

For these data sets, TC2D v5 offers an integrated automation that enables large numbers of data sets to be processed in an unattended fashion.

The primary automation uses MS Excel as the data source. If they are not already there, you simply place all your data sets in an Excel spreadsheet. These multiple sets are then processed with the ease of a single set. The automation output can be written to an MS Word (or generic RTF) file for all graphs and numeric summaries, and to MS Excel for numeric data. It is a simple matter to specify one sheet in the spreadsheet or the full file. Import from MS Excel 95, 97, and 2000 is supported. Export to MS Excel 95, 97, 2000 and MS Word 95, 97, and 2000 are supported. If you do not use MS Word, the exported reports can use a generic RTF format that can be read by any word processor, including Wordpad (ships with Windows).

Automation is integrated into each of TC2D’s major procedures. It is thus possible to denoise 100 data sets in one unattended step, and then fit these sets to a target equation in a second unattended step. Since Excel is used for the output of the automation, these files can be used as the input to any other TC2D procedure or in other programs that support MS Excel.

TC2D’s automation includes DLL support for those who want to write an external acquisition interface. TC2D v5 even includes a professional DLL automation interface for instrument manufacturers that want TC2D’s functionality available within their own software.

3. MS Excel and MS Word Enhancements¶

In addition to supporting all releases of Excel through 2000 for data import, data can now be saved from any procedure in an Excel 95 or Excel 97/2000 format for import by TC2D, for use in Excel, or in some other application.

Most procedures offer a new MS Word or RTF export. The MS Word export is a streamed export that automatically opens MS Word and transfers the graphs and reports directly, all with a single button click. The graphs can be half or full page, and a number of controls are offered including landscape or portrait mode. In fact, the file-based (RTF) export allows you to export a mixture of landscape graphs and portrait reports, all in a single document.

4. Derivative Enhancements¶

An exceedingly small measure of noise can play havoc with derivatives. That is one reason why fitting an effective parametric model is a good way to achieve a smooth derivative. Through v4, however, TC2D offered only numeric derivatives for its 3491 linear equations and its 174 built-in non-linear equations. TC2D v5 now offers analytic derivatives, both first and second, for all built-in equations. Analytic Derivatives are set in the Preferences option.

Although Savitzky-Golay smoothing has been long a part of TC2D's automated smoothing, v5 now offers a separate Savitzky-Golay Smoothing procedure. This is probably the best non-parametric approach for derivatives. A much greater control of the algorithm is now offered. First through eighth derivatives are available, SG model orders 2 through 12 can be selected, the filter pass count is adjustable, and an option has been added to manage edge effects. The option now includes residuals and a numeric summary includes the smoothing coefficients.

For interpolation of a smooth derivative, there is also a new Savitzky-Golay Spline Estimation procedure. A constrained cubic spline is automatically used in this option to interpolate SG processed data.

The Spline Estimation and Local Regression Estimation options now offer derivatives through order 4 for all algorithms.

The separate Inspect Derivative options that were present in v4 have been removed.

5. Denoising and Smoothing Enhancements¶

While the Fourier filtering and the various smoothing procedures in v4 were effective in noise removal, TC2D v5 goes several steps further in offering state of the art denoising.

Completely new to v5 is Eigendecomposition Denoising. This is a non-parametric procedure where the highest ranked eigenmodes capture the deterministic component of data (signal) where the least-ranked eigenmodes contain the stochastic (noise) component. Unlike the Fourier filtering, where the separation is based on frequency content, this method separates based on the strength of the components. When data sizes allow a sufficiently high order eigendecomposition, it is possible to filter virtually all noise from a given data set without distorting the underlying signal. This is the ultimate in noise removal.

TC2D v5 now includes a separate Fourier Denoising procedure. This is a simplified Fourier domain editing option designed exclusively for one-click noise removal. New to v5 is the option to apply a data taper and to filter based on either the magnitude of the FFT or the frequency thresholding previously offered. Exact n FFTs are now used to maximize the accuracy of the transform. Export now includes a much expanded FFT data summary and a new FFT numeric summary that includes a linear multi-sinusoidal fit that further optimizes the spectral information in the FFT. Up to 25 sine components are automatically least-squares fitted. The FFT can now be displayed in a variety of power, amplitude, dB, and other formats.

The Automated Smoothing procedure has been updated. The Eigendecomposition procedure has been added as well as an effective Kaiser-Bessel time-domain smoothing filter algorithm. Newly integrated is automatic spline interpolation for algorithms that require uniformly spaced data. Derivatives are now available for all of the smoothing procedures and are based upon a B-spline interpolant.

6. Reconstruction/Isolation of Signal Components¶

Eigenfiltering is useful for much more than noise reduction. Often a data set is a complicated mixture of a number of individual components that arise from a variety of sources. TC2D v5 includes a new Eigendecomposition Filtering procedure that makes it possible to isolate these discrete elements within a data stream. The separation is non-parametric and is often quite effective. While it is often most successful with oscillatory components of differing powers, mixtures of exponential decays can sometimes be effectively separated in this way.

When components have close to equal power, but separate spectral content, a Fourier domain filtration can often isolate the components. TC2D v5 offers an improved Fourier Filtering procedure. This is a full-power Fourier domain editor that represents a significant enhancement over the functionality in v4. Exact n FFTs, a selection of data tapers useful for reconstruction, and full graphical isolation of components have been added.

7. Improved Interpolation and Non-Parametric Estimation¶

The Spline Estimation option now includes a user-defined least-squares B-spline fitting procedure (the knot sequence is imported from any supported file format).

For data with oscillatory properties, TC2D v5 offers a Fourier Estimation procedure that directly evaluates the individual Fourier components. Any data range and density can be specified.

The local regression procedure has been revised. The Local Regression Spline Estimation procedure now uses spline interpolation to insure continuous and smooth derivatives.

Whereas the Evaluation procedure was needed in v4 for all interpolations, TC2D v5 now offers the means to directly specify and plot interpolated output. For this reason, the two Digital Filter options present in v4 have been removed. The Spline Estimation and Local Regression Spline Estimation now offer this same functionality.

8. Improved Evaluation Procedure¶

The Evaluation option now uses a single sorted list window. The columns can be sorted by x or y values, or by the magnitude of confidence interval. The numeric precision is adjustable. All roots are now done in the table generation procedure and the Direct to File option now offers all file formats. It is possible to generate a table by specifying either the sample increment or count; previous values are saved across sessions. Evaluation export is now spreadsheet-tailored to manage the different kinds of evaluations.

You can now save and recall evaluation sequences. Further, evaluation sequences are now automatically updated whenever an equation or algorithm setting changes.

9. Multiple Curve-Fit References in a Single Graph¶

In the Curve-Fit review, it is now possible to specify up to four Equation References in a given curve-fit plot. These can be other equations in the current fit, or equations saved to disk from prior fits. This will enable more extensive graphical comparisons between the various equations.

For comparing data rather than equations, the Compare with Reference option has been extended to also allow up to four data references to be compared with the current data set. References can now be from any data source. These can also include curve-fit references (equations saved to disk). For an equation reference, discrete data is generated.

10. Sort by Maximum Residual¶

For certain applications, the desired equation is one that minimizes the maximum error. To make this type of fitting easy, TC2D v5 includes a Sort by Maximum Error option for the Equation List. The maximum absolute error has also been added to the Numeric Summary.

11. RTF Windows¶

RTF Text windows can now be specified. This option is set in the Preferences. With RTF windows, you can now highlight and copy any part of a report for pasting into a different application. The faster display windows used in v4 are also available and are automatically used for large table sizes.

12. Mark and Refit¶

In v4, latent point processing made it possible to tag points (usually outliers) in the Review that would be rendered inactive when returning to the main menu. The data could then be refitted absent the influence of these points. This has been replaced in v5 by a direct Mark and Refit option. This option makes it possible to exclude points based outside a specified multiple of the fit standard error or outside a specified prediction intervals. Points can also be selected manually. The data can now be refitted within the Review. All equations are refit and all open windows are updated. Marked points are toggled on and off with successive refits, enabling the impact of excluding points to be more readily appraised.

13. Data Table Reset¶

In prior versions, once the data table was modified, the original data could only be restored from file. There is now a Reset XY Data button in the main toolbar and in the Data menu.This makes it possible to denoise or filter a data set and recover the raw data without reimporting it.

14. Improved Residuals¶

The Residuals procedure has been enhanced. It is now offered for most TC2D procedures rather than only in the Curve-Fit Review. A numeric window can now be displayed and is automatically updated. The histogram can now be exported.

15. New Connect Options in Graphs¶

All graphs containing discrete data now offer the option of connecting the data points with either a cubic spline or a cubic NURBS, in addition to a simple line. By default, imported data are now rendered with connected lines.

16. Differencing, Cumulatives, Normalizations¶

A Difference, Cumulative, Normalize procedure has been added. This puts most of the basic data manipulations operations in one place. Many of the operations are new to v5.

17. Generating Data¶

Prior to v5, data could be generated only using one-line View Function(X) expressions and by applying one-line calculations to the data table. The Edit menu now offers a Generate Data option. Apart from the absence of adjustable parameters, the format is similar to UDFs. You can thus generate a data set from complicated expressions

18. Importing Equation References as Data Sources¶

It is now possible to import references saved from TC2D v5 as a data set. You must specify starting and ending values as well as either an increment or data count. Data are then generated and imported.

19. More UDF Fn#s¶

For complicated UDFs, additional Fn#s can now be used. The allowed F1 through F9 function assignment in UDFs has been extended to F1 through F24.

20. XY Sectioning¶

For sectioning, there is a new XY sectioning mode. Here the mouse encloses a region in XY space that will either be included or excluded in the sectioning.

21. Preferences¶

TC2D v4 offered separate Review Preferences and General Preferences options. TC2D v5 combines these into a single Preferences option available from either the Edit menu or the File menu of the Curve-Fit graph.

22. Additional Mouse Functionality¶

If the Curve-Fit graph is the active window, you can use the rotating wheel on a wheel mouse to control movement through the equation list. Rotate the wheel forward (away from you) to advance up the equation list and back (toward you) to step down the equation list. You can also single click on the middle button of a three-button mouse (or depress the wheel of a wheel mouse) to advance or go back one equation, depending on the current direction through the list. The equation list window can be toggled on and off by double clicking the middle button or wheel.

23. Automatic Window Placement in Review¶

The Curve-Fit graph's control panel (or Window menu) now offers six predefined tile options to automatically open and place various Review windows.

The Window menu also offers options to save and import the window state (the windows currently open and their positions).

24. Revisions to Toolbars and Menus¶

The main toolbar and process toolbar can now be toggled on and off separately. This toggle has been moved to the Edit menu. The main toolbar now contains additional smoothing, denoising, and filtering options. The estimation options have been moved to the process toolbar. Some of the button images associated with denoising, smoothing, filtering, and estimation have been revised.

The Calculate, Table, View, and Non-Parametric menus have been replaced with Data, Filter, and Estimate menus. The items that modify the data table, but do not smooth, denoise, or filter the data are in the Data menu. The procedures that smooth, denoise, and filter the data are in the Filter menu. The items that offer interpolation and prediction are in the Estimate menu.

25. On-Line Documentation¶

Previous versions of TC2D offered both on-line help and a printed user's manual. Due mainly to memory considerations, the on-line help had been less detailed and lacked typeset equations, descriptions and graphs of the individual non-linear equations, quick-start information, tutorials, and other reference information.

With v5, all information is fully electronic. The on-line help system has been vastly expanded and now includes all reference information previously available only in the printed documentation. As a plus, the on-line help system exclusively uses color graphs to better illustrate the influence of parameters, intervals, residuals, and other aspects of fitting.

The tutorials are also electronic and have been increased in number. Several cover the new or expanded features in v5. These can be accessed directly from the Help menu.

Also included in the v5 distribution is the Adobe Acrobat reader and several key PDF documents. These include a document that lists all linear equations, another that details each non-linear model, and the curve-fitting pitfalls document. These have been reformatted in a standard letter size and use color graphs.