User-Defined Functions

You may install up to fifteen non-linear User-Defined Functions (UDFs) into TableCurve's equation set at any given time. This User-Defined Functions option is used as the control center for entering, editing, saving, and fitting UDFs. From this UDF dialog, you can work with individual functions or with UDF libraries. A TableCurve UDF contains all information necessary to fit the function, including the function name, the parameter count, the function's formula, and starting estimates and constraints for each parameter. A special Adjust item allows you to graphically adjust the starting estimates to better assure a successfully converged fit. You may also inspect the partial derivatives for the UDF to find instances of multiple constants, insignificant parameters, and to expose conditions where a fit would be likely to fail.

UDF Selection

The keypad with buttons labeled 1 through 15 is used to select the current UDF. In TableCurve's non-linear equation set, UDF#1 is Equation 8001, UDF#2 is Equation 8009, and UDF#3 through UDF#15 are Equations 8017-8029. UDFs active at any given time need not be sequential and they will remain in the position where you install them, even if lower number UDFs are empty.

Function Name

This name will be used to represent the function in the curve-fit graphs. You will wish to select a name that is meaningful to you.

Coefficient Count

This is the number of adjustable parameters in the UDF model. The UDF must contain the number of parameters entered. If, for example, you enter 4 as the coefficient count, the UDF must contain parameters #A,#B,#C,#D (or A0,A1,A2,A3). The maximum coefficient count is 10.

Entering the UDF

The UDF entry uses a simple ASCII multiline editor. You can use the Cut, Copy, and Paste items to move text about or to paste in the UDF formula if you placed it into the clipboard via another program. All of the functions and constants available within TableCurve can be accessed via a special Function Insert help. In a UDF no prefixes are automatically supplied.

You may define as many constants as you wish. For example, SQRT2PI=SQRT(2*PI) would be defined on one line and used in subsequent lines. Constant expressions are evaluated once and the numeric result is stored. Any assignment to a variable other than F1-F24 (or #F1-#F24) and Y is assumed to be a constant.

Any expression containing X or any of the adjustable parameters #A-#J (or A0-A9) must be assigned either to an F1-F24 expression or to Y. F1-F24 and Y expressions are compiled and are evaluated once for each data point in every iteration. The Y expression must always be the last line in the UDF. Here is a simple UDF example, first with the #A-#J nomenclature and then with the A0-A9 format:

S2=SQRT(2)

F1=ERFC(-#D/S2+LN(X/#C)/(#D*S2))

Y=#A+0.5*#B*F1

S2=SQRT(2)

F1=ERFC(-A3/S2+LN(X/A2)/(A3*S2))

Y=A0+0.5*A1*F1

The S2 expression is a constant that is evaluated numerically and stored for use in subsequent expressions. The F1 expression calls the complementary error function with an argument expression containing the third (#C or A2) and fourth (#D or A3) adjustable parameters. The last line is the required Y= function expression. It contains the first (#A or A0) and second (#B or A1) adjustable parameters. The F1 and Y expressions are compiled. In the fit, these expressions are evaluated at the X of every data point for each iteration's particular adjustable parameters.

UDFs Based on a TableCurve Built-in Equation

It is not necessary to manually enter any of TableCurve's built-in equations in a UDF. TableCurve can Generate UDFs for built-in equations with 10 or fewer parameters. The Save TableCurve UDF option found in the File menu of the Review Curve-Fit graph is used to generate such UDFs. A generated UDF will consist of the full mathematical equation expression rather than a reference to a built-in function. The estimates in the generated UDFs will consist of the values of the actual fitted parameters. The following generated UDF expression is an example for the Weibull function.

F1=(X-#B)/#C

F2=(#D-1.0)/#D

Y=#A*EXP(-(F1+F2^(1.0/#D))^#D+F2)*F2^(-F2)*(F1+F2^(1.0/#D))^(#D-1.0)

Directly Accessing TableCurve's Non-Linear Functions

For non-linear equations, generated UDFs will not execute as fast as the built-in functions available via the Non-Lin function insert help in the UDF entry dialog. The built-in non-linear functions also contain the conditional code insuring that the functions will return a valid value for all X whereas the generated UDFs will contain only the native equation without conditional statements. Note that the built-in non-linear functions available in UDFs are zero-intercept versions when both a zero intercept and intercept form exists. In UDFs where you use a built-in non-linear function, you will have to add an intercept term if such is desired. The following UDF expression is the equivalent of the above generated Weibull function.

Y=WEIBULL_(A0,A1,A2,A3)

Indirect Function References

The following functions utilize an indirect reference for the initial argument:

DX(n)

DX2(n)

AI(n,st,end)

AIP(n,st,end,prec)

QIP(n,st,end,prec)

SUM(n,st,end,inc)

SER(n,st,inc,lim)

PROD(n,st,end,inc)

IMPLICIT(n,yl,yh,prec)

In versions prior to v4.02, n had to be specified as an integer from 1 to 9. TableCurve 2D now supports entering Fn and #Fn in the initial argument position. For example, if F3 were to be processed by any one of these functions, the first argument could be 3, F3, or #F3.

UDFs with Derivative Functions

The first derivative function is DX(n) where n is from 1 to 9 and references an F1 to F9 expression. The second derivative function is DX2(n). The following example is for the first derivative of a Gaussian:

F1=A0*EXP(-0.5*((X-A1)/A2)^2)

Y=DX(1)

UDFs with Integration Functions

The primary integration function is AIP(n,start,end,prec). It first seeks to achieve the target precision using a successive step Gaussian Quadrature. If this is unsuccessful, a Romberg procedure follows. The following example uses the AIP() function to fit the cumulative of the Log-Normal distribution:

UPPER=40.0

F1=LN($/#C)/#D

F2=EXP(-0.5*F1*F1)

Y=#A+#B*AIP(2,X,UPPER,1E-6)

Note that the $ symbol is used as the variable of integration. In this example, the second function expression is integrated with $ ranging from X to this constant upper limit of 40 to a precision of 6 significant figures.

If you wish to limit the integration only to the much faster Gaussian Quadrature procedure, the QIP function can be used in place of AIP. Both functions return the integration for the best achieved precision when the target precision is not achieved. For the Gaussian Quadrature this will be a 128 step procedure. For the Romberg, this will be either 131072 or 177147 steps, depending on the type of integral.

UDFs with Implicit Functions

The IMPLICIT function is used to create models that must be implicitly solved for Y. The following UDF example is for the Michaelis-Menten model (MICHMENT.UDL):

F1=IF((Y.EQ.DATAERR).OR.(Y.LE.0),0,A0+A2*LN(A0)-A1*X-A2*LN(Y)-Y)

Y=IMPLICIT(F1,YMIN,YMAX,1E-8)

The following example is for the basic ligand binding model (LIGAND.UDL):

F1=10^X

F2=((A0*A1/(1+A0*(F1-Y*F1))+A2)*(F1-Y*F1))/F1-Y

Y=IMPLICIT(F2,YMIN,YMAX,1E-8)

Note that the Y= is not a part of the expression to be solved. The Fn is implicitly solved for Y assuming the Fn evaluates to 0. If your implicit function is of the form y=f(x,y) or 1=f(x,y), you must be sure to subtract Y or 1 from the Fn expression.

Validation

A UDF is extensively validated before it is compiled. If there is a math or parser error, you will be given a clear indication of the error and the cursor will be placed at the location where the validation failed. If you are having trouble with a UDF, you may save the failed UDF to disk. You can then recall it at some future time in an effort to fix what is wrong.

Starting Estimates and Constraints

You must enter starting estimates for the parameters in the model. Good starting estimates are sometimes necessary for convergence, and usually result in faster non-linear fits. Minimum and maximum constraints are optional. If a parameter violates a constraint that you set, a penalty function is added to the chi-square to attempt to bring the parameter back into a valid range.

Entering Formulas for Estimates and Constraints

You may enter formulas for the estimates and lower and upper limits in UDFs. These enable a UDF to be constructed so as to accommodate widely varying X and Y data ranges. Since these formulas are evaluated sequentially, any given formula can reference a previous estimate. For example, #D can reference #A, #B, or #C, but not #E and higher. More commonly, a UDF estimate formula will reference an X-Y data table constant which TableCurve computes for each data set. These can consist of the basic X-Y data table constants, such as XMEAN, XSTD, YRANGE, etc. as well additional constants used specifically for UDFs.

For peak functions, these constants include:

XCTR, X at Peak Center

XL50, X at Half-Maxima Left

XR50, X at Half-Maxima Right

XW50, X Width at Half-Maxima (FWHM)

For transition functions, the following constants are available:

X50, X at Ymin+Yrange/2

X25, X at Ymin+Yrange/4

X75, X at Ymin+3*Yrange/4

XWTR, X transition width X75-X25

For waveform functions, there are:

XWL, wavelength

XPH, phase for sine

XPH2, phase for sine-squared wave

When a UDF contains one or more formulas for the adjustable parameter estimates and limits, it is saved in a binary rather than ASCII form. These UDFs can only be edited inside of TableCurve 2D. The [.UDF] extension is used for both the binary and ASCII formats.

Reading a UDF

Use the Read button to read a UDF from disk. The UDF will be read into the current UDF position, replacing any UDF that might currently be in this same position. You may read any UDF into any of the 15 available positions, even if lower numbered positions are empty. This option will read both the ASCII format UDFs and the binary format UDFs containing one or more formulas for estimates or constraints. It is recommended that UDFs be created only within the program.

Clearing a UDF

Use the Clear Current UDF button to immediately free the currently selected UDF from memory and to reset the UDF entry screen.

Saving a UDF

Use the Save button to save a UDF to disk. If the estimates and constraints are numeric, the UDF is saved in an ASCII format with a [UDF] extension. If formulas are present, the UDF is saved in a binary format that requires approximately 4K of disk space. A UDF is always validated before a Save is made. If the validation fails, you are given an option to save the UDF even though it contains an error. You can then recall it at some future time in an effort to repair what is wrong.

Saving a UDF Library

To make it easy to work with up to 15 user functions, any set of installed UDFs can be saved as a user function library. UDF positions are preserved as you enter them and installation of empty UDF positions are permitted. UDF libraries are saved as binary [UDL] files, whether or not estimates contain formulas. In a UDF library, each UDF consumes about 4K, so a full 15 UDF set will require about 60K disk space. To save the current set of UDFs to a library, use the Save UDF Library button. For maximum flexibility, you may wish to save individual UDFs as separate files in addition to having them in libraries. If you choose to keep UDFs only in libraries, you can save out individual UDF components simply by reading the UDL library, selecting the UDF of interest, and then using the individual Save option.

Reading a UDF Library

To read the user functions within a TableCurve UDF library, use the Read UDF Library button. This will install and validate all of the UDFs in the UDL library. If a validation fails, you are notified of such and that specific function is not installed. To extract a UDF from a library, use this option, select the UDF desired, and then use the individual Save option to create the UDF file.

Clearing All UDFs

To clear all of the currently installed UDFs from memory and to clear the current entry screen, use the Clear All UDFs button. You must confirm this option before this the UDFs are freed. Note that it is not necessary to clear the existing UDFs when reading a UDF library since all current UDFs are cleared prior to this read operation.

Copying UDFs

The Copy -> button is used to copy the current UDF to the next empty UDF position. This option is particularly helpful when you wish to create a UDF library consisting of similar models.

Immediate Fit

Use the Fit UDFs button to immediately fit the UDFs. This is the same as the Curve Fit User Functions item in the Process menu. Note that this option fits all installed UDFs. The fitting only occurs if the current UDF is successfully validated.

Parameter Contribution Warning

Even with a successful UDF compilation, you may still get the warning Parameter n Makes Less Than .1% Fractional Contribution to Equation in X-Range of Data. Adjustment is Recommended. TableCurve determines the minimum and maximum partial derivative for each parameter. This range of partial derivative is multiplied by the current estimate for that parameter and then compared to the overall Y-data range. If a parameter does not make at least a .1% contribution relative to the Y-data range, this warning is given.

This warning is often a good indicator that the fit will fail because of poor starting estimates. It may also indicate a poorly designed model that contains a parameter that minimally impacts the equation in the X-range of the data. When this message occurs, it is recommended that you use the Adjust procedure to refine the estimates, and if necessary, the Adjust procedure's Derivatives option to see if an insignificant parameter is present.

If You are Uncertain of Starting Estimates

When working with a new or exploratory model, you may not know what values constitute good starting estimates. In such a case, for each parameter you are uncertain of, simply enter 1.0 for the starting estimate (or some other value for which the UDF is defined). If you achieve a successful compile but get the parameter contribution warning, use the Adjust item to graphically set the estimates.

If you do not get the parameter contribution warning, there is a good chance the fitting algorithm may successfully converge. There is no harm in letting the fitting algorithm do the work. Use the Fit UDFs option to fit only the UDFs. If you are successful, you may wish to take note of the successful parameters and modify the UDF accordingly. If the fit fails, you must return to the UDF screen and use the Adjust feature to graphically set the starting estimates to better reflect the data.

To assist in this adjustment, you may with to use the Find and Update option. This performs a limited convergence fit with a subset of the data and automatically places the parameters in the starting estimate fields. In rare cases, this reduced data set fit can wander off into oblivion producing meaningless estimates. If this occurs simply use the Reset button or Cancel out of the Adjust dialog to discard the modified estimates. In most cases, the Find and Update should produce excellent estimates. If there is a revision of a parameter initially entered as a formula in the main UDF dialog, either by manual adjustment or via the Find and Update option, you must confirm that the original formula for the estimate is to be replaced with the new numeric value. Note that any save operations occurring after this replacement will use this numeric value.

Graphical Adjustment of Starting Estimates

Use the Adjust item to open a graph of the X-Y data and the UDF. Do not be surprised if you do not see the UDF at first. If the estimates are too far off, it is possible that no part of the UDF graph will be anywhere within the field of the X-Y data.

The goal of graphical adjustment is to set the parameters so that the UDF starts to approximate the data. You may adjust each parameter with the scrollbar or you may enter the actual value. If you enter an actual value, there will be a brief delay before the screen is updated, allowing you to complete your entry.

For each parameter that has user-specified minimum and maximum constraints, as opposed to the defaults, the scrollbar adjustment will range from this minimum to maximum in 100 steps.

If such constraints are absent, the scrollbar's sensitivity will depend on computed partial derivatives. The partial derivatives are updated with each value entered. If you enter a value for which the partial derivative range is very nearly zero, or if such is true when you begin, the scroll bar adjustment may produce very large jumps. In this case you must enter a value that brings the UDF back into range.

Fn Scaling

If the starting estimates are so far off that the UDF curve doesn’t appear with the data, you can open the Scaling option for the graph and change the Y priority to Fn. This should allow you to at see the actual UDF, provided it is defined within the X range of the data. Once you can adjust the parameters to bring the UDF into the Y range of the data, the Find and Update option can often refine the estimates automatically.

Find and Update

The Find and Update button in the Adjust screen initiates a low convergence fit using a subset of the current data table. The results of this very rapid pre-fit are automatically placed within the starting estimate fields. While this is often a very attractive means for refining the starting estimates, the algorithm is as vulnerable to local minima as the main non-linear fitting engine. In rare cases, this reduced data set fit can wander off into oblivion producing meaningless estimates. If this occurs simply use the Reset button or Cancel out of the Adjust dialog to discard the modified estimates.

UDF Failures

The two most common causes of UDF failures are having the UDF produce essentially a constant value across the X-range of the data, or to have a parameter set at such a value, that its impact totally masks the impact of X varying across the range of the data. You have to be especially cautious when using a UDF with a very narrow range of X-data.

As you adjust the starting estimates graphically, the UDF is being updated. When you return to the UDF screen, the estimates will reflect those set in the graphical procedure. By careful attention in the Adjust feature, no valid UDF should fail to produce the intended fit.

Inspecting Partial Derivatives

From the graphical UDF Adjustment screen, you can choose the Derivatives option. This displays a similar set of adjustment controls except instead of seeing a graph, you see a table of partial derivative information.

The % of Y column consists of the % of the Y data range represented by the range of the partial derivative multiplied by the estimate. The last column reports a relative percent based on all non-constant parameters, and these sum to 100%.

The fitting algorithm relies on changes in partial derivatives to guide the revisions in the parameters that occur with each iteration. If a partial derivative is constant, then it should have a value of 1.0, meaning that it is a true constant in the expression. If you see two constants, then at least one of these parameters should be removed. If you find that across wide adjustments of a given parameter there is no significant contribution to the Y-range of the data, whereas the others are making such a contribution, you should seriously question whether this parameter belongs in your model.

Note that you can set the estimate of a given parameter so far out of range that no legitimate partial derivatives are found for any parameter in the X range of the data.