Techinal Note: Estimating Parameters of Nonlinear Segmented Models

Nonlinear models with a change point or lag are also called segmented models and are often used to describe fiber digestion as well as other biological processes. An example of this type of model is:

A model of this type could be used to describe fiber remaining at time t of fermentation, where B1 = available fiber for digestion, B2 = rate of fiber digestion (t⁻¹), B3 = lag time or change point (t), and B4 = the indigestible fraction. The data can be described by n points (P₁…P_n) and may be unequally spaced. The two coordinates of the th point are described by P_i = (t_i, d_i), where t_i is the measured time point and d_i is the measured value or fiber remaining at the th time.

The problem is how to partition the observation points between equation [1] and [2], for a given data set, to estimate B1, B2, B3, and B4 such that

is minimized, where t_k ≤ B3 < t_{k + 1}.

The segmented model (equation [1] and [2]) is continuous but not smooth. Continuity implies equation [1] and [2] have the same solution when t equals B3 (f₁[B3] = B1 + B4 and f₂[B3] = B1 + B4). Smoothness implies the first derivatives of equation [1] and [2] with respect to t are identical when t equals B3 (df₁[t]/dt = 0, and df₂[t]/dt = −B2*B1*exp[−B2*(t − B3)] = −B2*B1). The parameters can be estimated with statistical packages that have a nonlinear estimation procedure, such as SAS (SAS Inst., Inc., Cary, NC). The procedure used in SAS is PROC NLIN, and this procedure estimates parameters from segmented models that are continuous and smooth. Although the segmented model described by equation [1] and [2] does not meet the criteria for segmented models in SAS, this procedure is widely used. Minimization algorithms are derived under the assumption that the first partial derivative of residual sums of squares, with respect to the parameters, is smooth and continuous.

Selection of good starting values is important to obtain good nonlinear regression solutions. According to , “A poor selection of initial values may have several consequences: 1) in a well-behaved system, the amount of computer time required to reach a solution will be increased, but the solution will be correct; 2) in other situations, poorly selected initial values can lead the nonlinear regression program to go in the wrong direction or never converge on a solution; 3) it is also possible that poorly selected initial values can cause the program to converge on the wrong solution, a local minimum. The choice of initial values is less important with equations containing only one or a few parameters than with equations containing many parameters.” If case 3 occurs, the experimenter can usually suspect a wrong solution because the coefficient of determination is low or the standard errors of the estimates are high, but these evaluations do not always uncover wrong solutions. Problems of identifiability and ill-conditioning have been summarized by . Initial starting values generally are very important, and especially so with segmented models. Good overviews of nonlinear regression can be found in and . An applied SAS technique is not available that allows the selection of good starting values in a systematic, reliable procedure. However, transition functions have also been used to solve the change point problem ().

The objective of this article is to illustrate a technique to find good starting values around a local minimum that is lower or equal to the local minimum found using traditional methods. This technique will be illustrated by using a specific nonlinear segmented model and should work with other nonlinear segmented models.

Much has been written about change point problems (; ), but solutions to applied problems using statistical packages are limited. One solution to this problem is to fix B3 (lag) across a range of reasonable B3 values and solve for B1, B2, and B4. Then, the solution set with the lowest residual sums of squares is selected and the fixed B3 and the estimates of B1, B2, and B4 are used as starting values for the final solution. The selection of the reasonable range should be dictated somewhat by the experimenter's knowledge of the system. The number of fixed B3 to use was selected to be every 0.1 units from 0 to 8. However, an incremental unit of 0.1 was selected arbitrarily and the exact increment to use is more of a mathematical problem and is outside the range of this technical note. The advantage of this modified approach is that one should achieve a better solution than by selecting reasonable starting values for all parameters. The datum used for this technical note is a subset of data from and is given in Table 1. A traditional SAS program with a reasonable starting grid is under “Parameters” in Appendix 1. A modified SAS program to estimate the parameters is in Appendix 2. The modified SAS program generates a series of datasets using a different fixed lag or B3 for each dataset, and then B1, B2, and B4 are estimated for each fixed B3. The parameters estimated from the solution with the lowest residual sums of squares and the fixed B3 are used as the starting values for the final analysis.

Figure 1 was generated from the first PROC NLIN in Appendix 2 by using selected fixed B3 values near the two minima. Figure 1 shows one local minimum at about B3 = 3.5, and another local minimum at about B3 = 4.5. It is hoped that the local minimum at B3 = 3.5 is the global minimum, but the modified technique cannot guarantee the residual sums of squares is a global minimum. The parameters estimated (B1, B2, and B4) are assumed to approach their optimized values at each fixed B3. However, the parameter estimates might not be optimum, and the experimenter needs to be aware of other techniques to test their models, as described elsewhere (; ). Table 2 contains information based on the results from Appendix 1 and from the second PROC NLIN in Appendix 2. The results in Table 2 show the square root of mean square error for the traditional and modified techniques are 0.01820 and 0.01798, respectively. The estimates of B2 (rate of digestion) and B3 (lag) using the traditional analysis may lead to erroneous interpretation of the datum. The estimates of B2 and B3 are 0.062 and 4.5 for the traditional analysis compared with 0.056 and 3.5 for the modified analysis. Not all datasets will have more than one local minimum, but the technique described in this paper provides another method in an attempt to reach a lower local minimum than that found by using a traditional approach in selecting starting values. The frequency of other datasets that would benefit by a lower root mean square error from the application of this modified technique is not known, and a careful analysis to provide relevant statistics is outside the scope of this technical note. However, 20 datasets were generated from the dataset in Table 1 using various standard deviations, and the modified procedure worked in every case and provided lower or equal residual sums of squares compared with the traditional procedure. The experimenter should consider using the technique illustrated in this manuscript if the parameter estimates do not seem to make biological sense or do not compare with other replicates. Additionally, this modified technique should be used if one is interested in estimates of the rate constant (B2) and the lag (B3). If one is interested in the mean retention time primarily, then the traditional and modified techniques may provide similar results. For example, in this article, the percentage change from the traditional to the modified technique is about 10% for B2 and 22% for B3. However, the percentage change is only 3.4% for the mean retention time (1/B2 + B3). Although the technique developed in this article applies to SAS PROC NLIN, it can be modified to work with other statistical packages.