In model () we have assumed that the function is
observed through *linear* operators 's plus random errors.
Sometimes the function is observed indirectly which involves
*nonlinear* operators
(O'Sullivan, 1990; O'Sullivan and Wahba, 1985; Wahba, 1990; O'Sullivan, 1991; Wahba, 1987).

We consider the following non-parametric nonlinear regression
(NNR) model

(30) |

where is a known function of in an arbitrary domain , is a vector of unknown non-parametric functions which act nonlinearly as parameters of the function , and are random errors distributed as . The functions 's could have the same or different domains. We denote the model space of as

Let
and
.
We estimate
as the minimizer of the following penalized weighted
least squares

where is the orthogonal projection operator of onto in .

In the following we consider the special case when

where is a known nonlinear function, 's are linear operators. () holds for most applications and 's are usually the evaluational functionals. When () does not hold, using linearization method, we can approximate by a linear combination of linear operators.

When () holds, the solutions to ()
have the form (). Specifically,

where are bases of , , and is the rk of . We estimate coefficients 's and 's using () with 's being replaced by (). Since in () is nonlinear, an iterative method has to be used to solve these coefficients. Two methods are used: the Gauss-Newton and Newton-Raphson procedures. See Ke and Wang (2002) for more details.