Let be a function of multivariate variables . Let and assume that . Suppose that we model using an SS ANOVA decomposition. Specifically, we assume with given in ().

A semi-parametric linear mixed-effects (SLM) model assumes that

where , , are design points, is the design matrix for some fixed effects with parameters , is the design matrix for the random effects , , are random errors which are independent of and . The covariance matrices and are assumed to depend on a parsimonious set of covariance parameters . Regarding the fixed effects part as a partial spline, model () is essentially the same as the non-parametric mixed-effects model introduced in Wang (1998a).

For model (), the marginal distribution of
is
where
.
Given
and
,
we estimate fixed parameters and
as the minimizers of the following penalized weighted least
squares

Denote the estimates as and . We estimate as the posterior mean , where .

Again, the solution of has the form ().
Similar to Section 2.4, we use connections between a SLM and a
LMM to estimate
and
.
Consider the following LMM

where 's and 's are defined in (), , and , and are mutually independent. Then the GML estimates of and in () are the REML estimates of the variance components in () (Opsomer et al., 2001; Wang, 1998a). As in Section 2.4, we use