System Identification › Nonparametric Identification

Nonparametric methods refuse, up front, to fix a finite parameterization. Instead of asking "which of our candidate transfer functions is it?", they ask "what can the data tell us about the response function directly?" The question being dodged is the riskiest one in parametric practice: order selection, choosing how many poles, lags, or basis functions the model possesses. Guess too low and the model cannot represent the system regardless of optimization effort; guess too high and variance explodes with little improvement in fit. Nonparametric methods postpone or dissolve that commitment entirely, letting the data decide the effective complexity.

The oldest and most useful instance is the impulse response: the output sequence caused by a momentary unit input kick, $G(k) = C A^k B$ for a linear system in state-space form. With the convention $x_{t+1}=Ax_t+Bu_t$, $y_t=Cx_t$, zero initial state, and no direct feedthrough, the input-driven output is $y_{t+1}=\sum_{k=0}^{\infty}G(k)u_{t-k}$; equivalently, an unshifted impulse response places $CB$ at lag one and a direct-feedthrough term $D$ at lag zero. Absolute summability of the impulse-response weights is the input-output, or BIBO, stability condition that permits finite truncation with a bounded tail. For a minimal realization this agrees with internal asymptotic stability, but a nonminimal realization can hide unstable unobservable or uncontrollable modes. In practice the discarded tail is not literally zero: it is accepted only after its bound or estimated contribution falls below a declared numerical or noise tolerance. Visible unstable modes do not decay and therefore defeat this truncation argument. Estimating $G(k)$ from data is regression under linearity and a justified finite-horizon approximation: stack lagged inputs into the regressor and let least squares fill in the weights. Two properties follow immediately. Each tap is estimated independently, so a bad estimate at one lag says nothing about neighbors, and no finite state-space order has been imposed. The frequency response is the Fourier transform of the impulse response, the object engineers have long probed by driving systems with swept sinusoids and reading amplitudes and phases off instruments. Frequency-domain plotting carries real diagnostic power: resonant peaks mark lightly damped modes, phase crossings flag stability margins, and rolls in magnitude reveal bandwidth. Impulse- and frequency-domain estimators remain industrial defaults precisely because they answer "what does the system do?" without first demanding "what kind of system is it?"

The modern incarnation regularizes the impulse response toward smoothness. Under open-loop exogeneity, raw finite-impulse-response least squares is unbiased when the chosen truncation contains the true response, or when omitted-tail effects are orthogonal to the included regressors; otherwise truncation itself can introduce bias. Even in the favorable case, long-lag estimates are noisy because the true weights are small while the estimation difficulty is unchanged. Stable-kernel Gaussian-process methods address this dilemma by choosing a covariance construction whose induced impulse-response estimates satisfy the intended stability and decay conditions; the guarantee comes from that kernel construction, not from Gaussian-process regression in general. This gives a principled bias-variance dial: the kernel encodes qualitative prior knowledge without committing to a transfer-function order. The kernel choice occupies the role that model-class choice played in parametric work, but its consequences are typically softer than fixing an incorrect finite order.

Where nonparametrics shine and struggle deserves honest accounting on both sides:

• They excel as a first look, as the highest-fidelity available estimate when no reliable structure is known, and for validating parametric models afterward. A parametric model whose predicted frequency response deviates visibly from a kernel-smoothed nonparametric estimate has been caught misbehaving without the validator ever having committed to the parametric assumptions.
• They struggle for nonlinear systems, where convolution ceases to describe behavior, and for control design, where most synthesis theory wants structured matrices, not ten thousand FIR taps. Convolution is a statement about superposition, and nonlinear systems break superposition at the root; extending nonparametric ideas to the nonlinear world generally forces you to reintroduce some structural commitment, at which point much of the appeal evaporates.

It helps to hold parametric and nonparametric identification as two ends of the same bias-variance tradeoff rather than rival camps. Parametric estimators inject strong assumptions (linearity, order, form of the nonlinearity) and win dramatically when assumptions hold, collapsing variance; nonparametric ones hedge, paying variance for protection against misspecification. Modern practice mixes freely, for example estimating an impulse response nonparametrically to decide the model order, then fitting a parametric state-space model of that order and validating against the nonparametric estimate. The same dialectic reappears between physical priors and deep networks later in this chapter, with networks playing the flexible high-variance extreme and grey-box structure playing the biased-but-efficient one.