Nonparametric Estimation

views updated

Nonparametric Estimation

Nonparametric estimation is a methodology for estimating density functions or conditional moments of distributions without making any prior assumptions about functional forms. The data are allowed to speak for themselves in determining the shape of the unknown functions (Silverman 1986).

NONPARAMETRIC DENSITY ESTIMATION

In parametric estimation, if the underlying distribution is known to be normal, the data are used to calculate the mean μ and variance σ ² of the distribution, substituting them into the formula for the normal density. Suppose X is a continuous random variable, f (x ) is the probability density function, and F (x ) is the cumulative density function when X = x. With h as the width of a bin or interval, the nonparametric naive estimate of f (x ) is (probability that X belongs to the interval [ x –h /2, x + h /2]) = 1/h *(probability that (X –x )/h belongs to the interval [–1/2, 1/2]).

In other words,

where,

Alternately,

The graph of the estimated density function (Figure 1) from equation (1) is not a smooth curve. Thus the weight function I(.) is replaced by a kernel density function K(.) that satisfies the condition . The revised nonparametric estimate of the density function is expressed in equation (2). Alternative choices on the kernel density function are provided in equations (4) through (6). The graph of a density function using kernel density weights is smooth and differentiable as illustrated by Figure 2.

NONPARAMETRIC REGRESSION

A regression function is an equation that explains and predicts movements in one variable (the dependent variable) as a function of movements in another variable or a set of other variables (the independent or explanatory variables). Having observed the independent variable X_i, the regression function provides an average or expected value of the dependent variable Y_i, For a set of n data points the regression function can be modeled as

In equation (3), m is the unknown regression equation and ε represents unknown stochastic disturbances. The aim of regression analysis is to estimate the regression function m. The parametric approach assumes that the regression function has some prespecified functional form (such as logarithmic, inverse, quadratic, or cubic).

Nonparametric econometrics estimates the regression function m without assuming any specific form.

An example of the two different approaches is illustrated by Figure 3. The straight, negatively sloped, dotted line represents a linear parametric function, while the other curve is a nonparametric estimate. Both curves model the rank of a harness racing horse (1 being high and 20 being low) as a function of its average speed at a racing track. The linear model is unable to represent a U-shaped regression relationship between the rank and speed of racing horses for certain ranges of horse speed.

The assumption that the estimate of the regression function is linear, that is, m (X _i) = β₀ + β₁X _i, implies certain assumptions about the underlying data-generating process (Pagan and Ullah 1999). For example, if is a bivariate normal density, then it can be shown that the mean of the conditional density of Y _i given X _i is E (Y _i ǀ X _i) = α + βX _i, where α = EY _i–β (EX _i) and β = (Var(X _i))^-1 Cov(X _i, Y _i). Thus, the linear specification for the regression function is valid only if the underlying data-generating process is normal. If the true distribution is not normal, then the true functional form of m (X _i) is not linear, and least square estimates of the same, assuming a linear functional form, may be biased and inconsistent.

The question of which approach should be taken in data analysis was a key issue in a bitter feud between the statisticians Karl Pearson (1857–1936) and Ronald Aylmer Fisher (1890–1962) in the 1920s (summarized by Tapia and Thompson 1978). Fisher pointed out that the non-parametric approach gave generally poor efficiency that increased with the number of explanatory variables, thus giving rise to the well-known curse of dimensionality and requiring large data samples for accuracy. Moreover, the size of the required sample increases rapidly with the number of explanatory variables. At the same time, Pearson pointed out that the price paid for pure parametric fitting is the possibility of gross misspecification resulting in high model bias. The parametric and nonparametric estimation techniques support two different and yet very interesting viewpoints. The semiparametric estimation technique combines the two. Here, the relationship governing Y_i is expressed as a linear function of some explanatory variables and a nonlinear function of remaining explanatory variables where the nonlinearity is unknown. The coefficients of interest are the slope coefficients of the linear part. P. M. Robinson (1988) shows that it is possible to construct estimators of the linear part that exhibit consistency.

The basic principle behind the nonparametric estimation technique is to fit a window h around every observation of the dataset and estimate the relationship or moment of interest in each window. A kernel density function K(. ) is used to give high weights to data points close to the window and low weights to data points far from the window. Thus the regression relationship is estimated, piece by piece or window by window, as shown in Figure 4. One of the advantages of nonparametric estimation is that it estimates the regression coefficients at every data point. For example, if the researcher is interested in estimating the relationship between a firm’s size and its export intensity, the nonparametric estimation technique will provide an estimate of the slope coefficient for every firm at every time period, thus giving a broader picture for analysis. Both parametric and nonparametric techniques share a common foundation. Parametric estimates are obtained by minimizing the sum of squares of residuals (SSR). Nonparametric estimates are obtained by minimizing the SSR weighted

by the kernel density function at every data point. That is the reason why parametric estimates are a product of global fitting, while nonparametric estimates are obtained by local fitting. The conditional mean at point x is a weighted average data points, where . The weights w(X_i; x ) depend upon the kernel density function, the window width, X_i, and the point x at which the conditional expectation is evaluated.

CHOICE OF KERNELS AND WINDOW WIDTHS

Some examples of kernels commonly used in the literature (Silverman 1986) are:

It is well known in the literature that the choice of kernels does not influence significantly the efficiency of estimates. The choice of window width is, however, crucial. Small values of h cause oversmoothing and high values lead to undersmoothing of the estimates. The optimum h is the one that minimizes the integrated mean squared error of .

SCOPE OF NONPARAMETRIC ESTIMATION

The scope of applications for the nonparametric estimation technique is endless (Härdle 1990). It is particularly useful in time series applications, in treatment of extreme observations known as outliers, and in smoothing the gap of missing data by interpolating between adjacent data points. In general, nonparametric econometrics provides a versatile method of exploiting a general relationship between two or more variables without reference to a fixed parametric model.

SEE ALSO Functional Form; Properties of Estimators (Asymptotic and Exact); Semiparametric Estimation