Impact analysis of correlated and non-normal errors in nonparametric regression estimation: A simulation study
Abstract
In nonparametric regression, the correlation of errors can have important consequences for the statistical properties of the estimators, but the focus is on the on the identification of the effect on Average Mean Squared Error (AMSE). This is performed by a Monte Carlo experiment where we use two types of correlation structures and examine them with different correlation points/levels and different error distributions with different sample sizes. We concluded that if errors are correlated, then the distribution of errors is important with correlation structures, but correlation points/levels have a less significant effect, comparatively. When errors are uniformly distributed, AMSE is the smallest, followed by any other distribution, and if errors follow the Laplace distribution, then AMSE is the largest, followed by other distributions. Laplace also has some alarming effects. More specifically, the kernel estimator is robust in the case of a simple correlation structure, and AMSEs attain their minimum when errors are uncorrelated.
References
[1] Hardle W. Applied Nonparametric Regression. Cambridge University Press; 1990.
[2] Ullah A, Vinod HD. General nonparametric regression estimation and testing in econometrics. Handbook of Statistics 1993; 11: 85–116. doi: 10.1016/S0169-7161(05)80039-2
[3] Nadaraya EA. On estimation regression. Theory of Probability & Its Applications 1964; 9(1): 141–142. doi: 10.1137/1109020
[4] Watson G. Smooth regression analysis. Sankhyā: The Indian Journal of Statistics, Series A 1964; 26(4): 359–372.
[5] Royal RM. A Class of Nonparametric Estimators of a Smooth Regression Function [PhD thesis]. Stanford University; 1966.
[6] Mack YP. Local properties of k-NN regression estimates. SIAM Journal on Algebraic and Discrete Methods 1981; 2(3): 311–323. doi: 10.1137/0602035
[7] Mack YP, Muller HG. Derivative estimation in nonparametric regression with random predictor variable. Sankhyā: The Indian Journal of Statistics, Series A 1989; 51(1): 59–72.
[8] Ahmad IA, Lin PE. Fitting a multiple regression function. Journal of Statistical Planning and Inference 1984; 9(2): 163–176. doi: 10.1016/0378-3758(84)90017-X
[9] Gasser T, Muller HG. Estimating regression functions and their derivatives by the kernel method. Scandinavian Journal of Statistics 1984; 11(3): 171–185.
[10] De Brabanter K, De Brabanter J, Suykens JAK, De Moor B. Kernel regression in the presence of correlated errors. Journal of Machine Learning Research 2011; 12: 1955–1976.
[11] Maca JD, Carroll RJ, Ruppert D. Nonparametric Kernel and Regression Spline Estimation in The Presence of Measurement Error. Wirtschaftswissenschaftliche Fakultät, Humboldt Universität, Berlin; 1997.
[12] Wang XF, Wang B. Deconvolution estimation in measurement error models: The R package decon. Journal of Statistical Software 2011; 39(10): i10. doi: 10.18637/jss.v039.i10
[13] Li Y, Wong H, Lp W. Testing heteroscedasticity by wavelets in a nonparametric regression model. Science in China Series A: Mathematics 2006; 49: 1211–1222. doi: 10.1007/s11425-006-2016-2
[14] Park BU, Lee YK, Kim TY, Park C. A simple estimation of error correlation in non-parametric regression models. Scandinavian Journal of Statistics 2006; 33(3): 451–462. doi: 10.1111/j.1467-9469.2006.00506.x
[15] Altman NS. Kernel smoothing of data with correlated errors. Journal of the American Statistical Association 1990; 85(411): 749–759. doi: 10.2307/2290011
[16] Muller HG, Stadtmuller U. Detecting dependencies in smooth regression models. Biometrika 1988; 75(4): 639–650. doi: 10.2307/2336305
[17] Smith M, Wong CM, Kohn R. Additive nonparametric regression with autocorrelated errors. Journal of the Royal Statistical Society. Series B (Statistical Methodology) 1998; 60(2): 311–331.
[18] Su L, Ullah A. More efficient estimation in nonparametric regression with nonparametric autocorrelated errors. Econometric Theory 2006; 22(1): 98–126. doi: 10.1017/S026646660606004X
[19] Lee YK, Mammen E, Park BU. Bandwidth selection for kernel regression with correlated errors. Statistics 2010; 44(4): 327–340. doi: 10.1080/02331880903138452
[20] Chiu ST. Bandwidth selection for kernel estimate with correlated noise. Statistics & Probability Letters 1989; 8(4): 347–354. doi: 10.1016/0167-7152(89)90043-6
[21] Hart JD. Kernel regression estimation with time series errors. Journal of the Royal Statistical Society. Series B (Methodological) 1991; 53(1): 173–187. doi: 10.1111/J.2517-6161.1991.TB01816.X
[22] Herrmann E, Gasser T, Kneip A. Choice of bandwidth for kernel regression when residuals are correlated. Biometrika 1992; 79(4): 783–795. doi: 10.2307/2337234
[23] Opsomer J, Wang Y, Yang Y. Nonparametric regression with correlated errors. Statistical Science 2001; 16(2): 134–153.
[24] Benedetti JK. On the nonparametric estimation of regression functions. Journal of the Royal Statistical Society. Series B (Methodological) 1977; 39(2): 248–253. doi: 10.1111/j.2517-6161.1977.tb01622.x
[25] Silverman BW. Density Estimation for Statistics and Data Analysis, 1st ed. Chapman and Hall; 1986.
[26] Wand MP, Jones MC. Kernel Smoothing. Chapman and Hall/CRC; 1995.
[27] Loader CR. Bandwidth selection: Classical or plug-in? Annals of Statistics 1999; 27(2): 415–438. doi: 10.1214/aos/1018031201
[28] Kim TY, Park BU, Moon MS, Kim C. Using bimodal kernel for inference in nonparametric regression with correlated errors. Journal of Multivariate Analysis 2009; 100(7): 1487–1497. doi: 10.1016/j.jmva.2008.12.012
[29] Wu H, Zhang JT. Nonparametric Regression Methods for Longitudinal Data Analysis: Mixed-Effects Modeling Approaches. John Wily and Sons; 2006.
Copyright (c) 2023 Javaria Ahmad Khan, Atif Akbar, Nasir Saleem, Muhammad Junaid
This work is licensed under a Creative Commons Attribution 4.0 International License.
