Surveys in Mathematics and its Applications

ISSN 1842-6298 (electronic), 1843 - 7265 (print)
 Volume 3 (2008), 111 -- 122

THE EFFICIENCY OF MODIFIED JACKKNIFE AND RIDGE TYPE REGRESSION ESTIMATORS: A COMPARISON

Feras Sh. M. Batah, Thekke Variyam Ramanathan and Sharad Damodar Gore

Abstract. A common problem in multiple regression models is multicollinearity, which produces undesirable effects on the least squares estimator. To circumvent this problem, two well known estimation procedures are often suggested in the literature. They are Generalized Ridge Regression (GRR) estimation suggested by Hoerl and Kennard iteb8 and the Jackknifed Ridge Regression (JRR) estimation suggested by Singh et al. iteb13. The GRR estimation leads to a reduction in the sampling variance, whereas, JRR leads to a reduction in the bias. In this paper, we propose a new estimator namely, Modified Jackknife Ridge Regression Estimator (MJR). It is based on the criterion that combines the ideas underlying both the GRR and JRR estimators. We have investigated standard properties of this new estimator. From a simulation study, we find that the new estimator often outperforms the LASSO, and it is superior to both GRR and JRR estimators, using the mean squared error criterion. The conditions under which the MJR estimator is better than the other two competing estimators have been investigated.

2000 Mathematics Subject Classification: 62J05; 62J07.
Keywords: Generalized Ridge Regression; Jackknifed Ridge Regression; Mean Squared Error; Modified Jackknife Ridge Regression; Multicollinearity

Full text

Acknowledgement. The first author wishes to thank the Indian Council for Cultural Relations (ICCR) for the financial support. The authors would like to acknowledge the editor and the referee for their valuable comments, which improved the paper substantially.

References

  1. F. Batah and S. Gore, Improving Precision for Jackknifed Ridge Type Estimation, Far East Journal of Theoretical Statistics 24 (2008), No.2, 157--174.

  2. F. Batah, S. Gore and M. Verma, Effect of Jackknifing on Various Ridge Type Estimators, Model Assisted Statistics and Applications 3 (2008), To appear.

  3. D. G. Gibbons, A Simulation Study of Some Ridge Estimators, Journal of the American Statistical Association 76 (1981), 131 -- 139.

  4. J. Groß, Linear Regression: Lecture Notes in Statistics, Springer Verlag, Germany, 2003.

  5. M. H. J. Gruber, The Efficiency of Jackknife and Usual Ridge Type Estimators: A Comparison, Statistics and Probability Letters 11 (1991), 49 -- 51.

  6. M. H. J. Gruber, Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators, New York: Marcel Dekker, Inc, 1998. MR1608582 (99c:62196) . Zbl 0920.62085.

  7. D. V. Hinkley, Jackknifing in Unbalanced Situations, Technometrics 19 (1977), No. 3, 285 -- 292. Zbl 0367.62085 .

  8. A. E. Hoerl and R. W. Kennard, Ridge Regression: Biased Estimation for Non orthogonal Problems, Technometrics 12 (1970), 55 -- 67. Zbl 0202.17205.

  9. A. Hoerl, R. Kennard and K. Baldwin, Ridge Regression: Some Simulations, Commun. Statist. Theor. Meth. 4 (1975), 105--123.

  10. C. Leng, Yi Lin and G. Wahba , A Note on the Lasso and Related Procedures in Model Selection, Statistica Sinica 16 (2006), 1273--1284. MR2327490. Zbl 1109.62056.

  11. G. C. McDonald and D. I. Galarneau, A Monte Carlo Evaluation of Some Ridge-type Estimators, Journal of the American Statistical Association 70 (1975), 407--416. Zbl 0319.62049.

  12. L. C. Peele and T. P. Ryan , Comments to: A Critique of Some Ridge Regression Methods, Journal of the American Statistical Association 75 (1980), 96--97. Zbl 0468.62065.

  13. B. Singh, Y. P. Chaube and T. D. Dwivedi, An Almost Unbiased Ridge Estimator, Sankhya Ser.B 48 (1986), 342--346. MR0905210 (88i:62124).

  14. R. Tibshirani, Regression Shrinkage and Selection via the Lasso, Journal of the Royal Statistical Society B, 58 (1996), 267--288. MR1379242 (96j:62134). Zbl 0850.62538.

  15. H. D. Vinod, A Survey for Ridge Regression and Related Techniques for Improvements Over Ordinary Least Squares, The Review of Economics and Statistics 60 (1978), 121--131. MR0523503(80b:62085).

  16. H. D. Vinod and A. Ullah, Recent Advances in Regression Methods, New York: Marcel Dekker Inc, 1981. MR0666872 (84m: 62097).

  17. D. Wichern and G. Churchill, A Comparison of Ridge Estimators, Technometrics 20 (1978), 301--311. Zbl 0406.62050.


Feras Shaker Mahmood Batah Thekke Variyam Ramanathan
Department of Statistics, University of Pune, India. Department of Statistics,
Department of Mathematics, University of Alanber, Iraq. University of Pune, India.
e-mail: ferashaker2001@yahoo.com e-mail: ram@stats.unipune.ernet.in
Sharad Damodar Gore
Department of Statistics, University of Pune, India.
e-mail: sdgore@stats.unipune.ernet.in

http://www.utgjiu.ro/math/sma