Stat Papers DOI 10.1007/s00362-016-0821-4 REGULAR ARTICLE
Performance of the restricted almost unbiased type principal components estimators in linear regression model Yalian Li1
· Hu Yang1
Received: 9 July 2015 / Revised: 27 July 2016 © Springer-Verlag Berlin Heidelberg 2016
Abstract In this paper, two new classes of estimators called the restricted almost unbiased ridge-type principal components estimator and the restricted almost unbiased Liu-type principal components estimator are introduced. For the two cases when the restrictions are true and not true, necessary and sufficient conditions for the superiority of the proposed estimators are derived and compared, respectively. Finally, A Monte Carlo simulation study is given to illustrate the performance of the proposed estimators. Keywords Multicollinearity · Principle components regression · Equality Restrictions · Almost unbiased ridge estimator · Almost unbiased Liu estimator Mathematics Subject Classification 62J07 · 62J05
1 Introduction Consider the following multiple linear regression model: y = Xβ + ε,
(1)
where y is an n × 1 vector of responses, X is an n × p known design matrix of rank p, β is a p × 1 vector of unknown parameters, ε is an n × 1 vector of disturbances assumed to be distributed with mean vector 0 and variance covariance matrix σ 2 In , and In is an identity matrix of order n.
B 1
Yalian Li
[email protected] Department of Statistics and Actuarial Science, Chongqing University, Chongqing 401331, China
123
Y. Li, H. Yang
According to the Gauss-Markov theorem, the ordinary least squares estimate (OLSE) of (1) is obtained as follows: βˆ = (X X )−1 X y
(2)
It has been treated as the best estimator for a long time. However, many results have proved that the OLSE is no longer a good estimator when the multicollinearity is present. To overcome this problem, different remedies have been proposed. One accepted technique is to consider suitable biased estimators instead of the OLSE, since some biased estimators may improve the accuracy of the parameter estimation in linear models when multicollinearity exists. There are many new biased estimators have been proposed, such as principal components regression estimator (PCRE) (Massy 1965), ridge estimator (Hoerl and Kennard 1970), Liu estimator (Liu 1993), two-parameters (Özkale and Kaçıranlar 2007). Baye and Parker (1984) proposed the r − k class estimator which includes as special cases the PCRE, the ridge estimator and the OLSE. Then, Kaçıranlar and Sakallıo˘glu (2001) proposed the r − d estimator which is a generalization of the OLSE, PCRE and Liu estimator. Singh et al. (1986) introduced the almost unbiased generalized ridge estimator (AUGRE) by the jackknife procedure. Akdeniz and Kaçıranlar (1995) proposed the almost unbiased generalized Liu estimator (AUGLE). As special cases of the AUGRE and AUGLE, Akdeniz and Erol (2003) considered the almost unbiased ridge estimator (AURE) and almost unbiased Liu estimator (AULE), which are shown to be bias corrected estimators of the ridge estimator and Liu estimator, respectively. An alternative method to deal with multicollinearity problem is to consider parameter estimation with some restrictions on the unknown parameters, which may be exact or stochastic restrictions (Rao et al. 2008). By grafting the ridge estimator and Liu estimator into the restricted least squares estimate (RLSE) procedure, Sarkar (1992) and Kaçıranlar et al. (1999) introduced the restricted ridge estimator and the restricted Liu estimator, respectively. Hubert and Wijekoon (2006) proposed the stochastic restricted Liu estimator. Groß (2003) proposed a new restricted ridge estimator which satisfied the given linear restrictions. Xu and Yang (2011a, b) considered the restricted r − k estimator and the restricted r − d estimator, meanwhile, they introduced the restricted almost unbiased ridge estimator (RAURE) and the restricted almost unbiased Liu estimator (RAULE). The primary aim in this article is to introduce two new classes of estimators which one includes the RLSE, PCRE, and RAURE as special cases and the other one includes the RLSE, PCRE and RAULE as special cases and provide some alternative methods to overcome multicollinearity in linear regression. The rest of the article is organized as follows. The new estimators are introduced in Sect. 2. Then, superiority and some properties of the new estimators are discussed in Sect. 3. In Sect. 4, a Monte Carlo simulation is given to illustrate behavior of the estimators. Finally, some concluding remarks are given in Sect. 5.
123
Performance of the restricted almost unbiased type…
2 The proposed estimators For the unrestricted model given by (1), in order to overcome multicollinearity, some biased estimators are proposed such as the ridge estimator (RE) introduced by Hoerl and Kennard (1970) and the Liu estimator (LE) proposed by Liu (1993) βˆ (k) = (S + k I )−1 X y
(3)
βˆ (d) = (S + I )−1 (S + d I ) X y
(4)
and
respectively, where k > 0, 0 < d < 1, S = X X . Meanwhile, in order to reduce the bias of the RE and LE, the almost unbiased ridge estimator (AURE) proposed by Singh et al. (1986) and the almost unbiased Liu estimator (AULE) proposed by Akdeniz and Kaçıranlar (1995) are defined as βˆAU (k) = I − k 2 (S + k I )−2 βˆ
(5)
βˆ AU L E (d) = I − (1 − d)2 (S + I )−2 βˆ
(6)
and
respectively, where k > 0, 0 < d < 1. Now consider the spectral decomposition of the matrix given as r X X = Tr T p−r 0
0 p−r
Tr T p−r
(7)
where r = diag(λ1 , · · · , λr ), p−r = diag(λr +1 , · · · , λ p−r ) and λ2 ≥ λ1 ≥ .. · · · ≥ λ p > 0 are the ordered eigenvalues of S. The matrix T = Tr .T p−r p× p
is orthogonalwith Tr = (t1 , · · · , tr ) consisting of its first r columns and T p−r = tr +1 , · · · , t p consisting of the remaining p − r columns of the matrix T. Then Tr STr = r , the PCRE of β can be written as −1 βˆr = Tr Tr STr Tr X y = Tr r−1 Tr X y
(8)
The r − k class estimator proposed by Baye and Parker (1984) and the r − d class estimator proposed by Kaçıranlar and Sakallıo˘glu (2001) are defined as −1 Tr X y = Tr (r + k Ir )−1 Tr X y βˆr (k) = Tr Tr STr + k Ir
(9)
123
Y. Li, H. Yang
−1 βˆr (d) = Tr Tr STr + Ir Tr STr +d Ir Tr X y = Tr × (r + Ir )−1 Ir + dr−1 Tr X y
(10)
Following by Xu and Yang (2011a), the r − k class estimator and r − d class estimator can be rewritten as following βˆr (k) = Tr Tr T ( + k I )−1 T X y = Tr Tr βˆ (k) βˆr (d) = Tr Tr T ( + I )−1 I + d−1 T X y = Tr Tr βˆ (d)
(11) (12)
where βˆ (k) = T ( + k I )−1 T X y = (S + k I )−1 X y is the ridge estimator by Hoerl and Kennard (1970), βˆ (d) = T ( + I )−1 I + d−1 T X y = (S + I )−1 I + d S −1 X y is the Liu estimator proposed by Liu (1993). In addition to the model (1), assume it is known some prior information about β in the form of a set of j independent linear restrictions as follows: h = Rβ
(13)
where R is a j × p known matrix of rank j and h is a known vector. For the restricted model specified by (1) and (13), the restricted least squares error estimator (RLSE) introduced is defined as ˆ βˆ R = βˆ + S −1 R (RS −1 R )−1 (h − R β) −1 −1 −1 = AX y + S R (RS R ) h
(14)
where A = S −1 − S −1 R (RS −1 R )−1 RS −1 . The restricted almost unbiased ridge estimator and the restricted almost unbiased Liu estimator are proposed by Xu and Yang (2011b), which are denoted as βˆ R AU (k) = I − k 2 (S + k I )−2 βˆR βˆ R AU (d) = I − (1 − d)2 (S + I )−2 βˆR
(15) (16)
Now, we are to propose two new classes of estimators by combining the PCR estimator with the RAURE and RAULE, that is, the restricted almost unbiased ridge principal components estimator (RAURPCE) and the restricted almost unbiased Liu principal component estimator (RAULPCE) as follows:
and
βˆ R AU (r, k) = Tr Tr I − k 2 (S + k I )−2 βˆR = Tr Tr G k βˆR
(17)
βˆ R AU (r, d) = Tr Tr I − (1 − d)2 (S + I )−2 βˆR = Tr Tr Hd βˆR
(18)
respectively, where G k = I − k 2 (S + k I )−2 , Hd = I − (1 − d)2 (S + I )−2 .
123
Performance of the restricted almost unbiased type…
From the definition of the RAURPCE, we can easily obtain If r = p, then βˆ R AU (r, k) = βˆ R AU (k) , RAURE RLSE If k = 0, r = p, then βˆ R AU (r, k) = βˆR , RPCRE If k = 0, then βˆ R AU (r, k) = βˆ R (r ) =Tr Tr βˆR , where RPCRE means restricted principal component estimator. From the definition of the RAULPCE, we can similarly obtain RAULE If r = p, then βˆ R AU (r, d) = βˆ R AU (d) , RLSE If d = 1, r = p, then βˆ R AU (r, d) = βˆR , RPCRE If d = 1, then βˆ R AU (r, d) = Tr Tr βˆR , So the βˆ R AU (r, k)could be regarded as a generalization of RPCRE, RLSE and RAURE, while βˆ R AU (r, d) could be regarded as a generalization of RPCRE, RLSE and RAULE. Furthermore, we can compute that the bias, dispersion matrix and mean squared error matrix of the new estimators βˆ R AU (r, k) are Bias βˆ R AU (r, k) = E βˆ R AU (r, k) − β = Tr Tr G k − I β + Tr Tr G k S −1 R (RS −1 R )−1 δ D βˆ R AU (r, k) = Tr Tr G k · Cov(βˆ R ) · G k Tr Tr = σ 2 Tr Tr G k AG k Tr Tr and MSEM βˆ R AU (r, k) = σ 2 Tr Tr G k AG k Tr Tr + Tr Tr G k − I β + Tr Tr G k S −1 R (RS −1 R )−1 δ × Tr Tr G k − I β + Tr Tr G k S −1 R (RS −1 R )−1 δ
(19)
respectively, where δ = h − Rβ. In the similar way, we can get the MSEM of the βˆ R AU (r, d) as follows: MSEM βˆ R AU (r, d) = σ 2 Tr Tr Hd AHd Tr Tr + Tr Tr Hd − I β + Tr Tr Hd S −1 R (RS −1 R )−1 δ × Tr Tr Hd − I β + Tr Tr Hd S −1 R (RS −1 R )−1 δ
(20)
In particular, if we let r = p in Equations (19) and (20), then we can get the MSEM of the RAURE and RAULE by Xu and Yang (2011b) as follows: MSEM βˆ R AU (k) = σ 2 G k AG k + (G k − I ) β + G k S −1 R (RS −1 R )−1 δ (21) × (G k − I ) β + G k S −1 R (RS −1 R )−1 δ
123
Y. Li, H. Yang
and MSEM βˆ R AU (d) = σ 2 Hd AHd + (Hd − I ) β + Hd S −1 R (RS −1 R )−1 δ (22) × (Hd − I ) β + Hd S −1 R (RS −1 R )−1 δ
3 Superiority of the proposed estimators For the sake of convenience, we first list some notations, definitions and lemmas needed in the following discussion. For a matrix M, M , M + , rank(M), R(M) and N (M) stand for the transpose, Moore–Penrose inverse, rank, column space and null space, respectively. M > 0 means M is nonnegative definite and symmetric. Lemma 1 Let Cn× p be the set of n × p complex matrices, Hn×n be the subset of Cn× p consisting of Hermitian matrices and L ∈ Cn× p , L ∗ , M (L) and J(D) stand for the conjugate transpose, the range and the set of all generalized inverses, respectively. Let D ∈ Hn×n , a1 and a2 ∈ Cn×1 be lin/ M (D), let early independent, f i j = ai∗ D − a j , i, j = 1, 2 and if a2 ∈
∗ ∗ s = a1∗ I − D D − I − D D − a2 / a1∗ I − D D − I − D D − a1 . Then D + a1 a1∗ − a2 a2∗ ≥ 0 if and only if one of the following sets of conditions holds: (a) D ≥ 0, ai ∈ M(D), i = 1, 2, ( f 11 + 1) ( f 22 − 1) ≤ | f 12 |2 ; . (b) D ≥ 0, a1 ∈ / M(D), a2 ∈ M(D ..a1 ), (a2 − sa1 )∗ D − (a2 − sa1 ) ≤ 1 − |s|2 ; (c) D = U U ∗ − λvv ∗ , ai ∈ M(D), i = 1, 2, v ∗ a1 = 0, f 11 + 1 ≤ 0, f 22 − 1 ≤ 0, ( f 11 + 1) ( f 22 − 1) ≤ | f 12 |2 ; .. where U .v is a subunitary matrix (U possibly absent), a positive-definite diagonal matrix (occurring when U is present) and λ a positive scalar. Further, all expressions in (a), (b) and (c) are independent of the choice of D − ∈ J(D). Proof The Lemma 1 is due to Baksalary and Trenkler (1991).
3.1 Superiority of the estimators when the restrictions hold In this section, we are to investigate the superiority of the estimators when the restrictions hold, namely, δ = h−Rβ = 0. In this case, we have the MSEM of the RAURPCE, RAULPCE, RAURE and RAULE from (19) to (22): MSEM βˆ R AU (r, k) = σ 2 Tr Tr G k AG k Tr Tr + Tr Tr G k − I ββ Tr Tr G k − I (23)
123
Performance of the restricted almost unbiased type…
MSEM βˆ R AU (r, d) = σ 2 Tr Tr Hd AHd Tr Tr + Tr Tr Hd − I ββ Tr Tr Hd − I MSEM βˆ R AU (k) = σ 2 G k AG k + (G k − I ) ββ (G k − I )
(24)
MSEM βˆ R AU (d) = σ 2 Hd AHd + (Hd − I ) ββ (Hd − I )
(25)
(26) Let us consider the comparison between the RAURPCE and RAURE and the RAULPCE and RAULE, respectively. From Equation (23) to (26), we have 1 = M S E M βˆ R AU (k) − M S E M βˆ R AU (r, k) = D1 + b1 b1 − b2 b2 (27) 2 = M S E M βˆ R AU (d) − M S E M βˆ R AU (r, d) = D2 + b3 b3 − b4 b4 (28) where D1 = σ 2 G k AG k − Tr Tr G k AG k Tr Tr , D2 = σ 2 Hd AHd − Tr Tr Hd AHd Tr Tr b1 = (G k − I ) β, b2 = Tr Tr G k − I β, b3 = (Hd − I ) β, b2 = Tr Tr Hd − I β. Now, we shall use Lemma 1 to discuss the difference 1 and 2 . Since A Tr Tr + T p−r T p−r A = Tr Tr + T p−r T p−r = Tr Tr ATr Tr + T p−r T p−r AT p−r T p−r + Tr Tr AT p−r T p−r + T p−r T p−r ATr Tr We assume that Tr AT p−r = 0 and T p−r AT p−r is invertible, then AT p−r T p−r A = Tr Tr ATr Tr + T p−r T p−r
(29)
Meanwhile, It is noted that the are reasonable to the which is equivalent assumptions AT T T AT T r p−r r r r A Tr T p−r = is a partitioned matrix T AT = T p−r T p−r ATr T p−r AT p−r block diagonal matrix and the second main diagonal being invertible. Theorem 3.1 Suppose Tr AT p−r = 0 and T p−r AT p−r is invertible, then the new ˆ ˆ estimator β R AU (r, k) is superior to the β R AU (k) if and only if β ∈ N (F), where −1/2 . F = σ −1 T p−r AT p−r T p−r
123
Y. Li, H. Yang
Proof Since I − k 2 (r + k I )−2 − I Tr β + T p−r −2 − I T p−r × I − k 2 p−r + k I β
β b2 = Tr I − k 2 (r + k I )−2 − I Tr β − T p−r T p−r 2 −2 2 Tr ATr I − k (r + k I )−2 Tr G k AG k = Tr I − k (r + k I ) −2 −2 T p−r AT p−r I − k 2 p−r + k I T p−r + T p−r I − k 2 p−r + k I b1 = Tr
Tr Tr G k AG k Tr Tr = Tr Tr Tr I − k 2 (r + k I )−2 ×Tr ATr I − k 2 (r + k I )−2 Tr Tr Tr = Tr I − k 2 (r + k I )−2 Tr ATr I − k 2 (r + k I )−2 Tr Then we have −2 −2 T p−r AT p−r I − k 2 p−r + k I T p−r D1 = σ 2 T p−r I − k 2 p−r + k I And the Moore–Penrose inverse D1+ of D1 is −1 −2 −1 T p−r AT p−r D1+ = σ −2 T p−r I − k 2 p−r + k I −2 −1 × I − k 2 p−r + k I T p−r −2 Note that D1 D1+ = T p−r T p−r = I −Tr Tr , I −k 2 p−r + k I is positive definition / M(D1 ). matrix since p−r is supposed to be invertible and D1 D1+ a1 = a1 , so a1 ∈ Moreover, −2 T p−r β = D1 η1 b2 − b1 = −T p−r I − k 2 p−r + k I −1 −2 −1 β. This T p−r AT p−r where η1 = −σ −2 T p−r I − k 2 p−r + k I T p−r .. implies that b2 ∈ M(D 1 .b1 ). Sothe conditions of part (b) in Lemma 1 can be employed. Since I − D D − I − D D − = Tr Tr Tr Tr = Tr Tr and Tr b2 = Tr b1 , it is concluded that s = 1 in our case. Thus, it follows from Lemma 1 that the βˆ R AU (r, k) is superior to βˆ R AU (k) in the MSEM sense if and only if (b2 − b1 ) D1− (b2 − b1 ) = η1 D1 D1− D1 η1 = η1 D1 η1 ≤ 0. Observing that −1 η1 D1 η1 = σ −2 β T p−r T p−r AT p−r T p−r β = β F Fβ ≥ 0
123
Performance of the restricted almost unbiased type…
−1/2 . Thus the necessary and sufficient condition where F = σ −1 T p−r AT p−r T p−r turn out to be β ∈ N (F) . Theorem 3.2 Suppose Tr AT p−r = 0 and T p−r AT p−r is invertible, then the new ˆ ˆ estimator β R AU (r, d) is superior to the β R AU (d) if and only if β ∈ N (F), where −1/2 . F = σ −1 T p−r AT p−r T p−r
Proof In order to apply Lemma 1, we can similarly compute that I − (1 − d)2 (r + I )−2 Tr + T p−r −2 T p−r × I − (1 − d)2 p−r + I
Tr I − (1 − d)2 (r + I )−2 − I Tr β + T p−r −2 − I T p−r × I − (1 − d)2 p−r + I β
Tr I − (1 − d)2 (r + I )−2 − I Tr β − T p−r T p−r β 2 −2 2 Tr I − (1 − d) (r + I ) Tr ATr I − (1 − d) (r + I )−2 Tr −2 T p−r AT p−r ×T p−r I − (1 − d)2 p−r + I −2 T p−r × I − (1 − d)2 p−r + I Tr Tr Tr I − (1 − d)2 (r + I )−2 ×Tr ATr I − (1 − d)2 (r + I )−2 Tr Tr Tr Tr I − (1 − d)2 (r + I )−2 Tr ATr I − (1 − d)2 (r + I )−2 Tr
Hd = Tr
b3 =
b4 = Hd AHd =
Tr Tr Hd AHd Tr Tr = =
Therefore, the Moore–Penrose inverse D2+ of D2 is given by −1 −2 −1 D2+ = σ −2 T p−r I − (1 − d)2 p−r + I T p−r AT p−r −2 −1 × I − k 2 p−r + k I T p−r , then b ∈ Since D2 D2+ = T p−r T p−r 3 / M (D2 ). Moreover,
−2 T p−r β = D2 η2 b4 − b3 = −T p−r I − (1 − d)2 p−r + I −1 −2 −1 β. T p−r AT p−r where η1 = −σ −2 T p−r I − (1 − d)2 p−r + I T p−r . This implies that b4 ∈ M(D2 ..b 3 ). So theconditions of part (b) in Lemma 1 can be employed. Since I − D D − I − D D − = Tr Tr Tr Tr = Tr Tr and Tr b4 = Tr b3 , it is concluded that s = 1 in our case. Thus, it follows from Lemma 1 that the βˆ R AU (r, d)
123
Y. Li, H. Yang
is superior to βˆ R AU (d) in the MSEM sense if and only if (b4 − b3 ) D2− (b4 − b3 ) = η2 D2 D2− D2 η2 = η2 D2 η2 ≤ 0. Observing that −1 AT p−r T p−r β = β F Fβ ≥ 0 η2 D2 η2 = σ −2 β T p−r T p−r −1/2 . Thus the necessary and sufficient condition where F = σ −1 T p−r AT p−r T p−r turn out to be β ∈ N (F) . 3.2 Superiority of the estimator when the restrictions do not hold In this section, we are to compare the estimators when the restrictions do not hold, namely δ = h − Rβ = 0.From the Equations (19) to (22), we consider the following MSEM differences: 3 = MSEM βˆ R AU (k) − MSEM βˆ R AU (r, k) = D1 + b˜1 b˜1 − b˜2 b˜2 (30) 4 = MSEM βˆ R AU (d) − MSEM βˆ R AU (r, d) = D2 + b˜3 b˜3 − b˜4 b˜4 (31) −1 δ and where δ ∗ = R RS −1 R b˜1 = (G k − I ) β + G k S −1 δ ∗ −2 −I = Tr I − k 2 (r + k I )−2 − I Tr β + T p−r I − k 2 p−r + k I ×T p−r β + Tr I − k 2 (r + k I )−2 r−1 Tr δ ∗ −2 −1 p−r T p−r δ ∗ +T p−r I − k 2 p−r + k I b˜2 = Tr Tr G k − I β + Tr Tr G k S −1 δ ∗ = Tr I − k 2 (r + k I )−2 − I ×Tr β − T p−r T p−r β + Tr I − k 2 (r + k I )−2 r−1 Tr δ ∗ b˜3 = (Hd − I ) β + Hd S −1 δ ∗ = Tr I − (1 − d)2 (r + I )−2 − I −2 ×Tr β + T p−r I − (1 − d)2 p−r + I − I T p−r β +Tr I − (1 − d)2 (r + I )−2 r−1 Tr δ ∗ + T p−r −2 −1 p−r T p−r δ ∗ × I − (1 − d)2 p−r + I b˜4 = Tr Tr Hd − I β + Tr Tr Hd S −1 δ ∗ = Tr I − (1 − d)2 (r + I )−2 − I ×Tr β − T p−r T p−r β + Tr I − (1 − d)2 (r + I )−2 r−1 Tr δ ∗
123
Performance of the restricted almost unbiased type… Theorem 3.3 Suppose Tr AT p−r = 0 and T p−r AT p−r is invertible, then the new esti δ∗ + β mator βˆ R AU (r, k) is superior to the βˆ R AU (k) if and only if F˜ T p−r T p−r p−r
= 0. Proof It is easy to compute that −2 −1 p−r T p−r δ ∗ + p−r T p−r β = D1 η˜ 1 b˜2 − b˜1 = −T p−r I − k 2 p−r + k I −1 −2 −1 T p−r AT p−r where η˜ 1 = −σ −2 T p−r I − k 2 p−r + k I −1 p−r T p−r .. β . Then we have b˜ − δ ∗ + p−r T p−r 2 ∈ M(D1 .b˜ 1 ) and Since I − D D I − D D − = Tr Tr Tr Tr = Tr Tr and Tr b˜2 = Tr b˜1 , it is concluded that s = 1 ˆ in our case. Thus, it follows fromLemma 1 that the (r, k) is superior to βˆ R AU (k) β R AU − in the MSEM sense if and only if b˜2 − b˜1 D b˜2 − b˜1 = η˜ D1 η˜ 1 ≤ 0. Observing 1
1
that −1 δ ∗ + p−r T p−r β −1 η˜ 1 D1 η˜ 1 = σ −2 T p−r p−r T p−r AT p−r ∗ −1 p−r T p−r δ + p−r T p−r β δ ∗ + p−r T p−r β F˜ F˜ T p−r δ ∗ + p−r T p−r β ≥0 = T p−r −1/2 AT p−r −1 where F˜ = σ −1 T p−r p−r . Thus the necessary and sufficient condition ∗ turn out to be F˜ T p−r δ + p−r T p−r β = 0. Now, we are further to consider the difference given by Eq. (30). If −1 −2 −1 T p−r AT p−r η˜ 2 = −σ −2 T p−r I − (1 − d)2 p−r + I ∗ −1 p−r T p−r δ + p−r T p−r β We can compute that −2 −1 p−r T p−r δ ∗ + T p−r β = D2 η˜ 2 b˜4 − b˜3 = −T p−r I − (1 − d)2 p−r + I −2 − I Tr β Tr b˜4 = I − (1 − d)2 p−r + I + I − (1 − d)2 (r + I )−2 r−1 Tr δ ∗ = Tr b˜3
123
Y. Li, H. Yang
Fig. 1 Estimated MSE values of the estimators
. Then we have b˜4 ∈ M(D2 ..b˜3 ) and b˜3 I − D D − I − D D − b˜3 s = b˜3 I − D D − I − D D − b˜4 = b˜3 Tr Tr b˜4 /b˜3 Tr Tr b˜3 = 1 ˆ ˆ Thus, it follows from Lemma β RAU (r, d) is superior to β R AU (d) in the 1 that the − ˜ ˜ ˜ ˜ MSEM sense if and only if b4 − b3 D2 b4 − b3 = η˜ 2 D2 η˜ 2 ≤ 0. Observing that δ ∗ + p−r T p−r β η˜ 2 D2 η˜ 2 = σ −2 T p−r −1 −1 ∗ T T −1 AT δ + T β p−r p−r p−r p−r p−r p−r p−r δ ∗ + p−r T p−r β F˜ F˜ T p−r δ ∗ + p−r T p−r β ≥0 = T p−r δ∗ + Thus the condition turn out to be F˜ T p−r p−r T p−r β = 0 and we have: AT p−r is invertible, then the new estiTheorem 4 Suppose Tr AT p−r = 0 and T p−r δ∗ + ˆ ˆ mator β R AU (r, d) is superior to the β R AU (d) if and only if F˜ T p−r p−r T p−r β = 0.
123
Performance of the restricted almost unbiased type… Table 1 MSE values of the RLSE, RPCRE, RAURE, RAURPCE, RAULE and RAULPCE when σ 2 = 1 k/d
0.00
0.10
0.30
0.50
0.60
0.70
0.90
1.00
γ = 0.7 RLSE
0.0413
0.0413
0.0413
0.0413
0.0413
0.0413
0.0413
0.0413
RPCRE
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
RE
0.0619
0.0617
0.0612
0.0607
0.0605
0.0603
0.0599
0.0596
RAURE
0.0412
0.0413
0.0413
0.0413
0.0413
0.0413
0.0413
0.0412
RAURPCE
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
LE
0.0596
0.0599
0.0603
0.0608
0.0610
0.0612
0.0617
0.0619
RAULE
0.0412
0.0413
0.0413
0.0413
0.0413
0.0413
0.0413
0.0413
RAULPCE
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
0.0279
γ = 0.85 RLSE
0.0691
0.0691
0.0691
0.0691
0.0691
0.0691
0.0691
0.0691
RPCRE
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
RE
0.1085
0.1078
0.1063
0.1048
0.1041
0.1034
0.1019
0.1013
RAURE
0.0691
0.0691
0.0691
0.0691
0.0690
0.0690
0.0690
0.0690
RAURPCE
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
LE
0.1013
0.1020
0.1034
0.1049
0.1056
0.1063
0.1078
0.1085
RAULE
0.0690
0.0690
0.0690
0.0691
0.0691
0.0691
0.0691
0.0691
RAULPCE
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
0.0380
γ = 0.99 RLSE
0.8824
0.8824
0.8824
0.8824
0.8824
0.8824
0.8824
0.8824
RPCRE
0.3504
0.3504
0.3504
0.3504
0.3504
0.3504
0.3504
0.3504
RE
1.4636
1.3256
1.1039
0.9351
0.8653
0.8033
0.6983
0.6536
RAURE
0.8824
0.8788
0.8560
0.8208
0.8009
0.7801
0.7376
0.7164
RAURPCE
0.3504
0.3497
0.3446
0.3363
0.3314
0.3260
0.3145
0.3085
LE
0.6536
0.7194
0.8612
1.0165
1.0992
1.1852
1.3674
1.4636
RAULE
0.7164
0.7464
0.7986
0.8391
0.8545
0.8666
0.8806
0.8824
RAULPCE
0.3085
0.3163
0.3295
0.3396
0.3435
0.3465
0.3499
0.3504
γ = 0.999 RLSE
8.7377
8.7377
8.7377
8.7377
8.7377
8.7377
8.7377
8.7377
RPCRE
3.3710
3.3710
3.3710
3.3710
3.3710
3.3710
3.3710
3.3710
RE
14.5437
6.4864
2.4013
1.2539
0.9695
0.7731
0.5266
0.4462
RAURE
8.7377
7.0977
3.9484
2.4278
1.9728
1.6337
1.1731
1.0128
RAURPCE
3.3710
2.9566
1.8855
1.2448
1.0356
0.8740
0.6460
0.5641
LE
0.4462
0.9515
2.5650
4.9825
6.4927
8.2040
12.2295
14.5437
RAULE
1.0128
1.8797
3.9769
6.0744
6.9770
7.7227
8.6218
8.7377
RAULPCE
0.5641
0.9141
1.6980
2.4459
2.7618
3.0208
3.3311
3.3710
123
Y. Li, H. Yang Table 2 MSE values of the RLSE, RPCRE, RAURE, RAURPCE, RAULE and RAULPCE when σ 2 = 10 k/d
0.00
0.10
0.30
0.50
0.60
0.70
0.90
1.00
γ = 0.7 RLSE
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
RPCRE
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
RE
0.1957
0.1950
0.1935
0.1921
0.1914
0.1907
0.1893
0.1886
RAURE
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
0.1304 0.0652
RAURPCE
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
LE
0.1886
0.1893
0.1907
0.1921
0.1929
0.1936
0.1950
0.1957
RAULE
0.1304
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
0.1305
RAULPCE
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
0.0652
γ = 0.85 RLSE
0.2185
0.2185
0.2185
0.2185
0.2185
0.2185
0.2185
0.2185
RPCRE
0.0977
0.0977
0.0977
0.0977
0.0977
0.0977
0.0977
0.0977
RE
0.3432
0.3408
0.3360
0.3314
0.3291
0.3268
0.3224
0.3202
RAURE
0.2185
0.2185
0.2185
0.2184
0.2184
0.2183
0.2182
0.2181
RAURPCE
0.0977
0.0977
0.0977
0.0977
0.0977
0.0976
0.0976
0.0976
LE
0.3202
0.3224
0.3270
0.3316
0.3339
0.3362
0.3409
0.3432
RAULE
0.2181
0.2182
0.2183
0.2184
0.2184
0.2185
0.2185
0.2185
RAULPCE
0.0976
0.0976
0.0976
0.0977
0.0977
0.0977
0.0977
0.0977
RLSE
2.7903
2.7903
2.7903
2.7903
2.7903
2.7903
2.7903
2.7903
RPCRE
1.0858
1.0858
1.0858
1.0858
1.0858
1.0858
1.0858
1.0858
RE
4.6283
4.1918
3.4903
2.9557
2.7346
2.5381
2.2052
2.0634
RAURE
2.7903
2.7791
2.7068
2.5957
2.5326
2.4668
2.3321
2.2649 0.9534
γ = 0.99
RAURPCE
1.0858
1.0835
1.0677
1.0415
1.0257
1.0088
0.9724
LE
2.0634
2.2723
2.7217
3.2135
3.4753
3.7477
4.3242
4.6283
RAULE
2.2649
2.3601
2.5253
2.6533
2.7022
2.7405
2.7848
2.7903
RAULPCE
0.9534
0.9779
1.0199
1.0519
1.0641
1.0736
1.0845
1.0858
γ = 0.999 RLSE
27.6309
27.6309
27.6309
27.6309
27.6309
27.6309
27.6309
27.6309
RPCRE
10.6378
10.6378
10.6378
10.6378
10.6378
10.6378
10.6378
10.6378
RE
45.9913
20.5084
7.5831
3.9503
3.0497
2.4274
1.6459
1.3911
RAURE
27.6309
22.4445
12.4821
7.6698
6.2292
5.1557
3.6965
3.1887
RAURPCE
10.6378
9.3273
5.9401
3.9131
3.2514
2.7402
2.0185
1.7594
1.3911
2.9926
8.1014
15.7510
20.5286
25.9415
38.6728
45.9913
LE RAULE
3.1887
5.9349
12.5726
19.2082
22.0628
24.4212
27.2644
27.6309
RAULPCE
1.7594
2.8670
5.3468
7.7124
8.7114
9.5305
10.5117
10.6378
123
Performance of the restricted almost unbiased type…
4 Simulation results In order to illustrate the behavior of the RAURPCE and RAULPCE, we perform a Monte Carlo simulation study under different levels of multicollinearity. Following Liu (1993), we obtain the explanatory variables and the observations on the dependent variable using the following equation: xij = (1−γ 2 )1/2 z ij +γ z i5 , yi = (1−γ 2 )1/2 z ij +γ z i(j+1) , i = 1, 2, . . . ,n, j = 1, . . . , p where z ij are independent standard normal pseudo-random numbers and γ is specified so that the theoretical correlation between any two explanatory variables is given by γ 2 . In this study, we choose the restrictions h = Rβ, R = (1, −2, −2, −2)and r = 2, n = 100, p = 4 and different variances (σ 2 = 1, 10), respectively. Let’s consider the RAURPCE, RAULPCE, RAURE, RAULE, RE, LE, RPCRE and RLSE and compute their respective estimated MSE values with the different levels of multicollinearity namely γ = 0.7, 0.85, 0.9, 0.999 to show the weakly, strong and severely collinear between the explanatory variables (See Tables 1, 2). Furthermore, for the convenience of comparison, we plot the estimated MSE values of the estimators when γ = 0.999 and σ 2 = 10 in Fig. 1. Form Tables 1 and 2 and Fig. 1, we have: 1. With the increase of multicollinearity, the MSE values of RLSE, RPCRE, RE, LE, RAURE, RAURPCE, RAULE and RAULPCE are increasing. 2. For most cases, especially when there are severely collinear between the explanatory variables, i.e., γ = 0.9, 0.999, the new estimator RAURPCE has smaller MSE values than the RLSE, RPCRE, RE and RAURE, meanwhile, the new estimator RAULPCE has smaller MSE values than the RLSE, RPCRE, LE and RAULE. 3. Furthermore, the RAURPCE and RAULPCE also have more stable and smaller estimated MSE values than the RLSE, RPCRE, RE, RAURE, LE and RAULE from the Fig. 1. We can see that our estimator provide some alternative suitable methods to overcome multicollinearity which are meaningful in practice.
5 Conclusion In this article, our purpose is to provide some alternative unified estimators- RAURPCE and RAULPCE which include RAURE, RPCRE and RLSE and RAULE, RPCRE and RLSE, respectively. We aim to focus on the new ones reducing the MSE to deal with multicollinearity. Some properties of the two estimators have also been discussed under the restrictions are true or not true. The Monte Carlo simulation results show that the RAURPCE and RAULPCE may also, respectively, have smaller MSE value than the RE, RAURE, LE and RAULE and both are superior to RPCRE and RLSE. Acknowledgments We are grateful to the editor and the anonymous reviewers for the constructive comments and suggestions which have improved the quality of this paper. This work was supported in part by the National Natural Science Foundation of China (Grant No. 11671059) and the Fundamental Research Funds for the Central Universities (Grant No. 106112015CDJXY100004).
123
Y. Li, H. Yang
References Akdeniz F, Erol H (2003) Mean squared error matrix comparisons of some biased estimator in linear regression. Commun Stat Theory Methods 32(12):2389–2413 Akdeniz F, Kaçıranlar S (1995) On the almost unbiased generalized Liu estimator and unbiased estimation of the bias and MSE. Commun Stat Theory Methods 24:1789–1797 Baye MR, Parker DF (1984) Combining ridge and principal component regression: a money demand illustration. Commun Stat Theory Methods 13(2):197–205 Baksalary JK, Trenkler G (1991) Nonnegative and positive definiteness of matrices modified by two matrices of rank one. Linear Algebr Appl 151:169–184 Groß J (2003) Restricted ridge estimation. Stat Probab Lett 65:57–64 Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12:55–67 Hubert MH, Wijekoon P (2006) Improvement of the Liu estimator in linear regression model. Statistical Papers 47(3):471–479 Kaçıranlar S, Sakallıo˘glu S (2001) Combining the Liu estimator and the principal component regression estimator. Commun Stat Theory Methods 30:2699–2705 Kaçıranlar S, Sakallıo˘glu S, Akdeniz F, Styan GPH, Werner HJ (1999) A new biased estimator in linear regression and a detailed analysis of the widely-analysed dataset on Portland Cement. Sankhya Ind J Stat 61:443–459 Liu K (1993) A new class of biased estimate in linear regression. Commun Stat Theory Methods 22:393–402 Massy WF (1965) Principal components regression in exploratory statistical research. J Am Stat Assoc 60:234–266 Özkale MR, Kaçıranlar S (2007) The restricted and unrestricted two-parameter estimator. Commun Stat Theory Methods 36:2707–2725 Rao CR, Toutenburg H, Shalabh H (2008) C: linear models and generalizations: least squares and alternatives. Springer, NewYork Sarkar N (1992) A new estimator combining the ridge regression and the restricted least squares methods of estimation. Commun Stat Theory Methods 21:1987–2000 Singh B, Chaubey YP, Dwivedi TD (1986) An almost unbiased ridge estimator, Sankhya: Indian. J Stat B 48:342–346 Xu JW, Yang H (2011a) On the restricted r-k class estimator and the restricted r-d class estimator in linear regression. J Stat Comput Simul 81:679–691 Xu JW, Yang H (2011b) On the restricted almost unbiased estimators in linear regression. J Appl Stat 38(03):605–617
123
