Statistik 3 - Beispiel 26

From StatWiki
Jump to: navigation, search

Beispiel 26

Zeigen Sie, daß der LS-Schätzer \hat \beta\, für \beta\, im linearen Modell y = X \beta + u \, die

Beispiel 26 a - Bedingung 1. Ordnung


f(\beta) = (y - X \beta)'(y - X \beta) = \sum_{i=1}^n (y_i - \sum_{j=1}^k x_{i,j} \beta_j)^2

Ableiten:


\frac{f(\beta)}{\partial \beta_l} = 
\sum_{i=1}^n 2 (y_i - \sum_{j=1}^k x_{i,j} \beta_j) \cdot (-x_{i,l}) = 
-2 (y - X \beta)' X_{.,l}


Minimum finden:


\begin{pmatrix}
\frac{f(\hat \beta)}{\partial \hat \beta_1} \\ 
\frac{f(\hat \beta)}{\partial \hat \beta_2} \\ 
\vdots \\ 
\frac{f(\hat \beta)}{\partial \hat \beta_k} \\ 
\end{pmatrix} = 0


\begin{pmatrix}
\frac{f(\hat \beta)}{\partial \hat \beta_1} \\ 
\frac{f(\hat \beta)}{\partial \hat \beta_2} \\ 
\vdots \\ 
\frac{f(\hat \beta)}{\partial \hat \beta_k} \\ 
\end{pmatrix} = 

\begin{pmatrix}
(y - X \hat \beta)' X_{.,1} \\ 
(y - X \hat \beta)' X_{.,2} \\ 
\vdots \\ 
(y - X \hat \beta)' X_{.,k} \\ 
\end{pmatrix} = 

\begin{pmatrix}
X_{.,1} ' (y - X \hat \beta) \\ 
X_{.,2} ' (y - X \hat \beta) \\ 
\vdots \\ 
X_{.,k} ' (y - X \hat \beta) \\ 
\end{pmatrix} = 

X ' (y-  X \hat \beta)=
X'y - X'X \hat \beta = 0

Achtung: a'b = 0 \Rightarrow (a'b)' = b'a = 0


X ' y = X'X \hat \beta


\hat \beta = (X'X)^{-1} X'y

Beispiel 26 b - Bedingung 2. Ordnung


\begin{bmatrix}
\frac{f^2(\hat \beta)}{\partial \hat \beta_1^2} & 
\frac{f^2(\hat \beta)}{\partial \hat \beta_1 \partial \hat \beta_2} &
\cdots &
\frac{f^2(\hat \beta)}{\partial \hat \beta_1 \partial \hat \beta_k}\\ 

\frac{f^2(\hat \beta)}{\partial \hat \beta_2 \partial \hat \beta_1} &
\frac{f^2(\hat \beta)}{\partial \hat \beta_2^2} & 
\cdots &
\frac{f^2(\hat \beta)}{\partial \hat \beta_2 \partial \hat \beta_k}\\ 

\vdots & \vdots & \ddots & \vdots \\

\frac{f^2(\hat \beta)}{\partial \hat \beta_k \partial \hat \beta_1} &
\frac{f^2(\hat \beta)}{\partial \hat \beta_k \partial \hat \beta_2} & 
\cdots &
\frac{f^2(\hat \beta)}{\partial \hat \hat \beta_k ^2}  
\end{bmatrix} > 0

d.h. positiv definit.

Bedingung 1. Ordnung:


\frac{f(\hat \beta)}{\partial \hat \beta_i} = -2 X'(y - X \hat \beta) = -2 \sum_{t=1}^n (y_t - \sum_{l=1}^k \beta_l x_{t,l}) x_{t,i}

Bedingung 2. Ordnung:


\frac{f^2(\hat \beta)}{\partial \hat \beta_i^2 } = 

\displaystyle \frac{-2 \sum_{t=1}^n (y_t - \sum_{l=1}^k \beta_l x_{t,l}) x_{t,i}}{\partial \hat \beta_i} =

\displaystyle \frac{
 -2 \displaystyle \sum_{t=1}^n y_t x_{t,i} + 2 \displaystyle \sum_{t=1}^n 
 \left( \beta_i x_{t,i}^2 + \sum_{l=1,l \ne i}^k \beta_l x_{t,l} x_{t,i} \right) 
}
{\partial \hat \beta_i }
= 2 \sum_{t=1}^n x_{t,i}^2 = x_{.,i} '  x_{.,i}


\frac{f^2(\hat \beta)}{\partial \hat \beta_i \partial \hat \beta_j} = 

\displaystyle \frac{-2 \sum_{t=1}^n (y_t - \sum_{l=1}^k \beta_l x_{t,l}) x_{t,i}}{\partial \hat \beta_j} =

\displaystyle \frac{
 -2 \displaystyle \sum_{t=1}^n y_t x_{t,i} + 2 \displaystyle \sum_{t=1}^n 
 \left( \beta_j x_{t,i} x_{t_j} + \sum_{l=1,l \ne j}^k \beta_l x_{t,l} x_{t,i} \right) 
}
{\partial \hat \beta_j }
= 2 \sum_{t=1}^n x_{t,i} x_{t,j} = x_{.,i} '  x_{.,j}


\begin{bmatrix}
\frac{f^2(\hat \beta)}{\partial \hat \beta_1^2} & 
\frac{f^2(\hat \beta)}{\partial \hat \beta_1 \partial \hat \beta_2} &
\cdots &
\frac{f^2(\hat \beta)}{\partial \hat \beta_1 \partial \hat \beta_k}\\ 

\frac{f^2(\hat \beta)}{\partial \hat \beta_2 \partial \hat \beta_1} &
\frac{f^2(\hat \beta)}{\partial \hat \beta_2^2} & 
\cdots &
\frac{f^2(\hat \beta)}{\partial \hat \beta_2 \partial \hat \beta_k}\\ 

\vdots & \vdots & \ddots & \vdots \\

\frac{f^2(\hat \beta)}{\partial \hat \beta_k \partial \hat \beta_1} &
\frac{f^2(\hat \beta)}{\partial \hat \beta_k \partial \hat \beta_2} & 
\cdots &
\frac{f^2(\hat \beta)}{\partial \hat \hat \beta_k ^2}  
\end{bmatrix} = 2 X'X

X'X positiv definit wenn Rank X = k

A_{m \times m} \, heisst positiv definit, wenn v' A v > 0 \, für alle v \in R^m \, ausser v = 0\,.

Angenommen A\, ist nicht positiv definit, dann existiert ein \alpha \in \mathbb{R}^k, \alpha \ne 0\, mit 
\alpha'(X'X) \alpha = 0 \,
Siehe Statistik 3 - Beispiel 25#Weshalb ist X'X invertierbar?