Linear Regression Model (LRM)

Matrix form of Linear Regression Model

The compact form of the Linear Regression Model is

$$ y := X\; \beta + \epsilon, \; \epsilon :\sim i.i.d.(0,\sigma^2_{\epsilon} I) $$

with

$$ \begin{gather*}

y := (y_1, y_2, ..., y_T)', \text{observed}\\\\

X := \begin{pmatrix}

1 & x_{11} & ... & x_{1p} \\

... & ... & ... & ... \\

1 & x_{T1} & ... & x_{Tp}

\end{pmatrix}; \text{observed} \\\\

\beta := [\beta_0, \beta_1, ..., \beta_p]'; \text{unknown in real-life applications} \\\\

\epsilon := (\epsilon_1, ..., \epsilon_T)'; \text{unknown in real-life applications} \end{gather*} $$

Example and explanation

AR(1)

AR(1): $y_t := \phi_0 + \phi_1\times y_{t-1} + \epsilon_t; \; \epsilon_t \sim iid(0,\sigma^2_{\epsilon}).$

AR(1) as Linear Regression Model:

$$ \begin{align*}y :&= X\beta + \epsilon \\\\\underbrace{y}{(T-1)\times 1} &= (y_2, ..., y_T)'\\\\\underbrace{X}{(T-1) \times 2} &= \begin{pmatrix} 1 & y_1 \\ ... & ... \\ 1 & y_{T-1} \end{pmatrix} \\\\\underbrace{\epsilon}{(T-1)\times 1} &= (\epsilon_2, ..., \epsilon{T})' \, :\sim iid (0,\sigma^2_{\epsilon})\\\\\beta &= (\phi_0, \phi_1)'.\end{align*} $$

Explanation:

vector $y$ contains values (yields) for all $y_t$ (except $y_1$)
for example, $y_2 := \phi_0 + \phi_1 \times y_{1} + \epsilon_t;$
first column of $X$ is 1, because the $\phi_0$ has no coefficient
with this LRM we can handle the equations for all $y_t$ as one matrix

Matrix form of Linear Regression Model

Example and explanation

AR(1)

ARMA(1,1)