3
$\begingroup$

I'm trying to wrap my head around the general formula of mixed models and how it relates to the system of equations I'm used to.

The general formula read like this:

$$\mathbf{Y_{j}}=\mathbf{X_{j} \beta} + \mathbf{Z_{j}b_{j}} + \epsilon_{j},$$

where:

  • $\mathbf{J}$ is a cluster ${j = 1,...,J}$ and for each cluster $j$ there are ${i = 1,...,n_{i}}$ observations
  • $\mathbf{Y}$ is the response vector of length $n_{i} \times 1$, where $n_{i}$ is the number of observations.
  • $\mathbf{X}$ is the fixed-effects design matrix of dimensions $n_{i} \times p$, where $p$ is the number of fixed-effect parameters.
  • $\boldsymbol{\beta}$ is the vector of fixed-effects coefficients of length $p \times 1$.
  • $\mathbf{Z}$ is the random-effects design matrix of dimensions $n_{i} \times q$, where $q$ is the number of random effects.
  • $\mathbf{b}$ is the vector of random effects of length $q \times 1$.
  • $\epsilon$ is the vector of residual errors of length $n_{i} \times 1$, often assumed to follow $\epsilon \sim N(0, \sigma^2 \mathbf{I})$, where $\sigma^2$ is the variance of the residual errors.

However, I'm unsure, how the general formula relates to e.g. a model like this. A simple random slope model with a Level 1 predictor $X$:

L1:

$y_{ij} = \beta_{0j} + \beta_{10} \cdot X_{ij} + \epsilon_{ij} $

L2:

$\beta_{0j} = \gamma_{00} + U_{0j}$

$\beta_{10} = \gamma_{10} + U_{1j} $

For one cluster I would write the same model in the general formula like this. However I'm unsure about the random effects design matrix $\mathbf{Z}$.

\begin{equation} \begin{bmatrix} y_{1j}\\ \vdots \\ y_{n_{i}j} \end{bmatrix} = \begin{bmatrix} 1 & X_{1j} \\ \vdots \\ 1 & X_{n_{i}j} \\ \end{bmatrix} \cdot \begin{bmatrix} \gamma_{00} \\ \gamma_{10} \\ \end{bmatrix} + \begin{bmatrix} 1 & X_{1j}\\ \vdots \\ 1 & X_{n_{i}j} \end{bmatrix} \cdot \begin{bmatrix} U_{0j} \\ U_{1j} \end{bmatrix} + \begin{bmatrix} \epsilon_{1j}\\ \vdots \\ \epsilon_{n_{i}j} \end{bmatrix} \end{equation}

For a more complex model like this - a random slope model with a cross level interaction and a level 2 predictor $M$.

L1:

$y_{ij} = \beta_{0j} + \beta_{10} \cdot X_{ij} + \epsilon_{ij} $

L2:
$\beta_{0j} = \gamma_{00} + \gamma_{01} \cdot M_{j} + U_{0j}$

$\beta_{10} = \gamma_{10} + \gamma_{11} \cdot M_{j} + U_{1j} $

I would write this model for one cluster like so:

\begin{equation} \begin{bmatrix} y_{1j}\\ \vdots \\ y_{n_{i}j} \end{bmatrix} = \begin{bmatrix} 1 & X_{1j} & M_{j} & M_{j} \cdot X_{1j} \\ \vdots \\ 1 & X_{n_{i}j} & M_{j} & M_{j} \cdot X_{n_{i}j} \end{bmatrix} \cdot \begin{bmatrix} \gamma_{00} \\ \gamma_{10} \\ \gamma_{01} \\ \gamma_{11} \end{bmatrix} + \begin{bmatrix} 1 & X_{1j} \\ \vdots \\ 1 & X_{n_{i}j} \end{bmatrix} \cdot \begin{bmatrix} U_{0j} \\ U_{1j} \end{bmatrix} + \begin{bmatrix} \epsilon_{1j}\\ \vdots \\ \epsilon_{n_{i}j} \end{bmatrix} \end{equation}

Is my understanding/usage of $\mathbf{Z}$ correct? Why do I have to specify the $X_{ij}$ again for the $\mathbf{Z}$ matrix?

$\endgroup$
4
  • $\begingroup$ I'm pretty sure there is a R package that can help with this (I will look - all I remember is that "equation" is in the name - I think). $Z$ is very often block-diagonal so I'm not sure you have it right (using my phone at the moment so not easy to check). Also, take a look at this answer which perhaps which should be very relevant (if not an actual dupe): stats.stackexchange.com/questions/49935/… $\endgroup$ Commented Oct 17, 2024 at 18:25
  • $\begingroup$ Oh, hang on - your $Z$ is for just one cluster, so it won't be block-diagonal - but when you form the whole $Z$ then it will be (I think) $\endgroup$ Commented Oct 17, 2024 at 18:32
  • 2
    $\begingroup$ equatiomatic is the package I was thinking about: cran.r-project.org/web/packages/equatiomatic/index.html $\endgroup$ Commented Oct 17, 2024 at 19:21
  • $\begingroup$ Yes, if formed for the whole $Z$ it would be block diagonal. $\endgroup$ Commented Oct 28, 2024 at 12:54

1 Answer 1

2
$\begingroup$

If someone is also struggling with this, the post above is correct, and we can prove it by first writing the equation for the models as one equation and then solving the matrix for one person. Now we see, that equation (1) equals equation (2) and equation (3) equals equation(4).

For a random slope model:

L1:

$y_{ij} = \beta_{0j} + \beta_{10} \cdot X_{ij} + \epsilon_{ij} $

L2:

$\beta_{0j} = \gamma_{00} + U_{0j}$

$\beta_{10} = \gamma_{10} + U_{1j} $

As one equation: $$y_{ij} = \gamma_{00} + U_{0j} + \gamma_{10} \cdot X_{ij} + U_{1j} \cdot X_{ij} + \epsilon_{ij} \hspace{2cm} (1) $$


In general formula for one cluster:

\begin{equation} \begin{bmatrix} y_{1j}\\ \vdots \\ y_{n_{i}j} \end{bmatrix} = \begin{bmatrix} 1 & X_{1j} \\ \vdots \\ 1 & X_{n_{i}j} \\ \end{bmatrix} \cdot \begin{bmatrix} \gamma_{00} \\ \gamma_{10} \\ \end{bmatrix} + \begin{bmatrix} 1 & X_{1j}\\ \vdots \\ 1 & X_{n_{i}j} \end{bmatrix} \cdot \begin{bmatrix} U_{0j} \\ U_{1j} \end{bmatrix} + \begin{bmatrix} \epsilon_{1j}\\ \vdots \\ \epsilon_{n_{i}j} \end{bmatrix} \end{equation}

For person 1 in group j: $$y_{1j} = 1 \cdot \gamma_{00} + X_{1j} \cdot \gamma_{10} + 1 \cdot U_{0j} + X_{1j} \cdot U_{1j} + \epsilon_{1j} \hspace{2cm} (2) $$


For a random slope model with a cross level interaction and a level 2 predictor $M$

L1:

$y_{ij} = \beta_{0j} + \beta_{10} \cdot X_{ij} + \epsilon_{ij} $

L2:

$\beta_{0j} = \gamma_{00} + \gamma_{01} \cdot M_{j} + U_{0j}$

$\beta_{10} = \gamma_{10} + \gamma_{11} \cdot M_{j} + U_{1j} $

As one equation:

$$ y_{ij} = \gamma_{00} + \gamma_{01} \cdot M_{j} + U_{0j} + \gamma_{10}\cdot X_{ij} + \gamma_{11} \cdot M_{j} \cdot X_{ij} + U_{1j} \cdot X_{ij} + \epsilon_{ij \hspace{2cm} (3)} $$


In general formula for one cluster: \begin{equation} \begin{bmatrix} y_{1j}\\ \vdots \\ y_{n_{j}j} \end{bmatrix} = \begin{bmatrix} 1 & X_{1j} & M_{j} & M_{j} \cdot X_{1j} \\ \vdots \\ 1 & X_{n_{i}j} & M_{j} & M_{j} \cdot X_{n_{i}j} \end{bmatrix} \cdot \begin{bmatrix} \gamma_{00} \\ \gamma_{10} \\ \gamma_{01} \\ \gamma_{11} \end{bmatrix} + \begin{bmatrix} 1 & X_{1j} \\ \vdots \\ 1 & X_{n_{i}j} \end{bmatrix} \cdot \begin{bmatrix} U_{0j} \\ U_{1j} \end{bmatrix} + \begin{bmatrix} \epsilon_{1j}\\ \vdots \\ \epsilon_{n_{i}j} \end{bmatrix} \end{equation}

For person 1 in group j: $$ y_{1j} = 1 \cdot \gamma_{00} + X_{1j} \cdot \gamma_{10} + M_{j} \cdot \gamma_{01} + M_{j} \cdot X_{1j} \cdot \gamma_{11} + 1 \cdot U_{0j} + X_{1j} \cdot U_{1j} + \epsilon_{1j} \hspace{2cm} (4) $$

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.