Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Comparing Approaches to Measure Variance in Multilevel Regression, Slides of Computational Methods

An overview of different approaches to measuring the proportion of variance accounted for in multilevel regression models. The author compares the methods proposed by snijders and bosker, xu, and nakagawa and schielzeth using the bryk & raudenbush hsb data. The document also includes r code for computing the xu pseudo-r-square measure.

What you will learn

  • How does the Snijders and Bosker approach differ from the Xu approach?
  • What is the Xu pseudo-R-square measure and how is it calculated?
  • What are the different approaches to measuring variance accounted for in multilevel regression models?

Typology: Slides

2021/2022

Uploaded on 09/27/2022

prouline
prouline 🇬🇧

4.6

(7)

221 documents

1 / 3

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Newsom
Psy 526/626 Multilevel Regression, Spring 2019 1
Example R2 Computation
In multilevel models, there are no accepted standards for measures of multiple R2 (total variance accounted for
in the outcome), although several have been proposed (for reviews, see LaHuis, Hartman, Hakoyama, & Clark,
2014; Roberts, Monaco, Stovall, & Foster, 2011). Below I illustrate a couple of the possibilities using the Bryk &
Raudenbush’s HSB data. mathach is the outcome variable. Following recommendations of the developers of
these indices, the estimates below were obtained using full maximum likelihood (FML) rather than the default
restricted maximum likelihood estimation. To simplify calculations, the variance of the SES slope was not
estimated. Snijders and Bosker (2012; see also LaHuis et al., 2012) recommend against estimating variance of
slopes when computing R-squared estimates, but Hox (2010, p. 76) provides a formula for models with random
slopes. To save space I only include the output form HLM. Neither SPSS nor HLM give R-square measures.
Results from the Empty Model:
Final estimation of fixed effects:
Fixed Effect Coefficient
Standard
error
t-ratio
Approx.
d.f.
p-value
For INTRCPT1, β0
INTRCPT2, γ00
12.636972
0.244412
51.704
159
<0.001
Final estimation of varianc e components
Random Effect
Standard
Deviation
d.f. χ2 p-value
INTRCPT1, u0
2.93501
159
1660.23259
<0.001
level-1, r
6.25686
Test of Model using SES (level 1), MEANSES, and SECTOR (level 2) as predictors of MATHACH:
Final estimation of fixed effects:
Fixed Effect Coefficient
Standard
error
t-ratio
Approx.
d.f.
p-value
For INTRCPT1, β0
INTRCPT2, γ00
12.109382
0.198644
60.960
157
<0.001
SECTOR, γ01
1.224539
0.306080
4.001
157
<0.001
MEANSES, γ02
3.145129
0.384611
8.177
157
<0.001
For SES slope, β1
INTRCPT2, γ10
2.191168
0.108673
20.163
7024
<0.001
Final estimation of varianc e components
Random Effect
Standard
Deviation
d.f. χ2 p-value
INTRCPT1, u0
1.53905
157
600.10638
<0.001
level-1, r
6.08465
Computations
The measure suggested by Snijders and Bosker (1999, pp. 102-103) is perhaps the most widely used. This
approach distinguishes proportion of variance accounted for in the individual-level outcome Yij by the level-l
predictors from the variance accounted for in the group-mean level outcome by the level-2 predictors.
Variance accounted for in
ij
Y
by level-1 predictors:
( ) ( )
( ) ( )
0
0
22
2
122
37.02 2.36
139.15 8.61
39.38
1 1 .83 .17
47.76
ˆˆ
1ˆˆ
full full
Rnull null
στ
στ
+
=
+
= =−=
+
= +
pf3

Partial preview of the text

Download Comparing Approaches to Measure Variance in Multilevel Regression and more Slides Computational Methods in PDF only on Docsity!

Psy 526/626 Multilevel Regression, Spring 2019 1

Example R^2 Computation

In multilevel models, there are no accepted standards for measures of multiple R^2 (total variance accounted for

in the outcome), although several have been proposed (for reviews, see LaHuis, Hartman, Hakoyama, & Clark,

2014; Roberts, Monaco, Stovall, & Foster, 2011). Below I illustrate a couple of the possibilities using the Bryk &

Raudenbush’s HSB data. mathach is the outcome variable. Following recommendations of the developers of

these indices, the estimates below were obtained using full maximum likelihood (FML) rather than the default

restricted maximum likelihood estimation. To simplify calculations, the variance of the SES slope was not

estimated. Snijders and Bosker (2012; see also LaHuis et al., 2012) recommend against estimating variance of

slopes when computing R -squared estimates, but Hox (2010, p. 76) provides a formula for models with random

slopes. To save space I only include the output form HLM. Neither SPSS nor HLM give R -square measures.

Results from the Empty Model:

Final estimation of fixed effects:

Fixed Effect Coefficient Standard error

t -ratio Approx. d.f.

p -value

For INTRCPT1, β 0 INTRCPT2, γ 00 12.636972 0.244412 51.704 159 <0.

Final estimation of variance components

Random Effect

Standard Deviation

Variance Component

d.f. χ^2 p -value

INTRCPT1, u 0 2.93501 8.61431 159 1660.23259 <0. level-1, r 6.25686 39.

Test of Model using SES (level 1), MEANSES, and SECTOR (level 2) as predictors of MATHACH:

Final estimation of fixed effects:

Fixed Effect Coefficient Standard error

t -ratio Approx. d.f.

p -value

For INTRCPT1, β 0 INTRCPT2, γ 00 12.109382 0.198644 60.960 157 <0. SECTOR, γ 01 1.224539 0.306080 4.001 157 <0. MEANSES, γ 02 3.145129 0.384611 8.177 157 <0. For SES slope, β 1 INTRCPT2, γ 10 2.191168 0.108673 20.163 7024 <0.

Final estimation of variance components

Random Effect Standard Deviation

Variance Component

d.f. χ^2 p -value

INTRCPT1, u 0 1.53905 2.36866 157 600.10638 <0. level-1, r 6.08465 37.

Computations

The measure suggested by Snijders and Bosker (1999, pp. 102-103) is perhaps the most widely used. This

approach distinguishes proportion of variance accounted for in the individual-level outcome Y ij by the level-l

predictors from the variance accounted for in the group-mean level outcome by the level-2 predictors.

Variance accounted for in Yij by level-1 predictors:

( ) ( )

( ) ( )

0

0

2 2 2 1 2 2

full full

R

null null

σ τ

σ τ

= −

= − = − =

Psy 526/626 Multilevel Regression, Spring 2019 2

Where the full refers to the model tested and null refers to the model without predictors, or the empty model.

2

σ is the within-group variance and τ 20 is the between group (or intercept) variance.

Variance accounted for in Y. j by level-2 predictors:

2 2 2 0 (^2 2 ) 0 37.02 (^) 2. 1 45 39.15 (^) 8. 45 .82 2. 1 .87 8.

1 1 .34.

full B full

R

null B null

= −

= −

= − = − =

B is the average cluster size in the notation used by Roberts and colleagues. I cheated on the computation for

B , because I simply took the arithmetic average by dividing the total sample size, 7185, by the number of

groups, 160. This approach may be vulnerable to the influence of groups with very large or very small sample

sizes. The harmonic mean of group sample sizes (i.e., average nj ) is recommended as a more accurate

computation of B.

Xu (2003) proposed an overall measure of variance accounted for that does not require specific reference to

level-1 or level-2 predictors or outcomes. I have yet to see this measure reported much in the literature to date,

but Xu's simulation work suggests that it performs well. Only the within-group variance is used in this

measure, which I obtain from the output above.

0

2 2

r

Xu uses σ^2 for the full model residual variance and σ 02 for the null model residual variance. Clearly the

proportion of variance accounted for differs substantially from these different approaches, so the definition

used has important implications for the conclusions one might draw.

R code for computing the Xu pseudo-R-square measure

library(sjstats) #used for r2 computation below

#get empty (or null) model modeln <- lmer (mathach ~ 1 + (1 | schoolid), data=mydata,REML=FALSE) summary(modeln) [output omitted] #full model with ML modelf <- lmer (mathach ~ ses + sector + meanses + (1 | schoolid), data=mydata,REML=FALSE) summary(modelf) [output omitted] #get R-square approximations--use FIML estimation instead of REML (REML = FALSE) as in model #compute the Xu (2003) measure (given as R-squared in output) #r2 function requires sjstats package, loaded before lmer run #note: Omega-squared in output matches Snijders & Bosker R-sq r2(modelf, modeln) R-squared (tau-00): 0. R-squared (tau-11): NA Omega-squared: 0. R-squared: 0.

Comments

Roberts and colleagues (2010) review several approaches to quantifying the proportion of variance accounted

for in the dependent variable. Reporting of these values is by no means universal at this point. One reason is

that there has been disagreement about the best approach, because there is no simple parallel to the R^2

obtained with standard OLS regression. In some instances, these proportion of variance measures can be

negative too. The issue of what to do with slope variance has also been a hindrance. More simulation work and