Prepare for your exams
Get points
Guidelines and tips

Prepare for your exams

Study with the several resources on Docsity

Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips

Sell on Docsity

Log in Sign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search Store documents

The best documents sold by students who completed their studies

Search through all study resources

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents

20 Points

For each uploaded document

Answer questions

5 Points

For each given answer (max 1 per day)

All the ways to get free points

Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

University Rankings

Discover the best universities in your country according to Docsity users

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

From our blog

Exams and Study

Go to the blog

Comparing Approaches to Measure Variance in Multilevel Regression, Slides of Computational Methods

University of Exeter Computational Methods

An overview of different approaches to measuring the proportion of variance accounted for in multilevel regression models. The author compares the methods proposed by snijders and bosker, xu, and nakagawa and schielzeth using the bryk & raudenbush hsb data. The document also includes r code for computing the xu pseudo-r-square measure.

What you will learn

How does the Snijders and Bosker approach differ from the Xu approach?
What is the Xu pseudo-R-square measure and how is it calculated?
What are the different approaches to measuring variance accounted for in multilevel regression models?

Typology: Slides

2021/2022

Uploaded on 09/27/2022

prouline 🇬🇧

4.6

(7)

221 documents

1 / 3

This page cannot be seen from the preview

Don't miss anything!

Newsom

Psy 526/626 Multilevel Regression, Spring 2019 1

Example R2 Computation

In multilevel models, there are no accepted standards for measures of multiple R2 (total variance accounted for

in the outcome), although several have been proposed (for reviews, see LaHuis, Hartman, Hakoyama, & Clark,

2014; Roberts, Monaco, Stovall, & Foster, 2011). Below I illustrate a couple of the possibilities using the Bryk &

Raudenbush’s HSB data. mathach is the outcome variable. Following recommendations of the developers of

these indices, the estimates below were obtained using full maximum likelihood (FML) rather than the default

restricted maximum likelihood estimation. To simplify calculations, the variance of the SES slope was not

estimated. Snijders and Bosker (2012; see also LaHuis et al., 2012) recommend against estimating variance of

slopes when computing R-squared estimates, but Hox (2010, p. 76) provides a formula for models with random

slopes. To save space I only include the output form HLM. Neither SPSS nor HLM give R-square measures.

Results from the Empty Model:

Final estimation of fixed effects:

Fixed Effect Coefficient

Standard

error

t-ratio

Approx.

d.f.

p-value

For INTRCPT1, β0

INTRCPT2, γ00

12.636972

0.244412

51.704

159

<0.001

Final estimation of varianc e components

Random Effect

Standard

Deviation

Variance

Component

d.f. χ2 p-value

INTRCPT1, u0

2.93501

8.61431

159

1660.23259

<0.001

level-1, r

6.25686

39.14831

Test of Model using SES (level 1), MEANSES, and SECTOR (level 2) as predictors of MATHACH:

Final estimation of fixed effects:

Fixed Effect Coefficient

Standard

error

t-ratio

Approx.

d.f.

p-value

For INTRCPT1, β0

INTRCPT2, γ00

12.109382

0.198644

60.960

157

<0.001

SECTOR, γ01

1.224539

0.306080

4.001

157

<0.001

MEANSES, γ02

3.145129

0.384611

8.177

157

<0.001

For SES slope, β1

INTRCPT2, γ10

2.191168

0.108673

20.163

7024

<0.001

Final estimation of varianc e components

Random Effect

Standard

Deviation

Variance

Component

d.f. χ2 p-value

INTRCPT1, u0

1.53905

2.36866

157

600.10638

<0.001

level-1, r

6.08465

37.02297

Computations

The measure suggested by Snijders and Bosker (1999, pp. 102-103) is perhaps the most widely used. This

approach distinguishes proportion of variance accounted for in the individual-level outcome Yij by the level-l

predictors from the variance accounted for in the group-mean level outcome by the level-2 predictors.

Variance accounted for in

ij

Y

by level-1 predictors:

( ) ( )

0

22

2

122

37.02 2.36

139.15 8.61

39.38

1 1 .83 .17

47.76

ˆˆ

1ˆˆ

full full

Rnull null

στ

+

= −

+

=− =−=

+

= − +

Partial preview of the text

Download Comparing Approaches to Measure Variance in Multilevel Regression and more Slides Computational Methods in PDF only on Docsity!

Psy 526/626 Multilevel Regression, Spring 2019 1

Example R^2 Computation

In multilevel models, there are no accepted standards for measures of multiple R^2 (total variance accounted for

in the outcome), although several have been proposed (for reviews, see LaHuis, Hartman, Hakoyama, & Clark,

2014; Roberts, Monaco, Stovall, & Foster, 2011). Below I illustrate a couple of the possibilities using the Bryk &

Raudenbush’s HSB data. mathach is the outcome variable. Following recommendations of the developers of

these indices, the estimates below were obtained using full maximum likelihood (FML) rather than the default

restricted maximum likelihood estimation. To simplify calculations, the variance of the SES slope was not

estimated. Snijders and Bosker (2012; see also LaHuis et al., 2012) recommend against estimating variance of

slopes when computing R -squared estimates, but Hox (2010, p. 76) provides a formula for models with random

slopes. To save space I only include the output form HLM. Neither SPSS nor HLM give R -square measures.

Results from the Empty Model:

Final estimation of fixed effects:

Fixed Effect Coefficient Standard error

t -ratio Approx. d.f.

p -value

For INTRCPT1, β 0 INTRCPT2, γ 00 12.636972 0.244412 51.704 159 <0.

Final estimation of variance components

Random Effect

Standard Deviation

Variance Component

d.f. χ^2 p -value

INTRCPT1, u 0 2.93501 8.61431 159 1660.23259 <0. level-1, r 6.25686 39.

Test of Model using SES (level 1), MEANSES, and SECTOR (level 2) as predictors of MATHACH:

Final estimation of fixed effects:

Fixed Effect Coefficient Standard error

t -ratio Approx. d.f.

p -value

For INTRCPT1, β 0 INTRCPT2, γ 00 12.109382 0.198644 60.960 157 <0. SECTOR, γ 01 1.224539 0.306080 4.001 157 <0. MEANSES, γ 02 3.145129 0.384611 8.177 157 <0. For SES slope, β 1 INTRCPT2, γ 10 2.191168 0.108673 20.163 7024 <0.

Final estimation of variance components

Random Effect Standard Deviation

Variance Component

d.f. χ^2 p -value

INTRCPT1, u 0 1.53905 2.36866 157 600.10638 <0. level-1, r 6.08465 37.

Computations

The measure suggested by Snijders and Bosker (1999, pp. 102-103) is perhaps the most widely used. This

approach distinguishes proportion of variance accounted for in the individual-level outcome Y ij by the level-l

predictors from the variance accounted for in the group-mean level outcome by the level-2 predictors.

Variance accounted for in Yij by level-1 predictors:

( ) ( )

0

2 2 2 1 2 2

full full

R

null null

σ τ

= −

= − = − =

Psy 526/626 Multilevel Regression, Spring 2019 2

Where the full refers to the model tested and null refers to the model without predictors, or the empty model.

2

σ is the within-group variance and τ 20 is the between group (or intercept) variance.

Variance accounted for in Y. j by level-2 predictors:

2 2 2 0 (^2 2 ) 0 37.02 (^) 2. 1 45 39.15 (^) 8. 45 .82 2. 1 .87 8.

1 1 .34.

full B full

R

null B null

= −

= − = − =

B is the average cluster size in the notation used by Roberts and colleagues. I cheated on the computation for

B , because I simply took the arithmetic average by dividing the total sample size, 7185, by the number of

groups, 160. This approach may be vulnerable to the influence of groups with very large or very small sample

sizes. The harmonic mean of group sample sizes (i.e., average nj ) is recommended as a more accurate

computation of B.

Xu (2003) proposed an overall measure of variance accounted for that does not require specific reference to

level-1 or level-2 predictors or outcomes. I have yet to see this measure reported much in the literature to date,

but Xu's simulation work suggests that it performs well. Only the within-group variance is used in this

measure, which I obtain from the output above.

0

2 2

r

Xu uses σ^2 for the full model residual variance and σ 02 for the null model residual variance. Clearly the

proportion of variance accounted for differs substantially from these different approaches, so the definition

used has important implications for the conclusions one might draw.

R code for computing the Xu pseudo-R-square measure

library(sjstats) #used for r2 computation below

#get empty (or null) model modeln <- lmer (mathach ~ 1 + (1 | schoolid), data=mydata,REML=FALSE) summary(modeln) [output omitted] #full model with ML modelf <- lmer (mathach ~ ses + sector + meanses + (1 | schoolid), data=mydata,REML=FALSE) summary(modelf) [output omitted] #get R-square approximations--use FIML estimation instead of REML (REML = FALSE) as in model #compute the Xu (2003) measure (given as R-squared in output) #r2 function requires sjstats package, loaded before lmer run #note: Omega-squared in output matches Snijders & Bosker R-sq r2(modelf, modeln) R-squared (tau-00): 0. R-squared (tau-11): NA Omega-squared: 0. R-squared: 0.

Comparing Approaches to Measure Variance in Multilevel Regression, Slides of Computational Methods

Related documents

Partial preview of the text

Download Comparing Approaches to Measure Variance in Multilevel Regression and more Slides Computational Methods in PDF only on Docsity!

Example R^2 Computation

In multilevel models, there are no accepted standards for measures of multiple R^2 (total variance accounted for

in the outcome), although several have been proposed (for reviews, see LaHuis, Hartman, Hakoyama, & Clark,

2014; Roberts, Monaco, Stovall, & Foster, 2011). Below I illustrate a couple of the possibilities using the Bryk &

Raudenbush’s HSB data. mathach is the outcome variable. Following recommendations of the developers of

these indices, the estimates below were obtained using full maximum likelihood (FML) rather than the default

restricted maximum likelihood estimation. To simplify calculations, the variance of the SES slope was not

estimated. Snijders and Bosker (2012; see also LaHuis et al., 2012) recommend against estimating variance of

slopes when computing R -squared estimates, but Hox (2010, p. 76) provides a formula for models with random

slopes. To save space I only include the output form HLM. Neither SPSS nor HLM give R -square measures.

Results from the Empty Model:

Test of Model using SES (level 1), MEANSES, and SECTOR (level 2) as predictors of MATHACH:

Computations

The measure suggested by Snijders and Bosker (1999, pp. 102-103) is perhaps the most widely used. This

approach distinguishes proportion of variance accounted for in the individual-level outcome Y ij by the level-l

predictors from the variance accounted for in the group-mean level outcome by the level-2 predictors.

Variance accounted for in Yij by level-1 predictors:

full full

R

null null

Where the full refers to the model tested and null refers to the model without predictors, or the empty model.

σ is the within-group variance and τ 20 is the between group (or intercept) variance.

Variance accounted for in Y. j by level-2 predictors:

full B full

R

null B null

B is the average cluster size in the notation used by Roberts and colleagues. I cheated on the computation for

B , because I simply took the arithmetic average by dividing the total sample size, 7185, by the number of

groups, 160. This approach may be vulnerable to the influence of groups with very large or very small sample

sizes. The harmonic mean of group sample sizes (i.e., average nj ) is recommended as a more accurate

computation of B.

Xu (2003) proposed an overall measure of variance accounted for that does not require specific reference to

level-1 or level-2 predictors or outcomes. I have yet to see this measure reported much in the literature to date,

but Xu's simulation work suggests that it performs well. Only the within-group variance is used in this

measure, which I obtain from the output above.

r

Xu uses σ^2 for the full model residual variance and σ 02 for the null model residual variance. Clearly the

proportion of variance accounted for differs substantially from these different approaches, so the definition

used has important implications for the conclusions one might draw.

R code for computing the Xu pseudo-R-square measure

Comments

Roberts and colleagues (2010) review several approaches to quantifying the proportion of variance accounted

for in the dependent variable. Reporting of these values is by no means universal at this point. One reason is

that there has been disagreement about the best approach, because there is no simple parallel to the R^2

obtained with standard OLS regression. In some instances, these proportion of variance measures can be

negative too. The issue of what to do with slope variance has also been a hindrance. More simulation work and