Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Model Selection - Handout #7 - Clinical Trials | STA 6973, Study notes of Statistics

Material Type: Notes; Professor: Oliveira; Class: SP-STA: Clinical Trials; Subject: Statistics; University: University of Texas - San Antonio; Term: Unknown 1989;

Typology: Study notes

Pre 2010

Uploaded on 07/30/2009

koofers-user-6jz
koofers-user-6jz 🇺🇸

10 documents

1 / 4

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
HANDOUT 7: Model Selection
This handout illustrates some methods for model selection:
AIC, BIC and cross validation.
Cross Validation.
Consider again the spatial rainfall data set “darwinw.dat”.
We entertain four models for this data set. The models have all a constant mean function
and no nugget, but they differ in the semivariogram function:
Model 1: Power exponential model with θ2= 1 (kappa in geoR) ;
Model 2: Power exponential model with θ2= 1.5 ;
Model 3: Mat´ern model with θ2= 1 ;
Model 4: Rational quadratic with θ2= 2 ;
> library(geoR)
##
## A possible criterion to select model is based how well each
## semivariogram model fits the empirical semivariogram, measured by the
## weighted residual sum of squares evaluated at the estimated values
##
> darwin.wls1 <- variofit(darwin.mom9, cov.model = "exp", ini = c(800,10),
+ fix.nugget = T, nugget = 0, weights = "cressie") ## Model 1
> darwin.wls4 <- variofit(darwin.mom9, cov.model = "powered.exp", ini = c(800,10),
+ fix.nugget = T, nugget = 0, weights = "cressie", kappa = 1.5) ## Model 2
> darwin.wls5 <- variofit(darwin.mom9, cov.model = "matern", ini = c(100,5),
+ fix.nugget = T, nugget = 0, weights = "cressie", kappa = 1) ## Model 3
> darwin.wls6 <- variofit(darwin.mom9, cov.model = "cauchy", ini = c(100,1),
+ fix.nugget = T, nugget = 0, weights = "cressie", kappa = 2) ## Model 4
>
> darwin.wls1$value ; darwin.wls4$value ; darwin.wls5$value ; darwin.wls6$value
[1] 7.485468
[1] 4.564043
[1] 4.658527
[1] 3.461156
##
## According to this criterion, Model 4 provides the best fit
##
1
pf3
pf4

Partial preview of the text

Download Model Selection - Handout #7 - Clinical Trials | STA 6973 and more Study notes Statistics in PDF only on Docsity!

HANDOUT 7: Model Selection

This handout illustrates some methods for model selection: AIC, BIC and cross validation.

Cross Validation.

Consider again the spatial rainfall data set “darwinw.dat”. We entertain four models for this data set. The models have all a constant mean function and no nugget, but they differ in the semivariogram function:

Model 1: Power exponential model with θ 2 = 1 (kappa in geoR) ; Model 2: Power exponential model with θ 2 = 1.5 ; Model 3: Mat´ern model with θ 2 = 1 ; Model 4: Rational quadratic with θ 2 = 2 ;

library(geoR)

A possible criterion to select model is based how well each

semivariogram model fits the empirical semivariogram, measured by the

weighted residual sum of squares evaluated at the estimated values

darwin.wls1 <- variofit(darwin.mom9, cov.model = "exp", ini = c(800,10),

  • fix.nugget = T, nugget = 0, weights = "cressie") ## Model 1

darwin.wls4 <- variofit(darwin.mom9, cov.model = "powered.exp", ini = c(800,10),

  • fix.nugget = T, nugget = 0, weights = "cressie", kappa = 1.5) ## Model 2

darwin.wls5 <- variofit(darwin.mom9, cov.model = "matern", ini = c(100,5),

  • fix.nugget = T, nugget = 0, weights = "cressie", kappa = 1) ## Model 3

darwin.wls6 <- variofit(darwin.mom9, cov.model = "cauchy", ini = c(100,1),

  • fix.nugget = T, nugget = 0, weights = "cressie", kappa = 2) ## Model 4

darwin.wls1$value ; darwin.wls4$value ; darwin.wls5$value ; darwin.wls6$value [1] 7. [1] 4. [1] 4. [1] 3.

According to this criterion, Model 4 provides the best fit

plot(darwin.mom9, xlab = "distance (km)", ylab = "semivariogram (mm^2)",

  • main = "Several models fitted to the darwin data by WLS")

lines(darwin.wls1) lines(darwin.wls4, lty = 2) lines(darwin.wls5, lty = 3) lines(darwin.wls6, lty = 4) legend(1,400,c("model", "power exp. theta_2 = 1", "power exp. theta_2 = 1.5",

  • "matern theta_2 = 1", "rational quadratic theta_2 = 2"), lty = 0:4, bty = "n")

dev.print(file = "fig7.1.ps")

0 2 4 6 8 10

0

100

200

300

400

Several models fitted to the darwin data by WLS

distance (km)

semivariogram (mm^2)

modelpower exp. theta_2 = 1 power exp. theta_2 = 1.5matern theta_2 = 1 rational quadratic theta_2 = 2

Figure 1:

mean(darwin.cv1$error^2) ; mean(darwin.cv4$error^2) [1] 47. [1] 48. mean(darwin.cv5$error^2) ; mean(darwin.cv6$error^2) [1] 48. [1] 60.

The best model in terms of cross-validation performance is Model 1.

Note that the worst model is model 4, in spite of the fact that

this model provides the best fit to the empirical semivariogram

datalook.f(cbind(darwin.gdat$coords, round(darwin.cv1$std.error, 2)),

  • main = "Standardized prediction residuals in Darwin using model 1")

dev.print(file = "file = fig7.2.ps")

Plot of the standardized prediction residuals

mean(darwin.cv1$error) [1] 0.

2 4 6 8 10

2

4

6

8

10

Standardized prediction residuals in Darwin using model 1

x−coord.

y−coord.

−0.71 0.

−1. 0.11 −0.

0.89 0.

−0.

0.87 −0.39 −0.

−0.79 (^) 0.25 −0.

2.1 0. −0.58 −0.

−1.

−0. −0.

0.2 0.

Figure 2: