Lecture 11: PY 0782 - Advanced Quantitative Research Methods

Dr. Thomas Pollet, Northumbria University (thomas.pollet@northumbria.ac.uk)

2018-12-16 | disclaimer

1 / 33

PY0782: Advanced Quantitative research methods.

Last lecture: Multilevel: part I.
Today: Multilevel: part II.

I solved something for this lecture, remember those overflowing code bits, meet 'xaringan'

2 / 33

Reload the data.

setwd("~/Dropbox/Teaching_MRes_Northumbria/Lecture11")
library(mlmRev) # contains data
library(lme4)
Exam<-mlmRev::Exam
fixed_pred<-lmer(normexam ~ standLRT + (1 | school), data=Exam, REML=F)

3 / 33

Make a plot

library(ggplot2)
library(dplyr)
#subset 3 schools (just picked 3 from the dataframe)
subset<-filter(Exam, school=='1' | school=='17' | school=='18')
multilevelplot<-ggplot(subset,aes(standLRT, normexam)) + geom_jitter(alpha = 0.3) + facet_wrap(~school) + xlab("London Reading test") + ylab("Normed Exam Score") + geom_smooth(method="lm") + geom_hline(yintercept=0, linetype="dashed") + theme_bw()

4 / 33

Look at the pretty, pretty.

plot(multilevelplot)

5 / 33

Pros and cons.Positive is that it shows the actual data. But it doesn't really show what happens in a multilevel designs.
6 / 33

Pros and cons.

Positive is that it shows the actual data. But it doesn't really show what happens in a multilevel designs.
Can we have a graph with demonstrates everything again?

6 / 33

A random intercept and random slope.

Illustration based on this.

# pooled 
pooled.model <- lm(normexam ~ standLRT, data=Exam)
# Save the fitted values
Exam$PooledPredictions <- fitted(pooled.model)
# Intercept
varying.intercept.model <- lm(normexam ~ standLRT + school, data=Exam)
Exam$VaryingInterceptPredictions <- fitted(varying.intercept.model)
# Slope
varying.slope.model <- lm(normexam ~ standLRT:school, data=Exam)
Exam$VaryingSlopePredictions <- fitted(varying.slope.model)
# Interaction (both slope)
interaction.model <- lm(normexam ~ standLRT + school + standLRT:school, data=Exam)
Exam$InteractionPredictions <- fitted(interaction.model)

7 / 33

Build graph

We need a subset.

require(ggplot2)
require(dplyr)
subset<-filter(Exam, school=='1' | school=='18' | school=='21' | school=='40' | school=='55' | school=='59')
gg <- ggplot(subset, aes(x = standLRT, y = normexam, group = school)) +
    geom_line(aes(y = PooledPredictions), color = "darkgrey") +
    geom_line(aes(y = VaryingInterceptPredictions), color = "blue") +
    #geom_line(aes(y = VaryingSlopePredictions), color = "red") +
    #geom_line(aes(y = InteractionPredictions), color = "black") +
    geom_point(alpha = 0.3, size = 3) +
    facet_wrap(~school) + xlab("London Reading test") + ylab("Normed Exam Score") +
    theme_bw()

8 / 33

Graph: Random intercept.

print(gg)

9 / 33

Graph: Random slope.

require(ggplot2)
require(dplyr)
subset<-filter(Exam, school=='1' | school=='18' | school=='21' | school=='40' | school=='55' | school=='59')
gg <- ggplot(subset, aes(x = standLRT, y = normexam, group = school)) +
    geom_line(aes(y = PooledPredictions), color = "darkgrey") +
    # geom_line(aes(y = VaryingInterceptPredictions), color = "blue") +
    geom_line(aes(y = VaryingSlopePredictions), color = "red") +
    #geom_line(aes(y = InteractionPredictions), color = "black") +
    geom_point(alpha = 0.3, size = 3) +
    facet_wrap(~school) + xlab("London Reading test") + ylab("Normed Exam Score") +
    theme_bw()

10 / 33

Graph: Random slope

print(gg)

11 / 33

Graph: Slope and intercept.

require(ggplot2)
require(dplyr)
subset<-filter(Exam, school=='1' | school=='18' | school=='21' | school=='40' | school=='55' | school=='59')
gg <- ggplot(subset, aes(x = standLRT, y = normexam, group = school)) +
    geom_line(aes(y = PooledPredictions), color = "darkgrey") +
    # geom_line(aes(y = VaryingInterceptPredictions), color = "blue") +
    # geom_line(aes(y = VaryingSlopePredictions), color = "red") +
    geom_line(aes(y = InteractionPredictions), color = "black") +
    geom_point(alpha = 0.3, size = 3) +
    facet_wrap(~school) + xlab("London Reading test") + ylab("Normed Exam Score") +
    theme_bw()

12 / 33

Graph: Slope and intercept.

print(gg)

13 / 33

Try it yourself.

Use the 'Scottish schools' dataset and make those 3 graphs. (If you cannot load MLMrev, it should be available) from blackboard.

14 / 33

Common designs.You might have not cared so far as you only collect experimental data and multilevel models might not apply. Actually, they can be used for some of the designs you encounter.
15 / 33

Common designs.

You might have not cared so far as you only collect experimental data and multilevel models might not apply. Actually, they can be used for some of the designs you encounter.
Let us look at those.

15 / 33

Common designs.

You might have not cared so far as you only collect experimental data and multilevel models might not apply. Actually, they can be used for some of the designs you encounter.
Let us look at those.
Where subjects is each subject's id, tx represent treatment allocation and is coded 0 or 1, therapist is the refers to either clustering due to therapists, or for instance a participant's group in group therapies. Y is the outcome variable.

15 / 33

Repeated measures design.

16 / 33

Write some models.

A null model looks like this

# lme4
lmer(y ~ 1 + (1 | subjects), data=data)

A null growth model looks like this. ("Unconditional growth model")

# lme4
lmer(y ~ time + (time | subjects), data=data)

17 / 33

Conditional growth model.

Here we examine if treatment influences the outcome over time.

lmer(y ~ time * tx + (time | subjects), data=data)
# dropping a random slope.
lmer(y ~ time * tx + (1 | subjects), data=data)
# dropping a random intercept.
lmer(y ~ time * tx + ( 0 + time | subjects), data=data)

18 / 33

Three Levels.

Now imagine that we have therapists... .

lmer(y ~ time * tx + (time | therapist/subjects), data=df)

19 / 33

Crossed-over design (subject level)

In the previous example, a therapist could only offer either treatment or control. Randomization at therapist level

But often you'll have random allocation at the subject level.

lmer(y ~ time * tx + (time | therapist:subjects) + (time * tx || therapist), data=df)

20 / 33

Different level 3 variance-covariance strucures... .

We might hypothesize that therapists that are allocated participants that report worse symptoms at treatment start have better outcomes (more room for improvement). --> we solve this via modelling the variance-covariance matrix

lmer(y ~ time * tx + (time | therapist:subjects) + (time | therapist) + (0 + tx + time:tx | therapist), data=data)

21 / 33

Different level 3 variance-covariance strucures... .

It is also possible that when a therapist is successful with treatment A, that he/she will also be with B. We could model all such possible scenarios. This basically amounts to an unstructured variance-covariance matrix. (Luckily this is also the default for most packages.).

lmer(y ~ time * tx + (time | therapist:subjects) + (time * tx | therapist), data=df)

22 / 33

Glmer.

What if you don't have a normal distribution. For example, you have a forced choice task --> Binomial.

23 / 33

Glmer.

What if you don't have a normal distribution. For example, you have a forced choice task --> Binomial.

Extensions to non-linear models. Logit.

23 / 33

Glmer.

What if you don't have a normal distribution. For example, you have a forced choice task --> Binomial.

Extensions to non-linear models. Logit.

Example

# Example
m <- glmer(remission ~ IL6 + CRP + CancerStage + LengthofStay + Experience +
    (1 | DID), data = hdp, family = binomial)

23 / 33

family: Other models... .

Help! My data are not normal... . Pointers in Zuur et al. (2009).

Count data --> Poisson, Negative Binomial, -- 'Excess of zeroes'.
Ordinal --> probit / censored regression.
'Weird' functions. Gamma distribution.

24 / 33

Cool stuff, which I am unable to cover.

Machine learning. ('caret' package, Random forests) and text mining: check here.

25 / 33

Cool stuff, which I am unable to cover.

Machine learning. ('caret' package, Random forests) and text mining: check here.
Social network analysis. (Citation network analysis, for example)

25 / 33

Cool stuff, which I am unable to cover.

Machine learning. ('caret' package, Random forests) and text mining: check here.
Social network analysis. (Citation network analysis, for example)
Bayesian statistics. Check out McElreath, R. (2015). Statistical Rethinking. Texts in Statistical Science. CRC Press.

25 / 33

Cool stuff, which I am unable to cover.

Machine learning. ('caret' package, Random forests) and text mining: check here.
Social network analysis. (Citation network analysis, for example)
Bayesian statistics. Check out McElreath, R. (2015). Statistical Rethinking. Texts in Statistical Science. CRC Press.
Meta-analysis. Check out the amazing 'metafor' package.

25 / 33

Cool stuff, which I am unable to cover.

Machine learning. ('caret' package, Random forests) and text mining: check here.
Social network analysis. (Citation network analysis, for example)
Bayesian statistics. Check out McElreath, R. (2015). Statistical Rethinking. Texts in Statistical Science. CRC Press.
Meta-analysis. Check out the amazing 'metafor' package.
Statistical simulation. If you are interested, have a read of an example here.

25 / 33

Cool stuff, which I am unable to cover.

Machine learning. ('caret' package, Random forests) and text mining: check here.
Social network analysis. (Citation network analysis, for example)
Bayesian statistics. Check out McElreath, R. (2015). Statistical Rethinking. Texts in Statistical Science. CRC Press.
Meta-analysis. Check out the amazing 'metafor' package.
Statistical simulation. If you are interested, have a read of an example here.
Using R for writing. 'Shiny': App. building.

25 / 33

Rcmdr

SPSS-light.

(Thomas opens 'Rcmdr').

26 / 33

Running your analyses for your projects.Distinguish between exploratory and confirmatory analyses. Make analysis plan ahead.
27 / 33

Running your analyses for your projects.

Distinguish between exploratory and confirmatory analyses. Make analysis plan ahead.
Visual checks. Correlation matrices/plots. Any issues identified?

27 / 33

Running your analyses for your projects.

Distinguish between exploratory and confirmatory analyses. Make analysis plan ahead.
Visual checks. Correlation matrices/plots. Any issues identified?
Find the fitting analysis. (Most likely one we have seen?). Check assumptions! (Is it multilevel?)

27 / 33

Running your analyses for your projects.

Distinguish between exploratory and confirmatory analyses. Make analysis plan ahead.
Visual checks. Correlation matrices/plots. Any issues identified?
Find the fitting analysis. (Most likely one we have seen?). Check assumptions! (Is it multilevel?)
Run the analysis. Bootstrap if you can. Check if different methods lead to same conclusion.

27 / 33

Stuff which I have missed?

Any statistical tests you commonly employ that we have not covered?

28 / 33

Complete feedback form online.

What will (likely) change with regards to next year... .

Exercises (Do you want more?). Perhaps interactive. Might become mandatory.
...

Any feedback, points you want to raise?

29 / 33

Marks.

Still all to play for... .

Feedback via Turnitin.

You can post questions via blackboard. Book an appointment with me (check availability).

Questions on assignment via discussion board. (Unavailable Dec. 21st to Jan. 9th)

30 / 33

Exercise.

No set exercise, other than that I want you to explore an R package and see what it does. Alternatively, work through a tutorial. (see some examples here). or here

No inspiration then look through R-bloggers or datacamp.

Look at the vignette, example code, and try it on some data. Write it up in a small notebook.

31 / 33

References (and further reading.)

Also check the reading list! (many more than listed here).

Gelman, A., & Hill, J. (2006). Data analysis using regression and multilevel/hierarchical models. New York, NY: Cambridge University Press.
Hox, J. J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). London: Taylor & Francis.
Magnusson, K. (2015). Using R and lme/lmer to fit different two- and three-level longitudinal models http://rpsychologist.com/r-guide-longitudinal-lme-lmer
Nieuwenhuis, R. (2017). R-Sessions 16: Multilevel Model Specification (lme4) http://www.rensenieuwenhuis.nl/r-sessions-16-multilevel-model-specification-lme4/
Snijders, T. A. B., & Berkhof, J. (2008). Diagnostic Checks for Multilevel Models. In: Handbook of Multilevel Analysis (pp. 141–175). New York, NY: Springer New York. http://doi.org/10.1007/978-0-387-73186-5_3

32 / 33

References continued.

Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage Publications Limited.
Zuur, A., Ieno, E. N., Walker, N., Saveliev, A. A., & Smith, G. M. (2009). Mixed effects models and extensions in ecology with R. New York, NY: Springer.

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help