PSYC 3032 M
This is our LAST content-covering class 😩🥺(🥳?)
Next class, we will do a less formal website building Hackathon!
Data Analysis Assignment 2 is available
Participation: Please watch at least 3 other group presentations and add a comment or question for each
\(\sum_{i=1}^{100} (Please)\) fill out the course evaluations surveys sent by the university—it’s not for them, it’s for me!
Important
Returning to the example from last week
Recall that a researcher was interested in looking at the extent to which being a child of at least one alcoholic parent (compared to having no alcoholic parents) moderated the relationship between externalizing behaviour (focal predictor) and alcohol use in adolescents (outcome variable)
We previously dummy-coded coa=0 (no alcoholic parents) or coa=1 (at least one alcoholic parent).
We could extend this research question to re-operationalize CoA and instead code the variable based on the number of alcoholic parents, NoAP=0, NoAP=1, or NoAP=2
So, now, there is new categorical moderator, NoAP, with three different levels instead of our original CoA variable with only two levels
library(haven); library(tidyverse)
drink <- read_sav("drink.sav")
drink$NoAP <- as_factor(drink$numalc) # converting number of alcoholic parents var to
ggplot(drink, aes(x = NoAP, y = alcuse, fill = NoAP)) +
geom_boxplot( alpha = 0.85) +
geom_jitter(width = 0.15, color = "black", size = 1.5, alpha = 0.7) +
labs(x = "Number of alcoholic parents", y = "Alcohol Use", fill = "Number of alcoholic parents") +
theme_classic()
What do you make of this plot?
\[ \begin{array}{lcc} \hline &{\textbf{Dummy-code variable values}} \\ \hline \textbf{Level of IV} & \textbf{D1} & \textbf{D2} \\ \hline \text{No alcoholic parents} & 0 & 0 \\ \text{One alcoholic parent} & 1 & 0 \\ \text{Two alcoholic parents} & 0 & 1 \\ \hline \end{array} \]
\[\scriptsize \begin{array}{lcc} \hline &{\textbf{Dummy-code variable values}} \\ \hline \textbf{Level of IV} & \textbf{D1} & \textbf{D2} \\ \hline \text{No alcoholic parents} & 0 & 0 \\ \text{One alcoholic parent} & 1 & 0 \\ \text{Two alcoholic parents} & 0 & 1 \\ \hline \end{array}\]
\[\scriptsize \hat{Alcohol}_i= \beta_0 + {\color{darkcyan} {\beta_1}}Ext._i+ {\color{darkcyan} {\beta_2}}D1_{i} +{\color{darkcyan} {\beta_3}}D2_{i} + {\color{deeppink} {\beta_4}} (D1_{i} \times Ext._i) + {\color{deeppink} {\beta_5}}(D2_{i} \times Ext._i)\]
\(\hat{\beta}_0\) is the expected alcohol use for an adolescent with no alcoholic parents (NoAP=0) who has a score of 0 on externalizing behaviour
\(\hat{\beta}_1\) represents the relationship (simple slope) between externalizing and alcohol among the reference group (no alcoholic parents;NoAP=0)
\(\hat{\beta}_2\) and \(\hat{\beta}_3\) are the adjusted mean differences on alcohol use between those with one alcoholic parent (\(\hat{\beta}_2\)) or those with two (\(\hat{\beta}_3\)) and those with none who scored 0 on externalizing behaviour, respectively
Finally, the remaining interaction coefficients express the differences in slopes between those with one alcoholic parent (\(\hat{\beta}_4\)) or those with two (\(\hat{\beta}_5\)) and those with none, respectively, who also scored 0 on externalizing behaviour
Call:
lm(formula = alcuse ~ ext * NoAP, data = drink)
Residuals:
Min 1Q Median 3Q Max
-1.1881 -0.2272 -0.1301 0.1372 2.9590
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.10472 0.10582 0.990 0.3239
ext 0.10154 0.17165 0.592 0.5550
NoAP1 alcoholic parent -0.04595 0.15844 -0.290 0.7722
NoAP2 alcoholic parents 0.28481 0.37933 0.751 0.4539
ext:NoAP1 alcoholic parent 0.57232 0.22584 2.534 0.0122 *
ext:NoAP2 alcoholic parents 0.79183 0.57896 1.368 0.1733
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.6072 on 159 degrees of freedom
Multiple R-squared: 0.2165, Adjusted R-squared: 0.1918
F-statistic: 8.786 on 5 and 159 DF, p-value: 2.219e-07
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.10471789 0.1058161 0.9896214 0.32386281
ext 0.10153503 0.1716543 0.5915088 0.55501961
NoAP1 alcoholic parent -0.04595497 0.1584352 -0.2900554 0.77215181
NoAP2 alcoholic parents 0.28480643 0.3793336 0.7508073 0.45387837
ext:NoAP1 alcoholic parent 0.57231667 0.2258374 2.5341981 0.01223737
ext:NoAP2 alcoholic parents 0.79182829 0.5789603 1.3676729 0.17334452ext=0.102 is the simple slope of externalizing behaviour for the reference group (no alc. parents) when ext=0
NoAP1=-0.05 is the adjusted mean difference between those in NoAP=1 and those in NoAP=0 when ext=0
NoAP2=0.28 is the adjusted mean difference between those in NoAP=2 and those in NoAP=0 when ext=0
ext:NoAP1= 0.57 is the slope difference between those in NoAP=1 and those in NoAP=0 when ext=0
ext:NoAP2=0.79 is the slope difference between those in NoAP=2 and those in NoAP=0 when ext=0
With a dichotomous moderator variable, we focused on the regression coefficient (slope) for the single interaction term; if this value was large and statistically significant, we would conduct follow up tests (e.g., simple slope analysis) to probe the interaction
With three or more levels of a moderator, we cannot examine the statistical significance of a single regression coefficient to determine whether an interaction is statistically significant
Instead, we have to examine the joint significance of the regression slopes for each product term. In this example, this means we need a test for the joint impact of both \(\beta_4\) and \(\beta_5\)
We can, therefore, think of the two product terms, (\(Ext. \times D1\)) and (\(Ext. \times D2\)), as a set of variables
\[\scriptsize \hat{Alcohol}_i= \beta_0 + {\color{darkcyan} {\beta_1}}Ext._i+ {\color{darkcyan} {\beta_2}}D1_{i} +{\color{darkcyan} {\beta_3}}D2_{i}\]
\[\scriptsize \hat{Alcohol}_i= \beta_0 + {\color{darkcyan} {\beta_1}}Ext._i+ {\color{darkcyan} {\beta_2}}D1_{i} +{\color{darkcyan} {\beta_3}}D2_{i} + {\color{deeppink} {\beta_4}} (D1_{i} \times Ext._i) + {\color{deeppink} {\beta_5}}(D2_{i} \times Ext._i)\]
Analysis of Variance Table
Model 1: alcuse ~ ext + NoAP
Model 2: alcuse ~ ext * NoAP
Res.Df RSS Df Sum of Sq F Pr(>F)
1 161 61.227
2 159 58.613 2 2.6139 3.5453 0.03116 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\(\Delta R^2=0.035\), 95% CI[-.01, .08], F(2)=3.55, p = .031
Because the change in \(R^2\) is statistically significant, we can say that the interaction is statistically significant
Note, however, that it’s possible for the two interaction coefficients to be significant as a set, but then neither of the individual interaction coefficients is significant (and vice versa!)
If this occurs, we can still say that the categorical variable significantly moderates the relationship between the focal predictor and the outcome variable
sim_slopes() function from the interactions package:SIMPLE SLOPES ANALYSIS
When NoAP = no alcohol parents:
Est. S.E. 2.5% 97.5% t val. p
--------------------------- ------ ------ ------- ------- -------- ------
Slope of ext 0.10 0.17 -0.24 0.44 0.59 0.56
Conditional intercept 0.10 0.11 -0.10 0.31 0.99 0.32
When NoAP = 1 alcoholic parent:
Est. S.E. 2.5% 97.5% t val. p
--------------------------- ------ ------ ------- ------- -------- ------
Slope of ext 0.67 0.15 0.38 0.96 4.59 0.00
Conditional intercept 0.06 0.12 -0.17 0.29 0.50 0.62
When NoAP = 2 alcoholic parents:
Est. S.E. 2.5% 97.5% t val. p
--------------------------- ------ ------ ------- ------- -------- ------
Slope of ext 0.89 0.55 -0.20 1.99 1.62 0.11
Conditional intercept 0.39 0.36 -0.33 1.11 1.07 0.29
With a categorical moderator variable, we are able to examine simple intercepts and simple slopes for the focal predictor at each level of the moderator variable
But, continuous variables don’t have discrete, pre-defined levels…oh oh!
Instead, we have to choose the levels of the continuous moderator to examine…but, more on this shortly
For now, let’s look at a research example from Andy Field
How Does Callousness Affect the Relationship between Video Gaming and Aggression?
A research is interested demonstrating that callous traits (e.g., lack of guilt, lack of empathy, callous use of others for personal gain) moderates the relationship between amount of time spent playing video games (focal predictor) and aggression (outcome) among N=442 adolescents
Callousness is a continuous variable ranging from 0 to 43 points in the sample
Playing video games is also a continuous variable, measured in hours per week
Aggression is a continuous measure with a total possible range from 0 to 100 points
Descriptive Statistics
Variable n nNA pNA M SD Min Max Skew Kurt
ID 442 0 0.00% 221.50 127.74 1.00 442.00 0.00 -1.20
Aggress 442 0 0.00% 40.05 12.60 9.00 82.00 0.16 -0.01
Vid_Game 442 0 0.00% 21.84 6.96 1.00 38.00 -0.02 -0.17
callousness 442 0 0.00% 18.60 9.62 0.00 43.00 0.30 -0.77library(GGally)
my_fn <- function(data, mapping, method1="lm", method2= "loess",...){
p <- ggplot(data = data, mapping = mapping) +
geom_point(color="deepskyblue3") +
geom_smooth(method=method1, size=2, colour="black")+
geom_smooth(method=method2, size=2, span = 0.7, colour="purple", alpha=0.2, linetype="dashed")+theme_minimal()
p
}
ggpairs(games %>% select(Aggress, Vid_Game, callousness), aes(alpha = 0.5),
columnLabels = c("Aggression", "Hours Playing", "Callousness"),
lower = list(continuous = my_fn),
upper = list(continuous = 'cor'))
To best capture the relationship between each of the two continuous predictors—callousness and hours playing video games—with the outcome variable, aggression, we should look at the regression plane, as we did in Module 3
The difference, of course, is that, unlike in Module 3 (MLR with no interaction), the relationship between the focal predictor and outcome is not constant across all levels of the 2nd predictor
Because the relationship between hours playing video games and aggression varies at different levels of callousness, we should expect the regression plane to “warp”
The following figure demonstrates how the regression plane bends or twists when two continuous variables interact
The warped regression plane in the previous slide definitely demonstrates an interaction, but we would still want to get a stronger sense of the size of the effect and our inferential statistics for the regression model
So let’s estimate the model corresponding to the research example based on the following regression model:
\[\scriptsize \hat{Aggression}_i = \beta_0 + {\color{darkcyan} {\beta_1}}(Video \ games_i) + {\color{darkcyan} {\beta_2}}(Callousness_i) + {\color{deeppink} {\beta_3}}(Video \ games_i \times Callousness_i)\]
lm() Output
Call:
lm(formula = Aggress ~ Vid_Game * callousness, data = games)
Residuals:
Min 1Q Median 3Q Max
-29.7144 -6.9087 -0.1923 6.9036 29.2290
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 33.120233 3.427254 9.664 < 2e-16 ***
Vid_Game -0.333597 0.150826 -2.212 0.027495 *
callousness 0.168949 0.161049 1.049 0.294731
Vid_Game:callousness 0.027062 0.006981 3.877 0.000122 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 9.976 on 438 degrees of freedom
Multiple R-squared: 0.3773, Adjusted R-squared: 0.373
F-statistic: 88.46 on 3 and 438 DF, p-value: < 2.2e-16 Estimate Std. Error t value Pr(>|t|)
(Intercept) 33.12023276 3.427253805 9.663782 3.728541e-20
Vid_Game -0.33359731 0.150825524 -2.211809 2.749531e-02
callousness 0.16894909 0.161048775 1.049055 2.947311e-01
Vid_Game:callousness 0.02706219 0.006980821 3.876648 1.221295e-04Interpreting Effects
(Intercept)/ \(\hat{\beta}_0=33.12\): the expected aggression score for a person who has no callousness and doesn’t spend any time playing video games
Vid_Game/ \(\hat{\beta}_1=-0.33\): the effect of time playing video games on aggression among those who scored 0 on callousness; i.e., for every additional hour of playing per week, we expect an aggression score lower by 0.33 points among those who have no callousness
callousness/ \(\hat{\beta}_2=0.17\): the effect of callousness on aggression among those who don’t play video games; for every additional 1-point on the callousness scale, we expect an aggression score higher by 0.17 points among those who don’t play video games
Vid_Game:callousness/ \(\hat{\beta}_3=0.03\): the expected change in the relationship between playing video games and aggression corresponding to a one-point increase on the callousness scale; that is, the slope of playing video games is increasing/steeper by 0.03-points, for each additional 1 point increase on callousness
The interaction term is statistically significant, which means there is sufficient evidence that the relationship between time spent playing video games and aggression depends on an individual’s callousness
But, we still don’t know exactly how callous traits impacts the relationship between gaming and aggression. For that, we would need to probe the interaction and estimate the simple effects
Recall that, with a categorical moderator, we need to calculate \(K\) simple regression equations, one for each of the \(K-\)level moderator
But, because our moderator is now continuous, in theory, there could be an infinite number of simple effect equations (oh no!)
callousness=2.01, callousness=2.0099, callousness=2.00999, etc.An infinite number of simple slopes is not very useful, we’ll have to decide on a few “locations” on the moderator that make the most sense, but how!?
It’s our job to decide which “locations” (and how many) across the moderator the relationship between the focal predictor and outcome portrays the most truthful, yet interesting story about our data in relation to the research question
The number of values chosen is flexible, but common convention in psychology is to choose three levels across the continuous moderator
Examining the relationship at three levels allows us to get a snapshot of the relationship between the focal predictor and outcome for those with low, medium, and high levels of the moderator
Standard Deviations: We can choose one SD below the mean, at the mean, and one SD above the mean to represent low, medium, and high locations of the moderator variable, respectively
Percentiles: Alternatively, we could look at the 25th, 50th (i.e., median), and 75th percentiles values of the moderator (might be more appropriate if the moderator is not normally distributed)
Research Question- or Theory-based: If possible, you should use levels of the moderator that you believe to be meaningful within the context of your own research questions or theory (recommended!)
Best Practices in Levels Selection
The best strategy for choosing “levels” (the values themselves and number of values to examine) of your continuous moderator should be by thinking carefully about the moderator variable conceptually
For example, what would be meaningful values of “low” and “high”?
Another way you could make the decision about moderator levels is to use clinical criteria
e.g., Low = 15 (classified as mild depression), Medium = 25 (moderate depression), and High = 35 (severe depression)
And another thing, ensure that your selection is organic to the metric of the moderator, e.g., if your moderator has a discrete “flavour” (e.g., number of courses completed during your degree), use whole numbers for location values, even if your selection is guided by SD or percentiles (e.g., 22.57 courses completed is less practical)
In our particular research example, the measure of callous traits does not have a well-known metric to help us decide on what constitutes a “low”, “medium”, and “high”
And, because we have a large sample which spans across the full range of values and is roughly normally distributed, we will use the SD-based locations
| Min | Max | Mean | SD | SD_below | SD_above | Median | prcntl_25th | prcntl_75th |
|---|---|---|---|---|---|---|---|---|
| 0 | 43 | 18.6 | 9.62 | 8.98 | 28.21 | 18 | 11 | 26 |
Low callousness = 8.98 (callous mean minus 1SD)
Medium callousness = 18.60 (mean of callous)
High callousness = 28.21 (callous mean plus 1SD)
To get the simple intercepts and slopes for the relationship between video games and aggression, we could calculate the three simple regression equations corresponding to low, medium, and high callousness by plugging in the values we chose (above)
Or, we could let R do it for us…
Specifying Moderator Values with sim_slopes()
With a continuous variable, the default of sim_slopes() is to use the SD approach. But, you can easily change it manually by specifying the exact moderator values in the argument modx.values =, e.g., modx.values = c(11, 18, 26) for the 25th, 50th, and 75th percentile values as we observed in the descriptive statistics table in the previous slide
JOHNSON-NEYMAN INTERVAL
When callousness is OUTSIDE the interval [2.48, 17.38], the slope of
Vid_Game is p < .05.
Note: The range of observed values of callousness is [0.00, 43.00]
SIMPLE SLOPES ANALYSIS
When callousness = 8.97733 (- 1 SD):
Est. S.E. 2.5% 97.5% t val. p
--------------------------- ------- ------ ------- ------- -------- ------
Slope of Vid_Game -0.09 0.10 -0.29 0.10 -0.91 0.36
Conditional intercept 34.64 2.24 30.23 39.04 15.46 0.00
When callousness = 18.59502 (Mean):
Est. S.E. 2.5% 97.5% t val. p
--------------------------- ------- ------ ------- ------- -------- ------
Slope of Vid_Game 0.17 0.07 0.04 0.30 2.48 0.01
Conditional intercept 36.26 1.57 33.18 39.34 23.13 0.00
When callousness = 28.21272 (+ 1 SD):
Est. S.E. 2.5% 97.5% t val. p
--------------------------- ------- ------ ------- ------- -------- ------
Slope of Vid_Game 0.43 0.09 0.25 0.61 4.64 0.00
Conditional intercept 37.89 2.17 33.63 42.14 17.49 0.00Example Results Write-up
Time playing video games, callousness, and their interaction accounted for a substantial amount—about 38%—of the variability in aggression among adolescents, \(R^2=.38\), 95% CI[.31,.44], F(3, 438) = 88.46, p < .001. See Table X for the regression analysis results.
The relationship between laying video games and aggression use was significantly moderated by callousness, \(\hat{\beta}_3 = 0.027\), 95% CI [0.01, 0.04], t(438) = 3.88, p < .001, suggesting that for every one unit increase in callousness, the relationship between time playing video games and aggression is expected to be stronger by about 0.03-points (i.e., the slope will be steeper by 0.03 points); that is, the effect of time spent playing video games on aggression among teenagers becomes stronger as with higher levels of callousness traits.
Tests of simple slopes indicated that for adolescents who are lower on callousness (with a callousness score close to 9), the effect of time playing video games on aggression was small and not statistically significant \(\hat{\beta}_{callous-low} =-0.09\), 95% CI [-0.29, 0.10], t(438) = -0.91, p = .36. However, among adolescents with moderate callousness (with a callousness score close to 19), the effect of time playing video games on aggression was larger and statistically significant, \(\hat{\beta}_{callous-medium} =0.17\), 95% CI [33.18 39.34], t(438) = 23.13, p<.001; that is, among persons with an average level of callousness, we expect an aggression score that is higher by 0.17-point for every one additional hour of playing video games per week. This effect, although statistically significant, is very small (less than 0.25% of the scale range, i.e., 0-100) in relation to a full additional hour of gaming per week, and likely carries little to no meaningful implications.
…see next slide for more (sorry)
Example Results Write-up
…continued from previous slide
Finally, among adolescents with higher levels callousness (with a callousness score close to 28), the effect of time playing video games on aggression was larger and statistically significant, \(\hat{\beta}_{callous-high} =0.43\), 95% CI [33.63 42.14], t(438) = 17.49, p<.001; that is, among persons with high callousness, we expect a 0.43-points higher score on the aggression scale for every one additional hour of playing video games per week. Again, while this effect is statistically significant, it is still quite small (less than 0.5% of the scale range) and it is questionable whether it represents a meaningful difference in practice. See Figure X for a plot of the interaction.
Explaining Aggression from Playing Video Games and Callousness Among Adolescents
\[\scriptsize \begin{array}{lcccccc} \hline \text{Variable} & \hat{\beta} & SE(\hat{\beta}) & t & p & \text{95% CI} & sr^2 \\ \hline \text{(Intercept)} & 33.12 & 3.43 & 9.66 & <.001 & [26.38, 39.86] & \\ \text{Hours playing video games} & -0.33 & 0.15 & -2.12 & .027 & [-0.63, -0.037] & 0.01 \\ \text{Callousness} & 0.17 & 0.16 & 1.05 & .29 & [-0.15, 0.48] & 0.34 \\ \text{Hours playing video game} \ \times \text{Callousness} & 0.03 & 0.007 & 3.88 & <.001 & [0.01, 0.04] & 0.02 \\ \hline R^2 = .38, \ F(3, 428) = 88.46, \ p < .001 \end{array}\]
We have 2 remaining “issues”
e.g., callousness/ \(\hat{\beta}_2=0.17\): the effect of callousness on aggression among those who don’t play video games; for every additional 1-unit on the callousness scale, we expect an aggression score higher by 0.17 among those who don’t play video games
We can solve both “issues” by mean-centring the predictors
Mean-centring simply means (pun not intended) taking each observation and subtracting the mean of the variable from it, for example:
Callousness
| Min | Max | Mean | SD | SD_below | SD_above | Median | prcntl_25th | prcntl_75th |
|---|---|---|---|---|---|---|---|---|
| 0 | 43 | 18.6 | 9.62 | 8.98 | 28.21 | 18 | 11 | 26 |
Centred callousness
| Min | Max | Mean | SD | SD_below | SD_above | Median | prcntl_25th | prcntl_75th |
|---|---|---|---|---|---|---|---|---|
| -18.6 | 24.4 | 0 | 9.62 | -9.62 | 9.62 | -0.6 | -7.6 | 7.4 |
Estimate Std. Error t value Pr(>|t|)
(Intercept) 39.96710840 0.475056978 84.131189 1.717665e-272
Vid_Game_MC 0.16962470 0.068472677 2.477261 1.361583e-02
callousness_MC 0.76009260 0.049458276 15.368360 6.122177e-43
Vid_Game_MC:callousness_MC 0.02706219 0.006980821 3.876648 1.221295e-04e.g., callousness_MC/ \(\hat{\beta}_2=0.76\): the effect of callousness on aggression among those who play video games an average amount of time; for every additional 1 point on the callousness scale, we expect an aggression score higher by 0.76 points among individuals whose play video games roughly 21 hours per week
…But, these are not actually issues. Why, you ask?
In the context of moderation, multicollinearity does not affect the parameter estimates (including \(R^2\)), it only impacts the standard errors for the main effects (and thus their t and p values)
Why this isn’t a problem? in our model, the predictor-level parameter estimate and significance test that we care about the most is the slope of the interaction which is unaffected by the multicollinearity
And, as we discussed last lecture, the focus should be placed on the higher-order effects and simple slopes when an interaction is present!
A related issue is that it does not make theoretical sense to center a categorical variable, and so we shouldn’t do it…
When a model includes an interaction, we can use the same graphical and numeric procedures to check the linearity, homogeneity of variance, and normal distribution assumptions. Likewise, we can examine extreme observations in terms of their leverage, discrepancy, and influence
And don’t forget to check linearity in simple slopes. Remember linearity.check=TRUE?
Finally, you may have noticed the JOHNSON-NEYMAN INTERVAL section in the output of the sim_slopes()
The Johnson-Neyman (1936) method is a statistical technique used to identify the specific values or ranges on the moderator variable within which the simple slope of the focal predictor is statistically significant
In exploratory and/or predictive modeling, this might be helpful in determining where, along the continuum of the moderator, the interaction effect is strong enough to yield statistically significant simple slopes
…BUT (and it’s a big one folks), if our goal is explanatory and/or confirmatory modeling, we should lean on theory and practical significance, not rely on statistical significance to guide our selection of moderator locations (remember “Best Practices in Levels Selection”?)
