Test of Mediation

90
The ACL-BT comparisons had the highest kappas and the BCa -BT comparisons had
the lowest kappas. In general, the kappa values obtained in the pairwise comparisons support
what the Type I error and Power graphs (Figure 4.1 - 4.16) illustrated, which is that the ACL
and the BT performed the most similarly and that the BCa and the BT were the least similar
to one another in performance.
Table 4.7 shows how K was computed. The calculations in Table 4.7 are based on the
values from Condition 10 of the ACL-BT comparisons (Table 4.4). First, the marginals are
computed by adding the observed proportions within each row and within each column.
Next, the proportions of agreement between methods expected by chance (shown in
parentheses) are calculated for the two cells on the main diagonal by multiplying the
marginals. P0 (the proportion of agreement between the two methods) is computed by
adding the observed proportions in the agreement diagonal. Pc (the proportion of units for
which agreement is expected by chance) is computed by adding the parenthetical values in
the agreement diagonal.

91

Table 4.7

K for Condition 10 of ACL-BT Comparisons

Method 1: ACL

Significant Non-significant

Significant .047 (.002491) 0 XJ47
.947 (.902491) .953
Method 2: BT

Non-significant .006

XJ53 !947

p0= .047 + .947= 0.994

pc = .002491 + .902491 = .904982

; ; - ^ ~ ^ - -089018 „ 9 3 ?
\-pc .095018

92
The interpretation of kappas is problematic in some conditions. One factor that
impacted the interpretation of the kappa values in this study is range restriction for conditions
in which a very high proportion of "correct" decisions (e.g. both methods fail to reject Ho in
Tables 4.4 to 4.6) was observed. When base rates for the other outcome (e.g. Type I error in
Tables 4.4 to 4.6) are very small, kappa values are attenuated.
Conditions 2 and 3 in the BCa -ACL comparisons (Table 4.6) illustrate how, when
base rates for one of the outcomes (reject Ho, in this case) are very low, slight variations in
the off-diagonal cells affect the kappa values. Although in both conditions the two methods
agreed on 993 of 1000 cases (the cases where both methods concluded that there was no
significant effect), a slight variation in the way the two conditions did not overlap with one
another resulted in a kappa of .726 for Condition 2 and .249 for Condition 3.
As we will see in the next set of tables (Tables 4.8 to 4.10), low base rates in their
most extreme form (i.e. one outcome is not observed at all) leads to the computation of Pc=l
so that the denominator of kappa is zero, and kappa is undefined.

93

Table 4.8

Concordance between ACL andBT When an Indirect Effect Exists (ab>0)

Both reject Both commit A C L only - B T only -

cn a b Ho Type II error reject H 0 reject H 0 Po ACL-BT K
0.992 0.829
21 50 0.14 0.14 20 972 8 0 0.979 0.873
0.979 0.948
22 100 0.14 0.14 80 899 19 2 0.98 0.943

23 200 0.14 0.14 272 707 19 2

24 500 0.14 0.14 765 215 18 2

25 50 0.14 0.39 122 852 26 0 0.974 0.889
1 0.976 0.942
26 100 0.14 0.39 276 700 23 0 0.982 0.964
0 0.994 0.965
27 200 0.14 0.39 520 462 18

28 500 0.14 0.39 901 93 6

29 50 0.14 0.59 180 804 16 0 0.984 0.948
0 0.987 0.970
30 100 0.14 0.59 313 674 13 0 0.997 0.994
0 0.998 0.989
31 200 0.14 0.59 512 485 3

32 500 0.14 0.59 896 102 2

37 50 0.39 0.14 116 861 22 1 0.977 0.897
1 0.973 0.929
38 100 0.39 0.14 239 734 26 0 0.986 0.972
0 0.997 0.990
39 200 0.39 0.14 458 528 14

40 500 0.39 0.14 818 179 3

41 50 0.39 0.39 617 349 33 1 0.966 0.927
0 0.992 0.905
42 100 0.39 0.39 952 40 8 0 1 1.000
0 1 undefined
43 200 0.39 0.39 999 1 0

44 500 0.39 0.39 1000 0 0

45 50 0.39 0.59 799 182 19 0 0.981 0.939
0 1 1.000
46 100 0.39 0.59 978 22 0 0 1 undefined
0 1 undefined
47 200 0.39 0.59 1000 0 0

48 500 0.39 0.59 1000 0 0

53 50 0.59 0.14 117 866 17 0 0.983 0.923

54 100 0.59 0.14 179 809 12 0 0.988 0.960

55 200 0.59 0.14 342 646 12 0 0.988 0.974

56 500 0.59 0.14 718 282 0 0 1 1.000

57 50 0.59 0.39 651 326 23 0 0.977 0.949
58 100 0.59 0.39 920 74 6 0 0.994 0.958
59 200 0.59 0.39 996 4 0 0 1 1.000
60 500 0.59 0.39 1000 0 0 0 1 undefined

61 50 0.59 0.59 965 30 5 0 0.995 0.921

62 100 0.59 0.59 1000 0 0 0 1 undefined

63 200 0.59 0.59 1000 0 0 0 1 undefined

64 500 0.59 0.59 1000 0 0 0 1 undefined

94

Note. C = condition; a = population path coefficient for effect of X on M; b = population
path coefficient for effect of M on Y (controlling for X); ACL = asymmetric confidence
limits method; BT = basic test of mediation; P0 = percent agreement; K = Cohen's kappa.

95

Table 4.9

Concordance between BCa and BT When an Indirect Effect Exists (ab>0)

cn a b Both reject Both commit B C a only - B T only -

Ho Type II error reject H 0 reject H 0 Po BCa-BT K
0.974 0.583
21 50 0.14 0.14 19 955 26 0 0.968 0.813
0.928 0.830
22 100 0.14 0.14 78 890 30 2 0.943 0.827

23 200 0.14 0.14 267 661 70 2

24 500 0.14 0.14 765 178 55 2

25 50 0.14 0.39 113 821 66 0 0.934 0.738
1 0.933 0.841
26 100 0.14 0.39 265 668 66 0 0.953 0.906
0 0.988 0.933
27 200 0.14 0.39 512 441 47

28 500 0.14 0.39 895 93 12

29 50 0.14 0.59 172 773 55 0 0.945 0.829
0 0.954 0.895
30 100 0.14 0.59 300 654 46 0 0.965 0.930
0 0.991 0.955
31 200 0.14 0.59 491 474 35

32 500 0.14 0.59 884 107 9

37 50 0.39 0.14 110 830 59 1 0.94 0.753
1 0.938 0.840
38 100 0.39 0.14 230 708 61 0 0.947 0.894
0 0.988 0.960
39 200 0.39 0.14 443 504 53

40 500 0.39 0.14 808 180 12

41 50 0.39 0.39 593 311 95 1 0.904 0.793

42 100 0.39 0.39 950 40 10 0 0.99 0.884

43 200 0.39 0.39 999 1 0 0 1 1.000

44 500 0.39 0.39 1000 0 0 0 1 undefined

45 50 0.39 0.59 779 180 41 0 0.959 0.872
0 0.996 0.921
46 100 0.39 0.59 972 24 4 0 1 undefined
0 1 undefined
47 200 0.39 0.59 1000 0 0

48 500 0.39 0.59 1000 0 0

53 50 0.59 0.14 104 848 48 0 0.952 0.786
0 0.962 0.874
54 100 0.59 0.14 165 797 38 0 0.966 0.924
0 0.986 0.966
55 200 0.59 0.14 322 644 34

56 500 0.59 0.14 699 287 14

57 50 0.59 0.39 629 316 55 0 0.945 0.878
58 100 0.59 0.39 909 80 11 0 0.989 0.930
59 200 0.59 0.39 995 5 0 0 1 1.000
60 500 0.59 0.39 1000 0 0 0 1 undefined

61 50 0.59 0.59 955 41 4 0 0.996 0.951
62 100 0.59 0.59 1000 0 0 0 1 undefined
63 200 0.59 0.59 1000 0 0 0 1 undefined
64 500 0.59 0.59 1000 0
0 0 1 undefined

Note. C = condition; a = population path coefficient for effect of X on M; b = population
path coefficient for effect of M on Y (controlling for X); BCa = bias corrected and
accelerated bootstrap; BT = basic test of mediation; P0 = percent agreement; K = Cohen's
kappa

Table 4.10 97

Concordance between BCa and ACL When an Indirect Effect Exists (ab>0) BCa-ACL

cn a b Both reject Both commit B C a only - A C L only - K

Ho Type II error reject H 0 reject H 0 Po 0.645
0.975 0.824
21 50 0.14 0.14 24 951 21 4 0.967 0.852
0.865
22 100 0.14 0.14 88 879 22 11 0.936
0.957
23 200 0.14 0.14 283 653 56 8

24 500 0.14 0.14 781 176 41 2

25 50 0.14 0.39 133 806 46 15 0.939 0.777

26 100 0.14 0.39 280 649 52 19 0.929 0.836

27 200 0.14 0.39 522 425 37 16 0.947 0.893

28 500 0.14 0.39 897 83 10 10 0.98 0.881

29 50 0.14 0.59 181 758 46 15 0.939 0.817

30 100 0.14 0.59 308 636 38 18 0.944 0.875

31 200 0.14 0.59 492 451 34 23 0.943 0.886

32 500 0.14 0.59 885 94 8 13 0.979 0.888

37 50 0.39 0.14 127 819 43 11 0.946 0.793

38 100 0.39 0.14 246 689 46 19 0.935 0.838

39 200 0.39 0.14 453 485 43 19 0.938 0.876

40 500 0.39 0.14 810 169 10 11 0.979 0.929

41 50 0.39 0.39 617 278 72 33 0.895 0.763

42 100 0.39 0.39 956 36 4 4 0.992 0.896

43 200 0.39 0.39 999 1 0 0 1 1.000

44 500 0.39 0.39 1000 0 0 0 1 undefined

45 50 0.39 0.59 793 155 27 25 0.948 0.825

46 100 0.39 0.59 972 18 4 6 0.99 0.778

47 200 0.39 0.59 1000 0 0 0 1 undefined

48 500 0.39 0.59 1000 0 0 0 1 undefined

53 50 0.59 0.14 112 826 40 22 0.938 0.747

54 100 0.59 0.14 169 775 34 22 0.944 0.823

55 200 0.59 0.14 330 620 26 24 0.95 0.891

56 500 0.59 0.14 699 268 14 19 0.967 0.919

57 50 0.59 0.39 644 286 40 30 0.93 0.839

58 100 0.59 0.39 911 65 9 15 0.976 0.831

59 200 0.59 0.39 995 4 0 1 0.999 0.888

60 500 0.59 0.39 1000 0 0 0 1 undefined

61 50 0.59 0.59 956 27 3 14 0.983 0.752

62 100 0.59 0.59 1000 0 0 0 1 undefined

63 200 0.59 0.59 1000 0 0 0 1 undefined

64 500 0.59 0.59 1000 0 0 0 1 undefined

98

Note. C = condition; a = population path coefficient for effect of X on M; b - population
path coefficient for effect of M on Y (controlling for X); BCa = bias corrected and
accelerated bootstrap; ACL = asymmetric confidence limits method; P0 = percent agreement;
K = Cohen's kappa.

99
Because K was attenuated in some conditions by a very high rate of agreement that

clustered in one of the two agreement cells, the interpretation of apparently low kappas can

be facilitated by consulting the proportion of agreement (Po) between the two methods.
Summary statistics on the percent agreement between methods across conditions are included

in Table 4.11. These statistics confirm the high rate of agreement between methods.

Table 4.11

Summary Statisticsfor % Agreement

Minimum No Indirect Effect Indirect Effect Present
Maximum 95.2 89.5
Median 100 100
Mean 98.7 98.3
98.5 97.5

Disagreement between Methods
Agreement evident in the pairwise comparisons was quantified by the percent
agreement and Cohen's kappa. Disagreement between the pairs of methods was studied
through the construction of bar graphs. As expected, most of the disagreement between two
methods occurred at the smaller sample sizes. Figures 4.19 to 4.24 display the disagreement
between two methods for conditions where no indirect effect was modeled and sample size
equaled 50 and 100. Figures 4.25 to 4.30 display the disagreement between two methods for
conditions where an indirect effect was modeled and sample size equaled 50 and 100.
Figures A.l to A. 12 display the disagreement between two methods for the larger sample
sizes and are located in the appendix.

Figure 4.19
ACL-BTDisagreement: No Indirect Effect Exists (ab=0)

ACL-BT Comparisons: n = 50

a=59, b=0
a=0, b=.59
a=.39, b=0
a=0, b=.39

a=.14,b=0 k x ^ x
a=0, b=.14 kvTT

a=0, b=0

- S:::^^S^SBS^^;
yes,

Figure 4.20
ACL-BT Disagreement: No Indirect Effect Exists (ab=0)

ACL-BT Comparisons: n = 100
a=59, b=0
a=0, b=.59
a=.39, b=0
a=0, b=.39
a=.14, b=0
a=0, b=.14

a=0, b=0

Note: "no,yes" = ACL failed to reject Ho, and BT rejected Ho (Type I error);
"yes,no" = ACL rejected Ho (Type I error), and BT failed to reject Ho.

FBiCg^u,-rBeT4D.21isagreement: No Indirect Effect Exists (ab=0)

BCa-BT Comparisons: n = 50
a=.59, b=0

a=.39,b=0 W.
a=0, b=.39
a=.14, b=0

a=0, b=.14
a=0, b=0

Note: "no.yes" - BC, failed » ^ ^ f f i ^ W

"vyeess,:no" = BCa rejected H0 (Type 1 error;, ana D

Figure 4.22
BCa -BT Disagreement: No Indirect Effect Exists (ab=0)

BCa-BT Comparisons: n = 100

a=59, b=0

m
a=0, b=.59

•a= 39 b=0

a=0, b=.39

a=.14, b=0

—

a=0, b=.14 ^<^^^^^>:^><<^:^

a=0, b=0

j

10 15 20 25 30 35

Note: "no,yes" = BCa failed to reject Ho, and BT rejected Ho (Type I error);
"yes,no" = BCa rejected Ho (Type I error), and BT failed to reject Ho.

Figure 4.23
BCa-ACL Disagreement: No Indirect Effect Exists (ab=0)

BCa-ACL Comparisons: n = 50

a=14, b=0
a=0, b=14 V

a=0, b=0 V
0 5 10 15 20 25 30 35 40

Note: "no,yes" = BCa failed to reject Ho, and ACL rejected Ho (Type I error);
"yes,no" = BCa rejected Ho (Type I error), and ACL failed to reject Ho.

106

Figure 4.24
BCa-ACL Disagreement: No Indirect Effect Exists (ab=0)

BCa-ACL Comparisons: n = 100

a=.59, b=0
a=0, b=.59
a=.39, b=0
a=0, b=.39

a=.14, b=0
a=0, b=.14 P

a=0, b=0 t

Note: ""nyoe,sy,neos,«" -- RBCCaa ftaaiilleeda ttoo rreej,ect Hoo,, and ACL rejected Ho (Type I error);
= BCa rejected H0 (ly f
aile(J tQfej ect Ho

pe l error;, <uw

Figure 4.25
ACL-BTDisagreement: An Indirect Effect Exists (ab>0)

ACL-BT Comparisons: n = 50

a=. 59, b=59
a=59, b=.39

a=.39, b=39
a=59, b=.14 h::::.:.:.:.::::::::::::::::.:
a=.l4, b=.59
b=.39,a=.14 P

a=14, b=39 k:.:::.;:::::::.:.;.;.|
a=14, b=.14

Note- "no yes" = ACL failed to reject H„ (Type II error), and B T , ^ » I Ho;
"ves no" = ACL rejected H„, and BT failed to reject Ho (Type II error).

Figure 4.26
ACL-BT Disagreement: An Indirect Effect Exists (ab>0)

ACL-BT Comparisons: n = 100

a=59, b=.59
a=.59, b=.39

mmm*mm

a=39, b=.59
a=.39, b=.39

a=.59, b=.14

mmmmmmmmmmm^m

a=.14,b=59 :*:"x*:+:*:)
x-x:-.:x-x-.l
IPb=.39,a=.14
::;:;:;:;:B^^

W

„a=.14, b=.39 •:v:-::-:-:-:->:v:::::vXv:-:-:-:-x-::-:-:-:-:'X<-:-:-:-:-:-:-:-:-:-:-:-:-:v:v:':x^

a=.14,b=.14 x-x:-x:-x-x-x-x-x-xxx-x::-x-x-x-x-x-x::xx-xox-x-X'X-x-x-x-x-x::-x-x-xxj

Ii I

10 15 20 25 30

Note: "no,yes" = ACL failed to reject Ho (Type II error), and BT rejected Ho;
"yes,no" = ACL rejected Ho, and BT failed to reject Ho (Type II error).

Figure 4.27
BCa -BT Disagreement: An Indirect Effect Exists (ab>0)

BCa-BT Comparisons: n = 50

a=.59, b=59

a=.59, b=.39 yyy.-.yyyyyy)\-.:yyy.-y.yA.\y^

a=.39, b=.59 v^/.yyM<y.w/y^^

a=.39, b=39 I s w w i i i U a

a=.59, b=14 ^.v.v.v.v.vU:.:-:.:-:-:-:*:-:^-:^

b=.39,a=14 L | : . w : i ; : : . w : w i . : w . } ^

a=14, b=39 ^.y.yy.yJ:.ysA.y..:.l.yX\^

a=.14, b = 1 4 ••-.••••-•-•-••.•••.•••'••-.;•;•:•;.••---• •-••'••••-•-•-.-•-•I

-^^ , 1 , , 1 1 1

0 10 20 30 40 50 60 70 80 90 100

Note: "no,yes" = BCa failed to reject Ho (Type II error), and BT rejected Ho;
"yes,no" = BCa rejected Ho, and BT failed to reject Ho (Type II error).

Figure 4.28
BCa -BTDisagreement: An Indirect Effect Exists (ab>0)

BCa-BT Comparisons: n = 100

a=.59, b=59 =

a= 59 b= 39

umm&m#

a=.39, b=.59

mm

a=.39, b=. 39

a= 59 b= 14

mmmmmm^mmmmmmmmmi

a= 14 b= 59

mmmmtmmmmmmmmimmmm^
ab=.39, a=14

a=.14, b=39 i

a=.14, b=.14 •:•:•:•:•:•:•:•>:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:•:

—: 1- 1
10 20 30 40 50 60 70

Note: "no,yes" = BCa failed to reject Ho (Type II error), and BT rejected Ho;
"yes,no" = BCa rejected Ho, and BT failed to reject Ho (Type II error).

Figure 4.29
BCa-ACL Disagreement: An Indirect Effect Exists (ab>0)

BCa-ACL Comparisons: n = 50

a=.59, b=59

a=59, b=39

a=.39, b=.59

a=.39, b=.39

a=.59, b=.14

a=. 14, b=59

b=.39,a=14

a=14, b=.39

a=14, b=14

10 20 30 40 50 60 70 80

Note: "no,yes" = BCa failed to reject Ho (Type II error), and ACL rejected Ho;
"yes,no" = BCa rejected Ho, and ACL failed to reject Ho (Type II error).

Figure 4.30
BCa-ACL Disagreement: An Indirect Effect Exists (ab>0)

BCa-ACL Comparisons: n = 100
a=59, b=59
a=59, b=39
a=.39, b=.59
a=39, b=.39
a=.59, b=.14
a=.14,b=59
b=39, a=. 14
a=14, b=39
a=14, b=14

Note: "no,yes" = BCa failed to reject Ho (Type II error), and ACL rejected Ho;
"yes,no" = BCa rejected Ho, and ACL failed to reject Ho (Type II error)

113
In general, as one would predict, there was more disagreement between methods in
the two smaller sample sizes. In those conditions where no indirect effect was modeled,
there was more disagreement between methods when either path a or path b was large, a
pattern that was evident in the Type I error graphs (Figures 4.1 - 4.7). As expected, given
the relative Type I error rates (Figures 4.19 to 4.24 and Figures A.l to A.6), disagreements
mostly consisted of Type I errors by the more powerful method.
In those conditions where an indirect effect was modeled, disagreement was most
evident when one path was weak and the other was moderate or strong - and then only when
sample sizes were modest. When the BCa and the ACL was compared to the BT, the vast
majority of disagreements occurred when the more powerful method (BCa or the ACL)
correctly rejected Ho and the less powerful method (BT) committed a Type II error. When
the BCa and ACL were compared to one another there was more variability. Although the
BCa was the more powerful method overall, it was not uncommon (1-3% of samples in
Figures 4.29 and 4.30) for the ACL to correctly reject Ho when the BCa test yielded a Type II
error.
The pairwise comparisons of the three tests of mediation revealed that there was a
consistent pattern when the ACL and the BCa were compared to the basic test of mediation.
In the conditions where no indirect effect was modeled, the basic test of mediation rarely
committed a Type I error when the other method correctly failed to reject Ho. In the
conditions where an indirect effect was modeled, disagreements mostly occurred when the
more powerful method correctly rejected Ho and the basic test of mediation resulted in a Type
II error (up to 3% of samples when compared with ACL; up to 10% of samples when
compared to BCa). When the ACL and the BCa were compared to one another, the pattern of

114
results was not as clear. The ACL was usually the more conservative of the two when no
indirect effect was modeled (i.e. the ACL did not often result in a Type I error when the BCa
correctly failed to reject Ho). However, there was more variability in the BCa -ACL
comparisons when an indirect effect was modeled in that both methods sometimes committed
a Type II error when the other method correctly failed to reject Ho.

Summary
Overall, concordance between methods was high. Pairwise comparisons over 28,000
samples drawn from 28 different populations showed that for each of the three pairings, the
two methods yielded identical conclusions over 98% of the time (95% in the least concordant
condition) when no indirect effect was modeled (ab=0). Analysis of samples where
conclusions from two methods diverged indicated a relatively consistent pattern, with BCa
more likely to result in a Type I error than ACL, and ACL more likely to result in a Type I
error than BT. Only rarely did ACL result in a Type I error that was not also committed by
BCa and only rarely did BT result in a Type I error that was not also committed by ACL and
BCa.
For the 36,000 samples draw from 36 different populations in which an indirect effect
was modeled (ab>0) concordance was somewhat lower, with pairs of methods yielding
identical conclusions in more than 97% of the samples (90% in the least concordant
condition). In pairwise comparisons at samples sizes 50 and 100 both ACL and BCa had
slightly more power than BT. The pattern in the BCa and ACL comparisons was not as clear.
Although BCa more often reached the correct conclusion (reject Ho) in these samples, in a
small proportion of samples (about 1 to 3%, depending on population values of a and b),
ACL reached the correct conclusion whereas BCa did not.

115

Chapter Five

This study evolved as a response to researchers' assertions (Frazier et al., 2004;

Mallinckrodt et al., 2006; Preacher & Hayes, 2004) that tests of mediation that fall under the

single-test framework (methods that test the significance of the indirect effect often

quantified by the product term ab) are preferable to tests of mediation that fall under the

multiple-test framework (methods that test the significance of each path between variables in

a mediation sequence). The asymmetric confidence limits method (ACL) and the bias-

corrected and accelerated bootstrap method (BCa) exemplify the single-test framework while
the basic test of mediation (BT) exemplifies the multiple-test framework. I contend that this

position that single-test framework methods are superior to multiple-test framework methods,

which almost seems like a foregone conclusion in the recent literature on mediation, is

debatable.

The statistical properties of the mediation analyses included in this study have been

examined in simulation studies (MacKinnon et al., 2002; MacKinnon et al, 2004). Part of the

push behind the recommendation of single-test framework methods is the finding that these

methods are powerful at detecting mediation. In Chapter 2,1 summarized evidence from past

studies, and in Chapter 4 I presented evidence from a new simulation study comparing the

statistical performance of the ACL, the BCa, and the BT. In this chapter I will summarize
these findings and address two other standards by which these three methods should be

evaluated; the conceptual standard and the pragmatic standard. The conceptual standard asks

researchers to consider which method best captures the conceptual meaning of mediation.

The pragmatic standard asks researchers to consider which among the available statistical

116
methods is the minimally sufficient analysis appropriate to their data and their research
question.

The Statistical Standard
This study compared the performance of three tests of mediation that have been
recommended by researchers and have yet to be compared to one another. The basic test of
mediation (BT) (Kenny et al, 1998) is the latest iteration of the widely utilized approach
popularized by Baron and Kenny (1986). The bias corrected and accelerated bootstrap (BCa)
(Efron & Tibshirani, 1993) was recommended by Shrout and Bolger (2002) as it provided
more accurate confidence limits than the percentile bootstrap approach. MacKinnon and
colleagues (MacKinnon et al., 2002; MacKinnon et al., 2004; MacKinnon et al., 2007) have
refined a test of the indirect effect over the years, which is now described as the asymmetric
confidence limits test (ACL) (MacKinnon, 2008) and is implemented by the PRODCLIN
program.
MacKinnon et al. (2002) had compared the BT (which they referred to as the test of
joint significance) to the first iteration of the ACL as well as to other methods. One of the
conclusions from this study was that "the best balance of Type I error and statistical power
across all cases is the test of the joint significance of the two effects comprising the
intervening variable effect"(abstract). MacKinnon et al. (2004) compared the second
iteration of the ACL to other single sample tests of the indirect effect as well as several
bootstrap approaches (Study 2). The conclusion from this study was that "the single best
method overall was the bias-corrected bootstrap which had Type I error rates close to the
nominal level along with more power than the other methods" (p. 120). MacKinnon et al.
(2004) did also note that the Type I error rates for the bias-corrected bootstrap method were

117
elevated when one path was zero and the other path had a medium or large effect size.
Mallinckrodt et al. (2006) noted a gap in the literature on mediation in that no simulation
study had compared the basic test of mediation (which they refer to as the test of joint
significance) to any bootstrap methods. Mallinckrodt et al. (2006) found this gap in the
literature to be unfortunate because they speculated that the BT may perform better than the
bootstrap method with respect to Type I error because it would not be vulnerable to rejecting
Ho if one of the component paths was nonzero.

This study fills the gap in the mediation literature that Mallinckrodt et al. (2006)
noted. The Type I error and power rates in this study were similar to previous simulation
studies. Specifically, the Type I error and power rates for the BT were similar to those
obtained in MacKinnon et al. 2002. The Type I error and power rates for the BCa in this
study were compared to the results for the bias-corrected bootstrap in Study 2 of MacKinnon
et al. (2004) as results for the BCa were not published in the study due to space
considerations (the authors simply note that the BCa results were similar to but not better
than the bias-corrected bootstrap - see footnote on p. 116). The Type I error and power rates
for the ACL were compared to the results for the M method (which is what the second
iteration of the ACL was called) in study 2 of MacKinnon et al. (2004). Results of this study
were averaged across the permutations of path parameters that were present in MacKinnon et
al. (2004) (so as to parallel the way data were presented in MacKinnon et al. (2004)) in order
to compare the results of the two studies.

One difference between this study and MacKinnon et al. (2002) and Study 1 of
MacKinnon et al. (2004) is that MacKinnon and colleagues reported that for the conditions
that had the same effect size for the indirect effect but different path parameters (e.g. a = .14

118
and b = .59; a = .59 and b = .14), results for the conditions where path a was greater than
path b were similar to the results for the conditions where path b was greater than path a.
Partly because of this finding, MacKinnon et al. (2004) omitted the conditions where path a
was greater than path b in Study 2. A different pattern of results was found in this study. For
the same effect size for path ab, there was lower power in the conditions where path a was
greater than path b than there was for the conditions where path b was greater than path a
(reasons for which are described in Chapter 4, based on analyses by Kenny et al. (1998) and
Hoyle and Kenny (1999)).

The results of this study supported patterns of results found in previous studies in that
the BCa was found to be the most powerful of the three methods. It seems that power comes
with a price in that the BCa also had the highest Type I error rates, especially in the
conditions where sample sizes were small (N = 50 and N = 100) and one component path
was zero while the other was medium or large. Also, the gain in power by using the BCa is
small compared to the increase in Type I error rate. The BCa had power levels of .80 and
higher in 50 % of the conditions in which an indirect effect was modeled compared to 47%
for the ACL and BT. In contrast, the BCa had Type I error rates of .05 and higher in 61% of
the conditions in which no indirect effect was modeled compared to 54% for the ACL and
39% for the BT. In this study the Type I error rates for the BT and the ACL were usually
below the nominal level of .05. In general, it would not be wise to set the alpha level of a
study at levels below .05 because this would decrease the power of a study to detect an
effect. However, the finding that the BT and ACL often have Type I error rates lower than
.05 is not worrisome in that it seems the power of the methods to detect mediation is not
compromised.

119
It is important to note the null hypotheses that are evaluated by the three tests of
mediation in these analyses of Type I error rate and power. The null hypotheses of the BT
are a = 0 and b-0. If there is a failure to reject one of these null hypotheses, then the
conclusion is that there is no indirect effect. The null hypothesis of the BCa and the ACL is a
compound one. It is complex because it evaluates three different null hypotheses
simultaneously: (a) a = 0, b = 0; (b) a ^ 0, b = 0; and (c) a = 0, b 4- 0. One advantage of the
BT over the BCa and the ACL with respect to the testing of null hypotheses is that the results
of the BT clearly indicate which null hypothesis is rejected or not rejected while the results of
the BCa and ACL cannot provide this type of information.
The pairwise comparisons of the methods showed that when the ACL and BCa are
compared to the BT, the BT rarely makes a Type I error when the other method correctly
concludes that there is no indirect effect. In the conditions where an indirect effect is
modeled, the BT tends to make more Type II errors when the other method correctly
concludes that there is no indirect effect. When the BCa and ACL are compared to one
another, most of the disagreements occur when the BCa commits a Type I error and the ACL
correctly rejects Ho. In the conditions where an indirect effect was modeled there is more
variability in the disagreements in that sometimes the ACL commits a Type II error when the
BCa correctly rejects Ho and vice-versa.

The Conceptual Standard
Although there has been some debate as to how to define mediation (MacKinnon
2008), major contributors to the study of mediation agree that a mediation model describes
relationships between variables in causal terms (James & Brett, 1984; Kenny, 2008;
MacKinnon, 2008). In the simplest mediation model, the influence of a predictor variable on

120
a criterion variable is transmitted through an intervening or mediating variable. Mediation
hypotheses are fundamentally interesting to us because they represent our ideas of how
things work. Mediation analyses are often employed in the areas of theory building and
specification and program evaluation and development.

None of the three tests of mediation evaluated in this study provides any basis for
making causal inferences from correlational data. Providing evidence for causality is related
to model specification; how well researchers have taken care to establish that there are no
reverse causal effects in their model, address the issue of measurement error in the
measurement of their variables, and evaluate whether any omitted variables are potentially
causing both the mediator and the criterion variable. However, if one agrees that a mediation
hypothesis is a hypothesis about a chain of cause-effect relations, then a strong case can be
made that the basic test of mediation, and more generally, the multiple-test framework of
assessing mediation, best capture the conceptual meaning of mediation. If a model is made
up of component paths, then more information is gathered about the model if information
about the component paths is collected and studied. For example, in the area of program
development and evaluation, program developers can use the basic test of mediation to assess
the strength of the relationship between the program and the construct the program is
targeting and to assess the strength of the relationship between the targeted construct and the
outcome variable of interest.

MacKinnon (2008) described four major reasons why mediation analyses should be
stressed in the evaluation of prevention and treatment programs.

First, mediation analysis provides a check on whether the prevention or
treatment program has produced a change in the construct it was designed to

121
change. If a program is designed to change norms, then program effects on
norm measures should be found. Second, the results may suggest that certain
program components need to be strengthened or measurements need to be
improved. Failures to significantly change mediating variables occur either
because the program was ineffective or the measures of the mediating
construct were not adequate. Third, program effects on mediating variables in
the absence of effects on dependent measures suggest that program effects on
dependent variables may emerge later or that the targeted constructs were not
critical in changing outcomes. Finally, and most importantly, evidence
bearing on how the program achieved its effects can be obtained, (p. 35).
The emphasis of the four areas of prevention and treatment evaluation that
MacKinnon described is on the component paths of the mediation model, which in this case
are the relationship between the program and the targeted construct and the relationship
between the targeted construct and the outcome variable of interest. The basic test of
mediation can provide information on these components of the model by testing the
significance of these component paths and by providing effect size measures for these paths.
Neither the asymmetric confidence limits test nor the bias-corrected bootstrap method (or any
test that tests the significance of the indirect effect) would provide information that would
shed light on any of the four areas of research described by MacKinnon that the basic test of
mediation does not already provide.
In order to explore the value of estimating the size of the component paths that make
up a mediation sequence, consider a hypothetical example wherein the effect size of the
indirect effect of a program intervention on an outcome variable of interest is .16. Many

potential scenarios can account for this effect size. In one possible scenario, the path
coefficient for the effect of the intervention on the mediator (path a) is .2 and the path
coefficient for the effect of the mediator on the outcome (path b) is .8. In a second possible
scenario the values of the path coefficients are reversed; path a is .8 and path b is .2. In a
third possible scenario, both paths a and b are equal to .4. If the first scenario was true, then
the path coefficients tell us that the mediator targeted by the program is a strong mediator and
that more could be done on the program development side of things to try to strengthen the
relationship between the intervention and the mediator. If the second scenario was true, then
the path coefficients tell us that the targeted mediator is a poor mediator and that perhaps
another mediator would be better at influencing the outcome variable. If the third scenario
was true, then the path coefficients tell us that the intervention has a moderate influence on
the mediator which in turn has a moderate influence on the outcome. If only size of the
indirect effect is emphasized in a research report, then all the information about the strength
of the influence of the program on the mediator and of the mediator on the outcome variable
is lost. If the focus of the research report is on the strength and significance of the path
coefficients, not much is gained by reporting the product of the path coefficients and testing
its significance.

To put the matter succinctly, a mediator hypothesis concerns a chain of causation
between a predictor variable, one or more mediators, and a criterion variable. Because each
link in the causal chain is of interest, a conceptually sound test of mediation should examine
each link independently. This is the approach taken by multiple-test approaches to testing
mediator hypotheses.

123

The Pragmatic Standard

The pragmatic standard encompasses the recommendations of Wilkinson et al. (1999)

and Cohen (1990) regarding statistical analyses. Both Wilkinson et al. (1999) and Cohen

(1990) recommend choosing the minimally sufficient analysis appropriate for one's research

hypothesis. Cohen further exhorts researchers to stay in close touch with their data and to

understand the statistics that underlie their analysis. A case can be made that for the simplest

mediation model, the basic test of mediation is the minimally sufficient analysis to use and is
the easiest to understand. The basic test of mediation requires two regression equations

(Equations 2.2 and 2.3) to estimate and test the significance of the relationship between the

predictor and the mediator (path a) and between the mediator and the criterion variable (path

b). If desired, confidence intervals around the estimates of paths a and b can be constructed

using the standard errors of paths a and b. The two regression equations used by the basic
test of mediation are also used by the asymmetric confidence limits approach and the bias-

corrected bootstrap in order to estimate the parameters needed for their methods. For the

ACL approach, the additional steps required are that the researcher has to input the values of

path a, path b, the standard error of a, the standard error of b, the correlation between a and

b, and the desired Type I error rate into PRODCLIN in order to run the analysis. For the
bias-corrected bootstrap method run by the boot package in R, eleven arguments need to be

specified in order to run the analysis. Examples of these arguments include the number of

bootstrap replicates to be performed, the type of simulation to be used, and the data to be

bootstrapped in the form of a vector, matrix, or data frame.

Cohen (1990) not only exhorts researchers to choose the minimally sufficient analysis

appropriate for their data and research questions, he also encourages researchers to

124
understand the analyses they are performing. Among the three methods studied, the basic
test of mediation is the most accessible to the majority of producers and consumers of
research in counseling psychology. Although computer programs make the implementation
of the asymmetric confidence limits approach and various bootstrap methods possible, the
mathematics behind the construction of these methods are beyond what most counseling
psychologists have studied. Consider the explanation that MacKinnon (2008) provides for
distribution of the product of the indirect effect (path ab):

.. .the analytical solution for the product distribution is a Bessel function of the
second kind with a purely imaginary argument (Aroian, 1947; Craig, 1936).
Although computation of these values is complex, Springer and Thompson
(1966) provided a table of the values of this function when za= Zb= 0.
Meeker, Cornwell, and Aroian (1981, see pp. 129-144 for uncorrelated
variables) presented tables of the distribution of the product of two standard
normal variables based on an alternative formula more conducive to numerical
integration, (p. 96)
It is the tables provided by Meeker et al. (1981, as cited in MacKinnon 2008) that
forms the basis of the FORTRAN program around which the PRODCLIN program is built.
The mathematics on which the bias-corrected and accelerated bootstrap is built is also
complex. Consider this explanation of the bias correction (Efron & Tibshirani, 1993):
The value of the bias correction z0is obtained directly from the proportion of

bootstrap replications less than the original estimate 0,

125

z0=a> -1 #f*(b)<6\

vB J

O ' (.) indicating the inverse function of a standard normal cumulative

distribution function, e.g. 0"'(.95) = 1.645. (p.186)

And this explanation for the acceleration:

There are various ways to compute the acceleration a. The easiest to explain

is given in terms of the jackknife values of a statistic 6 = s(x). Let x(i) be the

original sample with the z'th point x, deleted, let 9{i) = s(x(j)), and define

#( j = ^"_, 6(t) I n, as discussed at the beginning of Chapter 11. A simple
expression for the acceleration is

a= U=11(4.)-4oJ
12
(p. 187)

Most counseling psychologists do not have enough background in math to understand
the theoretical underpinnings of the ACL and the BCa. If understanding the math behind the
methods is challenging, then understanding the critiques of the math underlying the methods
is even more so (see Gleser (1996) and Canty, Davison, and Hinkley (1996) for comments on
the BCa). In contrast, the basic test of mediation requires basic doctoral level statistics;
training in multiple regression is a common requirement in doctoral programs in counseling
psychology.

Limitations and Future Directions
The majority of the mediation hypotheses that are tested are usually cases of partial
mediation. The variables measured in mediation hypotheses are always measured with some
degree of error. These two aspects of mediation analyses speak to the limitations of this
study. In this study only full mediation was modeled. The direct effect of the predictor
variable on the criterion variable (path c') was set to zero for all conditions because the three
tests of mediation studied were only concerned with paths a and b. Also, MacKinnon et al.
(2002) reported that in their simulation, the results of the conditions where c' was set to zero
were similar to the results of the conditions where c' was varied. In this study measurement
error was not modeled. The results reflect perfect measurement of the predictor, mediator,
and criterion variables. Kenny and colleagues (Baron & Kenny, 1986; Kenny et al. 1998, and
Hoyle and Kenny, 1999) have repeatedly warned about the effects of measurement error on
the power to detect mediation.
The parameters of this study were chosen in order to replicate and extend the work of
MacKinnon et al. (2002) and MacKinnon et al. (2004). Future simulation studies could look
at other parameters for sample size and size of path coefficients. For example, the largest
sample size of this study (N=500) could be left out and smaller sample sizes (e.g. N=80 or N
= 150) could be added in order to reflect the more common sample sizes obtainable in
counseling psychology research. Also, the effect of measurement error could be studied in
order to extend the work of Hoyle and Kenny (1999).
The variables modeled in this study were simulated under the conditions of
multivariate normality. It is unclear how the three tests of mediation evaluated in this study
would perform with variables that do not have normal distributions.

Two important areas that researchers should pay attention to when conducting a
mediation analysis that are not discussed in detail here but are treated at length elsewhere are
design issues and specification errors (Baron and Kenny, 1986; Kenny et al, 1998). Design
issues encompass the effects of distal and proximal mediation and multicollinearity on the
estimation of paths a and b. Specification errors refer to instances when the three
assumptions that underlie the use of multiple regression to assess a mediation model are not
met. The three assumptions are that there are no reverse causal effects (e.g. that the criterion
variable does not cause the mediator); that the mediator is measured without error; and that
an omitted variable does not cause both the mediator and the criterion variable.

Conclusions and Recommendations
Researchers who would like to conduct a mediation analysis are faced with a
potentially confusing array of recommendations in the literature as to which method to use
when testing mediation. MacKinnon et al.'s (2002) finding that Baron and Kenny's (1986)
three-step procedure for testing mediation had low statistical power when compared to the
performance of other tests of mediation has been widely cited. This has led researchers
(Frazier et al. 2004; Preacher & Hayes, 2004; Mallinckrodt et al., 2006) to recommend the
use of other, more powerful tests of mediation such as bootstrap methods and the methods
developed by MacKinnon and colleagues (MacKinnon et al., 2002; MacKinnon et al, 2004;
MacKinnon et al. 2007). Readers (including editors and reviewers) may conclude from
reading some or all of these articles that all multiple-step approaches to testing mediation are
inferior.
Two points are important to consider when evaluating the recent recommendations
against the use of Baron and Kenny's (1986) three-step model for testing mediation. The

128
first is that Baron and Kenny's (1986) approach is found to be lacking in statistical power
when compared to other methods because of their requirement that a significant relationship
be found between the predictor variable and the criterion variable. Kenny et al. (1999)
relaxed this requirement. They stated that the essential steps in establishing mediation are
that a significant relationship is found between the predictor variable and the mediator and
that a significant relationship is found between the criterion variable and the mediator,
controlling for the effects of the predictor variable.

A second point is that Baron and Kenny's (1986) three-step method for testing
mediation has been incorrectly described as having four steps (Frazier et al. 2004;
Mallinckrodt et al. 2006). What has been discussed as the required fourth step is what Baron
and Kenny (1986) provided as an optional test of the indirect effect (ab) (which was to test
the significance of the indirect effect by using a variant of Sobel's equation for the standard
error of an indirect effect). It is this optional test of the indirect effect (ab) that has also been
found to be lacking in power when compared to other tests of the indirect effect (e.g. the BCa
and the ACL). This is because Sobel's equation for the standard error of an indirect effect
and its variants are based on the assumption that the indirect effect (ab) is normally
distributed when in fact it has been shown to not have a normal distribution. Frazier et al.'s
(2004) and Mallinckrodt et al.'s (2006) recommendation against the use of Baron and
Kenny's (1986) test of mediation is based on a rejection of a test of the indirect effect that is
not a part of Baron and Kenny's method.

MacKinnon et al.'s (2002) finding that Baron and Kenny's three-step method for
testing mediation had low power compared to other methods has been widely discussed.
What has been overlooked is MacKinnon et al.'s (2002) finding that the test of joint

129

significance provided the best balance between Type I error and power among all the

methods studied. This test ofjoint significance is the same method proposed by Kenny et al.

(1999) as a refinement of Baron and Kenny (1986). This test is referred to as the basic test of

mediation in this study.

The goal of this study was to examine how the basic test of mediation fared relative to

the strongest of the modern alternatives that have been proposed - in statistical terms, but

also relative to conceptual and pragmatic criteria. The basic test of mediation has distinct

advantages both conceptually and pragmatically. It is also arguably superior statistically as

the results of this study show that it provides a good balance between Type I and Type II

errors.

The results of this study show that if a researcher uses the basic test of mediation and

rejects Ho, more modern methods such as the BCa and ACL would come to the same

conclusion (i.e. the BT rarely rejects Ho when the BCa and the ACL do not). If a researcher
uses the basic test of mediation and fails to reject Ho, then using the BCa or the ACL might
yield a different result. However, the question remains as to whether this result is the correct

conclusion or a Type I error as more powerful methods for testing mediation usually have

higher Type I error rates.

When the results of the methods were compared to one another, the ACL and the BT

performed the most similarly and the BCa and the BT performed the least similarly. The
differences between the methods have to do with their relative Type I error rates and power.

These differences can be conceptualized as the ratio of Type I error and the ratio of power

between two methods. The power ratio as well as the Type I error ratio between the BCa and
the BT are greatest when sample sizes and effect sizes are small. What is also important to

130
note is that the Type I error ratio is five to ten times larger than the power ratio in these
conditions. The tendency of the BCa to commit a Type I error is greater than any advantage
in power it has when compared to the BT.

For the three-variable mediation hypothesis, the basic test of mediation (Kenny et al.
1999) has much to recommend it. It is a conceptually sound test of mediation in that it tests
the significance of each component path in a hypothesized causal chain. The two regression
analyses that comprise the basic test of mediation are easy to run on any statistical package
and the results are readily comprehensible and interpretable. The power of the basic test of
mediation is comparable to that of the bias-corrected and accelerated bootstrap and the
asymmetric confidence limits test, the only exception being when effect sizes and population
sizes are very small. Moreover, the Type I error rate of the basic test of mediation is low,
which is particularly helpful when researchers are exploring new theories about relationships
between variables.

Testing the significance of the indirect effect does not provide the researcher with any
information that the basic test of mediation does not already provide. Moreover, focusing on
testing the significance of the indirect effect reduces a mediation analysis to the question, "Is
the indirect effect significant?" and draws attention away from the aspects of mediation that
researchers should be concerned with, such as "What is the strength of the relationship
between the predictor and mediator? The mediator and the outcome?" Another drawback of
testing the significance of the indirect effect is that the methods most often recommended to
accomplish this such as the ACL and the BCa can be time intensive to learn and implement.
Researchers might be better off spending their time on matters related to the design of their
study and the measurement of their variables, factors which have a large impact on providing

131
support for their mediation hypothesis, rather than on learning modern statistical techniques

that provide them with nothing that simpler regression analyses do not already provide.

132

References

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social

psychological research: Conceptual, strategic, and statistical considerations. Journal

of Personality and Social Psychology, 6, 1173-1182.

Bollen, K. A., & Stine, R. (1990). Direct and indirect effects: Classical and bootstrap

estimates of variability. Sociological Methodology, 20, 115-140.

Canty, A.J., Davison, A.C., Hinkley, D.V. (1996). Bootstrap confidence intervals: Comment.

Statistical Science, 11, 214-219.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and

Psychological Measurement, 20, 37-46,

Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.

Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.

Davison, A.C., & Hinkley, D.V. (1997). Bootstrap Methods and their Applications. New

York: Cambridge University Press

Efron, B., & Tibshirani, R.(1993). An Introduction to the Bootstrap. New York: Chapman &

Hall/CRC.

Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in

Counseling Psychology research. Journal of Counseling Psychology, 51, 115-134.

Gleser, L. J. (1996). Bootstrap confidence intervals: Comment. Statistical Science, 77, 219-

221.

Hoyle, R. H., & Kenny, D.A.(1999). Sample size, reliability, and tests of statistical

mediation. In R. H. Hoyle (Ed.), Statistical strategiesfor small sample research

(pp. 195-222). Thousand Oaks, CA: Sage.

133
Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment

evaluations. Evaluation Review, 5, 601-619.
Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In

D.T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The Handbook of Social Psychology
(4th ed., pp. 233-265). New York: Oxford University Press.
Lockwood, C. M., & MacKinnon, D. P. (1998). Bootstrapping the standard error of the
mediated effect. In Proceedings of the Twenty-Third Annual SAS Users Group
International Conference (pp. 997-1002). Cary, NC: SAS Institute.
MacKinnon, D. P. (n.d.). Retrieved January 13, 2007, from
http://www.public.asu.edu/~davidpm/ripl/conference/david_mackinnon.htm
MacKinnon, D.P. (2008). Introduction to Statistical Mediation Analysis. New York:
Lawrence Erlbaum Associates, Taylor & Francis Group.
MacKinnon, D. P., & Dwyer, J. H. (1993). Estimating mediated effects in prevention studies.
Evaluation Review, 17, 144-158.
MacKinnon, D. P., Fritz, M.S., Williams, J., & Lockwood, C. M. (2007). Distribution of the
product confidence limits for the indirect effect: Program PRODCLIN. Behavioral
Research Methods.
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation,
confounding, and suppression effect. Prevention Science, I, 173-181.
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A
comparison of methods to test mediation and other intervening variable effects.
Psychological Methods, 7,82-104.

134
MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the

indirect effect: Distribution of the product and resampling methods. Multivariate
Behavioral Research, 39, 99-128.
MacKinnon, D. P., Warsi, G., & Dwyer, J. H. (1995). A simulation study of mediated effect
measures. Multivariate Behavioral Research, 30, 41-62.
Mallinckrodt, B., Abraham, W. T., Wei M., & Russell D. W. (2006). Advances in testing the
statistical significance of mediation effects. Journal of Counseling Pyschology, 53,
372-378.
Preacher, K. P., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect
effects in simple mediation models. Behavior Research Methods, Instruments, &
Computers, 36, 717-731.
R Development Core Team (2007). R: A language and environmentfor statistical computing.
R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-
project.org.
Ripley, B. (2007) boot: Bootstrap R. R package version 1.2-28.
Sobel, M. E. (1982). Asymptotic confidence intervals for indirect effects in structural
equation models. In S. Leinhardt (Ed.), Sociological methodology 1982 (pp. 290-
312). Washington, DC: American Sociological Association.
Sobel, M. E. (1986). Some new results on indirect effects and their standard errors in
covariance structural models. Sociological Methodology, 16, 159-186.
Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies:
New procedures and recommendations. Psychological Methods, 7, 422-445.

135
Venables, W. N., & Ripley, B. D. (2002) Modern Applied Statistics with S. New York:

Springer.
Wilkinson, L. & the Task Force on Statistical Inference (1999). Statistical methods in

psychology journals: Guidelines and explanations. American Psychologist, 54, 594-
604.

Appendix

R Syntax

# NOTE.- Use setwdO to point R to the folder where prodclin.exe lives.

library(boot)
library(MASS)

abprod = function(data,i) {
regrl = lm(scale(data[i,2])-scale(data[i,1])-l)
regr2 = lm(scale(data[i,3])-scale(data[i,2])+scale(data[i, 1] )-1)
return(regrl$coef*regr2$coef[1] )

}

"prodclin" <-
function(a,sea,b,seb,rho,alpha)

{

#Prodclin.r - MacKinnon, Fritz, Williams, and Lockwood, 2005
#Dept. of Psychology, Arizona State University
#Necessary files: prodclin.r, prodclin.exe

#NOTE: Use setwdO to point R to the folder where prodclin.exe lives.

write(c(a,sea,b,seb,rho,alpha),file="raw.txt",ncolumns=6)

shell("prodclin.exe")

critval <- read.table("critval.txt",header=FALSE)
lcrit <- critval[1,1]
ucrit <- critval[1,2]
ab <- a*b
sobel <- sqrt(a*a*seb*seb+b*b*sea*sea)
da<-a/sea
db<-b/seb
dadb<-da*db
sedadb<-sgrt(da*da+db*db+l)
low <- (lcrit-dadb)/sedadb
high <- (ucrit-dadb)/sedadb

prodlow <- ab+low*sobel
produp <- ab+high*sobel

nl<-qnorm(alpha/2)
normlow <- ab+nl*sobel
normup <- ab-nl*sobel

if(lcrit==99999 || ucrit==99999) {
out<- "Did not converge. Please check input values or try again with

fewer decimal places" }
else {

out<-list(a=a, sea=sea, b=b, seb=seb, ab=ab, "Sobel se"=sobel,
rho=rho,"typel error"=alpha,

"Normal lower limit"=normlow,"Normal upper limit"=normup,
lower critical value"=low,

"Prodclin upper critical value"=high,"Prodclin lower
limit"=prodlow, "Prodclin upper limit"=produp)}
out

}

# End function definitions

# Start program
samp=c(50,100,200,500)

nreps=1000
nboot=1000

# r.s contains 16 permutations of a, b, and c values.
corrs <- c(rep(0,4), rep(.14,4) , rep(.39,4), rep(.59,4),

rep(c(0,.14,.39,.59) ,4), rep(0,16))
r.s <- data.frame(a=corrs [1:16] , b=corrs [17:32] , c2=corrs[33:48])
r.s$c <- r.s$c2 + r.s$a*r.s$b
r.s$rYM <- r.s$b + r.s$a*r.s$c2

nrows = length(samp)*nrow(r.s) rep (NA, nrows)
rep(NA, nrows)
lia.results <• data.frame(run rep(NA, nrows)
iii rep(NA, nrows)
rep (NA, nrows)
n rep(NA, nrows)
pop.rMX rep(NA, nrows)
pop.rYM rep(NA, nrows)
pop.rYX rep(NA, nrows)
bca.pet.rej.pos rep(NA, nrows)
bca.pet.rej.neg = rep (NA, nrows
bca.pet.rej = rep(NA, nrows
acl.pet.rej.pos = rep(NA, nrows
acl.pet.rej.neg rep(NA, nrows)
acl.pet.rej rep(NA, nrows)
bt.pet.rej.pos rep(NA, nrows)
bt.pet.rej.neg = rep(NA, nrows
bt.pet.rej = rep(NA, nrows
ACLBT0 = rep(NA, nrows
ACLBT1 = rep(NA, nrows
ACLBT2 = rep(NA, nrows
ACLBT3 = rep(NA, nrows
ACLBT4 = rep(NA, nrows
ACLBT5 = rep(NA, nrows
ACLBT6 = rep(NA, nrows
ACLBT7 = rep(NA, nrows
ACLBT8 = rep(NA, nrows
ACLBCA0 = rep(NA, nrows
ACLBCA1 = rep(NA, nrows
ACLBCA2
ACLBCA3

138

ACLBCA4 = rep(NA, nrows),
ACLBCA5
ACLBCA6 = rep(NA, nrows),
ACLBCA7
ACLBCA8 = rep(NA, nrows),
BCABTO
BCABT1 = rep(NA, nrows),
BCABT2
BCABT3 = rep(NA, nrows),
BCABT4
BCABT5 = rep(NA , nrows),
BCABT6 = rep(NA , nrows),
BCABT7 = rep(NA nrows),
BCABT8 = rep(NA , nrows),
= rep(NA nrows),
= rep(NA nrows),
= rep(NA , nrows),
= rep(NA nrows),
= rep(NA nrows))

lia.descrs data frame(run _ rep(NA nrows),
iii
= rep (NA n r o w s ) ,
jjj = rep(NA nrows),
n = rep(NA nrows),
nreps = rep(NA nrows),
nboot = rep(NA nrows),
pop.rMX = rep(NA nrows),
pop.rYM = rep(NA nrows),
pop.rYX = rep(NA nrows),
a.min = rep(NA nrows),
a. lq = rep(NA nrows),
a.2q = rep(NA nrows),
a. 3q = rep(NA nrows),
a. max = rep(NA nrows),
a.mean = rep(NA nrows),
rYM.min = rep(NA nrows),
rYM.lq = rep(NA nrows),
rYM.2q = rep(NA nrows),
rYM.3q = rep(NA nrows),
rYM.max = rep(NA nrows),
rYM.mean = rep(NA nrows),
c .min = rep(NA nrows),
c. lq = rep(NA nrows),
c.2q = rep(NA nrows),
c.3q = rep(NA nrows),
c.max = rep(NA nrows),
c.mean = rep(NA nrows),
ab.min = rep(NA nrows),
ab. lq = rep(NA nrows),
ab.2q = rep(NA nrows),
ab. 3q = rep(NA nrows),
ab.max = rep(NA nrows),
ab.mean = rep(NA nrows),
acl.ciwi dth min
acl.ciwi dth iq = rep(NZ\, nrows) ,
acl.ciwi dth 2q = rep(N;\., nrows) ,
acl.ciwi dth 3q = rep(N;\, nrows) ,
acl.ciwi dth max = r e p ( M\, nrows) ,
acl.ciwi dth mean = rep(N;\, nrows) ,
= rep(NJ\., nrows) ,

bca.ciwidth.min = rep(NA, nrows),
bca.ciwidth.lq = rep(NA, nrows),
bca.ciwidth.2q = rep(NA, nrows),
bca.ciwidth.3q = rep(NA, nrows),
bca.ciwidth.max = rep(NA, nrows),
bca.ciwidth.mean = rep(NA, nrows),
acl_bca.diff.min
acl_bca.diff.lq = rep(NA, nrows),
acl_bca.diff.2q = rep(NA, nrows),
acl_bca.diff.3q = rep(NA, nrows),
acl_Jbca.dif f .max = rep(NA, nrows),
acl_bca.diff.mean = rep(NA, nrows),
acl_bca.diff.ltO = rep(NA, nrows),
lower.diff.min = rep(NA, nrows),
lower.diff.lq = rep(NA, nrows),
lower.diff.2q = rep(NA, nrows),
lower.diff.3q = rep(NA, nrows),
lower.diff.max = rep(NA, nrows),
lower.diff.mean = rep(NA, nrows),
lower.diff.ltO = rep(NA, nrows),
upper.diff.min = rep(NA, nrows),
upper.diff.lq = rep(NA, nrows),
upper.diff.2q = rep(NA, nrows),
upper.diff.3q = rep(NA, nrows),
upper.diff.max = rep(NA, nrows),
upper.diff.mean = rep(NA, nrows),
upper.diff.ltO = rep(NA, nrows),
acl.triesl = rep(NA, nrows),
acl.tries2 = rep(NA, nrows),
acl.tries3 = rep(NA, nrows),
acl.tries4 = rep(NA, nrows),
acl.tries5 = rep(NA, nrows),
= rep(NA, nrows))

# Outer loop steps through 16 rows of r.s
# For Condor version, input start, stop values from file starts
# File contains two integer values, separated by spaces:
# iii (between 1 and 16),
# jjj (between 1 and 4 ) .
# So condor version runs only one condition (iii_jjj).

x <- scant"startstop.txt", what=0, n=2)

for(iii in x[l] :x[l] ) {
sigma = matrix(c(l, r . s $ a [ i i i ] , r . s $ c [ i i i ] ,

r . s $ a [ i i i ] , 1, r.s$rYM[iii],
r . s $ c [ i i i ] , r.s$rYM[iii], 1), ncol=3)

# j j j loops through designated sample sizes

for (jjj in x[2] :x[2] ){

n=samp[jjj]