Regression With Multiple Categorical Independent Variables ...

Overview An Example

Multiple Categorical IVs s From Pedhazur, p. 513-514

Variable Coding “Assume that in an expe
associates, the independ
Statistical Tests exposures to a list. Spec
randomly assigned, in eq
ANOVA Example exposure to a list, so tha
q Example exposure, a second grou
q ANOVA so on to five exposures f
q Regression dependent variable meas
responses on a subsequ
q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

4:
eriment on the learning of paired

dent variable is the number of
cifically, 15 subjects are
qual numbers, to five levels of
at one group is given one
up is given two exposures, and
for the fifth group. The

sure is the number of correct
uent test.”

Slide 25 of 51

The Analysis

s Running an ANOVA (from A
Model...Univariate in SPSS

Overview

Multiple Categorical IVs

Variable Coding

Statistical Tests

ANOVA Example
q Example
q ANOVA
q Regression

q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

Analyze...General Linear
S) produces these results:

Slide 26 of 51

Overview The Interpretation

Multiple Categorical IVs s From the example, we coul

Variable Coding H0 : µ1 = µ

Statistical Tests s Here, F4,10 = 2.10, which g
s Using any reasonable Type
ANOVA Example
q Example fail to reject the null hypothe
q ANOVA
q Regression s We would then conclude th
exposures on learning (as m
q S S Differences
s Note that for this analysis th
q ANOVA Table produced (four degrees of f

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

ld test the hypothesis:
µ2 = µ3 = µ4 = µ5
gives a p-value of 0.156.
e-I error rate (like 0.05), we would
esis.
hat there is no effect of number of
measured by test score).
here were five coded columns
freedom for the numerator).

Slide 27 of 51

Overview A New Analysis

Multiple Categorical IVs s Instead of running an ANOV
the means of the test score
Variable Coding run an linear regression?

Statistical Tests s For the linear regression to
of X must fall on the linear
ANOVA Example
q Example s The key point is that the me
q ANOVA
q Regression s Using the difference betwee
Regression, I will show you
q S S Differences trend in the analysis.

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

VA to test for differences between
es at each level of X , couldn’t we
o be valid, the means of the levels

regression line.
eans must follow a linear trend.
en the ANOVA and the
u how you can test for a linear

Slide 28 of 51

Multiple Regression

s Running an regression (from
in SPSS) produces these re

Overview

Multiple Categorical IVs

Variable Coding

Statistical Tests

ANOVA Example
q Example
q ANOVA
q Regression

q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

n Results

m Analyze...Regression...Linear
esults:

Slide 29 of 51

Overview Multiple Regression
Multiple Categorical IVs
Variable Coding s From the example, we coul
Statistical Tests
ANOVA Example H0
q Example
q ANOVA s Here, F1,13 = 8.95, which g
q Regression
s Using any reasonable Type
q S S Differences reject the null hypothesis.

q ANOVA Table s We would then conclude th
Curvilinear Regression relationship between numb
Wrapping Up measured by test score).

Lecture #11 - 11/12/2008 s This conclusion is different
before.

s What is different about our

n Results

ld test the hypothesis:
0 : b1 = 0
gives a p-value of 0.010.
e-I error rate (like 0.05), we would

hat there is a significant
ber of exposures and learning (as

than the conclusion we drew

analysis?

Slide 30 of 51

Overview S S Differences

Multiple Categorical IVs s Notice from the ANOVA ana

Variable Coding s From the regression analys

Statistical Tests s Note the difference betwee
x The S Streatment is larger
ANOVA Example x S Sdeviation = S Streatmen
q Example x The difference between S
q ANOVA termed S Sdeviation.
q Regression
s Take a look at how that diffe
q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

alysis, the S Streatment = 8.40.
sis, the S Sregression = 7.50.
en the two.
r.
nt − S Sregression = 0.90.

S Streatment and S Sregression is
erence comes about.

Slide 31 of 51

S S Differences

s The estimated regression li

Overview Y′ =

Multiple Categorical IVs X NX X¯ Y′ X¯ − Y

Variable Coding 1 3 3.0 3.2 -0

Statistical Tests 2 3 4.0 3.7 0

ANOVA Example 3 3 4.0 4.2 -0
q Example
q ANOVA 4 3 5.0 4.7 0
q Regression
5 3 5.0 5.2 -0
q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

ine is:

= 2.7 + 0.5X

Y′ (X¯ − Y ′ )2 NX (X¯ − Y ′ )2
0.2 0.04 0.12
0.3 0.09 0.27
0.2 0.04 0.12
0.3 0.09 0.27
0.2 0.04 0.12

NX (X¯ − Y ′ )2 0.90

Slide 32 of 51

Data Scatterplot

Overview 6.00
Multiple Categorical IVs
Variable Coding 5.00 Ω
Statistical Tests
ANOVA Example number correct 4.00 Ω ΩM
q Example
q ANOVA 3.00 ΩM Ω
q Regression
2.00 Ω 2.00
q S S Differences
1.00 number
q ANOVA Table
Curvilinear Regression
Wrapping Up

Lecture #11 - 11/12/2008

Ω Ω Ω

ΩM ΩM ΩM

Ω Ω Ω

3.00 4.00 5.00

r of exposures

Slide 33 of 51

Overview S S Differences

Multiple Categorical IVs s The value obtained in the p
the S Sdeviation.
Variable Coding
s The S Sdeviation is literally th
Statistical Tests measures a variable’s devia

ANOVA Example s This value serves as a basi
q Example
q ANOVA “What is the difference b
q Regression confirm to a linear trend
restriction?” (Pedhazur, p
q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

previous slide, 0.90, was equal to
he calculation of a statistic that
ation from linearity.
is for the question of:
between restricting the data to

and placing no such
p. 517)

Slide 34 of 51

Overview S S Differences

Multiple Categorical IVs s When the S Streatment is ca
the means of the treatment
Variable Coding
s If the means fall onto a (stra
Statistical Tests difference between S Streatm
S Sdeviation = 0.
ANOVA Example
q Example s With departures from linear
q ANOVA larger than the S Sregression
q Regression
s Do you feel a statistical hyp
q S S Differences

q ANOVA Table

Curvilinear Regression

Wrapping Up

Lecture #11 - 11/12/2008

alculated, there is no restriction on
t groups.
aight) line, there will be no
ment and S Sregression ,
rity, the S Streatment will be much

n.
pothesis test coming on?

Slide 35 of 51

Overview Hypothesis Test

Multiple Categorical IVs s The S ST reatments can be pa
S SRegression (also called th
Variable Coding remainder, the S S due to d

Statistical Tests Source
Between Treatments
ANOVA Example
q Example Linearity
q ANOVA Deviation From Linearity
q Regression
Within Treatments
q S S Differences Total

q ANOVA Table s If the S S due to linearity lea
one can conclude a linear t
Curvilinear Regression regression is appropriate.

Wrapping Up

Lecture #11 - 11/12/2008

artitioned into two components:
he S S due to linearity), and the
deviation from linearity.

df 1 SS MS F
4 3 8.60
7.50 7.50
10 7.50 0.30 0.30
14 0.90 1.00
10.00
18.40

ads to a significant F value, then
trend exists, and that linear

Slide 36 of 51

Overview Curvilinear Regres

Multiple Categorical IVs s The preceding example dem
could be detected using a s
Variable Coding
s A linear trend is something
Statistical Tests encountered linear regress

ANOVA Example s Curvilinear regression anal
not-so-linear trends exist be
Curvilinear Regression
q The Polynomial Model s Pedhazur distinguishes bet
q New Example possible:
q Estimation In SPSS
q Parameter Interpretation x Intrinsically linear.
q Variable Centering
q Multiple Curvilinear x Intrinsically nonlinear.
Regression

Wrapping Up

Lecture #11 - 11/12/2008

ssion

monstrated how a linear trend
statistical hypothesis test.

we are very familiar with, having
sion for most of this course.

lysis can be used to determine if
etween Y and X .
tween two types of trends

Slide 37 of 51

Overview Curvilinear Regres

Multiple Categorical IVs s An intrinsically linear mode
parameters but not linear in
Variable Coding
x By transformation such a
Statistical Tests linear model.

ANOVA Example x Such models are the foc
lecture.
Curvilinear Regression
q The Polynomial Model s An intrinsically nonlinear m
q New Example coerced into linearity by tra
q Estimation In SPSS
q Parameter Interpretation x Such models often requi
q Variable Centering algorithms than what is p
q Multiple Curvilinear GLM.
Regression

Wrapping Up

Lecture #11 - 11/12/2008

ssion

el is one that is linear in its
n the variables.
a model may be reduced to a
cus of this remainder of this

model is one that may not be
ansformation.
ire more complicated estimation
provided by least squares and the

Slide 38 of 51

Overview The Polynomial Mo
Multiple Categorical IVs
Variable Coding s A simple regression model
Statistical Tests the polynomial model, such
ANOVA Example polynomial:
Curvilinear Regression Y′ = a+
q The Polynomial Model
q New Example s One could also estimate a t
q Estimation In SPSS Y ′ = a + b1 X
q Parameter Interpretation
q Variable Centering s Or a fourth-degree polynom
q Multiple Curvilinear Y ′ = a + b1 X1 +
Regression
Wrapping Up s And so on...

Lecture #11 - 11/12/2008

odel

extension for curved relations is
h as the following second-degree
+ b1 X1 + b2 X12

third-degree polynomial:
X1 + b2 X12 + b3 X13

mial:
+ b2 X12 + b3 X13 + b4 X14

Slide 39 of 51

Overview The Polynomial Mo
Multiple Categorical IVs
Variable Coding s The way of determining the
Statistical Tests applicable is similar to dete
ANOVA Example significantly improve the pre
Curvilinear Regression model.
q The Polynomial Model
q New Example s Beginning with a linear mod
q Estimation In SPSS estimate the model, denote
q Parameter Interpretation
q Variable Centering s The tests of incremental va
q Multiple Curvilinear each level of the polynomia
Regression x Linear: Ry2.x
Wrapping Up x Quadratic: Ry2.x,x2 − Ry2.x
x Cubic: Ry2.x,x2 ,x3 − Ry2.x,x
Lecture #11 - 11/12/2008 x Quartic: Ry2.x,x2 ,x3 ,x4 − R

odel: Estimation

e extent to which a given model is
ermining if added variables

edictive ability of a regression

del (a first-degree polynomial),
ed as Ry2.x .
ariance accounted for are done for
al:

x
x2

Ry2.x,x2 ,x3

Slide 40 of 51

Overview A New Example

Multiple Categorical IVs s From Pedhazur, p. 522:

Variable Coding “Suppose that we are int
spent in practice on the p
Statistical Tests discrimination task. Subj
different levels of practice
ANOVA Example visual discrimination is a
correct responses is reco
Curvilinear Regression there are six levels the h
q The Polynomial Model possible for these data is
q New Example to determine the lowest d
q Estimation In SPSS the data.”
q Parameter Interpretation
q Variable Centering
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

terested in the effect of time
performance of a visual
jects are randomly assigned to
e, following which a test of
administered, and the number of
orded for each subject. As
highest-degree polynomial
s the fifth. Our aim, however, is
degree-polynomial that best fits

Slide 41 of 51

Data Scatterplot

Overview 20.00
Multiple Categorical IVs
Variable Coding 15.00
Statistical Tests
ANOVA Example Task Score 10.00 Ω
Curvilinear Regression Ω
q The Polynomial Model 5.00 Ω
q New Example Ω 5.00
q Estimation In SPSS Ω
q Parameter Interpretation Pra
q Variable Centering 2.50
q Multiple Curvilinear
Regression
Wrapping Up

Lecture #11 - 11/12/2008

ΩΩ
ΩΩ
ΩΩ
Ω

Ω
Ω
Ω
Ω
Ω

7.50 10.00

actice Time

Slide 42 of 51

Estimation In SPSS

Overview s To estimate the degrees of
create new variables in SPS
Multiple Categorical IVs a given power.

Variable Coding s Then successive regression
adding a level to the equati
Statistical Tests
Model R2 Incr
ANOVA Example 0.883
X 0.943
Curvilinear Regression X, X2 0.946
q The Polynomial Model X, X2, X3
q New Example
q Estimation In SPSS s Because adding X 3 did not
q Parameter Interpretation stop with the quadratic mod
q Variable Centering
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

S

a polynomial, first one must
SS, each representing X raised to

n analyses must be run, each
ion:

rease Over Previous F
0.940 121.029 *
0.060
0.003 15.604 *
0.911

t significantly increase R2 , we
del.

Slide 43 of 51

Overview Estimation In SPSS

Multiple Categorical IVs s Of course, there is an easie
s In SPSS go to Analyze...Re
Variable Coding

Statistical Tests

ANOVA Example

Curvilinear Regression
q The Polynomial Model
q New Example
q Estimation In SPSS
q Parameter Interpretation
q Variable Centering
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

S

er way...
egression...Curve Estimation

Slide 44 of 51

Estimation In SPSS

MODEL: MOD_2.
Independent: x

Overview Dependent Mth Rsq d.f. 12
12
Multiple Categorical IVs y LIN .883 16
y QUA .943 15 8
Variable Coding y CUB .946 14

Statistical Tests

ANOVA Example

Curvilinear Regression
q The Polynomial Model
q New Example
q Estimation In SPSS
q Parameter Interpretation
q Variable Centering
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

S

F Sigf b0 b1 b2 b3
-.0127
21.03 .000 3.2667 1.5571 -.1384
23.55 3.4946 .1290
82.18 .000 -1.9000 1.8803

.000 .6667

Slide 45 of 51

Data Scatterplot

Overview
Multiple Categorical IVs
Variable Coding
Statistical Tests
ANOVA Example
Curvilinear Regression
q The Polynomial Model
q New Example
q Estimation In SPSS
q Parameter Interpretation
q Variable Centering
q Multiple Curvilinear
Regression
Wrapping Up

Lecture #11 - 11/12/2008

Slide 46 of 51

Overview Parameter Interpret

Multiple Categorical IVs s The b parameters in a polyn
impossible to interpret.
Variable Coding
s An independent variable is
Statistical Tests single parameter - what’s h

ANOVA Example s The relative magnitude of th
degrees cannot be compar
Curvilinear Regression degree polynomials explode
q The Polynomial Model
q New Example x X → sx2
q Estimation In SPSS x X 2 → (sx2 )2
q Parameter Interpretation x X 3 → (s2x )3
q Variable Centering x ...
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

tation

nomial regression are nearly
represented by more than a

held constant?
he b parameters for different

red because the SD of the higher
es.

Slide 47 of 51

Overview Variable Centering

Multiple Categorical IVs s Centering variables in a po
collinearity problems.
Variable Coding
s Centering does not change
Statistical Tests regression parameters.

ANOVA Example

Curvilinear Regression
q The Polynomial Model
q New Example
q Estimation In SPSS
q Parameter Interpretation
q Variable Centering
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

olynomial equation can avoid
e the R2 of a model, only the

Slide 48 of 51

Overview Multiple Curvilinear

Multiple Categorical IVs s Running multiple curvilinea
forward extensions from wh
Variable Coding Y ′ = a + b1 X + b2 Z

Statistical Tests s Note the cross-product X Z
s This cross product term is t
ANOVA Example
Z individually.
Curvilinear Regression
q The Polynomial Model
q New Example
q Estimation In SPSS
q Parameter Interpretation
q Variable Centering
q Multiple Curvilinear
Regression

Wrapping Up

Lecture #11 - 11/12/2008

r Regression

ar regression models are straight
hat was shown today:
Z + b3 X Z + b4 X 2 + b5 Z 2
Z.
tested above and beyond X and

Slide 49 of 51