The 2 Sigma Problem: The Search for Methods of Group ...

The 2 Sigma Problem: The Search for
Methods of Group Instruction as

Effective as One-to-One Tutoring

BENJAMIN S. BLOOM

University of Chicago and Northwestern University

T w o University of Chicago doc- The students were randomly dents under conventional instruc-
toral students in education, Anania assigned the three learning condi- tional conditions. (See Figure 1.)
(1982, 1983) and Burke (1984), com- tions, and their initial aptitude
pleted dissertations in which they tests scores, previous achievement There were corresponding
compared student learning under changes in students' time on task in
the following three conditions of in the subject, and initial attitudes the classroom (65% under convert-
instruction: and interests in the subject were tional instruction, 75% under Mas-
similar. The amount of time for in- tery Learning, and 90+ % under
1. Conventional. Students learn struction was the same in all three tutoring) and students' attitudes
the subject matter in a class with groups except for the corrective and interests (least positive under
about 30 students per teacher. work in the mastery learning and conventional instruction and most
Tests are given periodically for tutoring groups. Burke (1984) and positive under tutoring). There
marking the students. Anania (1982, 1983) replicated the were great reductions in the rela-
study with four different samples of tions between prior measures (ap-
2. Mastery Learning. Students students at grades four, five, and titude or achievement) and the sum-
learn the subject matter in a class eight and with two different subject mative achievement measures.
with about 30 students per teacher. matters, Probability and Cartog- Typically, the aptitude-achievement
The instruction is the same as in the raphy. In each sub-study, the in- correlations changed from +.60
conventional class (usually with the structional treatment was limited to under conventional to + .35 under
same teacher). Formative tests (the 11 periods of instruction over a mastery learning and +.25 under
same tests used with the conven- 3-week block of time. tutoring. It is recognized that the
tional group) are given for feedback correlations for the mastery learn-
followed by corrective procedures Most striking were the differ- ing and tutoring groups were so low
and parallel formative tests to ences in final achievement mea- because of the restricted range of
determine the extent to which the sures under the three conditions. scores under these learning condi-
students have mastered the subject Using the standard deviation tions. However, the most striking of
matter. (sigma) of the control (conventional) the findings is t h a t u n d e r the be._.~st
class, it was typically found that the learning conditions we can devise
3. Tutoring. Students learn the average student under tutoring was (tutoring), the average student is 2
subject matter with a good tutor for about two standard deviations ~ o v e th~ ayerage__~-'ntro']
each student (or for two or three above the average of the control student taught under conventional
students simultaneously). This class (the average tutored student group_methods of instru~io--m-~-.
tutoring instruction is followed peri- was above 98% of the students in
odically by formative tests, feed- the control class).1The average stu- The tutoring process demon-
back-corrective procedures, and dent under _mastery learning was strates that most of the students do
parallel formative tests as in the about one standard ¢[evlation above hav~~ntiai toreac~h
mastery learning classes. It should tl~ average of the control gcJ ~ ( t h e level of learning. I believe an impor-
be pointed out that the need for cor- average maste---6~earn]ng student tan~T-~--df~esearch and instruc-
rective work under tutoring is very was above 84% of the students in tion is to seek ways of accomplish-
small. the control class). ing this under more practical and
realistic conditions than the one-to-
Benjamin S. Bloom is Professor of The variation of the students' one tutoring, which is too costly for
Education, University of Chicago and achievement also changed under most societies to bear on a large
Northwestern University, 5835 S. these learning conditions such that scale. This is the "2 sigma" proh-
Kimbark Avenue, Chicago, Illinois about 90% of the tutored students lem. C'~ researchers and teachers
60637. His areas of specialization are and 70% of the mastery learning d~~nditio_~s
evaluation and the teaching-learning students attained the level of sum- that wl-g-6nable the majority of
process. mative achievement reached by students under group instruction to
only the highest 20% of the stu-
Educational Researcher

attain levels of achievement that effects on what the schools can and case, we contrasted these alterable
should do with the educational years
cah-~t~~-bbe_.._reached only each society requires of its young variables with the more stable
under good tutoring conditions? people.
variables (e.g., personal character-
It--ha s ~ l ~ d ~ c~-de and This paper is a brief presentation istics of teachers, intelligence
a half to develop the Mastery of the work on solutions to the 2 sig- measures, achievement tests for
Learning (ML) strategy to a point ma problem. It is hoped that it will grading purposes, socioeconomic
interest both educational research- status of the family, etc.) and in-
where large numbers of teachers at ers and teachers in further research dicated some of the ways in which
every level of instruction and in and application of these ideas. the alterable variables influence
learning and the processes by which
many countries can use the feed- The Search these variables have been altered.
back-corrective procedures to get
the 1 sigma effect (the average ML In a number of articles, my gradu- But not all alterable variables are
ate students and I have attempted likely to-have equal effects on learn-
student is above 84% of the stu- to contrast alterable educational ing. Our research summaries were
dents under conventional instruc- variables with more stable or static intended to emphasize the alterable
tion-even with the same teacher variables (Bloom, 1980). In our variables that have had the strong-
treatment of this topic, we sum- est effects on school learning. With-
teaching both the ML and the con- marized the literature on such alter- in the last 3 years, this search has
ventional classes). If the research on been aided by the rapid growth of
able variables as the quality of the meta-analysis literature. In this
the 2 sigma problem yields practical teaching, the use of time by teachers literature, each writer has summar-
methods (methods that the average and students, cognitive and affective
ized the research literature on a
teacher or school faculty can learn entry characteristics of students, particular set of alterable variables
in a brief period of time and use to indicate the effect size between
with little more cost or time than formative testing, rate of learning, control and experimental groups of
conventional instruction), it would and the home environment. In each students. They have standardized

be an educational contribution of
the greatest magnitude. It would
change popular notions about human
potential and would have significant

FIGURE 1. Achievement distribution for students under conventional, mastery learning, and tutorial
instruction.

TUTORIAL
I -I*

MASTERY LEARNING

1-30~ f

CONVENTIONAL
1-30*

Summative Achievem@nt Scores

*Teacher-student ratio

June/July 1984

the results in terms of the difference or peer group; and (d) the teacher fects of separable variables may b'e
and the teaching process. best combined.
between the experimental and con-
trol groups divided by the standard We have speculated that two vari- In our work so far we have
deviation of the control group. 2 ables involving different objects of restricted the search to two or three
the change process may, in some in- variables, each of which is likely to
In each study, the reviewer also stances, be additive, whereas two have a .5 sigma effect or greater.
analyzed the effect size under dif- variables involving the same object We suspect that the research, as
ferent conditions, level of school, of the change process are less like- well as the applications to school
sex of student, school subject, size ly to be additive (unless they occur situations, would get too complex if
of sample, and so on. Such reviews at different times in the teaching- more than three alterable variables
are very useful in selecting alterable learning process). Our research is are used. In any case, our work has
variables that are most likely to con- intended to determine when these begun with variables in the top half
tribute significantly to the 2 sigma rules are true and when they are of Tafle I. Perhaps as the research
solution. not. Several of the studies done so moves on, it will be necessary to in-
far suggest that they may be true. clude some of the variables in the
Table I is adapted from a sum- Thus the ML process (which affects lower part of Table I.
mary of effect sizes of key variables the learner most directly), when
by Walberg (1984) who, with other combined with changes in the In our research with two
co-authors, has contributed greatly teaching process (which affects the variables, we have made use of a
to this literature. In Table I he has teacher most directly), yield ad- 2 x 2 randomized design with ML
listed the selected variables in order ditive results. (See Tenenbaum, p. and one other variable. So far we
of magnitude of effect size. (We 13 of this article and Mevarech, p. have not done research with three
have added other variables and indi- 14 of this article). Although we do variables. Where possible, we try to
cated the equivalent percentile for not believe these two rules are more replicate the study with at least two
each effect size.) Thus, in the first subject fields, two levels of school-
than suggestive at present, future ing, or some combination of subject
entry, tutorial instruction, we have fields and levels of schooling. We
research on this problem will un- hope that others will take up this 2
indicated the effect size (2 sigma) doubtedly yield a stronger set of sigma search and that some guide-
and indicated that under tutorial in- generalizations about how the ef-
struction, the average student is
above 98% of the students under TABLE I
the control teaching conditions. A
list of effect size studies appears in Effect of selected alterable variables on student achievement
the Appendix at the end of this (see Appendix)
article.
Percentile
In our own attempts to solve the Effect size equivalent
2 sigma problem we assume that
two or three alterable variables Da Tutorial instruction 2.00 98
1.20 84
must be used that together con- D Reinforcement 1.00
A Feedback-corrective (ML) 1.00 79
tribute more to the learning than D Cues and explanations 1.00 73
any one of them alone. Because of (A)D Student classroom participation 1.00b 69
more than 15 years of experience A Student time on task 1.00 66
with ML at different levels of A Improved reading/study skills 62
education and in different coun- C Cooperative learning .80
tries, we have come to rely on ML D Homework (graded) .80 58
as one of the possible variables to D Classroom morale .60
be combined with selected other A Initial cognitive prerequisites .60 60
variables. ML (the feedback- C Home environment intervention .50b
corrective process) under good con- D Peer and cross-age remedial
ditions yields approximately a 1 .40
sigma effect size. We have system- tutoring .30
atically tried other variables which, .30
in combination with ML, might ap- D Homework (assigned) .30b
proach the 2 sigma effect size. So D Higher order questions .30
(D)B New science & math curricula .20
far, we have not found any two .20
D Teacher expectancy
variable combination that has ex- C Peer group influence .25
ceeded the 2 sigma effect. Thus, B Advance organizers
some of our present research
reaches the 2 sigma effect but does Socio-economic status
not go beyond it.
(for contrast)
We have classified the variables
in Table I in terms of the direct ob- Note. This table was adapted from Walberg (1984) by Bloom.
ject of the change process: (a) the aobject of change process--A-Learner; B-Instructional Material; C-Home en-
learner; (b) the instructional vironment or peer group; D-Teacher.
material; (c) the home environment bAveraged or estimated from correlational data or from several effect sizes.

6 Educational Researcher

lines for the research can be set up cognitive entry prerequisites at the a 2-week period of learning in the
to make the combined results more beginning of the course. Working advanced course.
useful and to reduce the time and with high school teachers in Alge-
costs for experimental and demon- bra 2 and French 2, they developed When one of the enhanced classes
stration studies. an initial test of the prerequisites was also provided with ML feed-
for each of these courses. The pro- back-corrective procedures over a
Improving Student Processing of cedure in developing the initial test series of learning tasks, the final
Conventional Instruction was to take the final examination in results after a 10- to 12-week period
the prior course (Algebra 1 or of instruction was that this experi-
In this section of the paper we are French 1) and have a committee of mental group was approximately
concerned with ways in which four to six teachers in the subject in- 1.6 sigma above the control group
students can learn more effective- dependently check each test item on the summative examination.
ly without basically changing the that they believed measured an idea (The average student in the ML plus
teaching. If students develop good or skill that was a necessary prere- enhanc~'d initial prerequisites was
study habits, devote more time to quisite for the next course in the above 95% of the control students
the learning, improve their reading subject. There was very high agree- on this examination.) There were
skills, and so on, they will be better ment on most of the selected items, also attitudinal and other affective
able to learn from a particular and discussion among the teachers differences in students related to
teacher and course--even though led to consensus about some of the these achievement differences.
neither the course nor the teacher remaining items. These included positive academic
has undergone a change process. self-concept, greater interest in the
Two of the classes were helped to subject, and greater desire to learn
For example, the ML feedback- review and relearn the specific more in the subject field.
corrective approach is addressed prerequisites they lacked. This was
primarily to providing students not done for the students in the In Leyton's (1983) study, he
with the cognitive and affective other two classes--they spent the found that the average effect of in-
prerequisites for each new learning time on a more general and infor- itial enhancement of prerequisites
task. As we have noted before, mal review of the content taught in alone is about .6 sigma (see dif-
when the ML procedures are done the previous course (Algebra 1 or ferences between conventional and
systematically and well, the school French 1). The method of enhanc- conventional plus enhanced prere-
achievement of the average student ing the prerequisites was much like quisites and between ML and ML
under ML is approximately i sigma the ML feedback-corrective process plus enhanced prerequisites in
(84 percentile) above the average where the teacher retaught the Figure 2). That is, we have two pro-
student in the control class, even items that the majority of students cesses -- ML and initial enhance-
when both classes are taught by the had missed, small groups of stu- ment of cognitive prerequisites --
same teacher with much the same dents helped each other over items that have sizeable but separate ef-
instruction and instructional that had been missed, and the stu- fects. When they are combined,
material. As we view the ML pro- dents reviewed items they were not their separate effects tend to be ad-
cess, we regard it as a method of im- sure about by referring to the desig- ditive. We believe these two
proving the students' learning from nated pages in the instructional variables are additive because they
the same teaching over a series of material. The corrective process occur at different times. The
learning tasks. took about 3 to 4 hours during the enhancement of the initial prere-
first week of the course. After the quisites is completed during the
The major changes under the ML students completed the corrective first week of the new course, while
process are that more of the process, they were given a parallel the ML feedback-corrective process
students have the cognitive prere- test. As a result of the corrective takes place every 2 or 3 weeks dur-
quisites for each new learning task, process, most of the students ing the course, after the initial
they become more positive about reached the mastery standard (80%) enhancement.
their ability to learn the subject, and on the parallel test given at the end
they put in more active learning of the first week of the course. In This solution to the 2 sigma prob-
time than do the control students. a few cases, students who didn't lem is likely to be applicable to se-
As we observe the students' learn- reach this standard were given fur- quential courses in most school sub-
ing and the test results in the ML ther help. jects. (In the United States, over
and the conventional class, we note two-thirds of the academic courses
the improvements in the student More important was the im- in elementary-secondary schools are
learning under ML and the lack of proved performance of the en- sequential courses.) This solution, of
such improvement in conventional hanced classes over the other two course, applies most clearly to the
classes. classes on the first formative test in second courses in a sequence. It
the advanced course (French 2 or probably will not work as well with
One of our University of Chicago Algebra 2). The two enhanced the third, fourth, or later courses in
doctoral students, Leyton (1983), classes, which had been helped on a sequence if there has been no
suggested that one approach to the the initial prerequisites, were ap- earlier use of initial enhancement of
2 sigma problem would be to use proximately .7 sigma higher than prerequisites or ML procedures.
ML during the advanced course in the other two classes on the first We hope these ideas will be further
a sequence, but in addition attempt formative test given at the end of explored in the United States as
to enhance the students' initial well as in other countries. We

June/July 1984 7

believe this solution is relevant at all cedures that work well in enhanc- student academic self-concept, and
levels of education, including ing these prerequisites in one school student attitudes and interest in the
elementary-secondary, college, and should work equally well in other learning.
even the graduate and professional schools. Further research is need-
school level. ed to establish the sequential Our graduate students have writ-
courses in which this approach is ten papers on several other ap-
We also regard this approach as most effective. proaches for improving student pro-
widely applicable within a country cessing of conventional instruction:
because the prerequisites for a par- Finally, the time cost of the initial
enhancement procedures is limited 1. Help students develop a stu-
ticular sequential subject or course to the class hours of the course dur- dent support system in which
are likely to be very similar even ing the first week of the sequential groups of two or three students
though different textbooks and course, while the time or other costs study together, help each other
teachers may be involved. Thus, a of the ML procedures have usually when they encounter difficulties in
well made test of the initial prere- been very small. We hope that this the-course, help each other review
quisites for a particular sequential approach to the 2 sigma problem in advance of taking tests, and
course--Arithmetic 2, French 2, will be found to be a widely ap- review their learning periodically. A
Reading 2, and so on--may with plicable as well as economical solu- student support system that pro-
only minor changes apply to other tion available to most teachers who vides support, encouragement and
versions of the same course within wish to improve student learning, even help when needed can do much
a particular country. Also, the pro- to raise the level of learning of the

FIGURE 2. Average summative achievement scores under different/earning conditions. Comparison of
tutoring studies, mastery/earning, and enhanced prerequisites•

100 .7¢ 2 S I GMA 2.0o"
9O 95%
80 E-. o6c .00"
70 H
0or H
~o 50%
o3
60 H
~ 50 C~
n~
4O
0 30 r.3

20 0 -4- r.9
I0 Z C~ I-I CO
4- H
4-
Z
O r.9
H
TUTORING STUDIES
Z ANANIA (1981) BURKE (1983)
O
Educational Researcher
LEYTON (1983)

participants. There is evidence that be provided in the form of objec- or 3 weeks during the course. We
these and other cooperative learn- tives, some ideas about what will be believe that each of these processes
ing efforts are almost as effective learned in the unit, or a brief discus- is somewhat independent of the
as ML procedures. (Cooperative sion of the relation between what other processes.
Learning -- Effect size .80 (79 per- has already been learned and what
centile) Slavin, 1980.) will be learned in the unit. Such ad- Other suggestions for the ira-
vanced organizers (Luiten, Ames, & provement of instructional materials
2. There is evidence that Ackerson, 1980) appear to have an and educational technology include
students who take special programs average effect size on achievement the following:
to improve their reading and/or of about .2 sigma. (Incidentally,
their study and learning methods such advance organizers have about 1. Some of our students have
tend to learn more effectively. a .4 sigma effect on retention of the used computer learning courses,
Ideally, such special programs learning.) Although this effect is such as the Plato system, which ap-
should be available at the beginning rather consistent, by itself it is not pear to work very well for highly
of each new school level, that is, enough to contribute significantly to motivated students. We believe that
junior high school, high school, and the 2 sigma effect. It is likely that it should be possible to determine
so on. One would hope that the a combination of advance whether particular computer
special programs would be closely organizers at the beginning of a courses enable sizeable proportions
related to the academic courses the new topic, further organizational of students to attain the 2 sigma
student is currently taking. (Im- aids during the chapter or unit, as achievement effect. The effec-
proved reading/study skills--Effect well as appropriate questions, sum- tiveness of the computer courses
size 1.00 (84 percentile) (Pflaum, maries, or other organizational aids can be determined in terms of the
Walberg, Karegianes, & Rasher, at the end of the unit, may have a time required, completion rates,
1980). substantial effect on the student's student performance on achieve-
learning of that chapter. ment tests, and student retention of
Improve Instructional Materials the learned material. It is hoped
and Educational Technology In Process that the more effective computer
courses will also have positive ef-
The textbook in the United One of our students, Carlos fects on such affective characteris-
States, as well as in most advanced Avalos, is working on a study of the tics as academic self-concept, in-
countries in the world, is an almost effect of organizational aids in the terest in the subject, and desire to
universal part of school instruction. instructional material combined learn further with computer learn-
There has been much work on the with the initial enhancement of ing methods.
improvement of the textbooks for cognitive prerequisites and the ML
reading and, to some extent, arith- feedback-corrective procedures. 2. Although the average effect
metic, mathematics, and science Avalos is planning a research size for new science and math cur-
subjects. Most of these are in rela- design that will enable him to deter- ricula in the United States is only
tion to special curricular improve- mine the separate effects of each of .3 sigma, some of the new curricula
ments, which include improvements the three processes, the effect of (or textbooks) in these and other
in the sequential nature of the any two of the processes, and the subjects may be much more effec-
topics, the attempt to find impor- combined effect of all three pro- tive than others. We propose a
tant ideas or schema that help to in- cesses. At the least, it is anticipated careful search of the new curricula
terrelate the different parts of the that the combination of any two of and textbooks to determine which
subject, and improvements in the il- the processes will be greater than ones are more effective and to
lustrations and exercises in the the effects of any one of the same determine what characteristics
books. However, as far as we can processes. It is hoped that the effect make them more effective than the
find, these improvements have not of any two will be above 1.3 sigma others.
had very significant effects on stu- (90 percentile). If this is found, it
dent achievement unless the teach- will provide several new solutions to Home Environment
ers were provided with much in- the 2 sigma problem--some of and the Peer Group
service education for the new cur- which can be done with very little
riculum or the new textbook. cost or effort by the teachers or the In this section, we are primarily
school system. concerned with the out-of-school
My graduate students and I have support that the student receives
been intrigued by the possibility Avalos expects the results noted from the home or the peer group.
that the organization of a particular above because the organizational We are interested in the ways in
section (or chapter) of the textbook •aids can be built into new textbooks which the student's achievement,
might be better integrated or the and can be used by the students academic aspirations and goals, and
parts of the section more closely with a minimum of emphasis by the progress in learning are influenced
related to each other. Pre- teachers. The initial enhancement by these types of support. We know
organizers or advanced organizers of the prerequisites is completed that the home environment does
(Ausubel, 1960), have been before the students begin the study have great influence on the pupil's
moderately effective when provid- of the new course subject matter, school learning and that this in-
ed in the textbook or provided by whereas the ML feedback-correc- fluence is especially effective at the
the teacher at the beginning of the tive procedures take place every 2 elementary school level or earlier.
new unit of the course. These may The peer group's influence is likely
to be strongest (both positively or
June/July 1984
9

Although it is difficult to influence the student's choice separately by a parent educator
offriends and peer groups, the availability in the school twice a month for 6 months. A third
of a variety of extracurricular activities and clubs... experimental approach was that
should enable students to be more selective in their peer newsletters about the same topics
choices within the school setting. were sent to the home twice a
month for 6 months.
negatively) at the secondary school for the child's school achievement,
level. and their interests in and The parents of all four groups
knowledge of the child's school were observed and interviewed at
Home Environment Processes experiences. the beginning and end of the
6-month period using the Dave
There have been a large number These studies of the home en- (1963) interview and observational
of studies of the home environment vironment processes began with the me~ods. Although the three ex-
processes that affect the students' work of Dave (1963) and Wolf perimental approaches show signi-
school learning. These studies in- (1964, 1966), and since then have ficantly greater changes in the par-
volve interviews and observations been replicated in other studies ents' home environment index than
directed at determining the rele- done in the United States and other the control group, the most effec-
vant interactions between parents countries (Marjoribanks, 1974; tive method was the series of meet-
and their children. The studies find Kalinowski & Sloane, 1981). ings between groups of parents and
correlations of +.70 to +.803 be- the parent educator. The changes in
tween an index of the home environ- These previous studies of the rela- the home environment of this group
ment processes and the children's tionship between the home and the were highly significant when com-
school achievement. Some of the children's school achievement sug- pared with the changes in the other
borne environment processes that gest a strong effect of the home en- three groups of parents.
appear to have high relationships vironment on the school learning of
with school achievement include the the children, but they do not pro- The fourth grade children of all
following: these parents were given a national
vide evidence on the extent to which standardized test on reading and
1. Work habits of the family-the the home environment can be mother tongue as well as arithmetic
degree of routine in the home man- altered and the effect of such altera- at the beginning and end of the
agement, the emphasis on regulari- tion on changes in the children's 6-month period. It was found that
ty in the use of space and time, and school achievement. the children of the meeting group of
the priority given to schoolwork parents had changed by 1 sigma in
over other more pleasurable A recent study done in Thailand achievement, as contrasted with the
activities. by Janhom (1983) involved a control change in the control group of chil-
group and three experimental dren. In comparison, the parent
2. Academic guidance and sup- groups of parents (and their educators' visit to each of the homes
port--the availability and quality of children). In this study, the most ef- every other week had only a .5 sig-
the help and encouragement fective treatment of the parents
parents give the child for his or her was for the group of parents to ma effect on the children's school
schoolwork and the conditions they meet with a parent educator for achievement.
provide to support the child's about 2 hours twice a month for 6
schoolwork. months. In these meetings, the Other methods of changing the
parents discussed ways in which home environment have been re-
3. Stimulation in the home--the they could support their children's ported by Dolan (1980), Bronfen-
opportunity provided by the home learning in the school. There was brenner (1974), and Kalinowski and
to explore ideas, events, and the usually an initial presentation made Sloane (1981). Again, the most ef-
larger environment. by the parent educator on one of the fective approaches to changing the
home environment processes and home environmentprocesses result
4. Language development--op- in changes in the children's school
portunities in the home for the then the parents discussed what achievement. (Home Environment
development of correct and effec- they did as well as what they hoped - - Effect size .50 (69 percentile),
tive language usage. to do to support their children's Iverson & Walberg, 1982.)
school learning.
5. Academic aspirations and ex- The methods of changing the
pectations-the parents' aspirations Another experimental approach home environments are relatively
for the child, the standards they set included visits to each home costly in terms of parent educators
meeting with groups of parents
10 over a.series of semi-monthlymeet-
ings, but the payoff of this approach
is likely to be very great. If parents
continue to encourage and support
each of their children to learn well
in school throughout the elementary
school years, this should greatly
help the children during the years
they will attend schools and
colleges.

Educational Researcher

Although such research has not Teachers are frequently unaware of the fact that they
been done as yet, we hope that are providing more favorable conditions of learning for
others will explore an approach to some students than they are for other students. General-
the 2 sigma problem of providing ef- ly, they are under the impression that all students in
fective parent education combined their classes are given equality of opportunity for
with the mastery learning method. learning.
Because parent support takes place
in the home and ML takes place in der tutoring. We have pondered the feedback on the clarity of his or
the school, we expect that these two these facts and believe that this in her explanations, the effect of the
effects will be additive. The result part results from the unequal treat- reinforcements, and the degree of
should be close to a 2 sigma im- ment of students within most class- active involvement in the learning
provement in student learning. rooms. from a small number of high achiev-
ing students in the typical class of
Ideally, if both methods began Observations of teacher interac- 30 students.
with first or second grade children, tion with students in the classroom
one might hope that the combina- reveal that teachers frequently Teachers are frequently unaware
tion would result in consistently direct their teaching and explana- of the fact that they are providing
good learning, at least through the tions to some students and ignore more favorable conditions of learn-
elementary school years, with less others. They give much positive ing for some students than they are
and less need for effort expended by reinforcement and encouragement for other students. Generally, they
the parents or by the use of ML pro- to some students but not to others, are under the impression that all
cedures in the school. and they encourage active par- students in their classes are given
ticipation in the classroom from equality of opportunity for learning.
Peer Group some students and discourage it One basic assumption of our work
from others. The studies find that on teaching is the belief that when
During the adolescent years, it is typically teachers give students in teachers are helped to secure a
likely that the peer group will have the top third of the class the more accurate picture of their own
considerable influence on the stu- greatest attention and students in teaching methods and styles of in-
dent's activities, behavior, at- the bottom third of the class receive teraction with their students, they
titudes, and academic expectations. the least attention and support. will increasingly be able to provide
The peer group(s) to which the in- These differences in the interaction more favorable learning conditions
dividual "belongs" also has some ef- between teachers and students pro- for more of their students, rather
fect on the student's high school vide some students with much than just for the top fraction of the
achievement level as well as further greater opportunity and encourage- class.
academic aspirations. These effects ment for learning than is provided
appear to be greatest in urban set- for other students in the same class- In some of our research on the 2
tings. Although it is difficult to in- room (Brophy & Good, 1970.) sigma problem, we have viewed the
fluence the student's choice of task of teaching as providing for
friends and peer groups, the avail- It is very different in a one-to-one more equal treatment of students.
ability in the school of a variety of tutoring situation where there is a We have been trying to give teach-
extracurricular activities and clubs constant feedback and corrective ers feedback on their differential
(e.g., athletics, music, science, process between the tutor and the treatment of students. We attempt
mathematics, social, etc.) should en- tutee. If the explanation is not to provide teachers with a mirror of
able students to be more selective understood by the tutee, the tutor what they are now doing and have
in their peer choices within the soon becomes aware of it and ex- them develop techniques for
school setting. (Peer Group Influ- plains it further. There is much rein- equalizing their interactions with
e n c e - E f f e c t size .20 (58 percentile) forcement and encouragement in the students. These include such
(Ide, Haertel, Parkerson, & Wal- the tutoring situation, and the tutee techniques as: (a)attempt to find
berg, 1981). must be actively participating in the something positive and encouraging
learning if the tutoring process is to in each student's response, (b) find
Improvement of Teaching continue. In contrast, there is less ways of involving more of the
feedback from each student in the students in active engagement in
When we compare student learn- group situation to the teacher--and the learning process, (c)secure
ing under conventional instruction frequently the teacher gets most of feedback from a small random sam-
and tutoring we note that approxi-
mately 20% of the students under 11
conventional instruction do about as
well as the tutored students. (See
Figure 1). That is, tutoring probably
would not enable these top students
to do any better than they already
do under conventional instruction.
In contrast, about 80% of the stu-
dents do relatively poorly under
conventional instruction as com-
pared with what they might do un-

June/July 1984

pie of students to determine when ing the frequency with which the time for the students in the en-
they comprehend the explanations experimental teachers were using hanced cue + participation classes.
and illustrations, and (d) find ways these ideas well or poorly. Similar-
of supplying additional clarification ly, he had students note the fre- In terms of final achievement, the
and illustrations as needed. The ma- quency with which they were ac- average student in the enhanced
jor emphasis in this work was not tively participating in the learning cue and participation group was 1.5
to change the teachers' methods of and any problems they had with un- sigma higher than the average stu-
instruction, but to have the teacher derstanding the ideas or explana- dent in the control classes. (The
become more aware of the ways in tions. average student in the enhanced
which he or she could more directly group was above 93% of the stu-
teach to a cross section of the stu- In this research he compared stu- dents in the control classes.) (See
dents at each class section. dent learning under conventional Figure 3.) Nordin (1979, 1980) also
instruction and under enhanced made use of the ML procedures in
The first of our studies on improv- cues (explanations) and participa- othe? classes and found that they
ing instruction was done by Nordin tion conditions. During the experi- worked even better than the en-
(1979, 1980), who found ways of im- ment, observers noted that the stu- hanced cue + participation proce-
proving the cues and explanations dent participation and the explana- dures. Unfortunately, he did not use
for students as well as increasing tions and directions were positive in the ML in combination with the en-
the active participation of students. about 57% of the observations in hanced cue + participation
the control class as compared with methods.
He found it helpful to meet fre-
quently with the teachers to explain about 67% in the enhanced cue + In any case, Nordin (1979, 1980)
these ideas as well as to observe the participation classes. Students in did demonstrate that teachers could
teachers and help them determine the control classes noted that the
when they still needed to improve cues and participation were positive be taught ways to be more respon-
these qualities of the instruction. He for them about 50% of the time as sive to most of the students in the
also had independent observers not- compared with about 80% of the class, secure increased participation
of the students, and insure that

FIGURE 3. Average summative achievement scores under different learning conditions. Comparison of
tutoring studies, mastery learning, and enhanced instructional methods.

1 O0 2 SIGMA 98%
...................................
96% . . . . . . . . . . . . . . . . . . . . . . . . .
_ 93%
2 o-
.5o"
9O .7(

84% 84%

80 1.0" 1.m"

70

o9 Z 50~
0
Z 60
a
<
> [D
~-4
50 EI~

O' S~
C~
[-~ 4o Z r.,.U :.3
H .-4
I.-I Z
0
H --1

r.D 30 >
r~ Z
0
0 I--I Z O9
I-4
NORDIN (1979) 4-
2o O9

<3 Z
Oa
TF~ENBAUM (1982)
lO

Z O
O9

O
r.D

12 TUTORING STUDIES
ANANIA (1981) BURKE (1983)

Educational Researcher

most of the students understood the securing overt as well as covert par- In these countries, subjects are
explanations and illustrations that ticipation of the students in the taught as methods of inquiry into
the teacher provided. The observers learning, seems to us to be an ex- the nature of science, mathematics,
noted that the students in the cellent approach. This may be in the the arts, and the social studies. The
enhanced participation and cue form of an observer's notes on what subjects are taught as much for the
classes were actively engaged in the teacher and students did, stu- ways of thinking they represent as
learning (time on task) about 75% dent observations of their own in- for their traditional content. Much
of the classroom time, whereas the teractions with the teaching of this learning makes use of obser-
control students were actively (preferably anonymous, but coded vations, reflections on these obser-
learning only about 57% of the time. as to whether the students are in vations, experimentation with phe-
the top third, middle third, or the nomena, and the use of first hand
In a later study, Tenenbaum bottom third of the class in achieve- data and daily experiences, as well
(1982) compared control groups, ment), such as their understanding as the use of primary printed
ML groups, and Enhanced Cues, of the cues and explanations, the ex- sources. All of this is reflected in the
Participation, and Reinforcement in tent of their overt & covert par- materials of instruction, the learn-
combination with ML (CPR + ML). ticipation, and the amount of rein- ing and teaching processes used,
Tenenbaum studied these three forcement they are getting. Per- and the questions and problems
methods of teaching with random- haps a video-tape or audio tape re- used in the quizzes and formative
ly assigned students in two differ- cording of the class could serve the testing, as well as on the final sum-
ent courses--sixth grade science same purpose if the teacher is given mative examinations.
and ninth grade algebra. brief training on ways of summar-
izing the classroom interaction be- In sharp contrast with some of
Tenenbaum also used student ob- tween the teacher and the students these other countries, teachers in
servation of their own classroom in the class. the United States typically make
processes on cues, participation, use of textbooks that rarely pose
and reinforcement. He found that It is our hope that when teachers real problems. These textbooks em-
under the CPR + ML, students are helped to secure a more ac- phasize specific content to be
responded positively about their curate picture of their own teaching remembered and give students lit-
own participation about 87% of the methods and styles of interaction tle opportunity to discover underly-
time as contrasted with 68% in the with their students, they will be bet- ing concepts and principles and
control classes. ter able to provide favorable learn- even less opportunity to attack real
ing conditions for most of their stu- problems in the environments in
The results of this study demon- dents. which they live. The teacher-made
strated large differences between tests (and standardized tests) are
the three methods of instruction Improvement of Teaching of largely tests of remembered infor-
with the final achievement scores of the Higher Mental Processes mation. After the sale of over one
the CPR + ML group about 1.7 million copies of the Taxonomy of
sigmas above the control students Although there is much of rote Educational Objectives-Cognitive
(the average student in this group learning in schools through the Domain (Bloom, Engelhart, Furst,
was above 96% of the students in world, in some of the national cur- Hill, & Krathwohl, 1956) and over
the control group.) The average stu- riculum centers in different coun- a quarter of a century of use of this
dent in the ML groups was the tries (e.g., Israel, Malaysia, South domain in preservice and in-service
usual 1 sigma above the control stu- Korea) I find great emphasis on teacher training, it is estimated that
dents. (See Figure 3). problem-solving, application of prin- over 90% of test questions that U.S.
ciples, analytical skills, and creativi- public school students are now ex-
We believe that this research ty. Such higher mental processes pected to answer deal with little
makes it clear that teachers in both are emphasized because these cen- more than information. Our instruc-
the Nordin and Tenenbaum studies ters believe that they enable the stu- tional material, our classroom
could (at least temporarily) change dent to relate his or her learning to teaching methods, and our testing
their teaching methods to provide the many problems he or she methods rarely rise above the low-
more equal treatment of the stu- encounters in day-to-day living. est category of the Taxonomy-
dents in their classes. When this These abilities are also stressed knowledge.
more equal treatment is provided because they are retained and used
and supplemented with the ML long after the individual has forgot- In the tutoring studies reported
feedback and corrective procedures, ten the detailed specifics of the sub- at the beginning of this paper, it
the average student approaches the ject matter taught in the schools. was found, that the tutored stu-
level of learning found under tutor- These abilities are regarded as one dents' Higher Mental Process
ing methods of instruction. set of essential characteristics need- (HMP) achievement was 2.0 sigma
ed to continue learning and to cope above the control students. (See
We believe there are a variety of with a rapidly changing world. Figure 4.) (The average tutored stu-
methods of giving feedback to Some curriculum centers believe dent was above 98% of the control
teachers on the extent to which they that these higher mental processes students on the HMP part of the
are providing equality of interaction are important because they make summative examination.) It should
with their students. The tactic of learning exciting and constantly be noted that in these studies higher
providing a "mirror" to the teacher new and playful. mental processes as well as lower
of the ways in which he or she is
providing cues and explanations, 13
appropriate reinforcement, and

June/July 1984

lOO 98% 2 SIGMA 96% 98%
7c 2~
9O 2@ .................................................. 88%
S0 8/,% 2(
7O 90%
6O 1.3~" lo 5o%

50 79% o
ao
73% .8(r TUTORING S]~JDIES
30 BURKE (1983)
50% N 50% ~I 5o%
20 ~
H # r.J
t0 + r~
~3
~o tJ3
>4
~ f~ H
E-~
LEVIN (1979) cO ~Z r.O (1,
~E O < £)
+
TENENBAUM (1982)
8

•• o,
o,
J

MEVARECH (1980)

FIGURE 4. Average higher mental process achievement under different learning conditions. Comparison
of tutoring studies, mastery learning, and higher mental process instructional methods.

mental process questions were in- learning experiences in which the Teaching + ML) was 1.3 sigma
cluded in the formative tests used above the control group (L.M.P.
in the feedback-correctiveprocesses students applied principles in a Teaching) taught primarily by
for both the ML and tutored groups. variety of different problem situa- learning algorithms--a set of rules
Again, the point is that students can tions. On the summative examina- and procedures for solving par-
learn the higher mental processes if tions, the students were very high ticular math problems (the average
they become more central in the on the knowledge of principles and student in this experimental group
teaching-learning process. facts and in their ability to apply the was above 90% of the control stu-
principles in new problem situa- dents).
Several studies have been made in tions. These experimental students
which the researcher was seeking to were compared with a control group In all of these studies, attempts to
improve the higher mental pro- that was only taught the principles improve higher mental processes in-
cesses. (but not their application). On the cluded group instruction emphasiz-
higher mental processes, the ex- ing higher mental pr.ocesses and
We have already referred to the perimental group was 2 sigma feedback-corrective processes,
Tenenbaum (1982) study, which era- above the control students (the which also emphasized higher men-
phasized changing teacher-student average experimental student was tal processes. In addition, the tutor-
interaction. In this study, the Cue- above 98% of the control students) ing studies included an instructional
Participation-Reinforcement + in the ability to apply the principles emphasis on both higher and lower
Mastery Learning student group to new problem situations. mental processes, as well as the
was 1.7 sigma higher than the con- feedback-corrective processes,
trol students on the higher mental A third study by Mevarech (1980) which included both higher and
process part of the summative ex- was directed at improving the lower mental processes. It was evi-
amination. (The average CPR + higher mental processes by em- dent in all of these studies that in
ML student was above 96% of the phasizing heuristic problem solving the formative feedback and correc-
control students on the higher men- and including higher and lower tive processes the students needed
tal processes.) (See Figure 4.) mental process questions in the for- and received more corrective help
mative testing and in the feedback- on the higher mental processes
Another study done by Levin corrective processes. On the higher questions and problems than they
(1979) was directed to improving mental process part of the sum- did on the lower mental process
the higher mental processes by em- mative tests, the group using the questions.
phasizing the mastery of the lower
mental processes and providing heuristic methods + ML (HMP Educational Researcher

14

Summary APPENDIX

The Anania (1982, 1983) and Effect Size References
Burke (1984) studies comparing stu-
dent learning under one-to-one Tutorial Instruction*
tutoring, ML, and conventional Anania, J. (1982). The effects of quality of instruction on the cognitive and
group instruction began in 1980. As
the results of these separate studies affective learning of students (Doctoral dissertation, University of Chicago,
at different grade levels and in dif- 1981). Dissertation Abstracts International, 42, 4269A.
ferent school subjects began to Burke, A.J. (1984). Students' potential for learning contrasted under tutorial
emerge, we were astonished at the and group approaches to instruction (Doctoral dissertation, University of
consistency of the findings as well
as the great differences in student Chicago, 1983). Dissertation Abstracts International, 44, 2025A.
cognitive achievement, attitudes,
and academic self-concept under Reinforcement
tutoring as compared with the Lysakowski, R.S., & Walberg, H.J. (1981).Classroom reinforcement: A quan-
group methods of instruction.
titative synthesis. Journal of Educational Research, 75, 69-77.
During the past 4 years, the grad-
uate students in my seminars at the Feedback-Corrective, Cues & Explanations,~nd Student Classroom
University of Chicago and North- Participation
western University considered vari-
ous approaches to the search for Lysakowski, R.S., & Walberg, H.J. (1982). Instructional effects of cues, par-
group methods of instruction that ticipation, and corrective feedback: A quantitative synthesis. American
might be as effective as one-to-one Educational Research Journal, 19, 559-578.
tutoring. This paper reports on the
research studies these students Student Time on Task (in the classroom)
have completed, the studies that are Frederick, W.C., & Walberg, H.J. (1980). Learning as a function of time.
still in process, and some of the
other ideas we explored in these Journal of Educational Research, 73, 183-194.
seminars.
Improved Reading/Study Skills
Although all of us at first thought Pflaum, S.W., Walberg, H.J., Karegianes, M.L., & Rasher, S. (1980).
it was an impossible task, we did
agree that if we succeeded in find- Reading instruction: A quantitative synthesis. Educational Researcher,
ing one solution, there would soon 9, 12-18.
be a great many solutions. In this
paper, I report on six solutions to Cooperative Learning
the 2 sigma problem. In spite of the Slavin, R.E. (1980). Cooperative learning. Review of Educational Research,
difficulties, our graduate students
found the problem to be very in- 50, 315-342.
triguing because the goal was so
clear and specific--find methods of Home Work (graded) and Home Work (assigned)
group instruction as effective as one- Paschal, R., Weinstein, T., & Walberg, H.J. (in press). Effects of homework:
to-one tutoring.
A quantitative synthesis. Journal of Educational Research.
Early in the work, it became evi-
dent that more than group instruc- Classroom Morale
tion in the school had to be consid-
ered. We also needed to find ways Haertel, G.D., Walberg, H.J., & Haertel, E.H. (1981). Social-psychological
of improving the students' learning environments and learning: A quantitative synthesis. British Educational
processes, the curriculum and in- Research Journal, 7, 27-36.
structional materials, as well as the
home environmental support of the Initial Cognitive Prerequisites*
students' school learning. This Leyton, F.S. (1983). The extent to which group instruction supplemented
paper is only a preliminary report
on what has been accomplished to by mastery of the initial cognitive prerequisites approximates the learn-
date, but it should be evident that ing effectiveness of one-to-one tutorial instruction (Doctoral dissertation,
much can now be done to improve University of Chicago, 1983). Dissertation Abstracts International, 44,
student learning in the schools. 974A.
However, the search is far from
complete. We look for additional Home Environment Intervention (parental educational program)
solutions to the 2 sigma problem to Iverson, B.K., & Walberg, H.J. (1982). Home environment and learning:
be reported in the next few years.
I hope some of the readers of this A quantitative synthesis. Journal of Experimental Education, 50, 144-151.
article will also find this problem
challenging. Peer & Cross-Age Remedial Tutoring
Cohen, P.A., Kulik, J.A., & Kulik, C.C. (1982). Educational outcomes of
June/July 1984
tutoring: A meta-analysis of findings. American Educational Research
Journal, 19, 237-248.

Higher Order Questions
Redfield, D.L., & Rousseau, E.W. (1981). Meta-analysis of experimental

research on teacher questioning behavior. Review of Educational Research,

51, 235-245.

New Science & Math Curricula and Teacher Expectancy
Walberg, H.J. (1984). Improving the productivity of America's schools.

Educational Leadership, 41, 8, 19-27.

Peer Group Influence ~-

Ide, J., Haertel, G.D., Parkerson, J.A., & Walberg, H.J. (1981). Peer-group

influences on learning: A quantitative synthesis. Journal of Educational

Psychology, 73, 472-484.

Advance Organizers
Luiten, J., Ames, W., & Ackerson, G. (1980). A meta-analysis of the effects

of advance organizers on learning and retention. American Educational
Research Journal, 17, 211-218.

*not effect size studies

15

Notes Dave, R.H. (1963). The identification Nordin, A.B. (1980). Improving learn-
and measurement of environment ing: An experiment in rural primary
1In giving the percentile equivalent process variables that are related to schools in Malaysia. Evaluation in
we make use of the normal curve dis- educational achievement. (Unpub- Education: An International Review
tribution. The control class distributions lished doctoral dissertation, Univer- Series, 4, 2, 143-263.
were approximately normal, although sity of Chicago).
the mastery learning and tutoring Pflaum, S.W., Walberg, H.J., Kare-
groups were highly skewed. Dolan, L.J. (1980). The affective cor- gianes, M.L., & Rasher, S. (1980).
relates of home concern and support, Reading instruction: A quantitative
2Mean experimental--Mean control = instructional quality, and achieve- synthesis. Educational Researcher, 9,
standard deviation of the control ment. (Unpublished doctoral disserta- 12-18.
tion, University of Chicago).
Mex--M~ Slavin, R.E. (1980). Cooperative Learn-
sigma of control = effect size. Ide, J., Haertel, G.D., Parkerson, J.A., ing. Review of Educational Research,
& Walberg, H.J. (1981). Peer group 50, 31-5-342.
3When questionnaires rather than in- influences on learning: A quantitative
terviews and observations have been synthesis. Journal of Educational Tenenbaum, G. (1982). A method of
used, the correlations are somewhat Psychology, 73, 472-484. group instruction which is as effective
lower, with the average being between as one-to-one tutorial instruction.
+.45 and +.55. Iverson, B.K., & Walberg, H.J. (1982). (Doctoral dissertation, University of
Home environment and learning: A Chicago, 1982). Dissertation Ab-
References quantitative synthesis. Journal ofEx- stracts International, 43, 1822A.
perimental Education, 50, 144-151.
Anania, J. (1982). The effects of quali- Walberg, H.J. (1984). Improving the
ty of instruction on the cognitive and Janhom, S. (1983). Educating parents productivity of America's schools.
affective learning of students. (Doc- to educate their children. (Unpub- Educational Leadership, 41, 8, 19-27.
toral dissertation, University of Chi- lished doctoral dissertation, Univer-
cago, 1981). Dissertation Abstracts sity of Chicago). Wolf, R.M. (1964). The identification
International, 42, 4269A. and measurement of home environ-
Kalinowski, A., & Sloane, K. (1981). The mental process variables that are
Anania, J. (1983). "The influence of in- home environment and school related to intelligence. (Unpublished
structional conditions on student achievement. Studies in Educational doctoral dissertation, University of
learning and achievement." Evalua- Evaluation, 7, 85-96. Chicago).
tion in Education: An International
Review Series, 7, 1, 1-92. Levin, T. (1979). Instruction which en- Wolf, R.M. (1966). The Measurement of
ables students to develop higher men- Environments. In A. Anastasi (Ed.),
Ausubel, D. (1960). The use of advanced tal processes. Evaluation in Educa- Testing Problems in Perspective.
organizers in the learning and reten- tion: An International Review Series, Washington D.C.: American Council
tion of meaningful verbal material. 3, 3, 173-220. on Education.
Journal of Educational Psychology,
51,267-272. Leyton, F.S. (1983). The extent to CARING
which group instruction supplement-
Bloom, B.S. (1980). The new direction ed by mastery of the initial cognitive A Feminine Approach to
in educational research: alterable vari- prerequisites approximates the learn-
ables. Phi Delta Kappan, 61, 6, ing effectiveness of one-to-one tu- Ethics and Moral
382-385. torial methods. (Doctoral dissertation,
University of Chicago, 1983). Disser- Education
Bloom, B.S., Engelhart, M.D., Furst, tation Abstracts International, 44,
E.J., Hill, W.H., & Krathwohl, D.R. 974A. by Nel Noddings
(1956). Taxonomy of Educational Ob- "Noddings is chiefly concerned
jectives: Handbook I, Cognitive Do- Luiten, J., Ames, W., & Ackerson, G. with reintroducing moral educa-
main. New York: Longman. (1980). A meta-analysis of the effects tion into our schools.To this end,
of advance organizers on learning and she would both alter structures
Bronfenbrenner, U. (1974). Is Early In- retention. American Educational and encourage dialogue, prac-
tervention Effective? In H.J. Leich- Research Journal, 17, 211-218. tice, and confirmation in service
ter, (Ed.), The Family as Educator. of the ethical ideal .... A timely,
New York: Teachers College Press. Marjoribanks, K. (1974). Environments accessible elaboration of a
for Learning. London: National Foun- female ethic, with ramifications
Brophy, J.E., & Good, T.L. (1970). dation for Educational Research. both personal and political."
Teachers' communication of differen-
tial expectations for children's class- Mevarech, Z.R. (1980). The role of --Kirkus Reviews
room performance: Some behavioral teaching-learning strategies and feed- "An excellent example of the
data. Journal of Educational back-corrective procedures in devel- new humanistic-feminist scholar-
Psychology, 61,365-374. oping higher cognitive achievement. ship. Of interest to all who are
(Unpublished doctoral dissertation, concerned with humane
Burke, A.J. (1984). Students' potential University of Chicago). behavior."---Library Journal
for learning contrasted under tutorial $15.95 at bookstores
and group approaches to instruction. Nordin, A.B. (1979). The effects of dif-
(Doctoral dissertation, University of ferent qualities of instruction on University of
Chicago, 1983). Dissertation Ab- selected cognitive, affective, and time California
stracts International, 44, 2025A. variables. (Unpublished doctoral dis- Press
sertation, University of Chicago).
16 Berkeley 94720

Educational Researcher