ERIC ED099820: Project Upswing After Two Years: An Evaluation

DOCUMENT RESUME 95 ED 099 820 AUTHOR TITLE INSTITUTION SPONS AGENCY REPORT NO PUB DATE CONTRACT NOTE CS 001 541 Paramore, B.; And Others Project U...

0 Downloads 9 Views
DOCUMENT RESUME 95

ED 099 820 AUTHOR TITLE INSTITUTION SPONS AGENCY

REPORT NO PUB DATE CONTRACT NOTE

CS 001 541

Paramore, B.; And Others Project Upswing After Two Years: An Evaluation. Operations Research, Inc., Silver Spring, Md. bureau of Educational Personnel Development (DHEW/OF). Washington, D. C. TR-811 Dec 73 OEC-0 -71 -4668 (607)

243p.; See related documents CS 001 539, 540, and 543

EDRS PRICE DFSCRIPIORS

IDENTIFIERS

MF-$0.75 HC-$11.40 PLUS POSTAGE Educational Research; Grade 1; Grade Repetition; *Learning Difficulties; Primary Education; *Program Evaluation; Reading Improvement; Reading Instruction; *Reading Programs; *Remedial Reading; Self Concept; *Tutorial Programs; Tutoring; Volunteers *Project Upswing

ABSTRACT

Project Upswing was a two-year tutoring experiment directed by universities in Denver, Oxford (Mississippi), St. Louis, and San Francisco and involving first graders identified by their teachers at the beginning of the school year as capable of normal achievement but having learning difficulties. Half of the students served as a control group and half were tutored twice a week, one hour each session, by adult volunteers. The project was evaluated for impact of tutoring on children's development of reading skills, visual-motor integration skills, and self-esteem and for effectiveness of tutor training. Analysis indicated that tutoring was effective in helping students improve their rates of progress in reading and their self-esteem but that tutoring had no significant effect on visual-motor integration skill. As a byproduct of the evaluation, the data suggested that grade retention had negative effects. The mean reading score at the end of the year for both retained and promoted children was in the averaje range. But at the end of the second year, the reading score for those children who repeated first grade dropped significantly into the low-average range; the promoted children maintained their previous level. (TO)

U S DEPARTMENT OF !SEALT14. EDUCATION &WELFARE NATIONAL INSTITUTE OF EDUCATION

1 .1

(,NA NT HAS HE EN RFPRO

FROM THi Pi RsON OR OROANIZA T 'ON OR,GIN

t 0 tItAtILv AS RECEIVED

N.,

T PIN TS 0/ vi.F0d OR OPINIONS

A I () DO NOT NE( F`AARIL. Y 14IPRF AL NATIONAL NSTI WY F Ot N'

10'I( ANON POSITION OR POL ICY

OPERATIONS RESEARCH, Inc. SILVER SPRING, MARYLAND I

I

Project Upswing after Two Years: an Evaluation B. Paramore, P. Plantec, and J. Hospodar

December 1973

Prepared under Contract No. OEC-0-71-4668 (607) for the U.S. Office of Education Department of Health, Education, and Welfare Washington, D.C. 20202

2

BEST COPY AVAILABLE

SUMMARY

Project Upswing was a two-year tutoring experiment conducted under the auspices of the Department of Health, Education, and Welfare, U.S. Office of Education, Bureau of Educational Personnel Development. projects directed by Taliversities in four cities:

St. Louis; and San Francisco.

There were Upswing

Denver; Oxford, Mississippi;

The children involved were first graders

identified by their teachers at the beginning of the school year as capable of normal achievement but having learning difficulties.

Half of the children

were tutored and half were taken as a control group. The tutors were volunteers, primarily adult women.

College students

also served as tutors, especially in the small community of Oxford, Mississippi, where the nonstudent population is small.

In the first year of the project,

half of the tutors received training, most of it before tutoring began; half had only a brief orientation.

In the second year, all tutors received train-

ing, much of it during the tutoring period. The children were tutored twice a week, one hour each session.

Tutors

worked out their own instructional approaches with suggestions from project staff and teachers or with more extensive help as requested.

Programmed

materials were made available by the project but were not heavily used. The project was evaluated for impact of tutoring on children's development of reading skills, visual-motor integration skills, and self-esteem.

The

evaluation was also concerned with whether traininc' increased tutor effective-

ness and with the preferability of different training approaches.

3

BEST COPY P""..lat

The children's skills iu the criterion areas were measured before and

after tutoring with a battery of standardized instruments (an experimental self-esteem measure was also administered).

Basic experiences also were

measured, by the Test of Basic Experiences, as a proxy for family background.

The pre- and post-tutoring test results for tutored and control group children were compared using multiple regression and analysis of covariance.

The latter

was used to control for initial skill level. A differel,t group of children was involved each year.

Those tutored

in the first year were tested at the end of second grade to determine effects of tutoring over time.

The analysis indicated that tutoring was effective in helping children improve their rates of progress in reading, and their self-esteem.

There was

no significant effect on visual-motor integration skill. A project impact beyond the influence of tutoring was observed.

In

the first year, tutored children made significantly greater progress in reading (as measured by the Wide Range Achievement Test).

In the second year,

both tutored and untutored made significant gains in test-observed reading and self-esteem.

Enlargement of basic experiences also occurred in both groups.

The progress of the control group in the second year appeared to be attributable to the influence of the project on teachers.

The data suggested that it does not matter so much how one goes about involving or training teachers or tutors.

It does seem to matter that the

sense of project entity be well-defined, that there be clear leadership; that involvement be reinforced periodically; and that the participants know that someone (preferably someone whom they see as authoritative) is paying attention to their efforts and the results of their efforts.

This also applies to the

children.

Trained tutors were no more effective with their pupils than untrained. However, tutors felt a need for training and teachers considered trained tutors more effective. training.

School principals also generally believed tutors should receive

The inservice approach was preferred.

Despite the value placed on training, many tutors did not attend regularly.

It appeared that training might be valued more in theory than in practice

ii

4

BEST COPY AVAILABLE

because training traditionally has been accepted as a good thing.

It also

appeared that training is valued more highly when little or none is available. The follow-up data were somewhat ambiguous because of the nature of attrition in the follow-up groups.

However, the children generally maintained

their reading standard scores established at the end of the year of Upswing. Thus it appears that the effects of Upswing tutoring probably are durable. Finally, there were two major findings that came as a byproduct of the Upswing evaluation k,5i rt.

It was difficult to determine the reasons for

retention of a substant:ial proportion of children involved in the first year of Upswing; and the data s'tggested retention had negative effects.

The mean

reading score from the test given at the end of Upswing was in the average range for retained children as for those who were not retained. kept in first grade lost ground during the follow-up year.

The group

Their mean read-

ing standard score at the end of the follow-up year dropped significantly into the low-average range; the other children maintained their previous level. The Upswing data also showed that remedial reading generally was not effective in bringing up the reading scores of follow-up children.

When the project was

present, the children who had remedial reading did show a substantial mean increase in reading score; children who had tutors instead of remedial reading showed almost as great a mean increase.

These results pose some questions

about the comparative value of remedial reading, considering the high cost. In the final analysis, Project Upswing is found to be a cost-effective means of serving the large number of children with early learning problems. The project seems to be effective in both urban and rural settings.

It could

be a means of reaching the vast numbers of children with learning problems throughout the nation who now receive no help.

iii

TABLE OF CONTENTS

Page

SUMMARY

I.

II.

LIST OF FIGURES

ix

LIST OF TABLES

xi

STATEMENT OF THE RESEARCH PROBLEM

1

PREVIOUS RESEARCH FINDINGS

2

OVERVIEW OF PROJECT UPSWING AND ITS SIGNIFICANCE

3

DESCRIPTION OF PROJECT UPSWING

7

INSTITUTIONS INVOLVED IN UPSWING

8

DESCRIPTION OF BASIC PROJECT ACTIVITIES

8

CHARACTERISTICS OF TUTORS AND TEACHERS INVOLVED IN THE PROJECT III.

THE EVALUATION METHODOLOGY

14 1.7

PRELIMINARY METHODOLOGY

17

PRELIMINARY EVALUATION RESULTS

21

FINAL EVALUATION METHODOLOGY

.

V

6

25

IV.

ANALYSIS OF THE EFFECTS OF TUTORING ON CHILDREN

39

STIMMARY OF FINDINGS FROM THE FIRST YEAR OF UPSWING

39

DESCRIPTION OF CHILDREN INVOLVED IN THE UPSWING FOLLOW-UP

42

FOLLOW-UP OBJECTIVES

42

FOLLOW -UP PROCEDURES AND DATA SOURCES

46

STATISTICAL HYPOTHESES

46

FOLLOW-UP RESULTS

47

LIMITATION OF THE FOLLOW-UP ANALYSIS

77

CONCLUSIONS ABOUT THE EFFECTS OF TUTORING OVER TIME

78

DESCRIPTION OF CHILDREN INVOLVED IN PROJECT UPSWING'S SECOND YEAR

81

OJBECTIVES OF THE SECOND -YEAR ANALYSIS OF TUTORING EFFECTS

86

DATA COLLECTION PROCEDURES AND SOURCES IN THE SECOND YEAR

86

STATISTICAL HYPOTHESES USED IN THE SECOND YEAR ANALYSIS

87

RESULTS OF THE ANALYSIS OF TUTORING EFFECTS ON CHILDREN IN THE SECOND YEAR OF UPSWING

88

CONCLUSIONS ABOUT THE EFFECTS OF UPSWING ON CHILDREN IN THE SECOa YEAR V.

UPSWING TRAINING AND PROJECT MANAGEMENT

108 113

WHAT WAS LEARNED IN THE FIRST YEAR OF UPSWING

113

NEW EMPHASES IN THE SECOND YEAR

116

ORIENTATION AND TRAINING FOR TUTORS

117

CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUTOR TRAINING

138

VI.

VII.

TRAINING/ORIENTATION FOR TEACHERS

139

CONCLUSIONS ABOUT THE EFFECTIVENESS OF TEACHER ORIENTATION/TRAINING

158

DESCRIPTION OF PROJECT MANAGEMENT AND COMMUNICATIONS STRUCTURE IN THE SECOND YEAR OF UPSWING

161

PARTICIPANTS' REACTIONS TO PROJECT COMMUNICATIONS IN THE SECOND YEAR

165

CONCLUSIONS ABOUT THE EFFECTIVENESS OF MANAGEMENT AND COMMUNICATIONS IN UPSWING

174

THE PROBLEM OF ATTRITION

175

WHAT TO EXPECT

175

PLANNING AHEAD

176

COST ANALYSIS

179

METHODOLOGY AND DATA SOURCE FOR THE COST ANALYSIS

180

THE COST OF A TUTOR IN EACH PROJECT

180

COST TO LOSE ONE TUTOR DURING THE YEAR

182

COST TO TRAIN ONE TEACHER FOR PROJECT

VIII.

UPSWING'

182

COST TO UPGRADE CHILD PERFORMANCE ON WRAT

186

POTENTIAL FOR COST REDUCTION

186

CONCLUSIONS AND RECOMMENDATIONS FROM THE UPSWING EVALUATION

189

CONCLUSIONS

189

UPSWING'S FUTURE

195

RECOMMENDATIONS FOR FURTHER RESEARCH

196

EDITORIAL CLOSING COMMENT

199

APPENDIX A:

BIBLIOGRAPHY

A-1

APPENDIX B:

DATA COLLECTION FORMS USED IN PROJECT UPSWING

B-1

vii

LIST OF FIGURES

Page 1.

2.

3.

4.

5.

6.

7.

8.

9.

Rate of Progress in Reading as Measured by WRAT: Tutored Versus Control Children

41

Reading Performance After A Year Without Tutoring, Former Tutored Children As Measured by the WRAT: Versus Former Control Children

49

Male-Female Comparison of Change in WRAT Score Over a Year Without Tutoring

54

Comparison of Self-Esteem Test Results for the Follow-Up Groups

55

Class Standing of Former Upswing Children, as Evaluated by Their Second-Year Teachers

.

63

Classroom Behavior of Former Upswing Children, as Evaluated by Their Second-Year Teachers

66

Former Upswing Children's Overall Self-Confidence in School, as Evaluated by Their Second-Year Teachers

69

Comparison of Pre- and Post-Tutoring Reading Score Distributions of Children Who Did and Did Not Receive Special Educations]. Services in the Follow-Up Year

75

Comparison of Reading Test Results in the Second Year of Upswing for Tutored and Control Group Children

91

ix

10.

11.

Comparison of Self-Esteem Test Results for Tutored and Control Group Children Comparison of Change in Children's Visual-Motor Integration Skills According to How Much Tutors Said They Worked on Such Skills with the Children .

12.

91

.

.

.

104

Comparison of Basic Experiences Test Results for Tutored and Control Group Children

106

Teacher Opinion About the Importance of Teacher Meetings in a Project Like Upswing

151

Teacher Opinion About Whether Teachers Generally Would Be Willing To Attend Meetings for a Project Like Upswing if they Were Not Paid

153

15.

General Management Structure of Project Upswing

162

16.

Generalized Model of Upswing Management and Communications in the Schools

164

Tutor Satisfaction with Various Channels of Communication within Upswing

166

13.

14.

17.

x

10

LIST OF TABLES

Page

Variables, Measures, and Sources of Data for the Analysis of Tutoring Effects on Children

28

Variables, Measures, and Sources of Data for the Analysis of Tutoring Effects Over Time

29

Variables, Measures, and Sources of Data for the Analysis of Project Management Strategies and Training

31

4.

Children Involved in the Upswing Follow -Up

43

5.

Sex of Children Involved in Upswing Follow-Up

44

6.

Ages of Children Involved in Upswing Follow-Up, at End of Follow -Up Year

45

Pretest WRAT Score and Change in Score Over the Follow-Up Year by City and Status

51

Type of Class Assignment for the School Year Following Upswing Tutoring, by Status in Upswing

57

Type of Class Assignment for the School Year Following Upswing Tutoring, By City

59

Special Services to Former Upswing Children During the Follow-Up Year, by Status in Upswing

60

Special Services to Former Upswing Children During the Follow-Up Year, by City

62

1.

2.

3.

7.

8.

9.

10.

11.

12.

City Comparison of Former Upswing Children's Class Standing, as Evaluated by Their Second-Year Teachers

65

.

City Comparison of Former Upswing Children's Classroom Behavior, as Evaluated by Their Second-Year Teachers

67

Basic Statistics on Follow-Up Children's Rates of Absenteeism From School

70

City Comparison of Follow-Up Children's Rate of Absenteeism Fi.om School

72

Reading Test Results for Upswing Follow Up Children Grouped by Type of.Class Assignment

74

Special Services to Children in Different Types of Classes

76

18.

Ages of Children in Project Upswing's Second Year

83

19.

Upswing Children's Kindergarten Experience

84

20.

Comparison of the Pretutoring Reading Levels of Tutored and Control Group Children

89

Reading Test Results for Tutored and Control Group nildren in Each City

93

Comparison of Pretutoring Self-Esteem Levels of Tutored and Control Group Children

98

Summary cf Tutor Opinion About Initial Training Related to Project Goals, Role Definition and Instruction of Children

120

Summary of Tutor Opinion about How Well Organizational Matters Were Covered in Preservice Training

121

Tutor Opinion about the General Value of Upswing Group Training Meetings

125

Aspects of Group Training Considered Most Useful by Tutors

127

13.

14.

15.

16.

17.

21.

22.

23.

24.

25.

26.

27.

Tutor Opinion about the General Value .)f Tn44.vidual Help Provided by Upswing Staff

xii

12

..

129

Aspects of Individual Help From Upswing Staff Considered Most Useful by Tutors

131

Tutors' Opinion as to What Approach Was Most Useful to Them in Tutoring

133

Training Materials Found to Have Most Value by Upswing Tutors

134

31.

Tutor Attendance at Preservice Training Meetings

136

32.

Tutor Attendance at Inservice Training Meetings

137

33.

Project Schedule for Teacher Orientation/Training

141

34.

Summary of Teacher oplaim About Initial Training/ Orientation Related to Project Goals, Research Aspect, and Role Definition

143

Summary cf Teacher Opinion About the Adequacy of Organizational Information They Received in PreService Training/Orientation

145

Teacher Assessment of the Upswing Training Program Content

148

Teacher Assessment of the Frequency, Schedule, and Environment of Upswing Teacher Meetings

150

Teacher Opinion About the Professional Benefits of Participating in Upswing

154

Attendance at Upswing Preservice Meetings for Teachers

156

Attendance at Upswing's Inservice Meetings for Teachers

157

41.

Tutors' Overall Satisfaction with Upswing

168

42.

Teacher Satisfaction with'Amount of Teacher/ Upswing Staff Contact

169

Teacher Opinion About Individual Support Given to Upswing Tutors by Project Staff

171

Teachers' Overall Satisfaction with Upswing

172

28.

29.

30.

35.

36.

37.

38.

39.

40.

43.

44.

13

45

Cost of One Tutor Who Was in Project Upswing for the Entire Year

181

Cost of One Tutor Who Dropped Out of Project Upswing During the Year

183

47.

The Average Monthly Cost of Tutors

184

48.

Teacher Costs in Project Upswing

185

49.

The Relative Cost of Point Gains on the WRAT by City for Pooled Tutored and Control Groups

187

46.

xiv

14

I.

STATEMENT OF Ti.E RESEARCH PROBLEM

A teacher working alone with a class of typical size (usually 25-30 students) does well to give 10 minutes a week of individual help in a subject area to each child.

Very nearly everyone in education

recognizes the value of individualized instruction, particularly for children who have learning problems; but tight school budgets rarely make this help possible.

Individual instructional needs of certain kinds can be satisfied by programmed materials, which provide feedback important for learning. However, there is a strong argument that needs for personal interaction are as important, if not more important than instructional needs.

Most

all teachers can point out children in their clafses whose need for positive reinforcement from an adult is so great that it impedes the children's academic progress.

Use of volunteers in the schools is a way of meeting special needs of individual children in a personal way without taxing school budgets. to do?

The question is, what can volunteers reasonably be expected

Traditionally, both volunteers and paid aides have not been

heavily involved in instruction.

Most commonly, they relieve teachers

of clerical and housekeeping chores, and supervif,..: children in follow-up

work or play, so that the teachers have more time for instruction.

Tutor-

ing by volunteers who do not necew3arily have teaching credentials is a

1

15

relatively recent trend.

If volunteers are effective in this role, a

more children could receive individualized instruction.

great

PREVIOUS RESEARCH FINDINGS

A review of the literature indicates that most tutoring projects that have been evaluated involved high school and upper elementary students tutoring younger students.

Often the student tutors themselves have had

problems in the subject of tutoring; and the experience is intended to give them a boost as well as their pupils.

Reading first, and then mathematics, seem to be the favorite subject areas for tutoring.

Most reports describe projects with quite narrowly

focused objectives--for example, children will be able to produce the sounds associated with the letters of the alphabet and name the letters, or children will increase their sight vocabularies by x number of words by the end of the tutoring period.

The tutoring periods most commonly were brief - 4 to

10 weeks.

A number of studies have considered the effects of tutoring on the self concepts of tutors, tutees, or both.

More positive self-concept

experience. has been regarded as a potential by-product of the tutoring No study was found of a project that defined and applied direct methods

of helping children improve their self-concepts. Although ORI's review of previous research was not exhaustive, it indicated that tutoring projects of all kinds generally have been found

successful in improving their target academic skills, sometimes successful in improving motivation to achieve, and rarely successful in improving self-concept.

The literature is not without contradictions, but these

trends are clear.

Generally the reported projects involved some kind of pre- and post-tutoring test (or tests) and a control group of children.

The projects

tended to be small, rarely involving more than 50 tutored children.

No

previous project was found to have the diverse settings and geographic dispersal of Upswing, which was conducted in four cities over the United States.

2

BEST COPY AVAILABLE

While it is easy to fault other's research methodology, it should be noted that in many cases evaluation reports leave important questions unanswered and indicate less than rigorous procedures.

The Upswing experience

makes it easy to sympathize with problems in ensuring "clean" data.

However,

some caution should be used in accepting the reported success in many cases, especially when very small numbers of children were involved. It is worth noting that several studies in which tutors were required to use a programmed instructional procedure suggest that this may be a more fruitful approach for tightly-defined, strictly academic objectives than leaving it up to the tutor to plan instruction.) with a strict programmed approach, however.

There are some problems

One of particular importance

is that it requires selection of children on a rather narrow set of characteristics.

A second major problem related to the first, is that it is

easy to overlook individual differences; adherence to the program may take precedence over the child, especially when the tutor is relatively unsophisticated.

Third, for a long-term project, strict adherence to a program can

bore both child and tutor.

A programmed approach may be essential when tutors

are young, and may be both more comfortable and more effective for some older tutors ir certain kinds of projects.

Upswing tutors had access to programed

materials and still preferred planning their own approaches, although they wanted guidance.

OVERVIEW OF PPOJECT UPSWING AND ITS SIGNIFICANCE Project Upswing was a two-year pilot program studying the effects of tutoring by adult volunteers on first-grade children who demonstrated nonspecifid learning difficulties that were expected to influence their reading achievement. could be established.

The idea was to intervene before a failure pattern Preserving or boosting children's self-esteem was an

important consideration.

Approximately 800 children received Upswing tutoring.

The children were quite heterogeneous in causes and manifestations of learning difficulties.

1 Harris, 1967; Ellison et al., 1969; American Institutes for Research in the Behavioral Sciences, 1971. See Appendix A for complete references.

3

17

BEST COPY AVAILABLE

Essentially, the evaluation of Upswing involved comparison of tutored children with a control group for changes in reading skill, visual-motor integration, and self-esteem.

The project design called for groups drawn

randomly from a pool of candidates for tutoring selected by teachers.

The

groups were checked for comparable initial reading and visual-motor skills, and comparable IQ, through analysis of a preliminary test battery.

A follow-up

analysis was done for the children tutored in the first year of Upswing to see if gains would hold over a year without tutoring.

A special feature of the Upswing evaluation was its focus on training issues.

It addressed the questions of whether training is necessary, if so,

why, and what kinds of training are preferable.

The evaluation also looked,

with particular care, into the relationship between reading difficulty and self-esteem, and the effect of the one-to-one relationship on self-esteem. Finally, the evaluation included analysis of operational data from the twoyear project development effort to determine whether different management approaches seem to influence tutors' effectiveness and satisfaction. The rationale for Project Upswing was as follows: 1.

If an inexpensive appr,ach can be developed to idencify children with nonspecific learning problems in the first grade, and if an inexpensive general treatment

can be applied effectively to that group, then large numbers of children can be helped through the crucial first year of school.

This could substantially reduce

the number of failure-oriented children who require expensive remedial treatment, which, when given, often is to no avail.

Such a program could be applied in

remote communities where professional help is scarce or nonexistent. 2.

It is clear that professional diagnosis and treatment cannot be employed in any ultra-low-cost programs.

As an alternative, Upswing used teacher screening as a rough form of diagnosis.

Teachers generally

have definite opinions as to which children in their

4

18

classes need special help

With minimll training in

the use of diagnostic chicklists, teachers can identify a hetergeneous populati)n of children with learning problems.

Although tha specific causes of the problems

may not be known, the general problems may be diagnosed. The best treatment rerhaps would be based on a full understanding of a child's problem, but this is prohibitively expens've in most cases. 3.

As an alternative to specific professional treatment, Upswing used a very general affective approach applied by volunteers.

The treatment was different for each

child, because each volunteer introduced his or her personality as a key feature of treatment.

The plan

was heavily dependent on the volunteer's ability to form a relaticnship with the child and perhaps boost self-esteem. 4.

The actual cost of such a program if put into practice on a moderate scale with a salaried management could be about $80 per child per year.

The issue of whether

the cost of such a program can be justified revolves around the following: a.

Can such a program help a significant number of children achieve at least at a passing level?

b.

Can such a program help children increase their reading ability?

c.

Can such a program help children raise low selfesteem?

d.

Can such a program reduce needs for remediation?

The task of evaluating Project Upswing was to determine if the effectiveness of the program in any or all of these areas was sufficient to justify its cost.

5

II.

DESCRIPTION OF PROJECT UPSWING

Upswing children were tutored individually twice a week--in school, but outside the classroom.

Tutoring sessions lasted 45 minutes to an hour.

The tutor was supposed to concentrate on establishing a warm, relaxed relationship with the child to help him Develop and maintain a positive attitude toward academic tasks

Acquire the reading skills expected at his age Develop his self - esteem as a person and, in

particular, as a learner.

Tutors received training for their work from a university school of education staff.

A broad-brush approach to training was used.

It covered

a variety of topics from child development, to description of specific types of learning problems, to techniques of teaching reading and writing, to behavior management and positive reinforcement.

The importance of fostering

ego strength was a constant theme.

In the first year of Upswing, half of the tutors received only a

brief orientation; this group provided a control for checking the need for training.

In the second year, all tutors were offered training of some

kind.

Teachers received a very brief orientation to project objectives and procedures in the first year, which they did not find adequate. second year they received a more extensive orientation. 7

In addition,

In the

BEST COPY AVAILABLE 141

follow-up was conducted throughout the tutoring period, generally in the form of group sessions, to ensure that teachers' questions and problems concerning the project were resolved and to give project staff an opportunity to receive teachers' suggestions.

In some cases, teachers received

instruction in techniques of diagnosing and helping to correct learning difficulties and in the philosophy and practice of positive response to children's behavior.

A new group of children was involved each year.

The effects of

tutoring were evaluated at the end of each year by comparative analysis of pre- and post-tutoring test results for tutored children and a control group.

Attitudes and opinions of tutors, teachers, principals and university

staff were weighed against the test results.

The original group of children

was followed through testing and through evaluation by their second-year teachers, to see if the effects of one fear of tutoring would endure. INSTITUTIONS INVOLVED IN UPSWING Upswing was conceived by Volunteers in Education, a unit of the U.S. Office of Education (USOE), Department of Health, Education and Welfare. USOE gave grants to university departments of education in five cities to plan and conduct the project.

They were:

University of Denver, University

of Missouri, St. Louis; California State University, San Francisco; University of Mississippi at Oxford; and University of Cincinnati.

The last

university was unable to meet the project requirements and was dropped from the study.

Although university staff had major responsibility for the project,

in all cases it was a cooperative effort of that group, the local school system, and, where one existed, the local school volunteer organization.

An

independent evaluation of Upswing was performed by Operations Research, Inc., (ORI), Silver Spring, Maryland, under a contract with USOE's Bureau of Education for the Handicapped, Division of Research. DESCRIPTION OF BASIC PROJECT ACTIVITIES Volunteer Recruitment

Volunteers were recruited by the university project staffs, in some cases with the help of the local school volunteer organizations.

Newspapers,

television and radio announcements were used, project directors spoke before 8

21

community groups, and in some cases university students were sought through classroom announcements and bulletins.

Recruitment campaigns took place in

the summer before each school year, although efforts to get enough volunteers often continued into the fall when, for example, elementary school principals were asked to refer potential volunteers to Project Upswing. Selection of Elementary Schools

The project directors worked with school system administrators and principals to select schools for the project.

The choices were made to

obtain a geographic and socioeconomic mix of schools in each city, within the limits of principals' willingness to participate.

Oxford, Mississippi,

has only two schools--one for the city and one for the county, the latter Both schools participated.

drawing children primarily from rural homes.

Nineteen Denver schools were involved in the first year of Upswing, eight St. Louis schools, and 10 in San Francisco.

In the second year, five to

six schools were involved in all cities except Oxford. reduced to make the project management easier.

The number was

Throughout the project,

more schools wanted Upswing than could be included. Selection of Teachers

Principals asked first-grade teachers to participate in Project Upswing.

Teachers could refuse, but few did.

Selection of Children

The first-grade teachers in participating schools were asked to !dentify children for the project.

They did so based on observation of

their students during the first month of school.

In Upswing's first year 9

child identification was left entirely to the teacher, except that they were asked to make decisions based on the following broad criteria:

The children should appear to have potential for normal functioning; i.e., they should not be severely handicapped intellectually or physically.

The children should show signs of being unable to function normally in the classroom setting because of some difficulty, such as a perceptual problem,

9

22

delayed establishment of left-right dominance, relative lack of basic experiences, etc.

In the second year of Upswing, teachers were instructed in techniques of observing children in an orientation/training meeting held shortly after school opened.

Teachers also were given a readiness inventory to use as a

check on their selections (the Cegelka Academic Readiness Evaluation). It should be noted that teachers did not fill out the readiness inventory for all of their pupils.

They first screened subjectively and

then completed the inventory for those children whom they considered to have the greatest number of problems or (within the limits of the Upswing criteria) the most serious problems.

The Upswing project directors divided the children referred by teachers into two groups, one that would receive tutoring and one that would not. This was supposed to be done on a random basis, within the following two constraints.

No teacher could have more than six students involved in Upswing,

to ensure that the project would not be a burden.

Further, insofar as pos-

sible, each teacher had an equal number of tutored and control group children, to hold constant the effects of classroom experiences. data indicates grouping was not always randomly done.

The analysis of test It appears that sometimes

project staff responded to pleas from teachers to give tutors to the children who had the worst problems, or that project staff did so on their own after examining the readiness inventories.

All children identified for the project were tested at the beginning of the school year.

The test battery included:1

Wide Range Achievement Test (WRAT), Level I, Reading Subtest--to measure change in reading skills component of school performance

1

In the first year of the project testing was done rather late.and went on after tutoring began. Most was completed in November and December 1971; some, however, was not done until January 1972. In the second year, all testing was done in October 1972.

10

23

BEST COPY AMIABLE

Beery-Buktenica Developmental Test of Visual-

Motor Integration--to measure change in psychomotor behavior defined as level of visualmotor integration skill Test of Basic Experiences (TOBE), Level L, General Concepts Subtest--to measure change in grasp of fundamental concepts relevant to school performance--such concepts as size, location, congruity, distance, etc.

The Funny Faces Game, an inventory of selfperceptions in relationships and situations encountered by children starting school.

An IQ test, a behavior rating scale, and the Metropolitan Achievement Test, reading subsection, were used in the first year, but dropped in favor of the TnBE, and the self-esteem inventory. The pretest battery was readministered at the end of each school year.

Post-testing was done at the end of April or in May.

Volunteer Training

Volunteers were introduced to the project in meetings held each year before tutoring began.

The purposes of these meetings were:

To acquaint participants with the goals of the project and its organization To give them some information about child development and the characteristics of children with learning difficulties

To introduce them to procedures they would be expected to follow in the schools To describe the kinds of materials available for use in tutoring.

11

24

BEST Cn°11 """ ABLE

Meetings were held during the tutoring period to provide further training in specific instructional techniques and to work on problems that arose in the tutoring situation.

In general, the first year's training was more formal and more heavily preservice.

Considerably greater emphasis was placed on continuing

support to volunteers during tutoring in the second year.

For the second

year, the project director and staff in each city developed the training program they felt would best meet the needs of their volunteers. As noted previously, half of the volunteers in the first year received orientation only.

Generally the project directors followed

volunteer preference in making group assignments, within the constraint that the groups had to be approximately equal in number.

This was not a

9,00d approach methodologically, since it admits the possibility of group

differences in motivation, self-confidence, or both.

However, there was

evidence that arbitrary assignments would cause people to quit, or at least promote some dissatisfaction, causing a different type of bias.

In terms

of sex, level of education, income, and relevant training and experience, the groups were reasonably well matched.

Teacher Orientation/Training One or more meetings for teachers was held before tutoring began, to acquaint them with the goals and organization of Upswing and to discuss selection of children as described above.

Teachers had only a brief orien-

tation in the first year of Upswing; in the second year teacher workshops were held in two cities throughout the tutoring period.

In some cases,

Upswing provided teachers with information on instructional techniques in addition to soliciting their ideas for tutors.

As was true for volunteers,

a great deal more emphasis was placed on continuing interaction with teachers in the second year of Upswing.

12

BEST COPY AVAILABIL

Tutoring Time and Place.

to an hour per session.

Tutors and children met twice a week for 45 minutes Tutoring was done at school, durirg school hours,

but outside the classroom in auditoriums, hallways, multipurpose rooms, cafeterias, or wherever space could be found. Lesson Plans and Content.

Tutors prepared their own plans.

were asked to set aside one hour per week for this purpose.

They

In some cases

they were required to use lesson plan forms and turn them in to Upswing staff for review.

That requirement generated some dissatisfaction among

the tutors, but some projects found it helpful in "quality control" and in providing more relevant assistance to tutors. Although tutoring was supposed to focus on reading, self-esteem, and visual-motor coordination, volunteers found themselves responding to a variety of needs of their individual pupils.

The feeling was that it is

difficult and inappropriate to isolate skills when working with children so young.

Tutoring Mater

Some materials were provided by the project- -

e.g., a commercial language development kit, a proeammed reading system, books, games and puzzles, etc.

Some schools provided paper and art supplies.

Volunteers at times arranged with teachers to use classroom materials. also bought, borroued, and made materials on their own.

They

They favored using

a variety of simple materials, and many put aside the project-supplied language and reading kits because they felt their pupils wanted a change or that those materials were complicated to use. Special Activities.

Some tutors got permission to take children

out of school on special trips, took them out on weekends, visited in their homes, or invited the children to visit them at home. on the volunteers' initiative.

Such extras were done

The project did not suggest or assist in

activities outside of school, but in a good number of cases tutor and child developed a special kind of friendship. Teacher-Tutor Interaction.

The degree of interaction between teachers

and tutors was according to individual predispositions.

13

Tutors called for

BEST COPY AVAILABLE

their pupils at the classroom and often talked briefly with teachers then.

Conferences had to be held at lunchtime, after school, or some other time when teachers were free.

Talephone conferences were fairly common.

In

general, there was more tutor-teacher interaction during the second year.

The increased interaction was fostered by project staff, after it was found in the first year that both tutors and teachers felt a need for more communication with each other.

Cormnication Between Upswi'mg Staff and Participants.

Upswing

staff often served as liaisons between teachers and tutors, in addition to talking regularly with each individually and maintaining communication with principals.

This was mtne true in the second year of the project,

when a staff member visited each school at least once a week, offering

tutors advice, assistance, am reassurance as needed, and checking with teachers for problems or observations.

CHARACTERISTICS OF TUTORS AND TEACHERS INVOLVED IN THE PROJECT The following data were taken from the "First Impressions" question nair's completed by tutors and teachers in the second year of Upswing.

These

groups were very similar from year to year; in fact, about 40% of the tutors and 72% of the teachers participated both years.

The characteristics of

children involved are described in Section IV, as a preface to analysis of tutoring impacts. Tutors

Almost all of the Upswing tutors were women--either college students or homemakers not otherwise employed.

The Oxford project supplied most of

the college students (about 70% of Oxford tutors were in that occupational category); the St. Louis project also involved a significant number of students

(about 40% of that city's tutors. The age distribution was skewed to the young side by the college student concentrations.

Still, considerable variation in age existed.

range was under 21 to over 60 years, with most between 21 and 50.

The

In short,

people of all ages were attracted to Upswing. The tutors tended to be well-educated. but had not completed undergraduate work.

About half attended college

About 30% had completed bachelor's

14

27

BEST COPY AVAILABLE

or advanced degrees.

About 45% of

Only two had not finished high school.

the tutors had previous direct relevant training in child development.

From

the distributions of response in each city, the 40% were not necessarily college students.

It was pre-

The tutors also tended to have relevant experience. viously noted that 40% worked in Upswing both years.

Roughly 50% had exper-

ience outside Upswing, as tutors, teacher aides, or teachers.

These forms

of experience were not cross tabulated, and some overlap can be assumed. However, it appears reasonable to estimate that 60%-70% of the tutors had relevant experience from some source.

Forty-one percent had worked outside

Upswing with children who had learning difficulties.

Socially, Upswing drew the kind of tutors one might expect to draw for tutoring during school hours.

The tutors generally appeared to be socio-

economically advantaged, well-educated, and community-service oriented.

They

were people with time and energy, to actuate their motivation to contribute to child.-en's education.

Unsuccessful efforts were made, particularly in the

first year, to attract people with more varied background.

Certainly different

recruitment approaches would be required to accomplish this and probably it would be necessary to have greater flexibility about certain features of Upswing organization (time commitment required, tutoring hours, etc.). The characteristics of Upswing tutors are important because of the evaluation findings about the insignificance of training in helping children. It may be that this finding is only valid for tutors like those currently in Upswing.

It is also cleat that it would be more difficult to plan train-

ing that would satisfy a majority of a less homogeneous group. and more varied training approaches would be needed.

Much different

The individual support

that was a feature of the second year of Upswing seems much more promising for tutors of divergent background. Teachers

Since most of the teachers involved in Upswing's second year also

were involved in the first, and background was not related to Upswing results, extensive data were not collected again.2

2

For details of the first-year population, see P. Plantec, et ca Yvaivati.on of Project dprwing: Interim Repor, Technical Report No. 700. Operations Research, Inc., January 1972. Silver Spring, Maryland: 15

28

BEST COPY

WARE

About 40% of the second year teachers said they had worked before their participation in Upswing, with children who had learning difficulties. About 60% said they had worked previously with a volunteer aide.

The most

striking response from teachers was that only 40% overall, and no more than 54% in any city, said they had any training in child development other that that provided by Project Upswing.

This is somewhat difficult to believe;

but if it is true, then Upswing would appear possibly to have made a very important contribution to the backgrounds of teachers who attended training.

16

29

III.

THE EVALUATION METHODOLOGY

In broad terms, Upswing's fundamental hypothesis'was that a child who experiences a one-to-one facilitating relationship with an adult tutor

will make better progress in overcoming learning difficulties than a child who does not experience such a relationship.

A further hypothesis was that

this kind of relationship in the first year of school can prevent failure and therefore is preferable to the common, but expensive, remedial approach. Beyond testing these hypotheses, the evaluation has tried to identify

variables that have impact on tutoring results--that is, the evaluation has tried to find out what, specifically, makes tutoring successful. PRELIMINARY METHODOLOGY Comparison Groups

In the first year of the study there were three comparison groups for the evaluation of tutoring effects on children.

Children were identified

by teachers in each city as candidates for Upswing tutoring.

These were all

children who appeared to be capable of normal functioning, but in whom the teachers recognized signs of various minimal learning difficulties, particularly in development of reading skills.

The children were divided randomly into

three groups: one that would be assigned untrained tutors, one that would be assigned trained tutors, and one that would have no tutors. then were tested.

All of the children

Analysis of test results on IQ, initial reading level, and

visual-motor integration, showed that the randomly drawn groups were very similar in these attributes. 17

30

The analysis of change in test results from initial to final battery was a simple, two-way design involving comparisons of all tutored children versus untutored children and then of those tutored by trained volunteers versus those tutored by untrained volunteers. was treated separately.

First, each city's project

At this level of analysis, no meaningful differences

between test results of comparison groups were found.

After that was determined,

the analysis was performed on aggregate data (children with trained tutors in all cities combined, etc.).

Pooling the data for the four cities produced

different score distributions for the comparison gr (vs, and pooling the city samples provided a larger sample base with a resulting increase in statistical precision.

The statistical procedures used, adjusted for prob-

ability of error, etc., uncovered small but significant group differences. Variables

The primary criterion or dependent variable of the evaluation was change in reading skill as measured by the WRAT.

Change in visual-motor

integration, as measured by the Beery-Buktenica Developmental Test, also was investigated.

In addition, the evaluation considered change in other

aspects of psychomotor control (distractibility and hyperkinesis), change in oral language skills, and change in self-esteem as observed by the adults in the study.

The primary independent variable was child's status vis-a-vis a tutor--i.e., whether the child had a tutor at all, and if so, whether that tutor was trained or untrained.

The evaluation also looked at impact on

tutored children of such tutor and teacher characteristics as age, level of education, experience working with children at the age level and with children who have learning problems, and (for tutors only) income level. Finally, it was hypothesized that a number of psychosocial variables would influence tutoring outcomes.

These variables included, for example, the

quality of relationships between tutor and child and between tutor and teacher, the teacher's feelings about working with volunteer aides in general, the tutor's confidence in his or her ability to help the child, the tutor's satisfaction with training or orientation and with Upswing in general, etc.

Data Sources

The reading portion of the Wide Range Achievement Test and the Beery-Buktenica Developmental Test of Visual-Motor Integration were administered at the beginning and end of tutoring to obtain measures of the dependent variables change in reading proficiency and change in visual-motor integration skill.

of tutoring.

The Slossen IQ test was administered only at the beginning Questionnaires completed by volunteer tutors and teachers

were the data source for measures of all other study variables. Statistical Techniques

Analysis of test score distributions was used to determine whether tutoring affected the children in the criterion areas of reading and visualmotor integration.

The Student's T-test was used to determine the significance

of differences between group mean test results.

A significant level of 5%

was selected as the criterion for accepting or rejecting all null hypotheses.'

Multiple linear regression analysis was used for deeper-level evaluation to identify variables that appeared to exert greatest influence on tutoring results.

For this purpose, change in tested reading level was

taken as the dependent variable.

initial test scores, change scores, indi-

cators of tutor and teacher background, and indicators of attitudes of par-

ticipants and their relationships all were included in the regression as independent variables.

In addition, tutors' and teachers' subjective assess-

ments of certain kinds of changes in the children were included to see how

1 The T-test is based on probability theory. Essentially the test estimates the likelihood that the means of two distributions differ by chance. When the differences between two means is found significant at the 5% level, that indicates the difference probably would occur 95 times out of every 100 times If the experimenter selected a significance the experiment was repeated. level of 107, he would make a decision to accept a difference between his study groups as real if the T-test showed it would be likely to occur 90 The percentage times out of every 100 times the experiment was repeated. represents the margin of error the experimenter is willing to tolerate. For statistical reasons too involved to explain here, the lower the significance level selected, the lower the chance that your "significant" findings are due to sampling error.

19

32

they would relate to test outcomes and to each other.

These "psychosocial"

indicators were created using a modified Likert procedure to scale response alternatives by how favorable to child progress they logically appeared

to

be.

Multiple linear regression procedures produce an equation that predicts the value of a dependent variable (in the Upswing case, number of points change in reading test score) from the values of ordered independent variables.

The mathematical model assumes that the relationships between

the dependent and independent variables are linE:r.

That is, the model

assumes that as the value of an independent variable x increases, the value of the dependent variable y will increase, or will decrease, in a straightline fashion for all observations.

The model is undermined to the extent

that increases (or decreases) in x sometimes are associated with increases in y and sometimes with decreases in y.

The regression output indicates the order of the independent variables in terms of "goodness of fit."

If there were a perfect relationship between

the true and predicted values of y, the fit would be perfect--i.e., if the two sets of values were plotted, the connected points would be superimposed. Goodness of fit is expressed as a multiple correlation coefficient, "multiple r."

Perfect fit yields a multiple r of 1.0, while no predictive power (no

apparent linear relationship between x and y) yields a multiple r of 0.

The

square of the multiple correlation coefficient expresses the amount of common variation between the independent and dependent variables. If the true values of y were predicted from the values of a single independent variable, there would be no need to consider other variables; knowledge of that x would tell the value of y.

In the absence of such a

perfect association, the regression keeps trying; it adds in the information about y from successive independent variables, gradually perfecting the fit between the prediction and the true values of y.

The procedure stops when

a perfect fit is obtained (multiple r = 1.0) or, as is most often the case, when none of the remaining independent variables increases the value of r, i.e., helps to improve the fit between the true and predicted values of y. Multiple linear regression was used as a detective device in the analysis of Upswing data.

It enabled us to identify which of the numerous

20

33

BEST COPY AVAILABLE

independent variables hypothesized to have some bearing on development of reading skills appeared worthy of further consideration.

It also provided

insights about relationships between the dependent variables.

Finally, it

gave an idea of how much of the variation in reading progress we could hope to explain by the kinds of factors or conditions that had been considered. PRELIMINARY EVALUATION RESULTS2 Reading

Analysis performed at the end of the first year of tutoring showed that tutoring did increase children's rate of progress in development of

The difference in point gain on

reading skills as measured by the WRAT.

the reading test was highly significant (a a 0.001).

In short, it is

highly unlikely that the test results turned out as they did due to chance error.

In an absolute sense, gains were real but modest all around--an

average of 7.5 age-adjusted standard score points for tutored children and an average of 4 points for untutored.

Still, the changes represent catching

up; if a child does not increase his rate of progress, but merely continues to develop at his original rate, his age-adjusted standard score will remain the same.

(If a child stops progressing, his score will drop, since he is

growing older, as of course it will if he regresses.)

The analysis found the expected inverse relationship between initial and final reading test score.

Children who scored lower at the beginning

tended to make higher scores at the end of the school year.

In a test-retest

situation involving large enough numbers of children, scores tend to converge toward the mean.

However, in Upswing a ceiling may have been operative as

well, because of the homogeneous IQ grouping and because of teacher or tutor expectancies. IQ.

Children were selected for inclusion on the basic of average

It appears that they may have been reaching their level of performance

potential or the level perceived as appropriate by those working with them.

2 A detailed presentation is available in Operations Research, Inc., Final Report on the Evaluation of Project Upswing's First Year, Technical Report No. 731, December 1972.

21

34

BEST COPY AVAILABLE

Gains in reading proficiency were established based administered test.

on an individually-

A group test of reading achievement also was used.

It

yielded erratic results initially and contradictory results at the end of the year.

In general, if any conclusion can be drawn from the group test

data, they showed the children doing very poorly in reading.

This occurred

probably, at least in part, because the test involved mark-sense coding and the children were first-graders with problems in visual-motor integration. Thus they were unable to keep from making stray marks on the test forms. caused machine scoring errors.

This

In addition, young children are unlikely to

be self-motivated to achieve, as required by the large-group mode, in a formal test situation.

Evidence indicates that children of that age often find a

test setting irrelevant or incomprehensible. It is possible that examiners prompted children iii individual testing.

However, all examiners had some training and experience in test adminis-

tration, and the results in four cities were consistent.

We conclude that

the results of the individually-administered test are reasonably reliable and that it is a much better test, at teas',: for young children.

The differences in outcoms c' beyond the scope of Upswing,

the two types of tests are important

since schools usually are evaluated on the

results of group-administered achievement teats.

(Although such tests

are norm-referenced, we do not betlieve Oat necessarily resolves the problem of unfair evaluation caused by measureoent of extraneous variance.) Relationship Between Tutor Training and Childrer's Progress in Reading Tutor training status ha6 no meaniugful effect on childret's tested progress in reading or in any other area un,Aer study.

-

This finding could

reflect on the type of training given (primaril7 preservice orientation in a group lecture mode, with minimal follow-up during tutoring).

Since train-

ing did not influence the tutors' success, it is most interesting that trained and untrained alike generally thought training important and wanted to be trained.

Teachers also thought trained volunteers generally were better and

said they preferred to work with them.

22

BEST COPY AVAILABLE

Visual -Motor Integration

Both the tutored and untutored groups made virtually no progress in acquiring visual-motor integration skills.

Both groups got about the same

results on the final test in this area as on the initial test; i.e., both groups continued to score well below expectancy for their age level. Relationship Between Reading and Visual-Motor Integration Test Results There appears to have been some negative relationship between visualmotor skills and reading proficiency.

A low negative correlation was obtained

between tested reading and visual-motor integration levels, both at the beginning and end of the tutoring period (r = -0.13) for the two sets of initial test results and r = -0.12 for the two sets of final test results.

Relationship Between IQ and Readiq,Test Results The Upswing data support the position that IQ, within the average range, is not an important factor in progress in development oi beginning reading skills (decoding). level and IQ was -0.09.

The correlation between change in tested reading

The correlation between initial reading score and

IQ was relatively high, 0.52, while the correlation of final reading score and IQ was lower, 0.40, as expected.

There was limited variance in IQ

among Upswing children since they were selected on potential for average functioning.

No more than 5% scored outside the average range (approximately

76-124) on the Slossen IQ measure.

Variance in reading scores increased

over the tutoring period, thus the lower correlation between IQ and final reading scores.

Influence of the Independent Variables Selected for Study The regression analysis identified 35 independent variables that together "explained" about 35% of the change in reading level.

The easiest

way to interpret that finding is to say that, theoretically, if one knew the values of the 35 independent variables brought into the regression, one could predict the amount of change in a child's reading achievement score with

23

36

about 35% accuracy.

This is quite a modest finding, but not insignificant

in light of the inconclusive findings available on factors associated with development of reading skills.

Much of the 65% unexplained variance probably

is attributable to psycho-environmental and historical factors we were unable to measure.

In common with most researchers, we found that characteristics of the child had most to do with the amount and direction of change in tested reading level.

For example, change in child's level of self-esteem (as

judged by volunteer) was found to be the second best predictor variable. Other comparatively important variables included change in child's ability to pay attention (distractibility), child's response to tutoring activities, child's overall progress--all as assessed by the volunteer tutor; and change in child's oral language skills and ability to pay attention as assessed by the teacher.

The best predictor, however, turned out to be an indicator based on the volunteer tutor's feelings about whether he or she was adequately prepared to use teaching methods and materials in tutoring.

We do not know

whether a tutor who feels adequately prepared is more effective or whether a tutor's opinion about adequacy of preparation depends on the child's progress. Probably that is an individual matter.

The correlation between reading

progress and tutor feelings about being well prepared was slightly higher than 0.3.

This is a reasonably high correlation for such data.

Thus the find-

ing would be significant if indeed it were clearly a matter of tutor preparation versus child progress.

However, the subjective measure of tutor-

perceived adequacy is probably highly related to feedback the tutor obtained from his or her perception of the child's improvement.

Thus this correlation

probably is mediated by the tutor's ability to observe child progress. The relationships between test results and tutor and teacher assessments of progress in reading were intriguing overall.

Both subjective assess-

ments showed rather low correlations with change in test score.

24

37

However,

BEST COPY AVAILABLE

the volunteers showed some ability to assess progress similar to that measured by the WRAT.

It was interesting to note that teachers appear better able

to perceive progress as iftessured by the group test of achievement, while

volunteers are more sensitive to the type of progress measured on the individual test.

This may say something about the perceptual set of the

observer.

FINAL EVALUATION METHODOLOGY

The remainder of this section describes the rationales and procedures used in the evaluation of Project Upswing's second year, which included first-year follow-up.

The findings from this evaluation are reported in

Section IV (tutoring effects on children and follow-up findings) and Section V (project management strategies and training approaches). The prelimii.ary analysis indicated it would be wise to separate the

evaluation of tutoring effects on child progress from evaluation management strategy and training approach.

of project

The latter domain is important

to a successful project in itself, regardless of the difficulty of relating it to what happens to the children.

A third domain of the final evaluation

was the staying power of tutoring effects on ch4ldreu from the first year of Upswing over a year without tutoring.

With regard to measurement of tutoring effects on children--both Immediate and over time--emphasis was put on improving the operational definitions of the variables by upgrading the test battery and reducing field error noise.

An experimental objective measure of self-esteem was added

to measure this important variable on an ordinal scale.

The Test of Basic

Experiences (TORE) was used to obtain a proxy measure of family background factors.

(In the first year information about family background was sought

through a questionnaire for parents, but the nonresponse rate was so high this approach was abandoned.)

Data were collected on special educational

services other than Upswing tutoring, since the evaluation team learned in site visits that it was not uncommon for both tutored and control children

25

38

to receive other forms of help.

Reduction of the field error in testing

was accomplished by requiring that all examiners be trained for Project Upswing and by rescoring a sample of all tests. For the evaluation of training an

project management, the opinions

of tutors, teachers, principals and project staff were sought through quesThe evaluation team made site visits to

tionnaires and informal interviews.

observe training throughout the year, and visited the schools in which Upswing tutors worked.

In addition, a field data coordinator, hired in each city to

assist in collection of evaluation data, was required to provide minutes of all training sessions. Comparison Groups

Only two groups of children were involved in the second year of Upswing--tutored and untutored.

The child groups were to be drawn randomly

by project staff from a pool of candidates identified by teachers, with the constraints that no teacher was to have more than six pupils involved and that half of each teacher's referrals should be assigned to the experimental group and half to the control group.

All volunteers were offered training.

This decision had to be made before the end of the first year of tutoring because of the timing of project grant decisions. evaluation results were not available.

Thus the first -year

The evaluation showed that trained

tutors were no more effective than untrained and, comparatively, cost a good deal more.

However, the evaluation also showed that the tutors wanted to

receive training and that teachers preferred to work with trained tutors. Variables and Data Sources for the Analysis of Effects on Children The primary criterion (dependent) variables established for the evaluation were:

Change in level of reading skill as measured by the WRAT

Change in level of visual-motor integration skill as measured by the Beery-Buktenica Developmental Test

Change in level of self-esteem demonstrated on the experimental "Funny Faces Game."

26

, 39

As a check on the self-esteem measure, data on children's self - confidence in the one-to-one tutoring situation were sought from the volunteer tutors, and and data on self-confidence demonstrated in the classroom b both tutored untutored children were elicited from teachers. (Copies of the data collec-

tion instruments--"Student Profiles"--used to obtain the tutor and teacher assessments of children are included in Appendix B.) The primary independent variable again was whether the child had a tutor.

Other variables hypothesized as likely to influence change in the

criterion areas are shown, with data sources indicated, in Table 1. Variables and Data Sources for the Anal sis of Tutorin

Effects Over Time

The follow-up analysis considered whether one year of Upswing tutoring influenced children sufficiently to distinguish them in the subsequent school year from the Upswing children who did not receive tutoring. The primary criterion variable was level of tested reading skill after the year

without Upswing, compared to tested reading skill immediately after the might be attributable Upswing experience. Other potential differences that to Upswing tutoring also were investigated, such as self-esteem, class standing, and need for special services in the second year.

Table 2 lists the

follow-up variables, measures, and sources of data. As Table 2 shows, the primary independent variable was dichotomous: tutored/untutored status. The assumption underlying the analysis is that psychoeducational variables other than tutored status would be randomly distributed in both populations.

The objective of the analysis was to

determine the staying power of the Upswing effect on reading level achievement compared with the "normal" progress of the control group. Variables and Data Sources for the Analysis of Project Management Strategy and Training The criteria for this part of the evaluation all had to do with the satisfaction of participants.

There were no subdivisions of manage-

of the numerous ment and training approaches within cities; further, because it undefinable environmental differences between cities, we did not feel

would be valid to evaluate the cities' often divergent methods based on changes in the children.

27

40

self-e.:,teem

Change in level of

Change in level of reading skill

gra tion skill

visual-motor inte-

C. Orange in level of

B.

A.

Dependent Variable

Motor Integration (VMI)

Difference between initial and final score on Beery-Buktenica Developmental Test of Visual-

Inventory

Difference between initial and final score on Self-Esteem

.

Same as criterion "A", plus volunteer's assessment of emphasis placed on motor and visual-motor coordination activities during tutoring

Same as criterion "A"

Child's sex

outside Upswing Whether child attended kindergarten

Special services child received

Records kept by ORI field data coordinator in each

Number of hours tutoring child received

Questionnaire data

Same as criterion "A"

naire for teachers Project records

Information from question-

naire for teachers

Information from question-

city

Lion (CARE)

score Initial scores from Cegelka Academic Readiness Evalua-

Initial WRAT reading

(TUBE)

Initial score from Test of Basic Experiences

MeasureData Source

BEST COPY AVAILABLE

tual-motor skills, initial auditory perception skills, initial languagespeech skills

Child's background of basic knowledge (an indicator of borne characteristics relevant to school performance) Child's initial level of reading proficiency Child's initial visual percep-

Difference between initial and final scores on Wide Range Achievement Test (WRAT)

Independent Variable

MeasureData Source

VARIABLES, MEASURES, AND SOURCES OF DATA FOR THE ANALYSIS OF TUTORING EFFECTS ON CHILDREN

TABLE 1

Type of class assignmeat after a year of Upswing tutoring

Upswing

to level of skill inatediately after

of l;psl.ving relative

skill atter a year out

Information from project directors

iame as criterion "B"; plus availability of special services in the school

Same as criterion "A"

Same as criterion "B" Project records Project records Same as criterion "A"

;Jame as criterion "A"

lame as criterion "B" Whether child received Up:wing tutoring Whether child received Upswing tutoring ;Arne as criterion "A"

Rating by second-year teacner

Questionnaire for secondyear teacher Questionnaire for secondyear teacher Rated by second-year teacher

"Funny Faces" self-esteem inventory administered a end of follow-up year

Cies.> standing at end of follow-up year

Classroom behavein follow-up year

Raze of absenteeism in follow-up year

Level of self -contidence in school

Levet °I self-esteem as measured by objective test

E.

F.

G.

H.

system

Project records

Child's sex

Project records

Project records

Project records Questionnaire for secondyear teacher

Project records

MeasureData Source

BEST PAW

D.

child in zallow-upeyear

Questionnaire for secondyear teacher

Rating based on questionnacre for second-year teacher

on WRAT given at end of Upswing tutoring the preceding year

Whether child received Upswing tutoring score on WRAT at end of Upswing tutoring

1innther child received Up:wing tutoring Child's sex 'Whether child received special services in follow-up year

Score on WRAT given, at end

of follow-up year versus score

Level of reading

C. Spo,:icil servi.:es given

B.

A.

Independent Variable

MeasureData Source

Dependent Variable

VARIABLES, MEASURES, AND SOURCES OF DATA FOR THE ANALYSIS OF TUTORING EFFECTS OVER TIME

TABLE 2

nynnagLE

Table 3 states the variables of the management and training analysis, their measures, and the sources of measurement data.

All measures included

in the table are subjective but one--for effectiveness of teacher training in how to identify minimal learning difficulties (dependent variable B).

The measure in that case was what the test data showed about the initial characteristics of the children teachers selected for Upswing. One additional variable was considered in the analysis of management strategy and training--volunteer attrition.

The attrition analysis was

performed separately, because attritees may be a separate population whose opinions about a project should not be mixed in with the opinions of those who did not drop out.

Nevertheless, rate of attrition may say something

important about recruitment, project operation, or both.

Attrition was

analyzed in terms of percentage losses, timing, and the reasons for leaving given by those who dropped out, considered in the context of project structure and procedures.

Statistical Techniques Tutoring Effects on Children.

As a starting point for the final

analysis, the distributions and basic statistics of pretest results were compared for tutored and untutored children to ensure that they were similar initially in the characteristics under study (tested reading level, self-esteem, visual-motor integration) and in background of basic knowledge as a proxy for school-relevant family background.

The group of children was partitioned on

both tutoring status and location of project ("city").

The latter classifi-

cation variable was applied to ensure that no city was for some reason changing the nature of the overall distributions.

The Student's T-test was used to

check the significance of differences between the mean initial test scores of all possible combinations of status/city groups.

F-ratios were computed to

see if any groups were distinguished by magnitude of variance.

For both T-tests

and F-tests, a significance level of 0.05 was adopted (i.e., it was decided to reject the null hypothesis that there was no difference between groups in mean or in variance if a = 0.05 or less).3

3 The information provided by the Student's T-test was very generally outlined for readers who are not familiar with the procedure, in the description

30

43

Volunteer's level of education

Volunteer' s age Volunteer' s occupation

Questionnaire data

Volunteer's feelings about support received from Upswing staff Volunteer's feelings about support received from teacher Volunteer's satisfaction with project communications Volunteer's confidence in effectiveness as a tutor Volunteer's experience and training relevant to children with learning problems

Modified Liken rating based on questionnaire data

Tutor role satisfaction

Questionnaire data Questionnaire data Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

MeasureData Source

Independent Variable

MeasureData Source

BEST COPY AVAILABLE

Dc.:pencient Variable

VARIABLES; MEASURES, AND SOURCES OF DATA FOR THE ANALYSIS OF PROJECT MANAGEMENT STRATEGIES AND TRAINING

TABLE 3

satisfaction

Dependent Variable Teacher role

Teacher's initial feelings about participation in

Modified Liken rating based on questionnaire data

amount of informal contact with Upswing staff Teacher's opinion of support provided to volunteers by Upswing staff Teacher's opinion of the value of volunteer tutors in working with children who have learning difficulties Teacher's previous experience working with a volunteer aide Number of years teacher has taught kindergarten and first grade Teacher experience and training relevant to children with minimal learning difficulties

Teacher's satisfaction with

meetings

Teacher's satisfaction with frequency of Upswing teacher

Project Upswing

Independent Variable

MeasureData Source

TABLE 3 (Cont)

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

MeasureData Source

BEST COPY AVAILABLE

teacrk:r training in pre:toting good volunteer-teacher working relationship

Effecr..u..,:.,ness of

difficulties

minimal learning

how to icienLify

from questionnaire

Teacher's assessment

VMI, Self-Esteem Inventory

WRAT, TOBE, CARE,

of c.-hilth-en selected for Upswing, from

Teacher's opinion about need for for teacher meetings in a volunteer program Teacher's satisfaction with frequency of Upswing teacher meetings

Type of Instruction

Initial characteristics

Effectiveness of tesicnr training in

program

Questionnaire data

Questionnaire data

ORI observation

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Questionnaire data

Volunteer's opinion about need for training Volunteer's satisfaction with amount of trainili; (group, individual Volunteer' - opinion of schedule, location, I. wsical environment, social environment of Upswing meetings Volunteer's opinion of content of training Volunteer's opinion of training materials used

Effectiveness of volunteer training

Rating based on questionnaire data Attendance at training meetings

MeasureData Source

Dependent Variable

MeasureData Source

BEST COPY AVAILABLE

Independent Variable

TABLE 3 (Cont)

-..vor; -ing relationship (cont)

v3lunzeer-teacher

promoting good

Teacher's assessment

Effectiveness of telcher training in from questionnaire

MeasureData Source

Deconslient Variable

Upswing meetings

location, physical environment, social environment of

knowledge of methods and materials used by volunteer Teacher's opinion of schedule,

Teacher's satisfaction with

Independent Variable

TABLE 3 (Cont)

Questionnaire data

Questionnaire data

MeasureData Sour.; e

13c..cT C.001

The next step in the analysis was to look in the same way at change scores for each test.

("Change score" is used to refer to the difference

between post-tutoring test score and pretutoring test score.)

Through this

step, the analytical procedure was much like that used in the preliminary analysis, except that city was used as a classification variable in the final analysis.

City was used as a classifier, despite the fact that the child

selection method was the same for all cities, because the data indicated that different groups of children were drawn.

This may have been true in the first

year of Upswing, but the preliminary analysis procedure was not sensitive enough to detect it.

Beyond this point, the final analysis took a different direction because of what was learned in the preliminary analysis.

The final evaluation

did not pursue identification of the imract of psychosocial factors such as volunteer-teacher relationship, etc.

It was decided that although we

believe this is a most important area for research, such inquiry would require attention focused on the development of stronger psychosocial indicators through careful attitude scale development.

There was not time or budget

for such an effort as part of the current Upswing evaluation.

Another important change was to drop multiple linear regression analysis and instead use analysis of variance with a covariance design, for exploration of the complex simultaneous influences of the independent variables un the criteria.

of the preliminary analysis. The F-ratio, a product of one-way analysis of variance, is an indicator of the relationship between score deviations from the mean within groups and score deviations from the mean between groups. Without getting elaborate about how the algorithm works, if the variance (calculated from the deviations of individual scores from the mean) within groups is small in relation to the variance between groups, one can conclAe that the groups represent different populations. A sig1, which takes into account the number of observations (in nificance Upswing's case test scores) in each group, can be calculated that indicates the percentage probability that differences between groups were random.

35

48

Analysis of variance involves dividing up the total variation of the dependent variable into its component parts.

These component parts

are determined by the number of independent variables being considered, and how they are classified.

For Upswing, the design was kept simple.

"Status" always was used as a classifier with each of the other independent variables in turn.

For example, the observations on a dependent variable,

say, change in reading test score, were grouped as follows:

change scores

for tutored females, untutored females, tutored males, untutored males; as another example, tutored children with low scores, mid-range scores and high scores on the TORE, untutored children who scored in those categories on the TOBE.

The procedure then looks at the differences in variance

between those groups versus their combined variance, and produces an F-ratio indicative of the probability that the groups are from the same population or from different populations. The covariance design adds another refinement.

It adjusts the

dependent variable (or variate) for the effect of an independent variable (the covariate) that may have nothing to do with treatment but, nevertheless, enters into the treatment result.

In the case of Upswing, we decided to

adjust change in test score for the effect of the covariate, initial test score, because of the strong inverse correlation of these two variables. The net result is an increase in the clarity of the findings. The final statistical procedure used was correlation analysis as a means of considering the relationships between dependent variables.

Of

particular interest was the relationship between change in self-esteem and change in reading test score, which, it was hypothesized, would have a reasonably high positive correlation. Follow -Up Analysis.

For the analysis of tutoring carryover effects

after a year without tutoring, essentially the same techniques were used as for the analysis of effects of the second year of tutoring. only two differences.

There were

First, three groups of children were involved; the

tutored group was split in the first year into children who worked with trained tutors and children who worked with untrained.

Although training

status of the tutor had no apparent effect on the children in the first year, the follow-up analysis took a look at the two groups separately, to

36 if

49

make sure.

We thought the tutor training status variable might have some

influence on, for example, type of class assignment in the second year, since teachers generally believed trained tutors were more effective. The other difference in follow-up analysis procedure was that it did not focus on change in test results, although change was considered. We worked with age-adjusted standard scores on the reading test, which would not necessarily be expected to change in the absence of "treatment." Although possible residual effects of treatment were investigated, the main question about reading was not how much children increased their rates of progress in the year following Upswing, but whether the children tutored in Upswing maintained the edge in standard score that they had established. There were no baselines for the other criteria used in the follow-up analysis. In the case of self-esteem, or class standing, for example, the analysis simply looked for significant differences between the children who were tutored in Upswing and those who were not. :ro!ect .'.1analement Strategy and Training.

evaluation was quite straightforward.

This part of the final

For the analysis of tutor and teacher

role satisfaction and perceptions of training, rating scales were designed on a modified Likert format.

These satisfaction ratings were cross-tabulated

with data on the independent variables that were hypothesized to influence satisfaction.

The evaluation team's subjective views of project organiza-

tion and the training programs were used to interpret these results.

37

BEST COPY AVAILABLE

IV.

ANALYSIS OF THE EFFECTS OF TUTORING ON CHILDREN

This section compares data on the tutored and untutored children who were involved in Project Upswing, in an attempt to deteruine the value of tutoring.

The section is divided into four major parts:

A review of findings about tutoring effects in the first year

An examination for residual effects among first-year children after they had been out

of the project for a year An analysis of the effects of involvement in the project on the new group of children tutored in the second year

Conclusions about the impact of the project over both years.

SUMMARY OF FINDINGS FROM THE FIRST YEAR OF UPSWING Children tutored in thf first year of Upswing made better progress in reading than the comparison or control group of children who were not tutored.

The gains on the Wide Range Achievement Test (WRAT), reading

subtest, were modest for both tutored and untutored children, but significant in relation to the standard error of the test. The initial test means for all Upswing groups fell into the lcw average range (80-90 points on the WRAT), with very small standard deviations.

Thus the project began with a total group of children whose

BEST COPY AVAILABLE reading difficulties apparently were not severe and who were very similar to each other in level of reading skill.

On the post-tutoring test, all

group means (for children tutored by trained volunteers, untrained volunteers or not tutored) fell in the average category of WRAT standard scores (90-109 points).

The mean gain for children with trained tutors was 7

standard score points; the mean gain for those with untrained tutors was The

8 points, and the mean gain for untutored children was 4 points.

difference between the amounts of gain made by tutored and untutored children were found to be statistically significant.

Figure 1 shows the relative

rates of progress.

Children who started with lower test scores tended to make greater gains than children who started higher.

It appears that, on a

group basis, some kind of ceiling on progress was operative--perhaps expectancies c..'7 tutors and teachers for the reading skills of children

of Upswing age.

None of the children improved their visual-motor integration skills in the first year of Upswing. low skills for that age.

They started and ended the school year with

Visual-motor integration (as measured by the

Beery-Buktenica test) and reading ability appear, for this population, to be separate domains.

It is possible that visual-motor integration

problems were easy for teachers to observe and contributed to the child selection for Upswing as much or more than low reading ability.

If so,

the low correlation between reading improvement and visual-motor integration gains would indicate that selection could be improved by concentrating on reading and ignoring visual-motor integration, if reading is to be maintained as the primary criterion variable. Tutors and teachers both believed that Upswing tutoring was especially beneficial to children's self-esteem.

Moreover, change in

self-esteem was one of the two variables most strongly associated with increase in reading test score. confidence of the tutor.)

(The other variable was related to the

There was no objective measure of self-esteem

in the first year of Upswing, and it was not possible to make comparable observations of that characteristic in control group children.

40

52

BEST COPY AVAILABLE

120

110

Mean Performance for Age Based on Standardization Sample

3:1

CC

100 5( = 94.1

90

X

87.6 WNW

X

--

Children Progress of Tutored

_

=1,

Min.

onwil

emamb ...

X = 90.6

Prowess of Control Children

85.8

80

70

60

50

44

1

NOV

DEC

JAN

FEB

MAR

(First Test)' Time

'Testing continued through January in some cities; thus the mean starting point of both groups is inflated to the extent that classroom instruction and tutoring (for the experimental children) increased the scores of the children children who were tested late.

FIGURE 1. RATE OF PROGRESS IN READING AS MEASURED BY WRAT: TUTORED VERSUS CONTROL CHILDREN

41

5.3

APR

MAY (Final Test)

Change in children's self-esteem presented itself as an intriguing area for further study; it appeared to be quite important, but the data available were limited.

DESCRIPTION OF CHILDREN INVOLVED IN THE UPSWING FOLLOW-UP

Each project located all children from the first year of. Upswing who remained in a local school system, for follow-up after a year without Upswing.

There were, for all projects combined, 242 children involved in the follow-up: 123 former tutored children and 119 former control

children.

Those numbers represent about half of the tutored group who

still were in Upswing by the end of the first year and about three-quarters of the control group.

Table 4 gives a breakdown by city and child status

vis-a-vis an Upswing tutor.

The mix of boys and girls was essentially the same in the three comparison groups.

The children in the three groups also were similar

in age distribution. respectively.

Tables 5 and 6 present the data on sex and age,

These variables were checked to ensure that their influence

did not obscure any effects of Upswing tutoring.

Not only were the com-

parison groups similar enough in sex and age composition to preclude that possibility, neither sex nor age differences were found to be associated with differences in the criterion areas.

There was only one minor excep-

tion to this finding--girls tended to make slightly greater gains in reading test score over the follow-up year, as is documented in the discussion of reading outcomes. FOLLOW-UP OBJECTIVES

The follow-up was conducted to find out whether: 1.

Tutored children would maintain their reading achievement edge over a year without Upswing tutoring.

2.

Children tutored in Upswing would indicate a different level of self-esteem than control group children on an objective test.

3.

Tutoring had an effect on the classroom assignments of children for their second year in school.

42

54

CA)

At end of first year Follow-up

Untutored children At end of first year Follow-up

T+U

Children who had untrained tutors (U) At end of first year Follow-up

Children who had trained tutors (T) At end of first year Follow-up

Status Group

163 119 (73%)

31

31(100 %)

48

38(79%) 28(67%)

22 (58%)

123(519')

46

18(55%)

32(49%)

37(48%)

36(53%)

243

52(46%)

113

71(55=t)

130

Total

38

33 65

6(50%)

77

14(499)

18(39%)

14(47%)

12

68

25

46

12(57%)

18(45%)

19(61%)

22 (58%)

30

21

40

31

xford

38

Denve.

BEST COPY AVAILABLE

City St. Louis San Francisco

(Percentages of first-year groups)

CHILDREN INVOLVED IN THE UPSWING FOLLOW-UP

TABLE 4

BEST COPY AVAILABLE

TABLE 5

SEX OF CHILDREN INVOLVED IN UPSWING FOLLOW-UP

Male

Female

Total

39/55%

32/45%

71/100%

Tutored by iintr lined volunteer

29/56%

23/44%

52/100%

Control group

67/56%

52/44%

110/100%

135/56%

107/44%

242/100%

Status in Project Upswing Tutored by trained volunteer

Total

44

56

Nonresponse cases omitted. Actual numbers of children were: 71T, 52U, 119C; grand total, 242.

** Rounding error.

*

232/101%** 2/1% 39/17%

I

108/47%

83/36%

Total

115/100% 0

19/16%

50/44%

46/40%

Control group (C)

50/100% 0

7/14%

67/99 % **

Total*

29/58%

2/3%

Greater than 8 yr, 6 mo

14/28%

13/19%

8 yr, 1 mo thru 8 yr, 6 mo

Tutored by untrained volunteer (U)

29/43%

7 yr, 7 mo to 8 yr

23/34%

7 yr, 1 mo to 7 yr, 6 mo

BEST COPY AVAILABLE

Tutored by trained volunteer (T)

Status in Project Upswing

Age Category

(Percentages based on number of children in each status group whose ages were reported.)

AGES OF CHILDREN INVOLVED IN UPSWING FOLLOW-UP, AT END OF FOLLOW-UP YEAR

TABLE 6

4.

Tutoring had an effect on children's later needs for special educational services such as remedial reading.

5.

Tutoring had an effect on children's class academic standing in their second year in school.

6.

Tutoring had an effect on children's classroom behavior.

7.

Tutoring had an effect on children's selfconfidence demonstrated in school in the second year.

8.

Tutoring had an effect on children's adjustment to school as demonstrated by rate of absenteeism in the second year.

FOLLOW-UP PROCEDURES AND DATA SOURCES

The children who could be located were administered two individual tests:

esteem.

the WRAT reading subtest and the "Funny Faces" measure of selfThe children's WRAT scores from the end of the first year (May 1972)

were taken as the baseline measure of reading skill.

There was no base-

line for self-esteem.

All children were follow-up tested at the end of

April or in. May 1973.

The examiners were project staff, all of whom had

training and experience in individualized test administration.

The children's teachers in the current school year were asked to give information about the children relative to the last six follow-up objectives listed above.

The teacher's observations were recorded on a

form, "Follow-Up on Children Tutored in the First Year of Upswing" (copy in Appendix B).

The form either was completed by the teacher as a question-

naire or completed by a project staff member in a personal interview with the teacher.

STATISTICAL HYPOTHESES1

In general, the follow-up analysis looked for differences in distributions of results for former tutored children and former control

1

All hypotheses except the last are stated positively because the null form may be confusing to some readers.

46

children in the eight criterion areas listed previously.

The hypotheses

were: 1.

That children who received Upswing tutoring would make higher reading test scores than the control group after a year without Upswing tutoring,

with a difference in group means significant at the 0.05 level or better and a variance ratio significant at the 0.05 level. 2.

That tutored children would score higher than control children on the test of self-esteem, with difference between means and variance ratio both significant at the 0.05 level.

3.

That there would be a positive relationship between the follow-up reading and self-esteem test scores with a correlation coefficient of 0.30 or higher.

4.

That there would be a positive relationship between change in reading test score and the follow-up self-esteem test score, with a correlation coefficient of 0.30 or higher.

5.

That, for all teacher-measured variables, the distributions of responses would differ for tutored and control group children with a significant variance ratio (a = 0.05).

6.

That there would be no statistically significant difference (a = 0.05) in any of the follow-up criterion areas between children who had been tutored by trained Upswing volunteers and those who had been tutored by untrained Upswing volunteers.

FnLLOW-UP RESULTS

In this description of results, the reading and self-esteem criteria are treated first, separately.

Then the second-year teachers'

observations about the children are presented.

47

The interactions among

follow-up variables are discussed next, and conclusions are drawn about the follow-up results.

Comparison of Reading Levels of Tutored and Untutored Children After A Year Figure 2 shows that the tutored and control children who could be followed held their own during; their second year in school.

The pre and

post means placed tutored and untutored alike at the low end of the average range on the WRAT (90-109 standard score points).

There was about

a two-point decline in standard score for the tutored group, but this is not significant in relation to the standard error of the WRAT. The standard deviations of tilt two groups also point up their

similarity in both initial and final reading test score distributions: Tutored Pretest

Post-test

Untutored

9.90

9.19

10.29

10.48

The standard deviations are consistently lower than the 15 points found in the normative population.

This indicates that, as expected, the Upswing

groups were more homogeneous than a normal population.

The increase in

standard deviation on the follow-up final test also was to be expected since environmental differences such as schooling, experiences, or maturation, cause greater variance in children's reading skills over time.

A

larger increase in standard deviation would be expected for a normal sample. It should be noted that the data in Figure 2 exclude the reading post-test scores of 26 children because there was no baseline measure for them; these children were not tested at the end of the year of Upswing tutoring.

Their "post" scores were checked to get at least a partial view

of the nature of bias that might have been introduced by their omission from the sample.

The 26 children included 16 who were tutored (10 by

trained volunteers, 6 by untrained, and 10 from the control group.

The

status group means for the 26 were higher than the means for the groups with complete data; however, there was a great deal of variance in the excluded scores.

The 10 children from the Upswing control group had a

higher mean reading post-test score than the 16 who received Upswi g tutoring, consistent with the trend for the larger follow-up groups.

48

Scort

WRAT Standard

41

60

80

100

120

160;

Jun

May

Jul

I

1972

Aug

I

Sep

1

Oct

1

Nov

1

Dec

1

Jan

1

Feb

1

--

Mar

1

1973

.M10111M.

Apr

i

May

1

= 91.0 (Control)

= 91.5 (Tutored)

BEST COPY AVAILABLE

FIGURE 2. READING PERFORMANCE AFTER A YEAR WITHOUT TUTORING, AS MEASURED BY THE WRAT: FORMER TUTORED CHILDREN VERSUS FORMER CONTROL CHILDREN

I

91.5 (Control)

I

x

= 93.7 (Tutored)

.1.0.

Mean Performance for Age Based on Standardization Sample

Control

Tutored

BE.;r

Rate

nul,!.;Thr

t'Proliress in Reading in the Follow -Up Year.

Both the

tutored and control groups increased their rate of progress in reading during the first year of Upswing tutoring as shown by the rising slope of the lines in Figure 1. for tutored children.

The increase in rate was significantly greater

In the follow-up year, thare was no difference in

the rates of progress of the two groups. rate of progress at all.

In fact neither increased its

Both simpl: held their own at the pretest level.

Since the WRAT is age-adjusted, an increased rate of progress is indicated by a gain in standard score.

If a child's standard score remains the same

from test to test over time, he is progressing at a uniform rate.

The

"average" child would be expected to obtain a standard score of 100 each time he is tested regardless of elapsed time or grade level.

The standard deviations of change scores for the two groups showed chat some children were increasing somewhat in rate of progress, while others were retrogressing.

Those values were almost identical:

tutored children and 7.45 for untutored children.

7.16 for

This result is another

indication of the close similarity of the tutored and untutored groups' reading skills.

In relation to the mean change for the groups (-2.3 points

for tutored children and -0.5 points for untutored children), the standard deviations represent some spread.

In relation to the range of possible

change, however, the standard deviations are quite small, indicating that nothing spectacular happened to the reading skills in any part of either group.

The change that occurred was negatively correlated with pretest

score (r = -0.3, statistically significant at the 0.001 level).

That is,

children who started with lower test scores tended to show greater score gains on the final test.

Differences in Children's Reading Levels From City to City.

Know-

ledge of the city in which a child lives is, in the case of the Upswing follow-up, a better indicator of his WRAT score than knowledge of his status vis-a-vis an Upswing tutor.

Table 7 shows, for example, that San Francisco

children averaged roughly 10 points lower in pretest score than Denver children or St. Louis tutored children. differences in change score.

was previously pointed out:

These differences in starting score in.lunnced The influence of initial score--the covariate.-

children who started lower tended to make

50

62

TABLE 7

BEST COPY AVAILABLE

PRETEST WRAT SCORE AND CHANGE IN SCORE OVER THE FOLLOW-UP YEAR BY CITY AND STATUS (1)

Status Group, by City

(2)

Mean Pretest Standard Score

(3)

Mean Point Change in Standard Score Pre to Post (a)

(b)

Unadjusted

Adjusted

Denver:

Children who had trained tutors (T)

96.6

-2.2

-1.2

Children who had untrained tutors (U)

95.9

-4.2

-3.7

Control group (C)

97.0

-0.2

-1.5

T

91.9

-2.1

-2.2

U

92.1

-3.6

-3.7

C

92.7

-2.3

-2.3

T

97.3

-4.1

-3.4

U

97.4

-1.6

-0.9

C

90.0

-1.5

-1.9

T

86.4

+3.0

+2.0

U

86.3

+0.2

-0.8

C

98.3

+3.9

+3.2

Oxford:

St. Louis:

San Francisco

51

BEST COPY AVAILABLE greater gains.

As Table 7 indicates (column 3a), San Francisco children

were the only children, regardless of status group, whose mean change in WRAT score was in a positive direction.

Their relatively lower pretest

scores account for part of this difference.

The effect of adjusting for

the covariate (column 3b) is to decrease the amount of positive gain indicated, giving a clearer picture of the influence of "city."

Column 3

of the table also shows that when amount of change is adjusted for pretest, the relative positions of the other city/status groups, in terms of how much their mean scores decreased, also changes.

In a sense one can say

that those who started higher had more to lose, so that adjustments for starting level reduces their amount of loss relative to those who started lower.

The foregoing rather complicated exegesis may be interpreted in essence as follows:

It appears that city possibly had some influence on

what happened to children's reading skills in the follow-up year.

This

influence is attributable to a combination of child characteristics, including pretest level of skill, anc. environmental characteristics that the Upswing evaluation cannot define.

When we controlled for the impact of

child's starting level, the influence of city on change in reading level over the year became miniscule.

This is attested to by the fact that, on

the final follow-up test there was no significant difference between the means of any two city/status groups or between the means by city for all children combined.

Differences in Reading Level Between Boys and Girls.

Girls

typically have been found to progress faster in the early years of school, and there wa' some tendency for girls involved in Upswing to reflect this early advantage.

Overall (without regard for status group) the pretest

means for boys and girls were almost identical.

However, girls tended

to gain more or lose less in standard score than boys over the followup year.

The margin was quite narrow; sex accounted for only about 2%

of the variation in change in reading score.

Still the difference was

enough to produce a significant F-ratio for the boys versus girls. status continued to have no apparent bearing on reading.

52

Upswing

Girls, regardless

BEST COPY AVAILABLE of whether they were tutored by trained volunteers, untrained

volunteers,

or were not tutored, tended to increase their final test scores or to lose fewer points on the final test.

Figure 3 compares the distributions of

change in reading test score for boys and girls.

Again, it should be

remembered that no change means no change in rate of progress; it does not indicate lack of development of reading skills.

The preceding Table 6 (under "Description of the Children ...") shows that, for all cities combined, the ratio of boys to girls was almost identical in the tutored and untutored groups.

There were imbalances in

Denver and San Francisco, but the data indicate that they do not affect the essential point; namely, that although sex of child appears to have had a very minor influence on follow-up results, the sex variable did not override the status variable to obscure any influences of tutoring. Comparison of Follow-Up Groups on Self-Esteem Test Results There were no differences between any two of the follow-up groups of children in measured level of self-esteem at the end of their second year in school (Figure 4).

The means for the three groups fell toward the

low end of the average range.

Although the standard deviations were fairly

large in relation to the range of actual test scores, +1 standard deviation places the children within the borderline to average range of the test. (Score categories were established in a pretest involving all first-graders in an elementary school--59 children--as the normative sample.)

of 13 Upswing children did not take the self-esteem test:

A total

one from the

group tutored by trained Upswing volunteers, seven from the group tutored by untrained volunteers, and five from the control group.

It is unlikely

that their scores would have changed, appreciably, the results shown in Figure 4.

Diffemnces in Self-Esteem From City_to City.

The self-esteem

scores of the three follow-up groups within each city were not so uniform as the WRAT scores.

Nevertheless, status still was not a significant

variable at the individual city level.

There were some comparatively large

differences between certain groups from city to city, but the pattern found in the WRAT scores (consistently lower or higher scores in a given

53

BEST COPY AVAILABLE

100;

Male

Female

55

Iwo Iwo dim* dimMt

50 45 40

% in

35

Change

Category

30 25

_

20

15 10 5

-"lb

-20 -15 -10

-5

0

5

10 15

20

25

No. of Points Change in Standard Score

FIGURE 3. MALE-FEMALE COMPARISON OF CHANGE IN WRAT SCORE OVER A YEAR WITHOUT TUTORING

(Percentages plotted at the mean values for the change categories)

54

6b

Serious Problem

1.

1

. 1

0

.

10.2

4

1.

1 11

12

I. 1

.

20

Self-Esteem Test Score

16

1

N = 114

N =45

N = 70

Borderline

8

SD= 11.6

SD

Problem

.

13 8 4

R = 20.0

X = 21.9

SD -- 12 4

1

.

1

28

Average

24

.

32

. 1.

.

1

36

High

40

. 1.

(Range of possible test scores = -46 through 41; range of Upswing follow-up scores = -13 through 41. Score categories based on pretest with all first-grade children in a Prince George's County, Maryland public school.)

FIGURE 4. COMPARISON OF SELF-ESTEEM TEST RESULTS FOR THE FOLLOW-UP GROUPS

Not Tutored

Tutored by Untrained Volunteer

Tutored by Trained Volunteer

21.6

BESTCOPYAVAILARLE city) was not evident in the self-esteem scores.

The F-ratio indicated

that the variance between city/status groups was significant (i.e., that certain groups apparently represented different populations in terms of 1::stribut::on of self-esteem).

However, the Student's T-test applied to

all possible pairs of city/status groups indicated that all differences between means on the self-esteem test were insignificant. Comparison of Follow-Up Groups Based on Information From Their SecondYear Teachers upswing status made no appreciable difference in any of the child characteristics about which the teachers were queried at the end of the follow-up year. ,x;e

Class Assianment for the Year After Upswing.

It was

hypothesized that status in Project Upswing might have some bearing on the type of class assignment children received for their second year in school, since the tutored children tended co gain more in reading and since teachers tended to regard Upswing tutoring as beneficial.

This

hypothesis assumes, of course, that class assignments are not random but are governed by achievement level, at least in the case of retention. It further assumes that reading proficiency is the achievement indicator given greatest weight, at least in the primary grades. Table 8 shows that, among the children who could be followed, Upswing tutoring had no important bearing on placement for the next school year.

The table also shows that data on type of class assignment by

themselves would suggest that Upswing children on the whole were considered a below-average group.

Placement in a combination first and

second grade class made up primarily of children with learning difficulties would seem to be a form of retention; the import of placement in a combination class of average or above-average children is less clear.

(Only

one child was reported to be in an above-average combination class.)

Ex-

cluding children in the latter, more ambiguous category from the retention group, the retention rate still is about 30%, which seems quite high. addition 10% to 15% of all the Upswing children were in second_grade classes made up primarily of children with learning difficulties.

56

Only

In

i

BEST COPY AVAILABLE

234/1007 115/100% 51/100%.

7/10%

36/53% 68/100%

Second grade, primarily children with learning difficulties

Second grade, normally distributed or above-average class

5/10%

6/11%

19/16%

8/7%

31/13%

22/10%

33/14%

* Non-response cases omitted. Actual total numbers of children were: 71 t; 'ired by trained volunteers, 52 tutored by untrained volunteers, 119 from control group; grand total, 242.

Total

113/48%

53/46%

24/47%

8/12%

Combination grade 1-2, normally distributed or above-average class

17/15 X,

8/12%

Combination grade 1-2, primarily children with learning difficulties 8/16%

18/16%

8/16%

9/13%

Retained in first grade

35/15Y

Not Tutored

Tota 1*

Tutored by Untrained Volunteer

Tutored by Trained Volunteer

Type of Class Assignment for the Year After Upswing

Upswing Status

(Percentages based on number of children in each status group whose teachers responded to question)

TYPE OF CLASS ASSIGNMENT FOR THE SCHOOL YEAR FOLLOWING UPSWING TUTORING, BY STATUS IN UPSWING

TABLE 8

BEST COPY AVAILABLE about half of all the Upswing groups were clearly promoted to normallydistributed or above-average classes.

Table 9 shows that there were differences in assignment pattern from city to city.

This probably reflects to some extent differences in

types of classes available and also differences in educational philosophy. The highest percentages of children were retained in Oxford and, more so, in St. Louis.

Denver had a considerably higher percentage of normal second-

grade placements than any other city.

There were some differences between status groups within cities, but the number of children per group are so small under the three-way classification (city x status x type of assignment) that conclusions about such differences are tenuous.

Differences mostly appear to be random,

but tend to occur more between groups tutored by trained versus untrained volunteers than between tutored versus untutored children, except in St. Louis.

The data suggest a possibility that there may have been greater

tendency in St. Louis to retain former control group children in first grade classes, while the alternative for former tutored children was more likely to be a combination first and second grade class normally distributed in achievement level. Special Services Given in the Follow -Up Year.

Eighty-six of the

242 children in the follow -up were reported to have received one or more

kinds of special educational service during their second year of school. The most commonly received services were (1) tutoring in a subject other than reading, (2) remedial reading, and (3) tutoring in reading.

Eighty

percent of the children who received special services had remedial reading or a reading tutor (in addition to other kinds of help in many cases). It was hypothesized that Upswing tutoring might make special services, at least in reading, unnecessary for many children in their second year of school.

However, as the data in Table 10 indicate, Upswing

status was not related to receipt of services.

There was virtually no

difference between tutored and untutored children in any category except, possibly, speech therapy.

The difference there occurred because about

three times as many children who had worked with trained Upswing tutors received speech therapy in the follow-up year as children who had untrained

58

BEST COPY AVAILABLE

234/100%

46/100% 69/100%

65/100%

54/100%

*Nor.response cases omitted. Actual total numbers of c:iildren were: Denver, 58; Oxford, 65; St. Louis, 70; San Francisco, 49; grand total, 242.

Total*

113/48%

22/48% 16/23%

31/13%

35/54%

10/22%

40/74%

9/13%

22/10%

33/14%

35/15%

Total

Second Grade, Normally Distrth,uted or AboveAverage Class

7/11%

6/13%

2/4%

6/13%

San Francisco

5/9%

10/14%

23/33%

11/16%

St. Louis

Second Grade, Primarily Children with Learning Difficulties

6/9%

6/9%

11/17%

Oxford

0

2/4%

7/13%

Denver

Combination Grade 1-2, Normally Dis--ibuted or Above-Average Class

1-2, Primarily Children with Learning Difficulties

Cornbination Grade

Retained in First Grade

Assignment

Type of Class

(Percentaged based on number of children in each city whose teachers reported type of class.)

TYPE OF CLASS ASSIGNMENT FOR THE SCHOOL YEAR FOLLOWING UPSWiNG TUTORING, BY CITY

TABLE 9

86/36%

42/35%

44/36%

tutor

Psychological services

education class

Part-tin-e in special

C.t:.._.r

Splaech therapy

Redwing

12:.::.:.:.-iial reading

Tutor in subject other than reading

If yes:

2/3%

1/2%

3/2%

2/2%

5/2%

7/3%

9/4%

4/3% 5/4%

2/4%

3/4%

2/2%

22/9% 6/5% 16/13`.13

3/6%

13/18%

5/4%

32/13% 17/11°4.

15/12%

6/12%

9/13%

4/8 %

37/15% ?3/19% 14/11%

4/8%

10/14%

1/1%

43/18%

22/18% 19/15%

7/13%

14/20%

16/31%

28/39%

Yes

156/64%

77/65%

79/C4%

36/69%

(N = 2421

Not Tutored (N = 119)

All Tutored Children (N = 123)

Total

BEST COPY AVAILABLE,

Tutored by Untrained Volunteer (N = 52)

43/61%

Tutored by Trained Volunteer (N = 71)

No

:Lad Special Services?

Upswing Status

(Percentages based on total number of children in each status group. Column totals omitted because some children received more than one kind of service.)

SPECIAL SERVICES TO FORMER UPSWING CHILDREN DURING THE FOLLOW-UP YEAR, BY STATUS IN UPSWING

TABLE 10

.

BEST COPY AVAILABLE tutors or no tutor.

We can only speculate about the reasons for this.

It appears that teachers may associate such problems as speech impairment with learning difficulty.

It also appears that there was a tendency to

assign the children perceived as having the greatest difficulty to the experimental group and, further, to trained tutors since they were regarded as most effective.

Since speech impairment is a readily apparent problem,

more speech-impaired children may have been perceived as in greater need of strong assistance.

Table 11 shows that there were differences between cities in types of special services offered and numbers of children served.

This was to

be expected because of differences in school system budgets and the involvement of the communities in providing volunteer aid in the schools.

Con-

siderably more special services were provided in Oxford and San Francisco, and the number of children who received tutoring in the latter city may indicate the contribution of the city's school volunteer organization- the San Francisco Education Auxiliary.

Considerably ftwer special services

were provided in St. Louis, perhaps because of limited budget. Generally, about the same percentages of former tutored and untutored children received special services within each city.

The only noteworthy

exception was that in Oxford a considerably larger proportion (58%) of children who had worked with trained Upswing tutors received special services than children who had untrained tutors (33%) or control group children (39%).

Data on different types of services partitioned by both

status and city are not appropriate for comparison since numbers of children in the comparison groups become so small. Standing.

Teachers were asked to evaluate the former Upswing

children's class standing at the end of the follow-up year.

Figure 5

illustrates that there were virtually no differences on this measure between the Upswing status groups.

The figure also shows that children tended

to the low side in class standing.

61

73

BEST COPY AVAILABLE

17/26%

4'7% 8/14% 4/7% 3/5%

Part-time in special education class Psychological services

Other

Speech '-lrapy

Read ing tutor

2/3% 0

0

1/2%

2/3% 7/11% 1/2%

6/9%

12/21%

If yes: Tutor in subject other than reading Remedial reading

14/29%

7/14% 4/8% 4/6% 1/1%

0

.0

37/15% 32/13% 22/9% 13/27%

3/4% 8/12%

7/3%

5/2%

5/10% 4/8%

9/4%

43/18%

20/41%

5/7%

86/36% 24/49%

16/23%

28/43%

18/31%

:'es

156/64%

25/51%

53/77%

37/57%

40/69%

(N = 49)

(N = 69)

No

Total (N = 242)

San Francisco

Oxford (N = 65)

Denver (N = 58)

Had Special Services?

City St. Louis

(Percentages based on total number of children in each city. Column totals omitted because some children received more than one kind of service.)

SPECIAL SERVICES TO FORMER UPSWING CHILDREN DURING THE FOLLOW-UP YEAR, BY CITY

TABLE 11

10

20

30

40

-

50 \-

60

100

T

32%

C

26%

U

20%

Low Average

T

21%

C

20%

T Average

U C

1

3%

High Average

6%

i

1%

Superior

4%

(Percentages based on total number of children in each group whose teachers responded to the question; no response for three children from the T group, one from the U group, and two from the C group.)

BY THEIR SECOND-YEAR TEACHERS

4%

Control group children (N- 1171

C

44%

Children who had untrained Upswing tutors (N =51)

U

FIGURE 5. CLASS STANDING OF FORMER UPSWING CHILDREN, AS EVALUATED

Low

U

27%

40%

45%

Children who had trained Upswing tutors (N' 68)

T

BEST COPY AVAILABLE Since there were significant differences in reading post-test score distributions from city to city, the possibility was considered that there might be city differences in teacher evaluation of children's class standing.

From Table 12, that did not turn out to be the case.

The data suggest a marginal difference between St. Louis and the other cities; however, even though fewer children there fall in the lowest class standing category, the general trend of average or below average standing obtained.

There were no significant differences in mean or

variance between status group distributions on class standing within city and from city to city.

It is possible (even probable), given the known

sampling bias, that real Upswing gains have been obfuscated. C!,-zssroom Behavior.

Teachers indicated that former Upswing control

group children, overall, exhibited slightly more disruptive classroom behavior than either group of tutored children (Figure 6).2

This result

can be traced to the combined effect of differences in two cities.

Denver

and St. Louis children, especially the former, presented more behavior problens, as shown in Table 13.

In both cities, more frequent disruptive

behavior was noted among control group children.

A counter trend occurred

in Oxford; however, it was not strong enough to neutralize or override the effect of the other two cities on project-wide results. Self-confidence Demonstrated in School.

The self-confidence measure

was obtained by asking teachers to rate children on a three-point scale where:

1 = child generally unsure of himself 2 = average

3 = child quite sure of himself. The children were rated in the following areas:

Friendship with other children in class Participation in group activities Approach to schoolwork Relationships with teacher Relationships with other adults.

It should be noted that nonresponse to the query about classroom behavior was sufficient to raise a question about the validity of this finding. 64

76

70/100%

64/100%

57/100%

45/100%

236/100%

* Nonresponse cases omitted. Actual total numbers of children were: Denver, 58; Oxford, 65; St. Louis, 70; San Francisco, 49; grand total, 242.

Total*

5/7%

2/3%

1/2%

Superior

8/3%

13/6%

4/9%

5/7%

1/1%

3/5%

High average

0

102/43 17/38%

35/50%

26/41Y

24/42%

Average

47/20'7

.

9/20%

16/23%

11/17%

11/19%

Total 66/28%

San Francisco

BEST COPY AVAILABLE

15/33%

Low average

Low

24/389

Oxford

Denver 18/32%

Class Standing

City St. Louis 9/13%

(Percentages based on total number of children in each city whose teachers responded to the question.)

AS EVALUATED BY THEIR SECOND-YEAR TEACHERS

CITY COMPARISON OF FORMER UPSWING CHILDREN'S CLASS STANDING

TABLE 12

7z1

02N

0

10

20

30

40

50

60

70

100i

1

times

Some-

28%

Frequently

14%

U

times

Some-

Frequently

15%

Amount of Disruptive Behavior, By Status Group

Never/ Rarely

20%

Never/ Rarely

48%

C

times

Some-

34%

(Percentages based on number of children in each status group whose teachers responded to question; no response for 4 children from the T group, 7 from the U group, and 6 from the C group.)

AS EVALUATED BY THEIR SECOND-YEAR TEACHERS

Frequently

18%

BEST COPY AVAILABLE

Children who had trained Upswing tutors (N 67) Children who had untrained Upswing tutors (N 45) Control group children (N=113)

FIGURE 6. CLASSROOM BEHAVIOR OF FORMER UPSWING CHILDREN,

Never/ Rarely

57%

60%

C

U

T

CJ

ab4

50/100% 61/100%

65/100%

225/100%

* Nonresponse cases omitted. Actualtotal numbers of children were: Denver, 58; Oxford, 65; St. Louis, 70; San Francisco, 49; grand total, 242.

Total*

49/100%

39/17% .

9/18%

11/17%

7/11%

12/24%

Frequently

67/30V_

15/31%

25/38%

17/28%

10/20%

Sometimes

119/53%

25/51%

29/45%

37/61%

28/56%

Never/rarely

in Class?

Disruptive Behavior

BEST COPY AVAILABLE (Percentages based on number of children in each city whose teachers responded to the question.) City . Total San Francisco St. Louis Oxford Denver

CITY COMPARISON OF FORMER UPSWING CHILDREN'S CLASSROOM BEHAVIOR, AS EVALUATED BY THEIR SECOND-YEAR TEACHERS

TABLE 13

BEST COPY AVAILABLE The individual item scores were tallied and also were summed to yiell an overall score.

Figure 7 illustrates that Upswing status had no influence on this measure of children's overall self-confidence that was discernible by their second-year teachers.

The results for individual items all were

virtually the same as the summed results.

The distributions of overall

and item self-confidence scores were about the same in each city as the distribution by status for all cities combined (represented in Figure 7). It is interesting to note that over two-thirds of all groups were considered average, with a significant minority below average.

These

overall findings agree with the self-esteem test findings, but the two measures do not correlate highly. ".onstructs of self-esteem.

Clearly we are dealing with two separate

One, the test is a standard set of criteria

for all children, and the other, teacher judgment, applies separate criteria to each child according to the teacher and his/her observational relationship to the child.

Both measures have their place, but the fair-

ness of a standard set of criteria is perhaps a more useful tool for this evaluation.

It also appears, from the statistical associations between

the teacher measures of self-confidence, class standing, and classroom behavior that the latter two variables entered into the subjective selfconfidence estimate in ways that, on the basis of self-esteem test results, may not have been warranted. Rate ol'Absenteeism.

The range on number of days absent from

school was 0-60, with a total follow-up sample mean of 9.6 and a standard deviation of 9.9.

There were modest differences between status groups in

rate of absenteeism.

Control children were a more heterogeneous group than

tutored children in terms of school attendance; the former group included enough children who were absent very frequently to increase their mean number of days absent, particularly over the mean for children who had worked with untrained tutors, as Table 14 shows.

68

80

65

*E.

1E

C

Cl)

tC

c, 0

0

10

20

30

40

50

60

70

T

Average

High

9%

Low

18%

U

Average

69%

High

12%

Low

23%

C

Average

68%

High

9%

Children who had trained Upswing tutors (N 69) Children who had untrained Upswing tutors (N 49) Control group children (N 117)

(Percentages based on number of children in each status group whose teachers responded to question. No response for two children from T group, three from U group, and two from C group.)

AS EVALUATED BY THEIR SECOND-YEAR TEACHERS

FIGURE 7. FORMER UPSWING CHILDREN'S OVERALL SELF-CONFIDENCE IN SCHOOL,

Low

23%

68%

C

U

T ,

TABLE 14

BEST COPY AVAILABLE

BASIC STATISTICS ON FOLLOW-UP CHILDREN'S RATES OF ABSENTEEISM FROM SCHOOL

(Percentages based on number of children in each status group whose teachers provided attendance data.) Upswing Status

Number of Children*

Mean Number of Days Absent

Standard Deviation

Children who had trained Upswing tutors (T)

63

8.8

8.0

Children who had untrained Upswing tutors (U)

43

6.9

5.8

109

11.2

11.9

215

9.6

9.9

Control group children (C) Total

* Nonresponse cases omitted. Actual numbers of children were: 71T, 52U, 119C; grand total, 242.

70

BEST COPY AVAILABLE

Absenteeism was higher overall in Denver and San Francisco than in oxford, particularly, and St. Louis. city.

Table 15 presents the data by

The rates in the first two cities were inflated by more absences

among control group children in Denver, and among children who had worked with trained Upswing tutors as well as the control group in San Francisco.

The data on number of days absent suggest a possibility that Upswing may have influenced school attendance to some modest extent, although characteristics of the city, or of the children in the city, were stronger influences.

If Upswing tutoring had an effect, it might be attributable

to added enjoyment of school because of the Lutoring experience; it might also, or in addition, be attributable to the fact that the schools,.volunteers, or both, frequently contacted the parents of tutored children to avoid volunteers making wasted trips to school.

Parents of tutored

children could have been made more conscious of school attendance because of the involvement with Upswing. Relationships Between Follow-Up Variables

IndirectEtypelsofila2ingStstys.

The only areas in which Upswing

status appeared to have any in1."..14-Ace in the follow-up year were classroom

behavior and absenteeism.

Control group children demonstrated slightly

more disruptive behavior and tended to be absent from school more frequently than tutored children.

However, the differences between the groups

in these respects were marginal.

Reading and the Other Measures.

Reading proficiency (as measured

by the WRAT) and tested level of self-esteem appeared to be unrelated, as reading proficiency and the teachers' estimates of children's selfconfidence in school.

Thus the hypothesized relationship between positive

self-concept and reading skill must be rejected for the Upswing follow-up population.

In assessing children's class standing, teachers apparently took into account some elements measured by the WRAT, for there was a significant positive correlation between the two (r = 0.4).

The WRAT results were not

related to any other variable under study except the child's class assignment and whether the child received special services.

71

The associations

REST COPY AVAILABLE

TABLE 15

CITY COMPARISON OF FOLLOW-UP CHILDREN'S RATE OF ABSENTEEISM FROM SCHOOL

(Percentages based on number of children in each city whose teachers provided attendance data.) City

Number of Children*

Mean Number of Days Absent

Standard Deviation

Denver

45

11.8

9.5

Oxford

63

6.8

7.7

St. Louis

59

8.8

9.7

San Francisco

48

12.4

12.2

215

9.6

9.9

Total.

* Nonresponse cases omitted. Actual total numbers of children were: Denver, 58; Oxford, 65; St. Louis, 70; San Francisco, 49; grand

total, 242.

BEST COPY AVAILABLE

found with these variables were contrary to what one would presume to be expecLed.

The decision to retain or to place a child in a learning difficulties class apparently was not based generally on deficiency in reading skill, or else the measure of reading skill used did not accord with the WRAT results. Table 16 shows that children who were retained tended to have slightly lower WRAT scores than children who were not, but the means for all groups fell at the low end of the average range, with small standard deviations. No significant difference was found in mean or variance of pretest reading scores between groups formed on type of class assignment. Table 16 also gives the reading post-test basic statistics for the type-of-class groups.

None of these group means went up significantly;

however, the mean for the group retained in first grade dropped significantly, indicating that these children lost ground in reading. Eighty percent of the 86 children who received special services had remedial reading or a reading tutor (in addition to other kinds of help in many cases).

The pretest WRAT scores of the 86 children tended

to be lower than the pretest scores of those who did not receive any special services.

Differences in pretest mean and variance for the two

groups were significant.

Those differences were maintained on the reading

post-test, which indicates that extra help in reading in the second year of school generally did not result in improved skills.

Figure 8 shows

the pretest and final score distributions for the children grouped on receipt of special services.

:ype of Class Assizment and Special Services.

The percentage of

children who were not retained and who received special educational services was twice as high as the percentage of retained children who received special services.

There was no particular difference between the other class assign-

ment groups in frequency of special services.

Table 17 presents these data.

Type of Class Assi3nment and Other Measures.

Type cz class assign-

ment had no bearing on tested self-esteem, self-confidence demonstrated in school, classroom behavior, or tcacher-assessed class standing.

73

TABL1: 16

BEST COPY AVAILABLE

READING TEST RESULTS FOR UPSWING FOLLOW-UP CHILDREN GROUPED BY TYPE OF CLASS ASSIGNMENT

Type of A:;signment

First grade Combination 1-2, primarily children w it a learning problems

Combination 1-2, normally-distributed or above-average

No. of Children, c/ of Total PopulationV

Reading Pretest

Reading Post-Test

= 84.3

35 17%

= 90.5 SD = 9.3

24 12%

= 89.5 SD = 7.6

21 10%

= 93.8

= 91.5

8.8

SD = 11.3

SD =

SD =

7.4

- 88.5

X

8.3

SD

class Second grade, primarily children with learning problems

Second grade, normally-distributed or above - average class

120/c

= 93.2 SD - 11.1

98 47%

93.7 SD = 10.5

24

=

93.bg/

SD = 12.0

=

94.5./

SD = 10.5

Percentages based on 208, the total number of children for whom we had both WRAT scores and a response from the teacher on 1-ype of class. The total follow-up population included 242 children; there was no pretest WRAT score for 26 children and no grade report for 8 children. Percentages do not sum to 100% because of rounding errors. One child excluded because of out-of-range post-test score. Two children excluded because of out-of-range post-test scores.

74

86

110

120

130

v0

60

70

.... .....

/

80

90

100

110

1.

WRAT Post-Test Standard Score

I

I

i

r I

i

I

/

I

/ i

...,_, / /

V 120

P

(Includes only children who took both tests. Percentages plotted on means for score categories.)

FIGURE 8 . COMPARISON OF PRE- AND POST -TEST READING SCORE DISTRIBUTIONS OF CHILDREN WHO DID AND DID NOT RE CE DIE SPECIAL EDUCATIONAL SERVICES IN THE FOLLOW-UP YEAR

WRAT Pretest Standard Score

100

10

10

90

15

15

80

20

20

70

25

25

60

30

30

I

35

35

I

100 I

Children who did not receive special services: N-140, Mean Score' 93.2, standard deviation 10.8

Children who received special services: N.,75, Mean Score = 87.7, standard deviation = 9.2

130

rn

CD 1""'

g

Zo MI

I

....

3

233/100/.

113/48%

* Noires7onse cases omitted. Actual number of children reported to have received special services was 86; number reported to have received no special services was 156; grand total = 242.

Tot...t1

83/100% 45/54%

Yes

Total* 150/100%

Second Grade, Normally-Distributed or Above-Average Class 68/45%

.

Retained First Grade

BEST COPY AVAILABLE

Nc

Services?

Received

Type of Class Assignment in Follow-Up Year Second Grade, Combination 1-2, Combination 1-2, Primarily Children Normally-Distributed or Primarily Children with Learning Difficulties Above-Average Class with Learning Difficulties

(Percentages based on number of children in each type of class whose teachers answered the question about special service .)

SPECIAL SERVICES TO CHILDREN IN DIFFERENT TYPES OF CLASSES

TABLE 17

BEST COPY AVAILABLE

Receipt of Srpcal Serviars and ether Variables.

Children who

received special services were on the low side in all follow-up measures

except tested level of self-esteem and degree of disruptive classroom behavior.

They obtained lower final WRAT scores, and lower class standing

and self-confidence ratings from their teachers. LIMITATION OF THE FOLLOW-UP ANALYSIS This analysis is seriously hindered by a nonrandom attrition of cltildren in the tutored and untutored groups.

Unfortunately it was not

possible to track children who left the school systems that hosted Upswing. Many children moved away, and an analysis of test scores indicates that the follow-up population is not truly representative of the original groups of tutored and control children.

This is particularly true of the

tutored group, which went down by half.

Moreover, the fact that, overall,

two times as many tutored children as controls moved away indicates that there were important demographic differences between the two groups not detected in the first-year evaluation. itinerant nature of family life, etc. ential attrition rate.

These differences might be in the The manifest effect was a differ-

It also is puzzling why, counter to the overall

trend, all of the control children in one city (San Francisco) remained There

while only about half of the tutored children in that city remained.

was no way to determine tie impact of differential losses from the comparison groups.

Thus it should be kept in mind that the follow-up results

Might be ouite 'Efferent had it been possible to track all children from the first year.

The impact of this attrition can be partially assessed through a comparison of the mean reading scores for the total population (all children completing Upswing's first year) and the post-attrition subsample.

This

comparison indicates that the attrition was concentrated among the low scorers for the control group and the high scorers for the experimental (tutored) group.

The net effect is to eliminate most differences.

77

BEST COPY AVAILABLE The self-esteem test was included in the follow -up on the assumption that initial level of self-esteem would be a randomly distributed variable in both groups, since the groups were supposed to be randomly drawn from a single population.

The hypothesis was that if the two

groups were comparable in self-esteem before Upswing, then differences after Upswing might be attributable to tutoring if it appeared that no unusual influe,Ices occurred in one or both groups during the second year.

Post-test differences between the two groups did not occur.

Differences

attributable to tutoring could have been wiped out because of attrition, as in the case of reading.

However, since reading skills and self-esteem

appeared to develop independently of each other in the Upswing population, there is no reason to assume attrition affected results in the two criterion areas in the same way.

All that can be said is that both groups of children

tended to be on the low side of average in test-observed self-esteem a year after Upswing.

It remains unknown whether the project in its first year

had any impact in this area and whether any greater benefits to tutored children endured.

CONCLUSIONS ABOUT THE EFFECTS OF TUTORING OVER TIME It is difficult to make definitive statements about the durability of tutoring benefits because of the follow-up limitations just described. However, the analysis suggests the following about carryover effects: The children who could be followed generally maintained their pretest levels of reading skill over the follow-up year (except for those retained in first grade).

If one can assume that

the attritees from the follow-up population would have doLe likewise, the tutored group

would have maintained its edge; i.e., the positive effect of Upswing tutoring on reading would have been stable over the next year.

There was nosignificant difference between the tutored and untutored groups in level of

78

BEST COPY AVAILABLE

self-esteem.

Moreover, the data give no reason

to believe that there were differences between the

two groups in this characteristic at the end of the tutoring period.

Had there been a positive correla-

tion between post-test reading score and self-esteem it might have been justifiable to speculate about

differences between status groups being obliterated by attrition.

However, neither the objective test

of self-esteem nor the teacher assessment of selfconfidence demonstrated in school showed any

association with reading level in the follow-up sample.

The only areas of actual differences between the follow-up sample groups were classroom behavior The differences were

and absences from school.

quite small--but control group children more commonly presented behavior problems in class and were more frequently absent from school.

The follow-up analysis yielded several interesting findings that

were outside the realm of the Upswing evaluation but nevertheless should be noted.

One of these is the lack of effect, possibly negative effect

of special educational services.

The data here are somewhat ambiguous

because the evaluation looked for effect on reading only and some children received special services in other areas.

However, 80% of those who

received any special service received either remedial reading or tutoring in reading (in addition to other kinds of help in many cases).

The

post-test reading score of children who received special services showed no improvement over their pretest scores.

They remained at a level sig-

nificantly lower than children who did not receive special services.

This

brings up the possibility that Project Upswing might be more beneflAal to children with reading difficulties than the traditional remedial reading approach or than other tutoring efforts.

79

BEST COPY AVAILABLE The negative effect potential is suggested by the fact that teachers evaluated children who received special services as lower in class standing and in self-confidence.

There were significant differences in both mean

and variance of these teacher ratings between the group of children who did and those who did not receive special services.

This may suggest

stereotyping.

Another issue that seems worthy of further exploration was raised by the follow-up analysis--namely, the basis for decisions to retain children or to place them in learning-difficulty classes.

It seems un-

likely that such decisions are made randomly, yet, for the Upswing followup sample, they did not appear to have been based ul'on reading skill. Here, too, the data are not free from ambiguity.

Certainly reading skill

would not be the only factor consAered in making class assignments.

How-

ever, it would seem that in the primary grades reading should be an important factor, perhaps the most important except for severe social immaturity.

There was no significant difference between the reading pretest scores of children who were retained or were in "slow" classes and children who advanced normally for their years in school.

One might

question the validity of the WRAT, which was used to measure reading in the Upswing evaluation.

Although the WRAT may not be the last word

in reading tests, it was carefully developed and standardized; moreover, it is a very straightforward mPasure of ability to decode.

There would

not seem to be much room for argument about WRAT results; either a child can decode or he cannot (except of course for contingencies like illness, examiner-introduced bias, etc.).

Thus we found it surprising, in view

of the predominance of "average" readers in the follow-up sample, that such high percentages were retained or were in classes made up primarily of children with learning difficulties.

We found it even more surprising,

even distressing, that the retained children, at least, did not demonstrate significantly lower reading skill than the others.

BEST COPY AVAILABLE Since the retained children apparently started out on an equal footing with the others in basic reading skill, it was especially dist.essing to observe thai: those who were retained showed significant loss

of skill by the end of the follow-up year.

This loss was real in an

The mean WRAT standard score of

absolute as well as a relative sense.

retained children dropped six points, with an even smaller standard deviation than for the pretest.

The foregoing tangential findings certainly seem worthy of serious consideration by the school systems involved in Upswing and by the educators in general.

Existing data on the value of special services

and on the reasons for and effects of retention certainly should be studied and perhaps further research would be in order.

DESCRIPTION OF CHILDREN INVOLVED IN PROJECT UPSWING'S SECOND YEAR There were 365 children who participated in the second year of Upswing- -181 who had tutors and 184 who did not.

The numbers of the two

Upswing status groups (tutored/control) were virtually the same in all cities--47 to 51 children in each group--except San Francisco, where there were 35 children in each group.

The groups were reasonably well matched

in each city (except for parameters later noted).

The term "city" has been

used to partition the total sample for "by-city" comparisons to assess the effect of gross environment on Upswing results. Sex

Sixty percent of the Upswing children were boys (219) and 40% girls (146).

The status groups showed about the same ratio of girls to boys as

the total population.

There were some differences in mix of the status

groups within cities.

For example, in the Oxford tutored group and the St.

Louis control group, the split was about 50/50, while in the Denver and St. Louis tutored groups, there were about 70% bcys to 30% girls.

However, since

sex of child had virtually no influence on the criterion measures, these differences are not considered important. Age.

Most of the children were between six and seven years old.

There

was a moderate tendency for control group children to be a little younger as

81

BEST COPY AVAILABLE shown in Table 18.

This difference was not important in the analysis

because the reading test results were converted to age-adjusted standard scores, and age within the narrow range of the Upswing children was found to be unrelated to the other criterion measures. Kindergarten Ex erience

Table 19 shows that about 70% of the children for whom data were available attended kindergarten.

The proportions of the two status groups

who had kindergarten experience are virtually identical.

It should be noted

that there was enough nonresponse that these data may not be valid for the population.

Ninety percent to almost 100% of both groups of children in

Denver and St. Louis (for whom there were data) attended kindergarten.

This

compares with about 807 to 90% of the San Francisco groups and only 16% to 227 of the Oxford groups. Family Background

The Test of Basic Experiences (TOBE), General Concepts subtest, was uses to obtain an indicator of elements of Upswing children's family background that might bear upon the children's progress in school.

A positive

correlation of about 0.5 has been shown between the TOBE and the Home Information Scale (HIS) derived from Richard Wolf's Environmental Process Scale. These family background measures, the latter depending upon home interview data, were designed to reflect such variables as amount of stimulation, quantity of educational materials, and parents' aspirations for their children's education.

Thus the TOBE appears to provide a reasonable proxy for family

background characteristics relevant to school achievement.

According to the

TOBE documentation, the test "undoubtedly reflects elements of socioeconomic status."

Overall, the 350 Upswing children who took the pretest tended to be significantly below averag. in terms of what the TOBE General Concepts test measures.

The Upswing population obtained a mean pretest score of 12.9, with

a standard deviation of 4.1.

The mean for the normative population (1,707 first

graders) was 19.2, with a standard deviation of 4.4. the children in the 8th percentile.

The Upswing mean places

180/101%**

358/100°/,

**Rounding error.

*Non-response cases omitted in percentage calculations. Actual numbers of children were: 181 tutored, 184 control; gran(' total, 365.

Total

178/100%

10/3%

8/4%

2/1%

Older than 71 yr.

58/16%

28/16%

30/17%

157/44%

7 yr., 1 mo. through 7 yr., 6 mo.

70/39%

87/49%

° 132/37%

1/0.3%

6 yr., 7 mo. through 7 yr.

1/1%

Control Group

59/33`%

0

Tutored

of total

population)*

Total (percentages

BEsT COPY AVAILABLE

6 yr., 1 mo. through 6 yr., 6 mo.

6 yr.

5 yr., 7 mo. through

Age Range

Upswing Status (percentages based on numiDer of children in status group)*

AGES OF CHILDREN IN PROJECT UPSWING'S SECOND YEAR

TABLE 18

I

309/100%

**Rounding error.

*Non-response cases omitted. Actual numbers of children were: 181 tutored, 184 control, grand total, 365.

180/101 'A**

37/12%

19/13%

18/11%

Don't know 158/99%**

53/17%

24/16%

29/18%

No

Total

219/71%

population) *

108/72%

Control Group

of total

(percentages

Total

111/70%

Tutored

.

1

Yes

Attended Kindergarten?

number of (percentages based children in status group)*

Upswing

UPSWING CHILDREN'S KINDERGARTEN EXPERIENCE

TABLE 19

BEST COPY AVAILABLE

There was no significant difference in mean or variance between the Upswing status groups, all cities combined.

or was there any significant

difference between status group means within any city. significant differences between cities.

There were, however,

All Denver children scores somewhat

higher (X= - 15) than those in any other city, although the only statistically significant differences were between Denver and St. Louis tutored children and Denver control group children versus the control groups in Oxford' and St. Louis. It is difficult and not :Particularly important to make sense out of these differences.

The important points to be made about the TOBE pretest results

are two: a

The results indicate that Upswing children came from "educationally-deprived" family backgrounds The results indicate that initial differences

between the Upswing status groups in family background can be considered inconsequential to the evaluation (i.e., our comparison groups are statistically equivalent). Academic Readiness

The readiness inventory used in evaluating the learning difficulties of the Upswing children is divided into three categories of true readiness behavior.

The categories pertain to visual-perceptual-motor behavior (such

as tieing shoes, holding pencil/crayon, cutting with scissors, etc.), auditory perception (e.g., rhyme recognition, ability to understand verbal messages),

and language and speech (development of speech patterns, production of sounds in correct sequence, blending, time required to speak or respond orally, etc.).

In addition, the inventory covers writing and spelling, reading, and mathematics skills that go beyond readiness.

The inventory was used (1) to help teachers structure their observations of :hildren in a consistent way on the behaviors traditionally associated with readiness ft.r. school; (2) to help tutors understand their pupils' specific

difficulties (teachers as well, of course, knew the CARE results and were given help in interpreting them by project staff whenever such help was requested).

85

Pt

97

BEST COPY AVAILABLE

It was intended to use

readiness components of the CARE in

establishing the comparability of the tutored and control groups in behaviors related to learning difficulties.

The instrument could not be used for that

purpose, however, because of its scoring mechanism.

The means for both groups

of children were outside the range of possible scores in all categories of the test.

This occurred because teachers skipped so many items in completing

the inventory, probably because (1) they described behaviors not yet relevant to the child (2) the child had no opportunity to demonstrate some behaviors, or (3) the child did not have the problems represented.

There was no provision

for "not applicable," "don't know," or "never" (displays the behavior) responses. (The behaviors are stated negatively; e.g., "Has difficulty staying within lines when writing."

The response choices are "Generally" and "Sometimes.")

In addition to out-of-range means, the standard deviations of the scores were nearly as large as the range of possible points. OBJECTIVES OF THE SECOND-YEAR ANALYSIS OF TUTORING EFFECTS This part of the analysis of the second year of Upswing had the following objectives: 1.

To verify the effect of tutoring on tested reading skill found in the initial evaluation.

2.

To obtain a more reliable measure of children's self-esteem and test the effect of tutoring on self -es teem.

3.

To determine if children's visual-motor integration skills would improve under tutorinc if tutors specifically directed part of their work with children to that goal.

4.

To continue study of relationships between the three major criterion areas.

DATA COLLECTION PROCEDURES AND SOURCES IN THE SECOND YEAR Children were identified by their teachers as candidates for Uln.wing tutorin:, and evaluated by the teacher on an acadumic readiness inventory

(the previously described Cegelka Academic Readiness Evaluation, or CARE).

86

98

BEST COPY MAILABLE The children were then administered a pretest battery that included two individualized instruments:

the WRAT and the "Funm. Faces Game" (self-

esteem inventory); and two individual- or small-group-mode instruments:

the

Beery-Buktenica test of visual-motor integration and the Test of Basic Experiences. (The same battery was administered at the end of the tutoring period.)

Teachers and tutors were asked for their opinions about the general benefits to children of Upswing tutoring on a questionnaire distributed near the end of the tutoring period. The teachers and tutors also were asked for their assessments of both tutored and untutored children's self-confidence via "Student Profiles" distributed with the final questionnaires.

These

forms were mailed by on-site data coordinators or given to respondents at Ppswing meetings.

They were returned directly to the evaluator.

The data

coordinator and project staff in each city conducted follow-up by telephone or personal contact at the Upswing schools or in meetings as notified of nonrespondents by the evaluation. STATISTICAL HYPOTHESES USED IN THE SECOND-YEAR ANALYSIS The analysis looked at the distributions of measurements in the three criterion areas listed above, for differences between the groups of tutored and untutored children. 1.

The hypotheses were:

That the tutored groups would make greater gains in reading, self-esteem, and visual-motor integration test scores over the tutoring period than the control group, with a difference between the group means for each test significant at the 0.05 level and a variance ratio significant at the 0.05 level (a = .05).

2.

That, for all tests, there would be a significant negative relationship between the children's pretutoring scores and the amount of gain shown on the post-test, with a correlation coefficient of 0.30 or higher, at a = .05.

3.

That there would be no significant correlation between change in tested reading level and change in tested level of visual-motor integration skill, at -t = .05. 87

o;

t,`

99

1

BEST COPY AVAILABLE 4.

That there would be a significant positive relationship between tested final level of self-esteem and change in both reading and visual-motor integration score, with correlation coefficients of 0.30 or higher, at a = .05.

5.

That there would be significant positive relationships between final self- esteem test score and both

the teacher and tutor assessment of child selfconfidence, with a correlation coefficient of 0.30 or higher, at a = .05.

RESULTS OF THE ANALYSIS OF TUTORING EFFECTS ON CHILDREN IN THE SECOND

YEAR OF UPSWING The results in each criterion area are presented separately with discussion of influences of the independent variables that were studied. relationships between criterion variables are explored.

Then

Finally, conclusions

are drawn about the second-year results, with reference to the first-year and the follow-up findings.

The Children's Reading Skills Before Upswing Only about a quarter of the Upswing children had initial reading test scores that reflected notably deficient skills.

About 35% scored in the lau-

average range, 40% in the average range, and 2% above average before tutoring began.

The percentage of children in each score category is given in Table 20.

It should be remembered that tests do not measure all aspects of reading.

It

ie quite possible for a child to know how to read but perform poorly in class because of shyness, etc.

Such children may have been recommended to Upswing

frequently, especially if they had other problems in class.

It appears from Table 20 that, although Upswing's primary tutoring goal was to help children overcome reading defizits, reading or reading-related behaviors may not have been the primary factors in selecting children for the project.

As will be demonstrated in this section, visual-motor integration

problems may have had more to do than reading problems.

children being identified for Upswing

Also, it way shown in the description of child char-

acteristics at the beginning of this analysis that the children tested quite

88

100

I

40/24% 10/6%

Inferior (70-79)

Defective (69 and below)

157/1003

6/4Y

23/15%

60/38%

63/40%

5/3/

Control Group

323/101*

16/5"/

63/20 /

112/35/,

126/3; ;._

6/2/

Total*

*Children who did not take the pretutoring WRAT, or whose scores could not be used because of error, or could not be converted to standard scores because age data were not available, were omitted from totals. There were actually 181 tutored and 184 control children in the project when it began; grand total, 365. **Rounding error.

166/100%

52/31%

Low average (80-89)

Total*

63/38%

1/1.-/,

Tutored

Status in Project Upswing

Average (90-109)

High average (110-119)

Score Category

W RAT Otandard

COMPARISON OF THE PRETUTORING READING LEVELS OF TUTORED AND CONTROL GROUP CHILDREN

TABLE 20

BEST COPY AVAILABLE low in basic experiences related to family background/socioeconomic status; evidently, these descriptors represent one or a cluster of primary selection These attributes are all highly visible in children; generally,

factors.

individual observation or diagnosis is not necessary to detect symptoms of at least some alements of VMI deficiency, and family background or socioeconomic characteristics usually are evident in children's dress and language patterns.

To determine whether first-grade children have acquired basic

reading skills, some kind of organized individual diagnostic effort must be made.

Sixty percent of the Upswing children did demonstrate low average

or lower reading skills, and the Upswing analysis gives several reasons to believe that without the project they would have remained there, or declined. However, it is interesting that teachers apparently did not focus on reading problems when selecting children.

Despite the "whole-child" approach that

distinguishes Upswing from many tutoring projects, indications were that the participants believed that children were chosen for Upswing because of reading problems.

Effect of Tutoring on Children's Reading Skills

The grouping of scores into the WRAT interpretive categories, as in the preceding Table 20, obscures a critical difference between the tutored and control group children.

Figure 9 reveals this difference, namely, that

the children who were not tutored made higher scores on the reading test both at the beginning and end of the project.

The pre- and post-tutoring dif-

ferences between the groups, mean and variance, were found to be statistically Both groups made significant progress in reading, and made

significant.

about the same amount of progress (8 points in standard score); thus, the control group's initial lead was maintained over the year.

These data

suggest that: I.

Children were not necessarily assigned to groups randomly, as was supposed to have been done.

It

appears that project staff must have been influenced in many instances to give tutors to children who demonstrated the greatest problems.

90

102

Score

WRAT Standard

41

60

80

100

120

160

,1

Immo

1972

Nov.

1

1

Jan.

Dec.

MMEW

1

M.N.*

Feb.

IMMEMIr

1973

Mar.

iiMMEMD

Apr.

MMEW

MEMED

May

x= 94.0, p = 10.7

MEINNIP

x = 97.3,p- 11.9

FIGURE 9. COMPARISON OF READING TEST RESULTS IN THE SECOND YEAR OF UPSWING FOR TUTORED AND CONTROL GROUP CHILDREN

I---

Oct.

x = 86.1, p = 11.7

MEM. MMEM

x = 89.3, p - 11.4 WIENS WM."

MEMMIr WINN.

Mean Performance for Age Based on Standardization Sample

Control

TO tor ed

BEST COPY AVAILABLE 2.

Some kind of "treatment" was given control group children.

The explanation for the consistent and

substantial gains among control group children will be developed as the section progresses.

The expected negative correlation was found between initial reading test score (r = 0.44, significant at the 0.001 level).

However, this tendency

was not strong enough to reduce the margin between the status groups at the end of the year, even though tutoted children more commonly had lower initial scores.

City Differences in Reading Test Results.

From a look at the data for

the individual cities, one finds that the difference between the initial test score distributions of the comparison groups is attributable to children in St. Louis and San Francisco.

As shown in Table 21, the control children in

St. Louis had a mean reading pretest standard score 7 points higher than that of the tutored children. points higher.

The San Francisco control group mean was about 6-1/2

There was virtually no difference between the groups' starting

levels in Denver, and the 2-point difference in Oxford, also in favor of the control group, was not statistically significant.

From the last column of Table 21, the control group children in all cities except St. Louis obtained mean change scores slightly higher than the tutored children.

The between-group differences in Denver, Oxford, and San

Francisco were not statistically significant.

The control childrer in St.

Louis made a minimal mean gain of just under three points, slightly less than the mean gain made by all control children in the first year of Upswing (four points) and slightly more than the mean gain made by the St. Louis control children in the first year (ont. point).

The control children in St. Louis

started out with higher reading scores, and they tended to maintain the same level of functioning in reading over the school year, while tutored children tended to raise their reading levels and caught up with the control group. In short, no treatment effect among St. Louis control children is evident. The St. Louis project model appears to account for that city's unique results. desig,

There, the second-year project was much closer to the original

than the second-year project in the other cities.

92

104

The St. Louis staff

Mean

86.1 92.6

San Fre_ncisco Tutored (N=33) Control (N=31)

Tutored' ( :: -, 44 )

81.2 88.3

37.7 88.9

88.8 88.1

Standard Score

:::cr.trcl (N-39)

St. :.cuis

Control (N=48)

0:.;:crd Tutored (N=46)

Tutored (N=43) Control (N=39)

Do ever

by City

Status Group

14.4 12.9

11.2 9.8

9.4 11.9

10.8 10.7

Standard Deviation

Pre-Tutoring

.

91.1 100.1

91.1 91.1

96.7 100.1

96.4 97.7

Standard Score

Mean

7.6 14.2

9.3 9.0

9.3 9.9

13.7 12.4

Standard Deviation

Post-Tutoring

.

5.0 7.5

2.7

9.3

9.0 11.3

7.6 9.6

Score

Mean No. of Points Change in Standard

_

9.7 12.7

8.7 7.6

6.8

8.0

11.2 8.6

Standard Deviation

Change in Score Pre to Post

READING TEST RESULTS FOR TUTORED AND CONTROL GROUP CHILDREN IN EACH CITY

TABLE 21

BEST COPY AVAILABLE visited the schools frequently, but their interaction was primarily with the tutors on an individual basis.

Their involvement with teachers was primarily

a "Hello, are there any problems about Upswing," kind of thing.

There was

The point is that,

no formal training and minimal orientation for teachers.

although Upswing ran smoothly in St. Louis, the project apparently was not felt as strongly as an entity in the schools and did not involve teachers as much or in the same ways as the projects in other cities.

The data indicate

that teachers made the difference for the control group in other cities, and further, that the teacher effect can be attributed to Upswing.

These obser-

vations will be developed more fully through the remainder of the report. 'lumber of Hours of Tutoring.

varied frog 4 to 50 hours.

The amount of tutoring children received

Half of the children were tutored 35-50 hours;

37% were tutored 20-34 hours.

The Upswing data suggest that 20-24 hours may

be the critical amount for influencing reading skills. lea

Children who received

gained only from 1-2 points (mean) on the post-tutoring WRAT.

The amount

of gain at the 20-24 hours mark jumped to a mean of 9 points and remained at about that level regardless of additional hours of tutoring. It should be noted that factors other than tutoring time undoubtedly are involved.

For example, children who received very little tutoring and

made negligible reading test improvement may have been absent from school a great deal because of illness or home problems.

Still, a trend is evident

in the data.

Influences on Reading Achievement of the Independent Variables Selected for Study Eight independent variables were selected as feasible to measure and as 'paving potential to influence reading test outcomes: 1.

Child's sex

2.

Child's age

3.

Whether child attended kindergarten

4.

Child's background of basic knowledge, an indicator of family characteristics relevant to school performance

94

106

BEST COPY AVAILABLE 5.

Child's initial level of reading proficiency

6.

Child's initial readiness skills

7.

Number of hours of Upswing tutoring child received

8.

Special services other than Upswing tutoring child received.

Although the status groups were comparable on the other independent variables (except initial reading level), all were checked through analysis of covariance for influence on change in reading level (1) independent of Ups';ing status and (2) interactive with Upswing status.

For example, (1)

Without regarc. for Ups-Ting status, did children who received special educa-

tional servicen other than Upswing tutoring tend to make greater gains in reading than children who did not?

(2) Did tutored children who received

other special services tend to make greater gains than control group children who received special services?

(3) Did kindergarten experience, starting

level of basic experiences or initial level of self-esteem bear on progress in reading?

The only such influence found was in the area of special services,

specifically, remedial reading; its influence was small. About a third of all the Upswing children received something outside the project (most commonly remedial reading or speech therapy).

Tutored

Upswing children more often got extra help than untutored, by a small margin (35% versus 29%).

There were sharp differences between cities, but, within

cities, except Denver, the school systems were even-handed in giving extra help to the two groups.

In Denver, half of the tutored group, versus 27% of

the untutored received extra help (thus the difference between comparison groups project-wide).

Only six Upswing children in St. Louis received special

services from the school system (four tutored and two control). There is a good argument that special attention of any kind may have far-reaching effects not necessarily logically related to the nature of the attention.

Thus we considered impact on reading of special services without

regard for type.

10 impact was found among either tutored or untutored Upswing

children.

We then looked at the impact of remedial reading only. was given to a total of 64 children: 30, or 16%, of the untutored.

This service

to 34, or 19%, of the tutored and to

The initial reading test scores of these two

95

'107

BEST COPY AVAILABLE subgroups were very similarly distributed and tended somewhat to be lower than the initial scores of the children who did not get remedial reading. The pretutoring mean score for the remedial group was 84 (in the middle of the low average range), with a standard deviation of 12.6. These data compare with a pretutoring mean of 87.6 for the total population--86.1 for all tutored and 89.3 for all control).

There was a significant difference between the distributions of change in reading score.

The mean gain for the remedial reading children

was 9.8, with a standard deviation of 11.4.

The mean gain for those who

did not get this service was 7.7, with a standard deviation of 9.1.

These

data indicate that remedial reading tended to have a positive impact, but that impact was marginal.

The frequency distributions indicated that some-

what more children in the remedial reading group made great progress, as reflected in the slightly higher mean and larger standard deviation for that group.

Comparing tutored and control group children who had remedial reading, the former gained 8.6 points in standard score on the final test (standard deviation 11.4) while the latter gained 11.3 points (standard deviation 11.3 as well).

Thus it appears that control group children tended to benefit a

little more from remedial reading than those with Upswing tutorq. be a case of "too many cooks" for the tutored children.

It might

However, the

difference is small.

It appears that Upswing involvement (not necessarily having an Upswing tutor) was as effective, or nearly so, as remedial reading for most children. Further, there apparently was no advantage to having both an Upswing tutor and remedial reading, in fact, the combination apparently had a negative effect in some cases.

The Independent Variables and Initial Reading Score.

Among the eight

variables at the beginning of this discussion were four that might have been expected to be related to pretutoring reading level if not to change in reading level over the year.

These variables were:

child's sex, whether the

child attended kindergarten, child's background of basic knowledge, and initial readiness skills.

The last could not be evaluated as explained previously.

96

108

BEST COPY AVAILABLE Of the others, only background of basic knowledge was found to have anything Lo do with initial reading level as measured by the WHAT.

There was

just under 20% associated variation between the basic experience and reading pretests.

Level of Self-Esteem Before Tutoring Began3

With data from all cities combined, there was no significant difference between tutored and untutored children in initial mean level of self-esteem. Both groups means fell in the "borderline problem" interval.`'

Although there

was considerable variance in the distributions (unlike the WRAT), the percentageF of children in each score category are roughly equivalent.

Table 22

shows that there was some tendency for control group children to score higher, but as just noted, the differences were not statistically significant.

The Oxtord children had the lowest initial self-esteem scores, with a mean for both groups of about 8 (standard deviation roughly 16.5).

Denver

and St. Louis control group children had comparatively high initial means (18.2 and 17.3, respectively), which placed these children in the range of average self-esteem (16 to 35 points) before tutoring began.

The Denver

and St. Louis tutored groups obtained initial means of 15.1 and 10.4, respectively (borderline problem).

The picture at the beginning of tutoring, then was a tendency toward below average self-esteem, mediated by considerable variation in scores (mostly in a negative direction).

Substantial self-esteem problems were

indicated for about a third of both groups and "superior adjustment for 5% to 7%.

Oxford children scored low; and the self-esteem results in St. Louis go

along with the reading results in indicating nonrandom assignment of children and tutors (i.e., lower children appear to have been assigned to the tutored group).

3

4

San Francisco children are excluded from the self-esteem test analysis because the pretutoring results were lost in the mail. The self-esteem measure--"Funny Faces Game"--is still under development. However, it was pilot tested with all first-grade children in a Prince Using that group as a preGeorge's County, Maryland, public school. liminary normative population, categories for score interpretation were established. The mean score for the normative group was 23, with a standard deviation of 11. The pilot test was conducted in September 1972.

97

r

'1

100

BEST COPY AVAILABLE

TABLE 22

COMPARISON OF PRETUTORING SELF-ESTEEM LEVELS OF TUTORED AND CONTROL GROUP CHILDREN

Self-Esteem Score Category

Status in Project Upswing

.

Total Tutored

Control Group

6/5%

8/7%

14/6%

Average (16 to 35)

46/36%

50/41%

96/38%

Borderline problem (6 to 15)

32/25%

23/19%

55/22%

22/17%

27/22%

49/20%

21/17%

15/12%

36/14%

127/100%

123/101%*

Superior adjustment (36 to 41)

Problem

(-5 to 5) Serious problem (-6 and below) Total

,

1 1

*Rounding error.

98

110:

250/100%

BEST COPY AVAILABLE

Kindergarten attendance, family background (as measured by the test of basic experiences), and initial reading level were related to initial self-esteem.

Children who attended kindergarten had a mean pretest self-

esteem score of 14.5, while children who did not had a mean pretest score of 7.2.

This result is.largely attributable to Oxford, where few children

attended kindergarten and where the lowest initial self-esteem scores were recorded.

These data are insufficient to conclude that kindergarten increases

children's self-esteem.

However, a fair portion of the test emphasizes com-

fort in social situations, which is presumably improved by the first school experience.

Kindergarten attendance bore no relation to amount of change

in self-esteem; thus it appears that if there is an influence, it may be rather quickly overridden.

Effect of Tutoring on Self-Esteem Figure 10 compares the development of self-esteem as indicated by preand post-tutoring test results for tutored and control group children.

The

results are comparable to those for reading, except that there was no significant margin between the two groups. groups of children gained in self-esteem.

For the project as a whole, both

The mean gain for tutored children

was about 7 points versus about 5-1/2 points for the control group.

Thus,

there was a tendency to reverse the slight initial edge of the control group (a tendency contributed by the St. Louis project, as will be discussed shortly). The gains were sufficient to bring the means from the borderline category into the average categ9ry, albeit at the low end of the average range. As for the individual projects, Oxford and Denver w(re very close to the overall pattern, although the Denver contrc.. group averaged very slightly

greater ga4n (the Oxford tutored group, on the other hand, augmented its slight initial lead).

These differences are negligible.

The St. Louis pattern,

however, is significant, particularly in relation to the reading test results. Once again the control group made minimal progress: 1.9 points with a standard deviation of 11.6). points with a standard deviation of 18.3.

mean change in score of

The tutorei group gained 7.1

The difference could in part be

due to a regression effect, since the control group started 'with a mean in the

average range while the tutored group mean was "borderline."

99

111

However, such an

problem area

SELF-ESTEEM borderline SCORE

average

49 to 0

2

4

6

8

10

12

14

16

18

20

41

i

Nov

Oct

I

Jan

I

Dec

GINNED

moo

Air-

Mar Feb

1973

I

I

...so

I

May

I

11

X = 19.2 p = 13.7

em ..... ...." ..... a... .00 ..... x = 18.4 p = 13.2

I

....

ow* ...."

Tutored (N = 127) Control (N = 123)

FIGURE 10. COMPARISON OF SELF-ESTEEM TEST RESULTS FOR TUTORED AND CONTROL GROUP CHILDREN

1972

I

I

p = 15.6

i = 11.3

006

p = 14 9....

ii- = 13.8

GIIIM

BEST COPY AVAILABLE

BEST COPY AVAILABLE effect was not evident in Denver.

There, the tutored group started "border-

line" and made :,?eo gain than the control group, which started average.

Effect of Other Study Variables on Change in Self-Esteem Children who tested higher initially in basic experiences (a proxy for family background at this age level) showed some tendency toward more progress in development of self-esteem.

However, only about 5% associated variation

was found between the two variables.

All of the other variables considered

in relation to reading were considered as well in relationship to self-esteem and none, including amount of tutoring and special services outside Upswing, showed any relationship to development of self-esteem.

The teacher and tutor ratings of children's self-esteem at the end of the school year tended to be in line with the final self-esteem test results, which lends credence to their validity.

The correspond'nce was by no means

perfect; low but significant correlations were obtained between the test outcome and both the teacher assessment (r = 0.24, a = .001) and the tutor assessment (r = 0.21, a = .05). Initial Level of Visual-Motor Integration Skill

The tutored and untutored children were virtually identical in initial level of visual-motor integration skill as measured by the Beery-Buktenica teat.

The pretest means (raw score) were 8.7 tutored and 8.9 control, with standard deviations of 2.4 and 2.3, respectively.

These statistics indicate a low level

of VMI skill for the children's age range.

A raw score of 9 on the VMI converts

to a chronological age equivalent of 5 years 3 months for both males and females.`'

The range of VMI raw scores was from 1 (age equivalent 2 years, 10

months male and female) to 14 (age equivalent fi years, 10 months male, and 6 years, 7 months female).

The Upswing children all (except one) were at least

6 years old at the time of the pretest, and 63% were over 6-1/2 years old. Thus there was a strong tendency for these children to show delayed development of visual-motor integration.

r'

At some points along the raw score continuum, the comfersion yields a slightly different age equivalent for boys and girls. The practice of disregarding sex in the Upswing analysis is justified by the even mix of girls and boys in the tutored and control groups..

101

113

BEST COPY fivII:vvitE

Of all the independent variables considered in this study, only basic experiences/family background had clearly any discernible relationship to initial VMI score.

(The VMI data were not compared for different age groups,

since that influence is known.

The Upswing age range is quite limited, and

the comparison groups were similarly distributed on age.)

There was about

just under 20% associated variation between initial VMI and initial TOBE scores.

This finding could well be due to a cultural differential in child

rearing practices.

Recent inquiry in this area has pointed out the importance

of nutrition and early exercise on development of VMI.6 Effect of Tutoring on Visual-Motor Integration

Tutoring once again had no impact on development of skills in the criterion area.

Both groups of Upswing children gained, but only marginally.

At the end of the school year the tutored children's mean VMI test score was 10.6, with a standard deviation of 2.6; the control group mean was 11.1, with a standard deviation of 2.1.

These means translate into an age equivalent of

about 6 years for males and 5 years, 10 months for females.

The children

still were from 6 months to 18 months below expectancy in visual-motor integration, although the difference between the pre- and post-test means represents about 9 months' growth in about 6 months.

This is an improvement over the

first year and suggests the groups were beginning to move toward expected level.

If the effect were continuous, we might expect the children to catch

up to peers in two or three years.

The amount (number of hours) of tutoring a child received apparently was unrelated to development of visual-motor skills over the year.

It is

possible, even probable, that most of the VMI improvement is attributable to a natural process of late maturing.

The test scores merely reflect a popula-

tion average, while many Upswing children appear to be 'late bloomers."

Further,

it appears that there were minimal if any benefits from emphasis on visual-

6

See Herbert G. Birch and Joan D. Gussow, Disadvantaged Children: Health 7utrition and :khool Failure. New York: Harcourt, Brace and World, 1970.

102

114

BEST COPY AVAILABLE motor activities in tutoring.

Ninety tutors said on the final questionnaire

that they and their pupils had "frequently" worked on activities to build visual-motor integration. "sometimes" did so.

Five said they "rarely" did so and 27 said they

If we assume that children who were worked with frequently

had more obvious problems than children who were never or rarely worked with,

we might expect, due to regression effects and treatment effects, that the "frequent" group would make significantly greater gains.

Figure 11 compares

the amount of change in VMI test score for the "frequently" and "rarely/ sometimes" groups.

The former group showed greater dispersion of change in

score, which could indicate both greater problems among children who frequently How-

worked on VMI skills in tutoring and some benefits of this "treatment." ever, the differences in the two distributions are marginal and could be attributable to influences other than tutoring activity.

The authors of the

VMI test define the domain of such problems and suggest general forms of corrective activity.

Although the Upswing tutors did not follow the test manual

in working on VNI, observation indicates they covered much the same ground. Visual-motor problems and corrective exercises were given substantial attention in Upswing training sessions.

The activities used in tutoring included color-

ing, cutting, tracing, putting puzzles together, tactile and kinesthetic games, writing and activities related to development of writing skills, etc. The two years of Upswing indicate that visual -motor integration problems

are difficult to resolve and are unlikely to be resolved by the efforts of volunteer tutors.

However, if one car,. assume that remedial reading and atten-

dance at a diagnostic learning center would involve work on visual-motor integration for children with problems in that category, then 75 Upswing children worked on VMI with sometone besides their tutor or teacher.

About two-thirds

of the children who received professional special services received one of those two kinds.

Yet special services also had no impact on development of

VMI skills.

The Upswing data also indicate that VMI problems did not impact children's self-esteem or hinder their development of reading skills.

Beery

obtained correlations between VMI and reading test results on the order of 0.4 to 0.5 for children from both low and mid-level socioeconomic backgrounds. However, the reading test used was from the Metropolitan Series.

103

115

The

Category

% in Change

10

20

30

40

100

-3 -1

0 1

3

4

5

Number of Points Change in VMI Score

2

21%

6

7

8

9

(Percentages based on numbers of children who took both the pre- and post-tutoring VMI and whose tutors responded to the questionnaire item on extent of VMI activity.)

FIGURE 11. COMPARISON OF CHANGE IN CHILDREN'S VISUAL-MOTOR INTEGRATION SKILLS ACCORDING TO HOW MUCH TUTORS SAID THEY WORKED ON SUCH SKILLS WITH THE CHILDREN

-2

22%

28%

(N= 90)

Frequently worked on VMS skills

Rarely or sometimes worked on VMI skills (N = 32)

BEST COPY AVAILABLE Upswing analysis found this measure to be ineffective for Upswing-type children; when TU. problems are present, the scoring system on the Metropolitan caused such error variance that the scores were more indicative of VMI than reading.

Thus we feel Beery's correlations may have been spuriously

high.

Effects of TutoringLon the Measure of Basic Experiences The Test of Basic Experiences proved to be an interesting measure that had more in common with all three of the criterion measures used in the evalThese relationships will be explored

uation than they had with each other.

shortly as the final element of the analysis of tutoring effects.

Before we

turn to that, however, the TOBE results deserve attention in their own right. It was stated in the description of children involved in this analysis that the TOBE was adopted as a proxy measure of family background characteristics relevant to school achievement.

The pretutoring TOBE scores of the

Upswing comparison groups were similar and the test results reflected (relative to the test's normative population) a paucity of environmental elements conducive to high performance in school. Both

Figure 12 presents the pre- and post-tutoring basic statistics. groups of Upswing children made meaningful gains on the 'MBE; the control

group mean score went up about 1 point more than that of the tutored group, but this difference in amount of change was not significant.

The differences

between the two groups' final scores however, were statistically significant. The slightly greater gains made by the control group had an impact when added to their initial lead.

This statistical distinction is less important than

the fact that both groups improved.

Percentile ranks give an indicator of

the kind of progress toward the norm that children made.

The final mean raw

score of the control group fell at about the 30th percentile; the final mean score for the tutored children fell at about the 22nd percentile.

The initial

means were at the 8th and 7th percentiles, respectively. This outcome was surprising since the TOBE documentation indicates improved scores are not anticipated without test-specific instruction.

One

might suppose that the tutoring relationship is particularly conducive to expanding a child's experience base; indeed, the goal was encouraged in all tutor training.

Yet amount of tutoring had no relationship to change in

105

117

TOBE Raw Score

0-10

11

12

13

14t

15

16

17

18

19

20

21

,

281

Nov.

Oct.

. Dec.

1

#00"

.00

1

Jan.

#00"

00"

Feb.

000

00"

1973

Mar.

4.10"

000

400'

Apr.

#00"

.000

de

Includes all children who took pretest (177 tutored, 173 control) and all children who took post-test (166 tutored, 159 control)].

May

x = 16.4 p = 3.9

oso'

x = 17.7 p = 3.5

FIGURE 12. COMPARISON OF BASIC EXPERIENCES TEST RESULTS FOR TUTORED AND CONTROL GROUP CHILDREN

1972

1

.00

1

= 12.5, p = 4.2

= 13.2, p =

also

.00

4.10.

.00

50th PERCENTILE

Control

Tutored

BEST COPY AVAILABLE

TOBE score; moreover, the gains of the untutored children remain to be explained.

The authors of the test found no impact of school itself on

change in score.

Further, if school itself made the difference, one would

expect kindergarten to make a difference in initial TOBE scores; it did not. It is true that teachers as well as tutors were given an opportunity to look at the tests used in Upswing and were given the results of the pretutoring battery.

However, no instructional program was worked out based on TOBE

data unless individuals did so on their own. We have posited that the influence of the project resulted in a different kind of teaching that caused control group children to gain as much as tutored.

The evidence for this is not so clear in the TOBE data as in the

reading and self-esteem data.

A difference in project presence, and there-

fore in project impact on teachers in St. Louis, was proposed to explain the lack of gain among control group children's tested reading and self-esteem. However, St. Louis children gained about the same on the TOBE as did the tutored (in fact the control group gain was a fraction greater).

Thus, the

TOBE results remain somewhat enigmatic. Relationships Between Criterion Measures AO.

There were low but significant correlations between all pairs of criterion measures before tutoring. ship to other measures initially.

The TOBE showed the strongest relationThere was about 20% associated variation

between the initial TOBE and both initial reading and visual-motor integration scores, and about 10% associated variation between the initial TOBE and self-esteem scores.

All sets of starting scores tended to be on the low

side in reference to normative data, although they were not as low as might have been anticipated except for the VMI and, particularly, the TOBE. When the effect of initial score on amount of change in self-esteem and visual-motor integration skill was controlled, there appeared to be a positive relationship between the amount of progress children made in development of those skills over the year and the basic experiences (as measured by TOBE) children brought to the program.

However, the associated variation

observed was miniscule (4%-5%) .

The characteristics measured in Upswing seemed to develop independently of each other.

The only areas in which amounts of change showed any associations

107

119

BEST COPY AO liLfAE were reading and basic experiences. be tenuous.

Here again the relationship appeared to

We obtained a correlation coefficient of 0 1; although it tested

as statistically significant, the value is too low to be important. In light of the weak albeit statistically significant associations between the initial test results (while scores on all tended to be low), and in light of the lack of association between tests in terms of gains, it can only be concluded that reading, self-esteem, and visual-motor integration are discrete areas of development for children like those in Project Upswing. It also appears that, although the basic experience factor, or family background, bears on level of functioning in all of these areas before formal schooling, the influence of background does not necessarily determine who makes the most progress in development of reading skills or self-esteem. Nor do basic experiences or background factors seem to be primary in visualmotor problems or their resolution.

The Upswing experience suggests that improvements in the areas studied have to be valued individually, for their own sake if they are to be valued at all, just as does kindergarten attende,srm.

For Upswing-age children, there is

no reason to assume from these data that any of the criterion attributes is a key to development of any of the others.

CONCLUSIONS ABOUT THE EFFECTS OF UPSWING ON CHILDREN IN THE SECOND YEAR The analysis found clearer evidence of more substantial progress made by children in the second year of Upswing than in the first.

However, children's

progress in the second year was not attributable necessarily to tutoring itself. Both the tutored and untutored groups of children develop significantly in reading, self-esteem and basic experiences.

There was an indication of growth in

both groups in visual -motor integration skills--an area in which there was

virtually no progress in the first year.

It is clear that Upswing had impact in its second year, but itwas impact due to the presence of the Upswing program, not just tutoring.

Upswing

seems to have altered the school environment in a way that benefitted children. The evidence for this is as follows. During the first year, the project had a low profile in the schools. Teachers were offered only brief orientation, which many did not attend.

108

120

They

BEST COPY AVAILABLE

expressed confusion about the goals and organization of the project.

There

were no Upswing liaison personnel working regularly in the schools except in San Francisco, where the liaison activity was concentrated in one school. short, the project lacked preaence.

In

Further, teachers were given only broad

guidelines for selecting children (average Ip, with minimal learning difficulties

was the stated combination of selection factors), and generally did not know which children were in the Upswing control group.

(Although they knew which

children were assigned tutors, the teachers apparently did not necessarily connnect "no tutor" with control group status.

The test factor may have been

confusing because testing in the first year was intended to be a screening device.)

In the second year, teachers were given training in child observation

and were given a set of behavioral descriptors to use, as a guideline in making selections.

Regardless of the quality of the training or the appropriateness

of the behavioral guideline, they served on function:

to focus teachers atten-

tion on the characteristics and needs of certain individual children.

In

addition, teachers were explicitly told which of their pupils were in the Upswing control group; in two of the four cities they received Upswing training throughout the year; and in all cities, an Upswing staff member visited each school at least once a week. As the foregoing suggests, the conclusion is that control group gains in the second year are attributable to teacher efforts with those children. The greater or different kind of teacher work with Upswing control children in turn appears to be attributable to increased teacher awareness of involvement in a national-scale pilot project, to increased attention to teacher needs and preferences by Upswing staff, and to the training offered teachers by the project.

Teachers did not always value Upswing training; however its possible

import is reflected in the fact that 60% of the teachers said they had no training in child development outside of Project Upswing.

The data show that the St. Louis control group made very minimal progress in the second year as in the first, while tutored children made significant gains, both years (although greater gains in the second). traced to project characteristics.

It appears that this can be

There was no teacher training program in

1 09

r:121

St. Louis.

BEST COPY AVAILAB LE Teachers were given project orientation individually, which included

discussion of the child characteristics relevant to selection of children for Upswing and use of the behavior inventory as a selection aid.

In the schools,

Upswing staff apparently focused their efforts on providing assistance to tutors, although staff maintained friendly relationships with teachers and checked on their satisfaction with the tutors.

St. Louis teachers generally were quite

satisfied with Upswing, but it seems that they were not involved in the same way as teachers in other cities.

It also seems that the project's identity

was different from and not so strong as it was in other locations. It is possible that teachers:

Intentionally compensated to control group children

Were made to feel special, and speciallyobserJed by involvement in Upswing and therefore performed better?

Benefitted professionally by their involvement through increased awareness of the need for and methods of child observation and prescriptive instruction

Began to believe more in the children's capacity to improve because the teachers made a conscious decision that each child had potential for normal functioning in the classroom.8

More than likely all of these explanations were operative. As for the children, tutored and untutored alike, we still do not know whether the progress they made is attributable to the simple fact that they were given special attention or to instruction.

Both vere no doubt involved,

Number of hours of tutoring a child

but in what proportions we do not kncw.

7

Something like the classic Hawthorne effect: F.J. Roethlisberger and Cambridge, Massachusetts: W.J. Dickson, 1970. Management and the Worker. Harvard University Press,

9

R. Rosenthal and L. Jacobson, 1968. York: Hold, Rhinehard, & Winston.

P?f(rmaZion !:n the Classroom.

110

122

New

BEST COPY AVAILABLE received was positively associated with test-observed reading gains (there This

seemed to be a minimum number of hours required for significant gain).

suggests instruction could have been more important in development of reading skills.

However, number of hours of tutoring apparently had nothing to do

with gains in self-esteem, visual-!otor skills, or basic experiences.

This

suggest,; that attention, with a strong classroom factor, could have been the more important variable in these areas.

The foregoing may seem to suggest that the work of the tutors was of little or no importance.

That is not our intent.

Tutoring itself had sig-

nificant impact in the first year and there is no reason to believe it did not in the second.

It is just that the untutored children got a share of

Upswing benefits from another source, and tutored children as 'ell may have received benefits of involvement from sourcws outside the tutoring relationship.

One additional piece of evidence for project impact in the second year comes from the follow-up data.

In the year following Upswing involvement,

neither the former tutored group nor the former control group increased its rate of progress in reading; in fact, there was a suggestion of a possible decline in skills for both groups.

This was true even for children who

received remedial reading in the follow-up year.

111

123

V.

UPSWING TRAINING AND PROJECT MANAGEMENT

The analysis identified two essential attributes of a successful in-school volunteer program:

Active leadership in the schools Continuing personal interaction between the leadership and the participant grogps.

The meaning of these two summary statements is developed in this section. It may be said at the outset that the important characteristics of this project appear closely related to the sense of involvement of various participant groups and to the way individuals perceive the importance of project objectives and/or'activities.

The following presentation

describes management and training strategies used in Upswing.

It re-

lates the evidence to our belief that Upswing was "successful." WHAT WAS LEARNED IN THE FIRST YEAR OF UPSWING' The original design involved project leadership most heavily at the beginning and end of tutoring.

1

The tutoring period itself was a

operations in the first year were described in detail in two previous reports: P. Plantec, et al_; Evaluation of Project Upswing: Interim 2ez-:rt (January 1972), and Final Evaluation of Project Upswing's First 7e.eqg (December 1972). Silver Spring, Maryland: UpLrations Research, Inc., Technical Reports 700 and 731.

113

124

kind of grey area in which needs for management apparently were not thought out fully.

It appears that the assumption was something like:

once you

get the machine running it will take care of itself; you only have to come back in to shut it off.

To get the projects started, the directors: Explained Upswing to school system administrators and got their agreement to receive the project in the schools

Contacted school principals for their agreement to participate and to recruit teachers (teachers wcria recruited by principals and, in some cases,

participated unwillingly)

Recruited volunteers to serve as tutors, through newspapers, television and radio, through local school volunteer organizations, churches, and clubs, and through the universities Bought tutoring materials Conducted approximately 15 hours of prescribed preservice training for the volunteer tutors Held a two-hour orientation meeting for some teachers who agreed to participate

Asked teachers to refer children who appeared to need extra help

Divided the children referred into experimental and control groups and assigned volunteers to children Tested children.

Most of the foregoing activities took place in the summer before school opened or immediately after school started.

Volunteer recruitment,

however, as well as selection and assignment of children, continued through October.

Tutoring began in November.

Many tutors recruited early lost

interest by the time actual tutoring started, and dropped out. Delays were encountered in another major start-up task-- pretert:ing of children.

Test results were intended to be used for making final selections

114

125

BEST COPY AVAILABLE

from among referrals and for ensuring matched groups for the evaluation.

However, tutoring had to get under way before testing could be finished and the results reviewed.

"Pretesting" went on from October into January.

The only other prescribed activity of project staff was to administer This was done in April and May.

the Lest battery at the end of tutoring. Management Problems

The directors -Ind staff typically found themselves involved in controlling minor crises from the beginning of tutoring. a lack of role definition.

There was apparently

School personnel seemed unsure of their respon-

sibilities to Upswing tutors, and there was limited opportunity for teachers and principals to communicate with tutors.

Although there were exceptions,

most teachers and tutors had great difficulty arranging to net and discuss the child being tutored.

In some cases there was little motivation on

either the teacher's or volunteer's part to discuss the child. Upswing staff were not often present in the schools.

Conflict

and even hard feelings arose in a number of cases because of poor communication and a lack of clear leadership for Upswing activities at the school building level.

Unbeknownst to project staff, there was considerable tutor

absenteeism and attrition; this seemed to be the problem that bothered ochool personnel the most.

The tutors often felt they were not getting

the advice and encouragement they needed to do a good job; questionnaires indicated that tutors felt somewhat isolated.

From talking with teachers and principals, it appears that such problems are common in school volunteer programs.

Supervision and support

of volunteers in the schools seem to be critical needs that are difficult, if not impossible, for regular school personnel to satisfy in view of their other responsibilities.

Problems in Training Tutors indicated that preservice training did not fully meet the needs they felt in working with their pupils.

Typical comments were that

the preservice training was good, but covered too much to absorb in so short a time period, was too general, and too abstract.

115

126

Many tutors called the project office for help.

In one city, the

untrained tutors felt especially at a loss, and the project director responded by conferring individually with all who wanted help, to diagnose children's problems and work out prescriptive tutoring plans.

Although

it is undocumented, ORI detected in field visits that a significant amount of individual prescriptive advice was given tutors informally by project staff.

However, it appears that such advice was given to the relatively

few individuals who actively sought it out.

There were a few inservice meetings held for the trained group.

Tutors generally were reluctant to call them "inadequate," but noted on questionnaires that these meetings occurred too late in the year and did not meet individual needs. Communications

The projects generally responded to problems as soon as they learned of them, but since channels of communication during tutoring were poorly .defined, it often was rather late before needs were brought to staff members' attention.

Generally the initiative had to be taken by the individual tutor,

by the teacher, or by a principal acting for a teacher.

Teachers particularly

tended to keep their problems with Upswing to themselves, probably because they were too busy with other activities which were regarded as more important or more directly their responsibility.

In personal interviews with a sample

of participants, the evaluation team found a good many dissatisfactions with Upswing that had not been expressed.

The team also found widespread appre-

ciation and a general high regard for the project despite the problems. The foregoing review of Upswing's first year focused on problems. We do not want to imply that problems were paramount.

The large majority

of participants in Upswing found it satisfying and worthwhile. uncertainty was found among teachers.

The most

Our purpose in pointing out the

problems is to show why the projects took the direction they did in the second year.

NEW EMPHASES IN THE SECOND YEAR

The evaluation indicates that the most important changes made in Upswing's second year may be summed up as increased interaction between 116

127

Upswing staff and all groups of participants, and clearer definition of the roles of staff and participants.

These changes were accomplished in some-

what different ways by different projects, but the analysis indicates that, with the exception of St. Louis, the differences in approach were small.

St.

Louis, which did not have any formal inservice training, will be discussed a little farther on in the narrative. ORIENTATION AND TRAINING FOR TUTORS

The U.S. Office of Education specified that a minimum of 30 hours of tutor training should be given.

The allocation of time to preservice

and inservice training was left to the discretion of the project directors,

as were the format of training and the content beyond certain minimum requirements.

Preservice Training

All projects tried to equip tutors with enough background information to give them confidence in stLr.ing to work with their pupils.

There

were different views of what constituted enough, as shown by the number of hours allocated to preservice training.

However, based on responses to

a questionnaire distributed about two months after tutoring began, tutors generally tended to be satisfied with the amounts of preliminary information received.

A set of minimum requirements for preservice training was established by the U.S. Office of Education and the project directors.

Those require-

ments were:

To describe Project Upswing's organization and objectives, including definition of roles and relationships of project participants To provide orientation information about the schools and the first-grade curriculum To define the rules of conduct and dress Upswing tutors would be expected to follow in the schools.

In addition, the preservice training offered by all projects included: Discussion of normal developmental needs and characteristics of Upswing-age children

117

128

Discussion of special characteristics and needs of children who have minimal learning difficulties. Evaluation team observations, backed up by questionnaire responses of the tutors, indicated that the depth and quality, or both, of coverage of these aspects of child development varied; however, all projects at least touched upon them.

Another common feature was that all projects used the readiness inventories completed by teachers in selecting children for Upswing.

Each

Lutor was given the inventory, and in some cases the pretest data of his or her pupil as well.

Staff helped tutors interpret the information for diag

nosis and planning of a prescriptive approach to tutoring.

This was not

always done in preservice training, but if not, it was done individually, shortly after tutoring began.

All projects paid careful attention to housekeeping details in the second year, since they had been troublespots in the first.

The staff made

sure in preservice training to tell tutors how to get to their school, the school telephone number and secretary's name, the name of their pupil's teacher, the procedure to use to report in at the school, the procedure to use if they could not keep a tutoring appointment, and the name and telephone number of an Upswing staff member to contact about questions or problems,

Tutor Opinion About Preservice Training Table 23 summarizes tutor opinion about the adequacy of training in defining the project and participants' roles, and in equipping them to start working with the children.

The data are from the "Volunteer Registration

and First Impressions" questionnaire, distributed about two months after tutoring began

The response rates were as follows:

(copy in Appendix B).

No. of Tutors in Project City

When

Form Was Distributed

No. of Tutors Who Returned Form

Response Rate

Denver

42

36

85%

Oxford

53

49

92%

St. Louis San Francisco

45

35

78%

34

25

74%

118

129

Table 23 indicates that all projects did quite well in clarifying project goals and the roles of tutors, teachers, and Upswing staff.

There

appear to have been some minor differences in effectiveness in establishing teacher and staff roles in the minds of tutors, but the differences are relatively unimportant in view of the overall picture of success. Differences were greater in tutor satisfaction with child-related information.

It is evident that the Denver project's comparatively extensive

and very well-planned preservice training program resulted in greater tutor satisfaction.

Denver offered 24 hours of training before tutoring began;

the agenda included observation of prospective pupils in the classroom followed by discussion.

The other cities offered 6 hours (San Francisco),

8 hours (Oxford), and 15 hours (St. Louis).

The weakest aspect of preservice training in all cities was how to recognize and deal with different kinds of learning problems.

That is also,

of course, the most complex and difficult to convey of all the topics addressed in the training.

Table 24 shows that the projects all were quite successful in covering organizational matters for a smooth beginning of tutoring.

The weakest

areas in all cases were preliminary arrangement for tutoring space and adequacy of tutoring space. arrangements made in advance.

Oxford was exceptionally successful in getting Tutoring locations had to be left up to

school staff, and in many cases there was not much they could do because there were few options and much competition for space outside classrooms. Apparently San Francisco schools were most accommodating or San Francisco tutors were most tolerant of less than optimal tutoring locations.

How-

ever, tut r comments indicated that in all cities they understood the problems of setting aside space for tutoring.

Denver and Oxford tutors apparently were less well informed about what tutoring materials were available to them in the schools.

The St.

Louis project distributed a typed list of materials to tutors and set up a supply closet in each school which was 100% effective in that area. Despite the foregoing differences, the major conclusion is that organizational matters were handled well in all cities.

119

Has Upswing taught you enough about now to recog,4--.a and deal with different kinds of lEarning problems?

grade children to guide you in dr:tcrraining what kinds of help your pups! needs?

and characteristics of first-

28/78%

32/90%

32/90%

Do you feel you were reasonably well prepared by Project Upswing to begin working with the child?

Have you received enough information about the normal needs

35/:? i'.

Do you have a clear picture f the kinds cf help you can expect from the Project Upswing staff?

from the child's teach.::r?

30/83%

36/100%

lias your role as an Upswing tutc,r been made clear to you?

Do you hate a clear pvture of the kinds of h, p i.;_ rtm expect

35/97%

Have the goals of Project L r,;wing been made clear to you?

Questions Asked

29/59%

38/78%

39/80%

45/92%

41/84%

47/96%

48/98%

13/37%

24/69%

24/69%

29/83%

26/74%

33/94%

33/94%

15/60%

14/56%

.

20/80%

23/92%

19/76%

24/96%

23/92%

Number & Percentage of Participants in Each City who answered Positively San Francisco St. Louis Oxford Denver

SUMMARY OF TUTOR OPINION ABOUT INITIAL TRAINING RELATED TO PROJECT GOALS, ROLE DEFINITION AND INSTRUCTION OF CHILDREN

TABLE 23

85/59%

108/75%

115/80%

132/91%

116/80%

140/97%

139/96%

Total

23/64% 32/90%

Has the school provided a satisfactory location for your tutoring?

Were you told what procedures you should use when you "reported in" for tutoring the first time?

32/90%

36/100%

Do you know whom to contact if you have a question or problem concerning Project Upswing?

32/90%

Did it appear that the child's teacher was prepared for your arrival?

materials available in the school.?

Do you know where to find tutoring

are available in the school?

27/77%

21/58%

DH arrungements for tutoring space seem to have been made in advance?

Do you know what tutoring materials

35/97%.

Denver

Were you notified what date and time you would begin tutoring sufficiently in advance?

Cuestions Asked

48/98%

43/88%

40/82%

38/78%

44/90%

34/69%

45/92%

47/96%

35/100%

32/91%

34/97%

35/100%

34/97%

25/71%

25/71%

32/91%

25/100%

22/88%

22/88%

23/92%

24/96%

21/84%

16/64%

24/96%

Number & Percentage of Participants in Each City Who Answered Positively San Francisco St. Louis Oxford

SUMMARY OF TUTOR OPINION ABOUT HOW WELL ORGANIZATIONAL MATTERS WERE COVERED IN PRESERVICE TRAINING

TABLE 24

144/99%

129/89%

128/88%

123/85%

134/92%

103/71%

107/74%

138/95%

Total

BEST COPY AVAILABLE Assessments of preservice training should be considered in relation to the overall assessments of training and to the findings about project effects on children.

The initial response to training will be

considered in those contexts as the analysis develops. Inservice Training for Tutors

The following topics were required to be covered in inservice training if they were not covered in preservice training:

Orientation to first-grade children (normal developmental characteristics and needs)

Orientation to characteristics and needs of children with minimal learning difficulties Techniques for developing and maintaining relationship with child, teacher Use of Peabody Language Development Kitt Techniques for supporting child's classroom learning activities (how to use actual classroom materials and related materials)

How to organize and pace tutoring sessions Techniques of positive reinforcement.

Beyond these requirements, the projects were free to provide any additional training, reinforcement of previous training, or problem-solving assistance they thought appropriate, with the condition that such additional assistance had to be given over the tutoring period, whether individually or in a group workshop type mode.

Three of the four projects established a schedule of regular meetings for tutors throughout the year. and San Francisco.

The meetings were held monthly in Denver

They were held weekly in Oxford, since Project Upswing

constituted a course at the University of Mississippi.

Although inservice

2 The Peabody Kit is a commercially developed package that includes picture cards, puppets, interlocking colored links, etc., designed to stimulate a child's thinking and verbalization. A set of lesson plans is provided with the kit, but most Upswing tutors preferred to ad lib, using the Peabody for a change of pace along with other tutoring activities. 122

133

BEST COPY AVAILABLE meetings were not attended regularly by all tutors, they built a reliable communications channel into the projects.

The data indicate that they

were, on the whole, responsive to tutors' needs and strengthened the projects. There was no inservice training program for tutors in the St. Louis project.

The approach to ongoing support there was entirely individualized.

Staff members visited the elementary schools twice weekly and helped individual tutors as needed.

There were one or two get-togethers for the tutors

in some schools, in which they shared ideas, tribulations, and successes; staff were there for consultation at those times.

The other projects also

had staff members visiting each school at least once a week.

Thus there was

double coverage in the other cities.

Tutor Opinion About the Overall Training Program

By the end of the year, opinion about training appeared to be somewhat less enthusiastic overall than at the beginning.

Nevertheless,

the tutors indicated that they generally considered training necessary and that they valued the training they received.

Questionnaire comments

indicated a good deal more satisfaction (or less dissatisfaction) with the In the following presenta-

training in the second year than in the first.

tion, two training modes are considered separately--group sessions, which were more in line with the standard implications of "training," and individual counseling.

The following data were taken from the "Volunteer Final Impressions" questionnaire (copy in Appendix B), which was distributed in May 1973. response rates were:

No. of Tutcrs in Project City

When

No. of Tutors Who Returned Form

Form Was Distributed

Response Rate

Denver

37

31

84%

Oxford

47

43

91%

St. Lou is

42

30

71%

San Francisco

20

16

80%

123

134

The

The rationale for collecting tutors' final impressions of the adequacy of ti

fining is that the experience gives perspective to the original impres-

sions.

It was expected that some first-impressiok optimism would be dampened

by the reality of facing the tutoring challenge, a :.d such was the case.

A

full understanding of the general final impressions of training and the reasons behind them can only be obtained through some detailed analysis.

So, proceed-

ing from the general to the specific, we present the findings below. Jeneral Value of Group Training Sessions.

In the final analysis, half

of the Upswing tutors still felt that the training sessions were very important to them, perhaps not perfect, but essential to performance.

Another 29% agreed

that the group training was quite useful, but felt that it was not essential to the task.

Bear in mind that both views probably were correct.

Upswing's

first year proved that some people do not really need much training, while others do.

No one said that training was completely unnecessary, only one

said that it was presented poorly, and only three said they needed no training.

Table 25 gives a breakdown of responses about the value of group training from each city and for all cities combined.

The percentage of responses

of each kind are given, but should be compared city to city only with caution; the number of respondents in each city, especially San Francisco, is so small a base, that percentages may be misleading.

The most important

point to be made about these data for each city are those just made about the overall responses.

About 60% of respondents in all cities except St.

Louis felt that the group training meetings were very important to their work as tutors. The difference in St. Louis is evidently related to the number, timing, and nature of meetings held there, as described previously; it does not necessarily mean that St. Louis tutors valued the training they received less, only that, apparently they found less of substance in the few jpo:ir meetings that were held.

The fewest "useful but not essential"

responses came from Denver (fewest in relation to number of respondents), which could indicate proportionally greater value placed on training meetings there.

However, the picture is obscured by comparatively high self-reported

124

135

AVAILABLE BEST COPY

TABLE 25

TUTOR OPINION ABOUT THE GENERAL VALUE OF UPSWING GROUP TRAINING MEETINGS

(Percentages based on number of respondents in each city who answered question. Only one, in Denver, did not.) Number and Percentage of Respondents Who Expressed Each Opinion Denver

Tutor Opinion

Did not attend enough meetings to have an opinion

I did not need training Training is not necessary for tutors Training was generally presented poorly Training was useful but

not essential

7

23% 2

Tctdl

2

5% 0

6%

St. Louis

San Francisco

0

0

0

1

20

7%

17%

1

0

16 37%

0

0

0

1

C

1

11

1% 5

37%

33%

7

9

61%

25 58%

24%

60%

31 100%

43 100%

30 100%

19

125

3

3%

3% 3

Total

10 33%

3%

0

10%

Training was very important to me as a tutor

Oxford

15

100%

35 29% 60 50%

119 100%

BESI Tro tvilo absenteeism in Denver.

(One cannot tell if the "I did not attend ..."

responses in St. Louis are truly absence-related or are related to the number of meetings held.)

The San Francisco training program was quite

similar to nenver's in content and approach, but the San Francisco project had a much lower staff-tutor ratio, and the staff gave more individualized assistance to tutors than was given in Denver.

It appears likely that group

meetings were considered less essential by San Francisco tutors because they had such strong support outside the meetings.

In Oxford, as noted previously, Upswing meetings, held weekly, constituted a university course.

Because of this frequency, non-student

volunteers did not always attend, and they may have perceived the meetings as not designed for them.

The university students involved in Upswing were

almost all upper-level students in the School of Education.

They had gen-

erally taken and currently were ta.ing other courses relevant to tutoring, so that the Upswing sessions may well have been somewhat less significant to them than to tutors not immersed in education.

To investigate the general impressions of group training, tutors were asked a number of questions about.specific characteristics of meetings. Content of Group Meetings.

The aspect that contributed most to satis*

faction of the trainees was the introduction of new ideas for use in tutoring, ideas about specific materials and techniques.

Other factors contributing

significantly to the utility of training as perceived by the trainees were (in order of importance):

how to size up the child, his needs and what to

expect; the confidence which comes from being trained; how to relate to the child and teacher; how to evaluate the child's progress; and how to handle behavior problems.

A ranked list of important factors is provided in

Table 26.

Tutors were given an opportunity to suggest alternate types of group sessions.

About 35% of the tutors used the opportunity to suggest what they

felt would be improvements.

Most of the suggestions for better training

came from St. Louis where very little formal training was offered.

It is

interesting to note that the suggestions for alternate types of training (from all cities) correspond almost exactly with the reasons volunteers were most satisfied with training, which indicates both that these things were important and they they were not adequately presented, to varying

126 11 7

4J

BEST COPY AVAILABLE

TABLE 26

ASPECTS OF GROUP TRAINING CONSIDERED MOST USEFUL BY TUTORS

(Percentages based on a total of 98, the number of respondents to this question.) Major Reasons for Usefulness of Sessions They acquainted me with a variety of materials and their uses They gave me knowledge of teaching techniques They helped me tutor appropriately to meet my pupil's spPcific learning needs They helped me know what to expect from my pupil

77%

60%

37% 32%

They helped me diagnose my pupil's specific learning needs They gave me confidence They helped me to have a better relationship with my pupil They helped me have a better relationship with my pupil's teacher They helped me evaluate my pupil's progress They helped me handle behavior problems *

Response Rates*

31%

21% 8%

8% 6%

5%

Respondents checked the three reasons they considered most important.

127

138

BEST COPY AVA:LABLE degrees.

It appears that more self-perceived needs were unmet in St. Louis

where little formal training was offered. began.

And all was offered before tutoring

These perceived needs did not affect ability to tutor, according to

the children's test results.

The most frequent suggestions ware, in order of importance: Meetings to discuss tutoring ideas Meetings for instruction in specialized techniques Help in planning lessons to cover specific problems Sessions to observe classroom teachers with firstgrade children

Meetings to discuss current school reading program Meetings between volunteers and teachers to discuss role expectations.

Schedule and Environmental Aspects of Training Meetings.

The tutors

were asked to assess the schedule, location, physical environment, and social atmosphere of Upswing meetings to make sure such variables did not impact their assessment of the value of training.

No negative influence was found.

About

85%-100% of the respondents in all locations said that these aspects of Upswing meetings were favorable.

The only factor that tutors said caused them to miss

meetings was schedule, and schedule conflicts did not appear to be a great problem.

general Value of Individual Counseling.

Overall, about 75% of the

responding tutors valued the individual help they received from Upswing staff. Forty-three percent considered it useful, although not essential, while 32% considered it very important (by implication, essential).

These data were not

cross tabulated with the data on the general value of group training.

How-

ever, the response patterns are about the same.

The distribution of opinion about the value of individual help is presented in Table 27 for each city and for all combined.

As for group train-

ing, the percentage of responses in each category for the individual cities are based on too low numbers to be substantial.

The most important features

of the data are that the respondents in each city responded favorably about the worth of the help received.

Only four respondents, dispersed over three

cities, said that individual help was not available to them, while only seven

128

1.39

Total

It was very important to me in my work as a tutor

essential

Individual help was not available to me I did not need individual help The Upswing staff generally did not give the kind of individual help I needed It was useful, but not

Tutor Opinion

43/100%

32/100%

15/50%

19/44%

14/44% 11/26%

0

2/7%

4/9%

1/30/0

10/31%

1/7%

4/13%

7/16%

6/19%

38/32% 119/100% 14/100%

30/100%

52/44%

7/6%

18/15%

4/3%

9/64%

I

Total

8/27%

4/29%

0

San Francisco

1/3%

St. Louis

2/5%

Oxford

1/3%

Denver

Number and Percentage of Respondents Who Expressed Each Opinion

(Percentages based on number of respondents in each city who answered the question. Only one, in San Francisco, did not.)

TUTOR OPINION ABOUT THE GENERAL VALUE OF INDIVIDUAL HELP PROVIDED BY UPSWING STAFF

TABLE 27

BEST COPY AVAILABLE respondents, from three cities, said the kinds of help offered did not meet their needs.

About 15% said that they did not need individual help.

A greater proportion of respondents in San Francisco than in any other city considered individual help very important. low tutor-staff ratio.

We relate this to the very

San Francisco had trouble recruiting volunteers and

about half of those who started tutoring dropped out.

The respondents from

that city probably represent a core group of tutors who were very dedicated to the project and who received a great deal of high-quality individual attention.

Interestingly, St. Louis tutors placed no more value on individual

help than on the group training offered there. f:ontent of Individual Counseling.

The tutors were asked to indicate

the ways in which individual help was useful to them if they had found it useful (Table 28).

They were given the same checklist of benefits as shown

in Table 26 for training meetings.

The dispersion of responses in Table 28

reflects the obvious--individual help was useful to different people for different reasons.

Comparison of Tables 26 and 28 shows that there was ex-

tensive overlap in the benefits of formal training and individual help as perceived by tutors.

Formal training was valued most for what it was intended

p rimarily to do--acquaint tutors with instructional materials and techniques.

ndividual help tended to be valued considerably less consistently for these re asons, valued somewhat more often than formal training for building tutor COnfidence,

and considerably more often for helping with tutor expectancies

for pupils, pupil evaluation, and the pupil-tutor relationship.

When asked where individual help could have been stronger, tutors name d the following (listed in order of importance):

Inservice hints or suggested approaches to dealing with the child (or school) (this seemed to refer to management of behavior or attitude

problems with the child; the school factors were not clear]

Discussions with child's teacher about techniques and progress

Extra individual instruction in reading techniques.

130

141

BEST COPY AVAILABLE

TABLE 28

ASPECTS OF INDIVIDUAL HELP FROM UPSWING STAFF CONSIDERED MOST USEFUL BY TUTORS

(Percentages based on a total of 98, the number of respondents who answered the question.) Major Reasons for Usefulness of Individual Help It helped me tutor appropriately to meet my pupil's specific learning needs It gave me knowledge of teaching techniques It gave me confidence It helped me diagnose my pupil's specific learning needs It acquainted me with a variety of materials and their uses It helped me to know what to expect from my pupil It helped me to have a better relationship with my pupil It helped me evaluate my pupil's progress It helped me handle behavior

Response Rate* 38%

34% 33% 29%

26%

26% 21%

20% 15% 15

problems

It helped me have a better relationship with my pupil's teacher

11%

*Respondents checked the three reasons they considered most important . 131

142

BESTOOPYAVAlUIBLE Anount.

Nearly 70% of the second-year tutors felt they had an adequate

rr:yint of individual counseling. help.

The other 30% expressed a need for additional

About this ratio prevailed in all cities except San Francisco, which

had the greatest proportion of tutors (12 out of 15) who expressed satisfaction with the amount of individual counseling received.

Again this seems to be

related to tutor-staff ratio.

Group Training Versus Individual Counseling.

Volunteers were asked

to compare the utility of group training and individual counseling.

As shown

in Table 29, nearly half the tutors preferred not to make a choice, seeing the two as equally useful.

About a third preferred the group training to

individual and the rest preferred individual counseling.

It is important

to note that in Denver, where group training was emphasized, about half of the tutors preferred that form of training; while in St. Louis, where individual counseling was stressed, about half of the respondents stated they preferred it that way.

This evidence supports the assumption that the specific training

approach should be developed to suit the experience and strengths of the key training individuals, rather than to follow an externally prescribed course. From what happened in the first year, it appears that individual contact is necessary; but it does not have to be (and should not be, in view of costs), the primary mode of delivering training.

Indeed, the individual

contact in all locations involved mostly problem-solving, response to questions, The data suggest that tutors tended to

and moral support rather than training.

perceive training and individual counseling as coequal and separate functions, both useful, but not necessarily essential to their work. Materaio Use(! in Trainina.

Eighty-two percent of the tutors who

responded to the questionnaire thought the materials used in the training they received were either good or outstanding.

Only 15% of all respondents con-

sidered the materials inappropriate or of little value.

Table 30 lists types

of materials used in training. The Peabody Language Development Kit was described previously.

The

I)ISTAR Reading System is a phonics-based program designed originally for

children who have fairly serious reading problems.

It is set up for group

instruction, and some of the Upswing tutors found it difficult to adapt for

132

143

BEST COPY AVAILAILK

TABLE 29

TUTORS' OPINION AS TO WHAT APPROACH WAS MOST USEFUL TO THEM IN TUTORING

(Percentages based on number of respondents in each city who answered the question; one in St. Louis did not.)

Preferred Training Approach

Number and Percentage of Respondents Who Expressed Each Opinion Oxford St. Louis San Francisco Denver

Individual Both

Neither Total

Total

13/30%

6/21%

4/27%

38/32%

5/16%

10/23%

14/48%

3/20%

32/27%

10/32%

20/47%

9/31%

8/53%

47/40%

15/48%

Group

I

,

1/4%

0

0

0

1/1%

,-

31

100%

43 100%

29 100%

133

14 4

15

100%

118 100%

BEST COPY AVAILABLE

TABLE 30

TRAINING MATERIALS FOUND TO HAVE MOST VALUE BY UPSWING TUTORS

Material

Percentage of All Respondents Who Rated Material Most Useful

Peabody

28

Games, puzzles

15

D [STAR

14

Flash cards Arts and crafts Library and other non-textbooks

9

Workbooks

4

Exercises

3

Tokens

3

Chalkboard

3

Instructo

3

Tape recorder

2

Other

2

9 7

100%

Total

134

145

-

one-to-one instruction.

DISTAR depends on rythmic oral repitition to establish It is highly structured, and the pub-

sounds and words in a child's memory.

lishers stress that strict adherence to the program on a regular basis is essential. Generally DISTAR lessons are given daily rather than twice a week. tutors considered the program too rigid. chase of one for every tutor.

Many Upswing

The cost of these kits prohibited pur-

One or two of each was kept in all Upswing schools.

They could not be taken home by the tutors for preparation except by special arrangement.

This was considered an impediment to using the materials, especially

DISTAR, by many tutors.

"Instructo" is another commercially-developed kit used primarily in St. Louis (a few kits may have been purchased for use in Oxford).

It includes

picture cards, work cards, a flannel/chalk board combination, etc.

Lesson

plans (for group use) are provided, covering color recognition, development of sight vocabulary, initial sounds, etc.--the usual components of a beginning reading program.

The other materials listed in Table 30 are self-explanatory.

The table

gives the percentage of respondents (from all projects combined) that considered each the most useful tutor - training material.

It is clear from the dispersion

of opinion in Table 30 that the specific materials used as a springboard for training are not critical.

Variety seems to have been the key for reaching

Upswing tutors.

Attendance at Tutor Training Meetings

Tables 31 and 32 show the rates of attendance at training meetings held before and during tutoring.

Attendance was consistently better before tutoring.

The reasons for variation beyond that are not clear.

Oxford is a special case,

since Upswing was a university course and the majority of tutors were university students for whom attendance was more or less compulsory.

San Francisco had

the best preservice attendance rate with the fewest number of sessions and hours. However, San Francisco also had a better inservice rate than Denver, with more sessions.

San Francisco held three inservice meetings in October, two in

November, one in December and January, two in February and March, and one in April.

Denver held only one inservice meeting each month from December through

May.

135

146

BEST COPY AVAILABLE

TABLE 31

TUTOR ATTENDANCE AT PRESERVICE TRAINING MEETINGS

(Percentage based on number of tutors reported by each project to be in tutor group at time training was offered.) Percentage of Training Tutor Attended

Tutors in Each Attendance Category, by City San Francisco St. Louis Oxford Denver (total hours (total hours (total hours (total hours offered = 6) offered = 22) offered = 8) offered = 15) 12/31%

34/74%

10/20%

80%-89%

8/20%

9/20%

0

0

70%-79%

3/8%

2/4%

7/14%

0

60%-69%

5/13%

0

6/12%

0

50%-59%

3/8%

0

0

1/2%

< 50%

8/20%

1/2%

8/16%

0

0

0

19/38%

1/2%

46 100%

50 100%

41 99%*

Total *

39/95%

90%-100%

39 100%

Rounding error.

136

147

0

BEST COPY AVAILABLE

TABLE 32

TUTOR ATTENDANCE AT INSERVICE TRAINING MEETINGS

(Tutors reported to have attrited before January 1, 1973, omitted.) Tutors in Each Attendance Category, by City Denver

Oxford

(total hours offered = 10)

(total hours offered = 22)

San Francisco (total hours offered = 39)

90%-100%

7/15%

11/24%

4/14%

80%-89%

4/8%

17/38%

4/14%

70%-79%

0

7/16%

3/10%

60%-69%

9/19%

5/11%

4/14%

50%-59%

0

3/7%

6/21%

< 50%

20/42%

2/4%

8/27%

0

8/17%

0

0

48 101%*

45 100%

Percentage of Training Tutor Attended

Total *

Rounding error.

137

29 100%

BEST COPY AV 'A; AV: The amount of preservice training offered in Denver, with rather good attendance (about 80% of the tutors took part in 60% or more) may have something to do with the comparatively low attendance at the monthly inservice meetings.

Denver had a better turnout for 22 hours of preservice training

than St. Louis had for 15 hours.

This may have occurred because the meetings

in the former city were spread over a 4-6 weekTeriod in the fall, while they were concentrated into four 3-hour sessions in one week during the late summer in St. Louis, with a final 3-hour meeting on a Saturday morning shortly after school started.

This is interesting since teachers seemed to give

better attendance where meetings were concentrated rather than shorter and held over a longer time period.

On the whole, attendance at training was not very good (except in Oxford, as explained previously).

It certainly casts doubt on the value of

investing time in development of an extensive training program even without the earlier finding that trained tutors, as a group, are no more effective in helping children improve their academic performance. The program in St. Louis was particularly unfruitful, with 38% attending no training meetings and an additional 16% attending less than half.

From observation during a preservice site visit, it was generally not

tutors involved ip Upswing's first year who missed the sessions.

Thus, St.

Louis offered a real comparison group made up predominately of tutors who had little if any formal training.

As shown in Section IV, the St. Louis

tutored group of children improved as much as the tutored group in any other city.

CONCLUSIONS ABOUT THE EFFECTIVENESS OF TUTOR TRAINING The tutors' questionnaire responses, and their attendance at training meetings,were the criteria for evaluating the effectiveness of training in the second year of Upswing.

Under these criteria, it must be said that the

programs were no better than 50%-60% effective. do with the quality of training offered.

This rating has nothing to

Considerable time and effort were

put into offering professional quality training responsive to the expressed needs of tutors.

The training in the second year generally was markedly

higher in quality than that offered in the first.

138

149

BEST COPY AVAILABLE

It appears that people value training more in theory than in practice and perhaps want it to be available even if they do not take advantage of it. Along these lines, it appears that training may have been considered more important in the first year because half of the tutors in the program were excluded from attenting.

It is clear that it does not pay to expend too much

in training tutors, although some preparation is necessary.

Tutors seem to have definite ideas about what training they want, and they focus clearly on techniques --what to do with the child.

They seem to

prefer that discussions of theory be minimal, and tied to observation of children.

They generally want very much to know how instruction is handled

in the classroom and want to keep their efforts consonant.

(There is a strong

argument that "more of the same" in tutoring is not a good thing, especially when a child is having difficulty in class.

However, the Upswing tutors and

teachers most frequently seemed to support tutoring as a classroom back-up.) Such features as physical and social environment, training materials used, presentation format, and schedule seem to admit a good deal of latitude. There was a reduction in the rate of tutor attrition from the first year to the second year of Upswing.

It appears that this is attributable to

the combined effects of bringing preservice training closer to the start of tutoring, conducting more extensive inservice training, and increasing the amount of regular communication between tutors and project staff.

The

relative contributions of the training program and the communications structure outside training are considered in the conclusion of this section. Attrition is considered. separately in some detail in Section VI.

In summary, it appears that the training of volunteers need not be elaborate or expensive. fact of training.

The content of training is less important than the

Thus, training should probably be geared to the voiced

needs of the volunteers themselves, concentrating training on solving inservice problems.

TRAINING/ORIENTATION FOR TEACHERS It was prescribed that all projects provide at least 10 hours of teacher training, covering, as a minimum, the following topics:

139

BEST COPY AVAILABLE Hcv to recognize minimal learning difficulties Techniques and materials covered in tutor training

How to employ behavior modification (one of the training topics for volunteers).

There was no stipulation about whether any of the teacher training should be provided during tutoring.

All teachers were paid about $50 for

time spent on Upswing during the year.

Table 33 shows the schedules for

teacher orientation/training in the four cities. Preservice

Before tutoring began, the projects:

Defined the goals and organization of Upswing to teachers

Explained Upswing research aspects, with special emphasis on the reasons for establishing a.control group (which was found in the first year to be a very difficult concept for many teachers to accept) Explained what would be expected of teachers involved in the project:

--Selection of children --Communication Frith and encouragement of tutors,

and guidance/assistance to the extent guidance

was needed and the teacher had the time and interest

--In two of the four projects, attcalance at meetings throughout the year Provided instruction/discussion/suggestions about how to observe children's behavior in a structured way and draw diagnostic conclusions about behavior Reviewed use of the Cegelka Aeddemic Readiness

Evaluation (CARE) as a screening aid in identifying learning difficulties in children.

Three of the four projects also discussed with teachers the principles and teUiniques of positive behavior management that tutors would be taught and encouraged to use.

140

151

TABLE 33

BEST COPY AVAILABLE

PROJECT SCHEDULE FOR TEACHER ORIENTATION /TRAINING

City

1973

1972 Aug

Sept

Oct

Nov

Dec

Jan

Feb

Mar

Apr

May

Total

EIN.",.;ER

Preiervice: No. of meetings No. of hours Inservice; No. of meetings No. of hours

2

1

1

4 mtg

3

1

1

5 hr 1

1

1

1

1

1

1

1

Social get-together at each school

4 mtg

5 hr 8 mtg 10 hi

Totals OXFORD

Preservice: No. of meetings

No. of hours Inservice: No. of meetings No. of hours

1

1 mtg

8

8 hr

1

1

1

1

2 mtq

32 hr

3 mtq 10 hr

Totals ST. LOUIS

Pre service:

,:::. :! mc:tings

1

1 mtg

No. of hours inservice: No. of meetings No. of hours

3

3 hr

No formal inservice training meetings; 1-2 informal social/discussion gatherings at some schools 1 mtq

Totals

3 hr

SAN FRANCISCO

Pre service

N. of f:et:tings

2

2 rag

No. of hcurs Inservice: No. of rr:Peti-qs '1o. of hours T".ils

5

5 hr

1

2}

1 1

1

1

3

21

1

2i

5 mtg 13 lir

7 mtg 18 hr

141

152

The initial interaction with teachers was done in one to four group meetings, with individual follow-up as needed in Denver, Oxford, and San In St. Louis, staff visited each school to provide teachers

Francisco.

individually with the information indicated above.

A joint teacher-tutor

meeting was held in St. Louis just before the start of tutoring to get acquainted, discuss the characteristics of children with learning difficulties, demonstrate Peabody materials, and review the completed readiness inventories and their implications for instruction.

Teacher Opinion About Preservice Training/Orientation Teachers also received a "Registration and First Impressions" questionnaire (see Appendix B) that asked questions about the adequacy of the information they were given before tutoring began. ments were very favorable.

Overall, their assess-

The rates of response for this questionnaire

were:

NO. of Teachers City

in Project

No. of Teachers Who Returned Form

Response Rate

Denver

19

16

84%

Oxford

18

18

100%

St. Louis

15

11

73%

San Francisco

18

16

89%

Table 34 indicates the degree of positive response to specific questions about substantive elements (as opposed to organizational elements) of the preparation Upswing teachers received.

On the whole, the projects were quite

successful in providing teachers with satisfactory information.

There were

-T.a-r,:z :*ie self-perceived weaknesses in understanding of child selection

parameters among teachers in Oxford and St. Louis.

The pretest data indicate,

however, that the Oxford and St. Louis teachers probably had as good a grasp on child selection parameters as the teachers in the other cities.. tion criteria were not particularly specific in any project.

142

153

The selec-

15/94%

9/56%

15/94%

15/94%

Do you understand why Project Upswing has a contol group of children?

Do you know which of your pupils are in the control group for Project Upswing

Has the teacher's role in Project Upswing been made clear to you?

Has it been nade clear to you what the Upswing volunteers are supposed to do?

the volunteers will be using?

14/88%

16/100%

Did you have a clear understanding of what characteristics to look for in selecting children for Project Upswing?

Do you have adequate information about the materials and techniques

16/100%

Denver

Have Project Upswing's goals been mac.e clear to you?

Questions Asked

16/90%

17/94%

17/94%

17/94%

18/100%

14/78%

18/100%

10/91%

11/100%

11/100%

7/64%

10/91%

9/82%

11/100%

16/100%

16/100%

16/100%

13/81%

14/88%

16/100%

16/100%

Number & Percentage of Respondents in Each City who Answered Positively San Francisco St. Louis Oxford

56/92%

59/97%

59/97%

46/75%

57/93%

55/90%

61/100%

Total

SUMMARY OF TEACHER OPINION ABOUT INITIAL TRAINING/ORIENTATION BEST COPY AVAILABLE RELATED TO PROJECT GOALS, RESEARCH ASPECT, AND ROLE DEFINITION

TABLE 34

The other area where substantial comparative weaknesses appear is knowledge of which children were assigned to the control group.

That is

not an important training assessment criterion, although the evaluation needed to know how many teachers had that information.

We would have preferred a

blind or, better, double-blind experiment, but either would have required more testing than the project staffs or the schools could handle comfortably. Teachers in the first year were confused about the control group, and in a good number of cases, were disturbed because some pupils they referred did not get a tutor.

Thus, it was decided to explain the use of controls explicitly

and tell teachers which of their pupils were in that category.

(The teachers

should have known anyway if they remembered which of their pupils were tested for Upswing; apparently some did not remember.)

Table 35 presents questions about the adequacy of organizational information provided to teachers at the start of tutoring, with the percentages of positive response.

In general, teachers were not as well informed about

organizational matters as about substantive; still, the overall picture is reasonably good.

The responses in Table 35 represent a great improvement over

the first year. Inservice

As noted previously, there was no requirement for the projects to offer an inservice program for teachers participating in Upswing.

The San

Francisco and Denver projects did; the Oxford and St. Louis projects did not.

There were two meetings for teachers in Oxford right after tutoring

began, but these are not regarded as inservice training since they covered mostly the same material as the preservice meeting.

In at least one St.

Louis school there were two social gatherings of Upswing teachers for informal discussion of the project; one took place at Christmas time and another during a visit by members of the evaluation staff.

In all cities, the Upswing staff

assigned to the schools as coordinators regularly checked with teachers to

make sure there were no problems and to receive teachers' suggestions for the project.

The focus of the teacher programs in Denver and San Francisco were somewhat different.

The Denver meetings (a one-hour session every month

144

155

8/50%

15/94%

12/75%

8/50%

16/100%

Did you know what time to expect each Upswing tutor assigned to one of your pupils?

Did you know what the procedure would be on the first day of tutoring?

Were arrangements made in advance for tutoring space outside your classroom?

Do you know whom to contact if you have a question or problem concerning Project Upswing?

Denver

Did you know what days to expect the Upswing tutor(s) assigned to your pupils?

Questions Asked I

18/100%

18/100%

16/89%

18/100%

14/78%

I

10/91%

8/73%

9/82%

11/100%

10/91%

16/100%

15/94%

14/88%

13/81%

12/75%

Number & Percentage of Respondents in Each City Who Answered Positively San Francisco St. Louis Oxford

SUMMARY OF TEACHER OPINION ABOUT THE ADEQUACY OF ORGANIZATIONAL INFORMATION THEY RECEIVED IN PRESERVICE TRAINING/ORIENTATICN

TABLE 35

60/98%

49/80%

51/84%

57/93%

44/72%

Total

BEST COPY. AVAILABLE

EMT COPY 4t41".4r:Lf

except December) were explicitly to provide a workshop-type experience.

The

teachers were asked to state what topics they wanted to cover in these sessions and to share ideas about materials and instructional approaches they had found useful.

The teachers were not particularly responsive to this request and

the amount of interest they displayed at the sessions was disappointing to project staff.

Teachers apparently found monthly meetings burdensome, and the

stipend apparently did little to make them feel better about this demand.

The San Francisco meetings were explicitly to inform teachers of the content of tutor training, to obtain teacher suggestions about what should be covered in tutor training and what the children needed in tutoring.

In

accomplishing its objectives for the teacher inservice program the San Francisco project provided some training, or review of training had elsewhere. Teachers, like those in Denver, did not, as a group, show any gr.:at enthusiasm for the meetings.

The San Francisco meetings, like those in Denver, were held after school.

They were longer than the Denver meetings, generally 2-1/2 hours.

Most took place toward the beginning of the project:

four in September

(three of those before tutoring began), one in October, and one in November. The teachers were brought together again in February and for a final session on the first of May. possibly in September.

Such a schedule would not appear burdensome, except Still a significant proportion of the teachers

indicated they felt there were too many Upswing meetings. Teachers' Final Assessment of the Orientation/Training They Received From PrL,,ect Upswing

Teachers gave their retrospective opinions about Upswing orientation and training on a "Final Impressions" questionnaire (copy in Appendix B). response rates were:

No. of Teachers Who Returned Form

No. of Teachers City

in Project

Response Rate

Denver

19

19

100%

Oxford

18

18

100%

St. Louis

15

11

73%

San Francisco

18

18

100%

146

157

The

The data indicate, overall, that the project;- were reasonably

successful in realizing the stated objectives for teacher training.

It is

perhaps most interesting that the projects which held no formal meetings for teachers after tutoring began were rated about the same on most items as those which put comparatively much greater effort into conducting a program for teachers.

However, the data also indicate that teachers in the cities

with more extensive programs may have gained more in certain respects from their participation.

Further, there is strong evidence that the formal

programs for teachers resulted in learning gains for children. Table 36 presents final questionnaire results that pertain to the substantive aspects of Upswing training.

Ninety-two percent of all teachers

valued the readiness inventory (the Cegelka Academic Readiness Evaluation, or CARE) they were trained to use as an aid in child selection.

The small per-

centage differences from city to city are not significant, since about 90% to 100% of teachers in all locations considered the inventory useful.

(The

CARE was not evaluated by teachers in relation to any of the many other existing readiness measures.

We believe they were responding favorably to the concept

of using some such inventory of learning behaviors as an aid in identifying children's needs.)

There was a somewhat broader range of response to the question on adequacy of information about tutors' techniques and materials.

The Oxford

program may have been comparatively weaker, although not deficient, in Lescribing them to the satisfaction of teachers.

However, the percentage

difference between Oxford and St. Louis or Oxford and San Francisco is not great enough to be very important in view of the numbers of people involved. The Denver program, which stressed instructional approaches, appears to have been stronger in putting across tutors' methods and materials, but here as well, one should not make too much out of 10% differences.

The important

point is that the projects apparently were successful in providing adequate information on this point.

Table 36 also shows that the teachers considered Upswing training moderately to very effective in helping them to work well with the tutors. The San Francisco project put special emphasis on clarifying the roles of tutor and teacher, with especially fine presentations on what each should expect from and give to the other.

The Denver inservice meetings stressed

147

158

In

general, were the Upswing meetings for teachers useful in helping you and the volunteer tutor(s) work well together?

about the teaching approaches and materials used by the Upswing volunteer(s) who tutored your pupil(s)?

Did you have enough information

Was the CARE used in identifying children with learning problems or potential learning problems?

Questions Asked

12/63%

17/90%

17/90%

Denver

15/88%

12/71%

17/100%

9/82%

9/82%

10/91%

8/95%

15/83%

16/89%

Number & Percentage of Respondents in Each City who Answered Positively San Francisco St. Louis Oxford

TEACHER ASSESSMENT OF THE UPSWING TRAINING PROGRAM CONTENT

TABLE 3 6

53/82%

53/82%

60/92%

Total

techniques for diagnosing and instructing to children's needs, which emph.sis may have overridden the tutor-teacher relationship issue.

Responses to the first

questionnaire indicated that teachers there believed they had a clear view of their role and the tutor's at the start of the project (Table 34).

The questions presented in Table 37 were asked to discover whether any dissatisfactions with teacher orientation/training were caused by logistics or environmental factors rather than content.

was about frequency of meetings.

The most important finding

There were more teachers dissatisfied in

this respect in Denver and San Francisco, and all of them checked that they felt there should be fewer meetings (11 and 7 respondents, respectively). Two teachers in St. Louis thought there should be more meetings for teachers, while one in that city and one in

Oxfol thought there should be none.

(Two

respondents in Oxford and one each in St. Louis and San Francisco skipped this question.)

The teachers checked which, if any, of the last four items in Table 37 caused them to miss Upswing meetings.

Only 18% said they found any of those

aspects so distasteful as to cause them not to attend.

The most commonly

troublesome was schedule; three respondents in Denver, four in San Francisco, and one in St. Louis said the schedule prevented them from attending one or more Upswing meetings.

Two other teachers cited location.

Table 37 shows that the Denver and San Francisco projects both were more successful in establishing a favorable group atmosphere, which would be expected since they had more opportunities to develop it.

(The question-

naire item defined "social atmosphere" as "friendliness, opportunity to express your ideas, etc.".)

However, neither social nor environmental con-

ditions influenced de:isions to attend, according to the questionnaire responses.

There was an interesting pattern of opinion about the importance of meetings for teachers in a project like Upswing.

The fewer the meetings,

the more important teachers thought they were (Figure 13).

This trend apparently

does not point to a trend of teachers wanting something they felt was lacking, since only two, in St. Louis, said they thought there should have been more Upswing meetings.

149

160

18/95%

19/100%

Was the physical environment favorable? *

Was the social atmosphere favorable?* 1

14/82%

14/74%

Was the location favorable?*

4

9/82%

9/82%

9/82%

9/82%

7/64%

17/94%

14/78%

17/94%

15/83%

10/55%

59/91%

55/85%

54/83%

51/80%

53/82%

Total

*Question not asked in the format presented here. Tha attributes were listed and teachers were asked to respond yes/no or favorable /unfavorable. (See "Teacher Final Impressions" questionnaire in Appendix B.)

14/82%

14/82%

12/71%

15/79%

Was the schedule of Upswing teacher meetings favorable?*

14/82%

8/42%

Were you satisfied with the frequency of Upswing teacher meetings?

Question Asked

Number and Percentage of Respondents in Each City Who Answered Positively San Francisco St. Louis Oxford Denver

TEACHER ASSESSMENT OF THE FREQUENCY, SCHEDULE, AND ENVIRONMENT OF UPSWING TEACHER MEETINGS

TABLE 3 7

% in

10

20

30

Category

F esponse

40

50

60

Very

important

15%

Not Important

32%

NR

0% Very Important

47%

Not Important

12%

OXFORD

Essential

Helpful, Not

35%

No They are helpful, but not essential They are very important

NR

Vury Important

46%

Not Important

0%

ST. LOUIS

Essential

Helpful, Not

35%

NR

18%

)

)

Very Important

33%

Not Important

11%

SAN FRANCISCO

E$sent.a:

Helpful, Not

56%

FIGURE 13. TEACHER OPINION ABOUT THE IMPORTANCE OF TEACHER MEETINGS IN A PROJECT LIKE UPSWING

DENVER

EILIRit 41

Not

Helpful,

53%

b. c.

a.

program in the schools? (Checl.-1.inei

WDo you think teacher meetings are important for a smoothly running volunteer tutoring

NR

0%

Figure 14 shows a similar pattern of opinion about whether teachers in general would be willing to attend meetings for a project like Upswing if they were not paid.

Those who had more meetings to attend, more often considered

pay necessary.

The foregoing evidence suggests that teachers tended to dislike having to commit time after school hours to Project Upswing.

Nevertheless, there

also were indications that the "medicine" of Upswing training had some good effects.

Teachers were asked whether they nought participation in Upswing

was beneficial to them professionally, and, if so, how. reasons.

Table 38 shows their

The differences in response pattern from city to city seem to rPflect

benefits of the inservice programs in San Francisco and, especially, Denver, where there was emphasis on offering teachers training per se. From Table 38 the Denver project was rated beneficial in more ways by more teachers than any other project.

The San Francisco project also clearly

provided more teachers with new knowledge of instructional materials and techniques than the St. Louis or Oxford project.

In site visits, we noted that the St. Louis schools offer comparatively little in the way of individualized instruction; thus whatever information Upswing offered in that regard would be more meaningful there than in the other cities, where the schools seemed to put a good bit of emphasis on individualization.

It should be noted that helping teachers themselves to individualize

instruction was not a goal in any project; however, it was thought that increased awareness of the issue might result from involvement in a one-to-one tutoring program.

Thq high Oxford response on use of helping personnel may have a reason similar to the reason suggested for St. Louis's high response on individualization.

Oxford was the only Upswing location in which the schools had never before

had a volunteer program.

Working with volurrPers was a totally new experience

for most Oxford teachers.

The most commonly valued aspect of Upswing participation was "additional understanding about how to diagnose specific learning needs."

Observation

indicated that this was stressed by all projects during the child selection period and that the use of the readiness inventory was important to teachers.

152

163

BEST COPY AVAILABLE

100

9.

Do you think teachers generally would be willing to attend meetings as part

of their involvement in a project like Upswing if they could not be paid? a.

80

b.

Yes No

) 1

76%

70

64%

60 58%

50 % in Response

Category 40

30

20

10

yes

no

DENVER

y Is

NR

no

NR

ST. LOUIS

OXFORD

yes

NR

SAN FRANCISCO

FIGURE 14. TEACHER OPINION ABOUT WHETHER TEACHERS GENERALLY WOULD BE WILLING TO ATTEND MEETINGS FOR A PROJECT LIKE UPSWING IF THEY WERE NOT PAID

153

161

)

1

1

I

12/63%

8/42% 9/47% 6/32%

Gave me greater insight into how to individualize instruction for my pupils

Increased my ability to use helping personnel effectively

Motivated me to further my education

11/60%

11/60%

Denver

Cave me additional understancliro; about how to diagnose si,t;cifie learning needs

teaching techniques

Ac:quaintac. we with additional

Acquainted me with a greater variety of instructional mIteIle;::

"iype of Benefit

2/12%

13/76%

7/41%

8/47%

2/12%

2/12%

Oxford

2/18%

5/46%

7/64%

6/54%

3/27%

3/27%

St. Louis

3/17%

6/33%

4/22%

8/44%

8/44%

10/56%

San Francisco

Number and Percentage of Respondents in Each City Who Checked Benefit (They were asked to "check all that apply.")

13/20%

33/51%

26/40%

34/52%

24/37%

26/40'.

Total

TEACHER OPINION ABOUT THE PROFESSIONAL BENEFITS OF PARTICIPATING IN UPSWING

TABLE 38

Tt appeared that, al thou6 teachers were familiar with the inventory concept and with many of the behaviors it included, many had not used such an instrument in structured observation of children.

Anothci factor may have been that

all projects made it a point to inform teachers of the results of the Upswing pretest battery (which was not done generally in the first year, to teachers' dissatisfaction).

Attendance at Upswing Orientation/Training Tables 39 and 40 summarize project records on teachers' attendance at Upswing meetings before and during tutoring.3 Attendance was better at preservice training than at inservice, and generally was better where there were fewer meetings.

The data suggest that

attendance had little to do with the content of meetings and seemingly much to do with their timing during the year, their frequency, and, probably, the nature of the city.

Oxford offered the most preservice training--8 hours

(or 10 hours if one accepts the position that the two follow-up hours after tutoring began were more in the spirit of preservice training)--and Oxford Denver fared a bit better on pre-

was the only city with 100% attendance.

service attendance than San Francisco, even though the former city held four sessions and the latter two for five hours of training.

Table 39 shows roughly

the same attendance rates for those two cities, but San Francisco had over three times as much inservice training as Denver.

It could b3 that the same

rates would have prevailed if Denver had held as much inservice training as San Francisco.

However, based on questionnaire data presented previously,

Denver teachers appeared to feel somewhat more burdened by the training requirement, even though considerably fewer hours were involved.

It appears likely

that this happened because San Francisco put more hours in at the beginning

We have said that there was no inservice program for teachers in Oxford. Two one-hour meetings were held at each school very shortly after tutoring However, because of the timing began, and these are recorded in Table 33. and content of these meetings, they were more in the nature of preservice training as previously discussed.

155

166

BEST COPY AVAILABLE

TABLE 39

ATTENDANCE AT UPSWING PRESERVICE MEETINGS FOR TEACHERS

Percentage of Training Teachers Attended 90%-100%

Number of Teachers in Each Attendance Category San Francisco Oxford St. Louis Denver (two 2i-hr (five 1-hr (one 8-hr (one 3-hr meetings) meetings) meeting) meeting) 18

10

9

12

80%-89%

6

0

0

70%-79%

0

0

0

60%-69%

1

0

0

50%-59%

0

0

0

< 50%

2

0

6

0

0

3

0

12 100%

18 100%

Totc.I

19

100%

18 100%

156

167

BEST GI AVAILABLE

TABLE 40

ATTrNDANCE AT UPSWING'S INSERVICE MEETINGS FOR TEACHERS

Percentage of Training Teachers Attended

Number of Teachers in Each Attendance Category San Francisco Oxford Denver (four 2i -hour (two 1-hour (four 1-hour meetings and meetings) meetings) one 3-hour)

90%-100%

6

80%-89%

0

0

70%-79%

6

3

60%-69%

0

0

50%-59%

3

6

< 50%

3

2

0

1

0

Total

18

7

19

18

18

100%

100%

100%

157

of the project and held longer meetings separated by greater time intervals. Another point about session length suggests itself:

the teacher meetings

were held right after school and teachers had to drive to them; if a teacher were delayed even a short time, there would not be much point in trying to get to a 1-hour meeting.

CONCLUSIONS ABOUT THE EFFECTIVENESS OF TEACHER ORIENTATION /TRAINING

The foregoing description of teacher orientation/training provides a framework for assessment of the projects' effectiveness in that activity. Two criterion areas were established for the assessment: 1.

Effectiveness in training teachers to identify children with minimal learning difficulties in

accordance with the Upswing parameters for the child population 2.

Effectiveness in promoting good tutor-teacher working relationships.

Criterion 1

The measure of project effectiveness in training teachers to identify appropriate children was the initial characteristic of the children demonstrated on the tests in the Upswing battery.

As described previously, the

tests covered reading, basic experiences, and self-esteem.

It has been noted

that, in addition, the readiness inventory data were to have been used in this assessment, but its use did not prove to be feasible.

The test data indicate that teachers in all projects did select children in accordance with the basic Upswing parameters (potential for normal functioning in school, with one or more minimal learning disorders expected to impact school functioning in the absence of "treatment" that a volunteer tutor could reasonably be expected to provide).

The pretest results were as follows:

The children generally exhibited visual-motor integration skills at about the level expected at age five (exceptions:

Denver and St. Louis control

group children had a mean VMI age equivalent score of 5 years, 7 months).

All but one of the Upswing

children were over 6 years old, and about 60% were over 6-1/2

(The control group children tended to

be younger than the tutored by a narrow margin.)

158

169

The children tested quite low in basic experiences. The mean TOBE score for all children was in the 8th percentile.

Scores on the "Funny Faces Came" (self-esteem measure) were predominately in the borderline range between average and low.

Only about 25% of the children exhibited serious reading problems on the pretest.

About 35%

scored in the low-average range and about 40% tested as average readers for their ages.

Upswing was conceived as a project that focused on reading problems related to reading. tutoring.

Work on reading skills was to be the primary intent of

However, the selection criteria imply that just about any deficit

qualified a child for Upswing tutoring, as long as it was not so severe as to require a specialist.

As discussed in Section IV, it appears that the more

visible difficulties were primary selection factors.

We do not wish to place too much emphasis on reading skills.

Cer-

tainly reading problems are not the only justification for tutoring, but reading was set up as the primary criterion area for the evaluation of Upswing. Further, more significant gains were made in reading than in any of the other criterion areas which apparently were stronger selection factors.

Thus there

was some fuzziness in the definition of project purposes and activities that is Aifficult to clear away.

Reading was the focus of efforts to help children,

although it was not necessarily their area of greatest need.

This was true

despite the fact that Upswing emphasized the importance of looking at children as complete individuals and responding to learning need$ of whatever kind as they arise.

Perhaps child characteristics were believed to contribute to

reading problems that in fact have no bearing on them; certainly, all of the variables measured in the Upswing evaluation proved to be orthogonal. In any case, teachers in all cities did successfully select children Thus, the training they were given

according to the project requirements. in how to select children was effective.

It is another series of debates

whether the selection criteria were overly generalized, whether the stated goals of the project were operative, and whether the child selection criteria were appropriate for the project goals.

The Upswing approach to child selection

159

170

reflects a view that nonspecific learning difficulties are relevant selection factors and, by extension, that one can help children to successful functioning in school without determining what aspects of school achievement are impacted by their behavioral symptoms. The content of instruction on how to select children for the project was very similar from city to city and, as shown in the description of preservice training, the teachers perceived their understanding of selection parameters as good in all cities.

(Roughly 80% to 100% said they had a

clear understanding of how to choose children.)

The format of presentation

of this training apparently had little to do with its impact (i.e., whether teachers were informed individually, as in St. Louis, or in groups). Criterion 2

The measure of training program effectiveness in promoting good tutor-teacher working relationships was the teacher assessment of the value of training in that regard, given in response to a questionnaire item.

Over-

all, 82% of the teachers said that the orientation or training they received was helpful to them in working well with the volunteer tutors.

There were

di.ferences from city to c'ty, with the lowest percentage of favorable responses from Denver teachers (63%) and the highest from San Francisco teachers (95%).

The corrparativej low assessment in Denver may have been

related to dissatisfaction with the greater frequency of teacher meetings they were asked to attend and with a concomitant tendency (again, comparatively speaking) to consider teacher meetings less essential in a project like Upswing.

(San Francisco teachers, however, tended to share in some

negative feelings about the frequency and importance: of meetings, although to a lesser extent.)

Probably a more important reason for the difference

in Denver was that teacher meetings there focused on how to work with children--identification of learning needs and appropriate instructional strategies.

In San Francisco, on the other hand, role definitions and

expectancies were expressed, and teachers' opinions about what the volunteer tutors should be doing were sought actively.

160

171

The data indicate that, on the whole, teacher training resulted in improvements over the first year in teacher satisfaction with the tutor-teacher relationship.

(Tutors indicated satisfaction with the nature of the inter-

action but often wanted -ore? of it.)

Tutors, overall, expressed need for uore

guidance from teachers about as frequently in the second year as in the first, as will be illustrated shortly.

Denver teacher training had a different

focus; probably for this reason teachers saw it as having comparatively less impact on tutor-teacher relationships.

(Thirty percent of Denver tutors in-

dicated they would have preferred more guidance from teachers.

However, in

Oxford, where teachers rated their training comparatively high on promoting good tutor-teacher relationships, about 50% of the tutors said they needed more guidance from their pupils' teachers--it is a matter of which side is perceiving.

nne intervening factor is that as the tutor-teacher relation-

ship grows, the tutor may see the teacher's time as increasingly valuable to her.

Thus, as the relationship quality increases, the satisfaction with

available time could decrease.

DESCRIPTION OF PROJECT MANAGEMENT AND COMMUNICATIONS STRUCTURE IN THE SECOND YEAR OF UPSWING The project directors were free to set up any management and communications structure they wanted, but all adopted essentially the same approach. Figure 15 is a generalized management flow chart.

There were variations to the structure illustrated in Figure 15, but these were more in detail than essentials.

In Oxford there was no school

volunteer organization other than Upswing, and the school liaison staff all were graduate students in the university's Department of Special Education. In Denver an Upswing volunteer from the first year, who continued to tutor, was one of the liaison staff along with two graduate students. there was a "lead" volunteer tutor in at least one school.

In addition,

In San Francisco,

the assistant project director was a school system psychologist who devoted part of her time to Upswing.

In addition, there was a very active, full-time

project administrative assistant, who served also as a liaison person in the schools.

Also in San Francisco, the preexisting school volunteer organization

took a much more active role than its counterparts in St. Louis and Denver. A representative of the

'an Francisco organization participated in Upswing

161

172

REST COPY AVAILABLE

Public School

41.1.. *mow,

......

Upswing

m....., lamino worwror Iios 40

Project Director or Coordinator

Administrators

4

F

OM/. PIMP.

Public Sr.hool Volunteer Proprem

(None in Oxford)

Project

Administrative Assistant (San Francisco only)

4..... .......... .

Secretary

1

1

L.

Volunteer Tutors

4 +F..

Direct Responsibility

41. Cooperation

FIGURE 15. GENERAL MANAGEMENT STRUCTURE OF PROJECT UPSWING

162

173

1

planning and training meetings, called tutors about attending training meetings and to check on suggestions, problems, etc.

In St. Louis, the

assistant project director, as well as all other staff members, was a graduate student in the Department of Special Education.

Responsibilities

assumed varied with the personalities and expertise of the people involved, but despite such differences, the projects were more alike than dissimilar in management framework.

There were three essential management tasks in running Project Upswing--(l) start-up activities including agreements with the school system and the individual school administrators, defining needs for staff and the roles of individual members, hiring staff, the recruiting campaign, tutor and teacher orientation, organization and management of the child selection process and conduct of testing, etc.; (2) planning and conduct of ongoing

training for tutors and teachers; and (3) supervision of tutoring activity in the schools.

The project directors, with varying degrees of help from key staff members, took major responsibility for start-up tasks and the training activities.

As the project developed, staff generally took more and more

responsibility for planning and conducting training, but the directors maintained leadership of formal training activities.

Staff soon functioned

independently in their school liaison roles except if difficult problems arose.

The essence of Upswing was the activity in the schools, and it appears that although some training is desirable to establish project identity and give tutors a feeling of competence, the training component can be a very small part of program activity.

From a management point of view, a strong

director seems essential at the beginning of the project, but this highlevel person probably need spend relativel! little time on it as staff in the schools take over routine activity.

However, observation indicated that

a readily ac,:essible, informed central figure is necessary.

The time demands

on staff working in the school liaison function, however, remain fairly heavy throughout the tutoring period.

Figure 16 shows the general nature of management and communications during tutoring. stronger.

Again, there were differences, but the similarities were

Each project assigned staff to visit each school regularly to

163

174

BEST COPY AVAILABLE

Frer.uent, regular communication Periodic communication as needed

4. . Minor communication FIGURE 16. GENERALIZED MODEL OF UPSWING MANAGEMENT AND COMMUNICATIONS IN THE SCHOOLS

164

175

assist tutors and check on attendance, to make sure project materials were in order, to resolve problems that might be of concern to teachers and principals, etc.

The school liaison personnel were more deeply involved in some cities

than others, but their basic function was the same everywhere - -to be a re-

sponsible project representative, readily available to keep things running smoothly.

PARTICIPANTS' REACTIONS TO PROJECT COMMUNICATIONS IN THE SECOND YEAR Tutors and teachers were asked questions related to project communications on their respective "Final Impressions" questionnaires.

Their opinions

about this aspect of Upswing are reported here, as well as general satisfaction with the project.

Principals' reactions also were sought and are presented

as the final elements of data for evaluation of Upswing training and management.

Tutor Opinion About Communication

Most of the tutors either said they were "satisfied" or "strongly satisfied" with the amount of communication they had with the Upswing staff, their tutee, the tutee's teacher, and other Upswing volunteers.

San Francisco

tutors who responded to the questionnaire seemed to be somewhat more satisfied in each of these categories than the respondents in the other three cities. Substantial proportions of Denver, Oxford, and St. Louis tutors did have some difficulty with teacher communication.

Comments from these cities

indicated that tutors felt teachers did not provide enough feedback on the progress the children were making.

Thus, tutors were concerned that they might

not be progressing with the children in the best direction.

About 30% to 50%

of tutors everywhere but San Francisco wanted more teacher guidance than they received.

Figure 17 is a graphic representation of the mean satisfaction scores for each type of communication.

Tutors indicated which channels were most important to the Upswing process.

They indicated clearly ,.hat communication with the child was most

important; this was followed closely by communication with the teacher.

Of

lesser importance was communication with Upswing staff and of relatively little importance was communication with other tutors. need of improvement is between tutor and teacher. lack of time for talk on both parts.

The channel most in

The reason is probably a

The solution will not be simple. 165

176

t

ll)

(1.3)

A

(1.2)

A

(1.4)

A

i t i I i

1.5 1

0

-.5

(0.8)

A

-1

-1.5

i

2

strongly dissatisfied

itiiiitii

dissatisfied

it!' I till iti 1h ilt

.5

neutral

Amount of Satisfaction

The relative importance of each channel is ranked on a scale from 1-100, based on the frequency with which tutors selected them as most important.

2

satisfied

FIGURE 17. TUTOR SATISFACTION WITH VARIOUS CHANNELS OF COMMUNICATION WITHIN UPSWING

Tutor-Other Tutors

Tutor-Teacher

Tutor-Upswing Staff

Tutor-Child

Channel of Communication

strongly satisfied

6

70

33

90

Importance Ranking of Each Channel

.

By city, pupil communication was rated firs: also. choice varied among, the cities.

But the second

oxford respondents telt communication with

Upswing staff was second in importance while respondents in the other three cities chose teacher communication.

Tutors' Overall Satisfaction With Upswing About 90% of the tutors said they found their experience in Project Upswing satisfying or strongly satisfying.

The split between the two degrees

of satisfaction was virtually even for the project as a whole.

There defi-

nitely was a greater degree of satisfaction expressed in Oxford, where almost 607 said they were strongly satisfied, and the remainder responded "satisfied." Only four tutors found the experience unsatisfactory (one out of 31 respondents to the questionnaire in Denver and three out of 15 respondents in San Francisco). Four others felt neutral about it (two in Denver and two in St. Louis.)

Table

41 gives the responses by city and project-wide. Cross-tabulated data indicated that satisfaction with communications of all kinds, satisfaction with the amount of guidance received from the teacher, and the tutor's perception of how much tutoring helped the child, were the most important determinants of overall satisfaction with the project.

Comparison

between neutral or dissatisfied tutors and those who expressed satisfaction is not meaningful because there were so few in the first category.

However, the

just-mentioned variables clearly had an impact on whether a tutor was generally "satisfied" or "strongly satisfied." Teacher Opinion About Communications Nearly all teachers were satisfied with the amount of interaction between them and Project Upswing staff (Table 42).

As shown previously, teachers also

generally considered their relationship with the volunteer tutors good, although there was less unanimity in this area.

When asked to rank forms of tutor-teacher communication in order of desirability from the teacher point of view, teachers showed considerable diversity of opinion.

However, more were inclined to prefer informal, one-

to-one interaction on an as-needed basis.

There was some tendency for teachers

to want to maintain control of this interaction (657. ranked teacher-initiated,

informal one-to-one meetings as first or second choice, while 53% ranked tutorinitiated meetings of this kind as first or second).

was Pegular tutor-teacher meetings, again one-to-one. 167

178

The other favored option Eighty-four percent

18/58% 2/6%

1/3% 31/99%***

Satisfied

Neutral

Dissatisfied

Total**

43/100%

0

0

18/42%

25/58%

Oxford

55/47% 4/3% 4/3 X

117/99%***

7/47% 5/33% 0

3/20% 15/100%

12/43% 14/50%

2/7%

28/100%

0

54/46%

San Francisco

***Rounding error.

Total**

St. Louis

*There was a "strongly dissatisfied" response option, but no one chose it. **Two non-response cases in St. Louis omitted.

10/32%

Denver

Strorgly Satisfied

Degree of Satisfaction

Number and Percentage of Respondents in Each Category*

(Percentages based on number of respondents in each city who answered the question.)

TUTORS' OVERALL SATISFACTION WITH UPSWING

TABLE 41

Total

No response to question

than necessary

3tz.::::: contacted no more

contact

D,-/sired more informal

Sausfieli with contact

Teacher Gpinion

19/100%

0

0

2/10%

17/90%

Denver

17/100%

0

1/6%

11/100%

1/9%

0

1/9%

18/100%

0

0

0

18/100%

9/825;

14/82% 2/12%

San Francisco

St. Louis

Oxford

Number and Percentage in Response Category

(Other than in meetings) Total

65/100%

1/17_,

1/1%

5/8'A,

58/90e.

TEACHER SATISFACTION WITH AMOUNT OF TEACHER/UPSWING STAFF CONTACT

TABLE 42

ranked this choice first, second, or third, but it was less favored than the as-needed options as first or second choice.

Teachers also were asked for an indication of their opinion about management and support of tutors in the schools by Upswing staff.

Seventy-

five percent considered the staffs' in-school activity the key reason for project effectiveness, while another 5% considered it useful but not essential. fable 43 gives a complete breakdown of the responses.

This question was asked to see whether teachers considered the school liaison role filled by someone outside the regular school staff as important as it was hypothesized to be.

Principals were asked a similar question.

The

responses indicate that the liaison function as defined in Upswing was critical in the opinion of most school personnel involved.

Teachers' Overall Satisfaction With Upswing Dverall, teachers indicated about the same level of satisfaction from their involvement in the project as tutors.

Eighty-nine percent either

were "satisfied" or "strongly satisfied," with about an even split between these two responses (Table 44).

San Francisco and Oxf)rd teachers expressed

somewhat lower satisfaction than those in Denver and St. Louis, but these possible differences are difficult to interpret in view of the small numbers involved.

Level of satisfaction with Upswing was not related to opinion

about the effectiveness of tutoring in helping children.

Only two teachers

considered tutoring ineffective, and both of them said they were strongly 01.~1:-.3:7,ed with their experience in Project Upswing.

The data from teachers

did not indicate the primary contributors to high satisfaction as did the tutor data.

Principals' Opinions About Project Upswing The principals of the 19 schools that participated in Upswing's second year also were sent a questionnaire at the end of the tutoring period (copy in Appendix B).

All 19 returned the form.

The overall impression gained

from the questionnaire responses was that the principals were enthusiastic about Project Upswing and were anxious to see it continue.

One-to-one tutor-

ing was seen as having "great potential" for helping children overcome learning difficulties by 18 of the 19 respondents and as having "moderate potential" 1 7 0

181

Total

No response to question

I don't know

needed, but the support given by the Upswing staff was not effective

than tat school's personnel is

i..!ividual support by someone other

19/100%

1/5%

0

1/5%

0

It is detrimental

i---

6

3/162,

it is ,:seful, but net essential

It is net needed

1.: /74%

Denver

17/100%

2/12%

4/:3%

1/.3 %

1/6%

0

0

9/53%

Oxford

11/100%

0

2/18%

0

0

0

0

9/82%

St. Louis

18/100%

0

1/6%

0

0

0

0

17/94%

San Francisco

65/100%

3/5%

7/11 A

2/3-:,

1,/1%

0

3/1:_-;,

49/75%

Tot::'.

BEST COPY AVAILABLE

Number and Percentage in Response Category

It is tie reason for Upswing's effctiver.:2ss

Teacher Opinion

TEACHER OPINION ABOUT INDIVIDUAL SU z PORT GIVEN TO UPSWING TUTORS BY PROJECT STAFF

TABLE 4 3

1/5%

Neutral

19/100%

16/101%***

0

2/13%

8/50%

6/38%

Oxford

11/100%

1/9%

0

4/36%

6/55%

St. Louis

16/100%

***Rounding error.

1/6%

2/13%

8/50%

5/31%

San Francisco

*There was a "strongly dissatisfied" response option, but no one chose it. **Three nonresponse cases omitted, one in Oxford and two in San Francisco.

Total**

0

8/42%

Satisfied

Dissatisfied

10/53%

Denver

62/100%

2/3%

5/8 X,

28/45%

27/44 '4

Total**

BEST COPY AVAILABLE

Number and Percentage of Respondents in Each Category*

Strongly Satisfied

Degree of Satisfaction

TEACHERS' OVERALL SATISFACTION WITH UPSWING

TABLE 44

by one.

The 18 assessed teacher attitudes toward the project as "positive"

co "very positive."

The major problem seemed to be space for tutoring.

The questionnaire included a check on whether Upswing training and management practices would give Upswing tutors preferred status.

The

principals' endorsement of volunteer tutors was not restricted to Upswing; however, all but one said they welcomed volunteer tutors in general. Reservations were expressed about high school student tutors and untrained tutors.

'fp;''n,: 2nd Surerv:son.

Almost all of the principals considered

training essential for tutors, and Upswing training was assessed as high in quality.

The inservice approach was endorsed.

The Upswing staff's individualized support to tutors in the schools was considered the key reason for project effectiveness by 16 of the 19 respondents, and useful although not essential by two.

One principal felt

that the Upswing staff did not do a good job, but someone other than regular school personnel is needed to perform this function.

Six principals said

they did not believe a program like Upswing could be run at all on an individual school basis, while 11 thought it could be done if funds were available for additional staff to conduct training and supervise tutors, as well as for materials. Trrro-tements Frogs the First to Second Year.

All but one of the

principals participated in Upswing both years, and 11 noted improvements in the second year.

It was observed that the tutors seemed more confident,

better prepared, more organized, more interested and conscientious, and more dependable.

These opinions were expressed by principals in all four cities.

Principals from St. Lciis, San Francisco and Denver found an increase in the supervision by.the Upswing personnel working with the teachers and tutors.

Principals in San Francisco, St. Louis, and Oxford reported increased en thusiasm among teachers for the Upswing tutors and/or better rapport between tutors and teachers during the second year of the project. Principals registered the most unanimous enthusiasm for Upswing of any participant group.

Comments included at the end of the questionnaire

touched upon the high caliber of the Upswing tutors, the rapport established

173

184

between rotors and teachers, and the enjoyment that the participating children leceived from the addid attention given them.

A quote from one

of the St. Louis principals sums up the prevailing attitude:

"[I have]

nothing to add but positive thoughts and feelings about the Project." CONCLUSIONS ABOUT THE EFFECTIVENESS OF MANAGEMENT AND COMMUNICATIONS IN UPSWING Although there are ambiguities in assessing any social system, general principles can be accurately evaluated. a clear pattern has developed.

In the case of Upswing,

The project is perceived as desirable,

effective, and useful, and these qualities are closely associated with the unique way in which the project was run. to the building-level coordinators.

Great responsibility was delegated

Much of the activity in Upswing was

almost autonomous, but run in cooperation and communication with the teachers and the system.

The management system is so flexible that Upswing has

been able to adapt to each school's unique needs and problems.

Vast dif-

ferences in attitude and environment found at schools within the same system are not 'j usual.

They do, however, present a problem to volunteer

programs which are centrally organized (as many are).

Close, continuous

communication between Upswing personnel and school personnel appear to have been essential to effective management. It is apparent that the person who is given responsibility for coordinating volunteers in each school should be sensitive to the political realities that affect teacher and principal attitudes.

This sophistication

appears to have been a very important factor in keeping Upswing a minimal burden on the odiools.

Because school personnel apparently were able to

see Upswing as a small effort for large return, the attitudes of teachers and principals seem to have grown steadily more positive.

174

185

VI.

THE PROBLEM OF ATTRITION

Upswing, 111,e all volunteer programs, has the inherent problem of attrition.

This section of the report uses the Upswing experience as a base

for discussion of what level of attrition can be expected under various circumstances, and present3 some ideas on how to compensate for the problem. WHAT TO EXPECT The average Upswing Project can expect to lose 20%-40% of its original trained volunteers by the end of the school year.

Attrition is

influenced by many environmental and personal factors, from moving to lack of motivation.

These can be classified into two major areas:

attrition

that can be anticipated, compensated for, and to some extent controlled; and attrition beyond project control.

The Upswing experience indicates

that about 107 of attrition will be in the former category and about 70% will be uncontrollable for an average program. Early Losses

In the Upswing projects that depended on adult volunteers from the community,

1 large number of people who agreed to participate dropped out

before tutoring began

This was especially true in the first year, when

there was a lengthy delay between orientation /pre ;ervice training and the

start of tutoring.

Even without such a delay it would be wise to anticipate

175

186

that probably 25% to 50% .of the people who show initial interest will not beRin tutoring.

Evidence suggests that about 10% of the people who begin train-

ing can be expected to leave for reasons associated with dissatisfaction with training or the project management.

Such problems as the training

schedule, transportation, training too simple or advanced, poor instructions or guidance, etc., have been voiced by volunteers.

An awareness of such

attitudes on the part of the project manager can significantly improve retention of volunteers.

Attrition After Tutoring Begins Roughly 207 of beginning tutors can be expected to drop out of the program completely some time during the school year.

In addition to these

true "dropouts," a number of tutors appear to have dropped out for several short periods during the year, but returned to the pupil. was not reported until too late to document reasons.

Such activity

Accepting the premise

that if a volunteer fails tc meet the child for more than one-half of the appointed times he is not an active volunteer, fully 50% of the original volunteers can be expected to be lost or go inactive by the school year's end.

Much of the gradual attrition is attributable to moving and job acquisition, or the ambiguous "personal problems."

These do not appear to

he areas of loss over which the project manager can effectively exert control. Heavy use of college student tutors (perhaps younger students as well) changes the picture entirely and seems to make management of attrition much simpler.

For example, Oxford, which used many students in the project

had virtually full control of attrition.

All of the people leaving the

project did so according to schedule in December. and plans were made to compensate for it.

The loss was expected,

In this case attendance was

controlled through the granting of college credit to students who faithfully completed a full term of tutoring.

The loss came at the end of the

quarter when students left or changed schedule. PLANNING AHEAD It is clear that an effective Upswing project must foresee attrition and compensate for it.

Each of the four cities in Upswing did this with

176

87

lottroes of success. t- eu:er the pro.

one of C10 major problems was that of preparing (t

e..

t-kai was compounded

because the evaluation needed consistent treatment of tutors with regard to trainitut, at the city level, for clarity of findings.) in several ways.

This was approached

For example, videotape was used to record the original

This was not highly effective because of the length of tapes (video-

les:,ons.

tape is very expensive and difficult to edit) and the lack of a chance to respond.

Sometimes the nonprofessional quality of the tapes was a further

drawback.

In one case the new recruit viewed tapes with the presence of an

in,;tructor prom the Upswing staff.

This was a clear improvement, as it allowed

interaction based on the taped material.

In other cases new volunteers were given a brief overview, with no ,ttemot at full coverage of the preservice training. to the need for extensive tutor training.

No evidence points

It appears that orientation with

a minimal amount of inservice training, preferably more toward the beginning of tutoring, would he effective.

In fact, orientation-only evidently is

suf icient if adequate supervision in the schools is provided.

Perhaps

the ideal plan is to have a continuous program of recruitment and orientation so that Upswing can maintain a smooth continuity of service to the children.

177

188

VII.

COST ANALYSIS

This section very simply describes the basic expenses in Project Upswing's second year.

The data do not provide a foundation for detailed

analysis; further, pilot study costs under a fixed grant arrangement do not necessarily give a true picture of what the project might cost in another framework.

In fact, since there appear to be only a few required features

of Upswing, costs could vary greatly, depending on local circumstances and the preferences of those setting up the project.

Even under the fixed grant,

(each project had about $30,000), allocations of funds differed considerably from city to city.

Ultimately, it must be left to the educational planner to weigh the benefits children derived from Upswing and decide if it is a competitive alternative for providing special help to children.

This decision would be

helped by comparison of the cost (as estimated for local conditions) and results of Upswing with the cost of remedial reading or with the cost of alternatives for boosting children's self-esteem or experience base (where such alternatives exist and their costs can be specified).

An estimate like cost per unit of

mean gain per child, as illustrated here, should suffice.

It also should be

noted that the characteristics of the project and therefore its budget should depend on what one wants it to accomplish.

It appears that if benefits to

tutored children are the sole concern, then a model such as that used in St. Louis during the second year would be appropriate, and it could cost considerably less than it did in the pilot project framework. 179

However, if one is

interested in the kind of generalized benefit observed in the other three it seems necessary to train teachers or provide some mechanism for strong teacher involvement.

The project would then cost more, and it may

be that Upswing is not thu best way to modify the school environment so as to achieve the generalized effect.

Other factors might also be considered

in the decision to use or not use Upswing--as one example, the benefit to the school system, to society, or both, of increasing the number of opportunities for community service.

We cannot be exhaustive here, but these a:e

some of the kinds of questions that would be in order in making a decision about whether to adopt a program like Upswing. METHODOLOGY AND DATA SOURCE FOR THE COST ANALYSIS Each of the four projects sent ORI a cost breakdown (see Appendix B) for a copy of cost report form).

The breakdown included estimates of per-

centage of time spent on various activities required to run Upswing, the cost of materials, miscellaneous expenses incurred during the year, etc.

The total

cost incurred for each activity was then distributed among the following: Trained tutors who remained active Trained tutors who dropped out Teachers who remained active.

Tutor costs were calculated on the basis of number who were involved in the project at the time the costs were incurred.

Teacher costs were based

on the total number of teachers trained, since very few left during the year. THE COST OF A TUTOR IN EACH PROJECT Table 45 presents the expenditures made in training and assisting one tutor in each project during the 1972-73 school year.

Denver and Oxford spent

approximately the same amount, while St. Louis and San Francisco spent more per tutor.

The latter cities would have had lower costs if they had filled

their goal quota of 50 volunteers per city, and if they had experienced less tutor attrition.

In all cases, the greatest expenditures were for training

and for supervision/assistance provided to tutors.

180

190

BEST COPY AVAILABLE

TABLE 45

COST OF ONE TUTOR WHO WAS IN PROJECT UPSWING FOR THE ENTIRE YEAR

Cost Category

Denver (40)*

Oxford (47)*

St. Louis

San Francisco

(32)*

(32)*

Recruiting

$

3.i

$ 19

$ 60

$ 55

Preservice training

$ 59

$ 19

$ 51

$ 64

Inservice training

$ 98

$ 63

$ 88

$153

$ 74

$ 60

$ 92

$166

Running project

$151

$317

$255

$301

Materials

$ 24

$ 13

$ 22

$ 26

Other

$ 10

-

$447

$491

Supervision

& assistance

Total

$

8

$576

$

1

$766

*Weighted average of the number of tutors during the 7-month span of the project.

181

191

cnsT TO LOSE ONE TUTOR DURING THE YEAR Training tutors who end up not tutoring is an expensive proposition. the cost-effectiveness of an Upswing program is definitely impacted by attrition. year.

Tutors left: Project Upswing for various reasons throughout the

Table 46 indicates the cost of losing one volunteer in each city.

The cost of attrition was lower in Denver than St. Louis or San Francisco. Denver only lost five volunteers whereas the other two cities lost 18 and 15 volunteers, respectively.

The coat of attrition in Oxford was considerably

lower than in my of the other three cities because tutors there tended to be lost early in tho ;Iroject and training as well as supervision costs were spread evenly over the year.

The other cities' losses were distributed

through the year, and there tended to be heavier outlays at the beginning. Interpreting the meaning of lost volunteer costs can be difficult. Costs increase the longer the person remains in the program before leaving, but consider the services performed by the late dropout.

A large number of

early dropouts can be disastrous because they cost, but give little or no service.

Table 47 shows that Oxford's costs were the lowest; they got about

2-1/2 months service for $72, or about $37 a month per person.

Denver, on

the other hand, averaged roughly 4 months service or about $90 a month per person.

St. Louis paid more than $175, while San Francisco paid about $121

per month for each dropout.

Table 47 provides a quick comparison of the

cost per month for both dropouts and "stay-ins."

Attrition was quite ex-

pensive, except in Oxford, which planned its attrition carefully and actually appears to ha-

profited from its dropouts.

This appearance, however, is at

least partly attributable to the university's bearing some of the cost of training, since Upswing constituted a university course. the meeting room was free.)

(For example,

The cost of recruitment was lower as well be-

cause a large number of the tutors were obtained from among the students of the School of Education.

Thus, public media did not have to be relied upon

so heavily.

CoST TI) TRAIN ONE TEACHER FOR PROJECT UPSWING

As the city models have indicated, teacher training varied considerably. From Table 48, Oxford spent less to train and provide liaison with teachers compared to the other three cities.

There were two reasons for this:

182

192

BEST COPY AVAILABLE

TABLE 46

COST OF ONE TUTOR WHO DROPPED OUT OF PROJECT UPSWING DURING THE YEAR

St. Louis

San Francisco

(5)*

Oxford (12)*

(18)*

(15)*

$ 90

$ 38

$111

$119

Inservice training

$ 98

$

8

$ 89

$120

Supervision

$ 58

$

8

$ 69

$130

$119

$ 38

$258

$236

$365

$ 92

$527

$605

Cost Category Recruiting & preservice training

& assistance Running project Total

Denver

*Number of tutors lost during the 1972-73 school year.

183

193

$81.0 $106.0 $121

$175

$37

$91

Dropouts

-

$109

5.5

San Francisco Mean per Month

$82

3

St. Louis

$70

2.5

Oxford

$63

4

Denver

BEST COPY AVAILABLE

Non-dropouts

service by:

LriL.n.L11 for tutor

Average cast per

months service for tutor dropouts

Average number of

Category

THE AVERAGE MONTHLY COST OF TUTORS

TABLE 47

TABLE 48

TEACHER COSTS IN PROJECT UPSWING

BEST COPY AVAILABLE

Denver

Oxford

St. Louis

San Francisco

(19)

(18)

(15)

(18)

$130

$57

$123

$278

Inservice training

231

31

210

124

Liaison activity in schools

127

65

131

250

55

57

67

60

3

13

13

-

4

-

-

-

6

-

-

$550

$229

$544

$712

Cost Category

Preservice orientation, training,

(staff, guest speakers, facili-

ties, etc.)

Stipends

Materials (office..

supplies, reprints, film, videotape used in training, etc.) Equipment

(projector, coffeemaking equipment,

etc.) Overhead Total

185

195

There were only two public elementary schools in the Oxford project, which cut costs on staff interaction with teachers (the other cities averaged five to six schools).

Less extensive training was given teachers in Oxford than in Denver or San Francisco.

The highest teacher costs were in San Francisco.

Preservice training and

liaison activity were more expensive there because of the salary levels of personnel involved.

The San Francisco project paid for the services of a

university faculty member to help with preservice training.

One of the staff

members responsible for school liaison was a school system psychologist who served half time as assistant Upswing director.

The other staff member

responsible for school liaison was the full-time project administrative assistant.

The other projects used graduate assistants earning half- or

quarter-time stipends for the school liaison work. Teacher training costs in St. Louis are somewhat puzzling.

They

are roughly equivalent to the costs in Denver, where a great deal more work with teachers was done.

The activity in St. Louis was much more like that

in Oxford, with one preservice orientation meeting (8 hours in Oxford and 3 hours in St. Louis) and no inservice activity except at the individual school level.

It should be remembered that the cost data represent rough

allocations of a fixed budget which vary in accuracy according to the time spent preparing the allocation and the precision of local recordkeeping. CnST TO UPGRADE CHILD PERFORMANCE ON WRAT Perhaps the most interesting way of looking at Upswing's cost is in terms of how much the gains cost.

Table 49 gives a brief rundown of the

wide range of cost from $33 to $89 per WRAT standard score point per child. The standard score increase indicates improvement beyond the expected level of performance.

POTENTIAL FOR COST REDUCTION

Since Project Upswing was designed to be flexible, one would expect costs to vary according to location and conditions.

More important, one

would expect costs per child served to vary according to the management 186

196

65 89

32.4

Grand

Total

mean

San Francisco

129

-

7.75

323 -

-

34.2

6

64

6

83

St. Louis

$58

-

33

30.8

10

94

Oxford

i

$43

9

Cost/Child/Point

Mean d+ Points

BEST COPY AVAILABLE

Total Cost (in thousands) $31.6

Score

82

Total Children

Denver

City

THE RELATIVE COST OF POINT GAINS ON THE WRAT BY CITY FOR POOLED TUTORED AND CONTROL GROUPS

TABLE 49

expertise, etc.

The findings in this cost analysis, however, are distorted

by the fixed funding level of each project.

The artificial level of just

over $30,000 caused great fluctuations in the per-child costs depending on the number of children served.

Further, the findings are misleading

because of the added cost of high-level administration personnel not required (at least not full-time and probably not even half-time) after the program becomes operational.

Evidence indicates that through the use of volunteer help to run portions of an Upswing program, use of token payments to training people and a single, carefully chosen paid administrator, Project Upswing could be run for less than $14 per child per month if 150 children were served.

It

would be possible for two or more small communities to cooperate on an Upswing program to save money.

Considering that about 20% of the average class

usually falls into the definition of an Upswing child, there would have to be about 750 first-graders in a community to require a full-time administrator.

Smaller communities could, by not using a full-time administrator,

keep costs to a bare minimum.

188

198

VIII.

CONCLUSIONS AND RECOMMENDATIONS FROM THE UPSWING EVALUATION

CnNCLUSIONS

This section has three parts. years of Upswing are reviewed. swing are put forward.

First, major conclusions from the two

Then recommendations about the future of Up-

Finally, the experience suggests recommendations for

further research--some that have to do with evaluation of Upswing-type programs and some that have to do with other issues that arose from the Upswing data. These recommendations are stated briefly in hopes that they may be useful to educational research planners and practitioners. Upswing tutoring was effective, and the project seems to have given unexpected benefits beyond the good of tutoring. Upswing developed in such a way that it had two results--one intended and one a byproduct.

The two-year experience indicated that tutoring does

help the children who receive it, and the tutors who give it, to an important degree.

The experience also indicated that a project like Upswing, under cer-

tain conditions, can extend its impact to teachers and through them to children

who do not receive tutoring.

The tutored groups of children in the first

year made significantly greater gains in reading than did the untutored comparison group.

In the second year, tutored and untutored alike demonstrated

189

clear progress in reading and in development of self-esteem and basic experiences.

The project's goal was to help children realize their potential for normal functioning in the skill areas studied.

The test results indicate

this goal generally was accomplished in the areas of reading and self-esteem. End-of-year test means in both areas were in the average range, whereas at the

beginning of the year the mean scores were in the low-average or borderline range.

The children had farther to go in terms of basic experiences and did

not pull up to average, but they did make significant progress in this area.

The gains of control group children in the second year have been attributed to Upswing, with teachers as the agents.

The project greatly

strengthened its efforts to involve teachers in the second year, and it appears that teachers related differently to at least the Upswing control group children because of the influence of the project.

It is impossible to

say what portion of the teacher effect is attributable to Upswing training in itself.

We would guess that little is, based on the findings about the effec-

tiveness, as measured by child progress, of tutor training.

However, it was

intriguing (and rather shocking) to find that 60% of the teachers who returned evaluation questionnaires said they had no training in child development other than that received in Upswing (all of which was provided in the second year). With regard to the amount of progress measured in the evaluation--it was modest, but indicative of significant growth. and appear to represent true effects.

The test data behaved ',ell

Other studies have reported more dramatic

gains under tutoring, but we tend to be wary of accepting such findings.

Enor-

mous strides may well be made by some individual children; this happened in Upswing.

On an aggregate basis, however, such large gains seem questionable. Sometimes amount of progress has to do with tutoring goals.

If, for

example, the goal is to help children overcome deficiency in recognizing vowel sounds, one can anticipate great success from a program of one-to-one drill over a few weeks.

The subject matter of tutoring is quite limited; if the

pre- and post-tutoring measurement device directly addresses the subject matter that was covered in tutoring, a high level of accuracy on the posttutoring test would be quite reasonable except among children with certain types of learning problems.

Upswing's goal for reading was much broader, and

190

200

a multitude of nonspecific deficiencies were being addressed. measurement device was a standardized instrument.

Further, the

Tutoring was not measure-

referenced; rather, the attempt was to help children acquire skills that could be generalized.

In these circumstances, the amount of mean progress

demonstrated in Upswing seems both reasonable and quite heartening.

We do

not assume that volunteer tutors generally work miracles where the proftssionals have failed, although with the right timing and a fortuitous combination of personalities, "miracles" have been wrought in tutoring as well as in the classroom.

Does Upswing tutoring work?

The data indicate that it does in the

areas of reading, self-esteem, and basic experiences.

The only area studied

in which tutoring did not appear to have appreciable impact was visual-motor integration; although there was a glimmer of improvement in the second year. Beyond these basic questions, the evaluation yielded Several most interesting findings.

These will be reviewed before recommendations for the project and

for further study are made.

The attributes studied in Upswing developed independently of each other. None was a oataZyst. Children experienced different amounts of success in different areas. The Upswing children tended, at the beginning of the year, to show low skills in all criterion areas.

However, the initial associations are mis-

leading lt that development in the various areas seems to go on independently. In other words, the criterion variables were orthogonal.

Other researchers

appear to disagree; they have found rather strong associations.

The TOBE and

the VIII documentation cite correlations on the order of 0.4 to 0.5 between those

measures and reading measures.

An r of about 0.3 was obtained between the TOBE

and the introversion-extroversion scale of the Classroom Behavior Inventory (Earl S. Schaefer).

The difference in findings probably is attributable to

two conditions: 1.

The Upswing evaluation looked at amount of change while the test developers cited looked at actual scores at a given point in time.

191

2.

There were considerable differences in the operational definitions (measures) used.

Knowledge of the amount of association in terms of change or development would seem to have more utility in helping children, but the necessary measurements are more time-consuming to obtain and more complicated in interpretation. Thus, the development approach often is not used. Probably most people in education and educational research believe that there is an association between school achievement level and self-esteem.

The

Upswing data do not belie this; they simply indicate that development of selfesteem can go on, or may not go on, regardless of a child's academic development.

Individually, Upswing data are not conclusive, but the overall pattern

is clear, that significant improvement in self-esteem will occur with personal attention from someone the child perceives as important in the given en-Aronment (be it tutor, teacher, or another) and who believes in the child's capabilities and values his accomplishments whatever they may be. may be preceded by an improvement in skills.

This improvement

In son? children it may be con-

current with, or followed by, improved skills; in others, the skills appear unchanged within the short study period.

Judging from the content of reading readiness inventories, it appears that visual-motor integration skills are commonly considered important in the development of reading skill.

In the two years of Upswing, no support for that

belief was found.

It often has been put forth that "cultural deprivation" blocks development of reading skills.

The Upswing evaluation tried to measure factors repre-

sented by that term through the Test of Basic Experiences, which, given the age group, would have to come from home and kindergarten.

Kindergarten exper-

ience showed no relationship to initial TOBE results for the Upswing children, so it is assumed that the measure was of home-contributed basic experiences. Although the correlations between initial TOBE scores and initial scores from the other tests were significant, "treatment" appeared to override the influences of basic experiences or family background.

The children made unexpected gains

192

202

BEST COPY AVAILABLE on the TOBE itself.

Moreover, progress in other areas shwed no meaningful

relationship to initial TOBE score. It looks as if tutoring could be an effect:.ve substitute for remedial reading. Moreover, it appears that the project presence may have made remedial reading work better. The above statements may seem extravagant; but the Upswinl.. data showed

that tutored children averaged about as much gain in reading test standard score as children who had remedial reading.

Moreover, children who had both a tutor

and remedial reading averaged no more gain than those who had just one or the other.

In fact, there was a suggestion that having both might be detrimental. The follow -up on children tutored in the first year of Upswing added

an interesting twist.

These children had no involvement with the project during

their second year in school except that they were tested at the end of the year. Of the fellow -up group, 86 had special services and 80% of those had either remedial reading or a reading tutor (mostly the former).

This group showed a

slight decline in reading skill on the follow-up test in relation to the test given at the end of their association with Upswing.

Although these data are

not conclusive, they strongly suggest that under ordinary circumstances, remedial reading may not be effective.

It appears that being identified with

a well-defined special project makes a difference in the kind of attention given children and the benefits they derive from it.

Further, in view of

the follow-up data, one must consider the possibility that it was not remedial reading that caused children in the second year of Upswing to gain but instead something in their relationship with tutors or teachers.

Children generally held their ground in the year after Upswing; they did not continue to increase their rates of development in reading. The follow -up data show that the children tended to maintain their age-

adjusted standard scores in reading during the year after Upswing.

That i3)

they continued to acquire reading skills at the rate they had established by the end of the Upswing year.

This was true of both former tutored and former

193

203

BEST COPY AVAILABLE control group children.

(There was a slight drop in the mean scores, but it

was not incerpreble as a tendency to lose ground:

average point loss on the

reading test of about two points for tutored and half a point for control). The nature of attrition from the follow-up group, in conjunction with the general trend for children to hold their own in the year after tutoring, indicates that the benefits of Upswing tutoring may well be stable. :7hildren whc were retained in first grade retrogressed in reading during the followup year.

The decline in mean reading scores of the follow-up groups is mostly attributable to losses by children who were retained.

The mean standard score

of these children was in the average range at the end of Upswing. the group had fallen back into the low-average range. points in standard score.

A year later,

The mean loss was six

(The initial standard deviation was small:

points; the final standard deviation was even smaller:

7.4 points.

9.3

This un-

fortunately indicates the group was becoming more homogeneous in reading; i.e., the highest children dropped the most, to meet up with the lower ones.)

Children

placed in combination first and second grade classes averaged about one point lower standard scores.

This amount of loss could be test error.

Children who

went on to second grade averaged about half a point gain, also not intrpreble. The decision to retain does not appear to have been based on reading level.

However, retention had a definite negative effect on children's read-

ing skills.

We would judge that their reading losses are attributable to lower

teacher expectancies, to insufficiently challenging reading material, and to the influence of the reading level of classmates. these children lost skills in other areas as well.

It seems reasonable to assume If the trend should continue

the children will lose status in the eyes of their peers, teachers, families, and, inevitably, in their own eyes, although they did not show lower self-esteem on the test or teacher assessment of self-confidence after one year.

We do not

know why these children were retained or if there was any objective foundation for the decision, but the evidence we have indicates that it was likely a bad decision.

194

204

BEST COPY AVAILABLE

The people invrlved in Upswing found it worth whi le and wanted the project to continue. Tutors, teachers, principals, and project staff made it clear in questionnaire responses and conversation during site visits by the evaluation team that they consider Upswing both valuable to children and personally rewarding. Another Upswing was established with the help of the Denver project director in a community just outside the city during the second year of the evaluation. That project continues to operate.

Tutors in the Denver project planned to

continue Upswing on their own in several schools.

The Denver project director

is seeking foundation support in establishing a statewide tutoring program along the lines of Upswing and would eventually like to see it expanded to regional scale.

A tutoring program has been established tentatively in Oxford as a result of Upswing.

The Oxford project director indicated that there were no

local funds for any additional school programs.

The county school system

has applied for a Title VII grant for a full-time coordinator of volunteer activities.

They were successful in getting money to hire a person for this

school year.

The feeling in St. Louis was that to operate the project successfully, at least one full-time project director and two half-time assistants should be hired.

The school system has not acted on this as far as we know.

Neverthe-

less, two St. Louis schools are planning to continue the project on their own, and a third plans to follow suit.

Upswing had strong support in San Francisco from the Departments of Curriculum and Pupil Personnel Services, the members of the school board and school disctict administrators.

The project director, assistant director and

administrative assistant wrote a preliminary operational proposal and budget and presented it to the city.

The San Francisco schools employ learning

specialists to serve small clusters of schools.

It was hoped that these people

might take over Upswing coordination as part of their responsibilities. UPSWING'S FUTURE

It appears at this writing that Upswing has a future; it should.

This

evaluation, after much deep searching has concluded that Upswing has value. 195

205

BEST COPY AVAILABLE Compared to its expensive alternatives, it probabl, is a good investment for most communities.

Perhaps one of its most beneficial future uses will be in

the less populous, less wealthy areas of the country, where children in need often do not receive professional help.

Upswing offers them paraprofessional

help that could be just as effective at a small fraction of the cost. Perhaps sprawling suburban school systems should reevaluate their vast expenditures for first- and second-grade reading remediation.

A small

portion of that budget might be well spent in establishing an Upswing type volunteer program.

As we :lave seer, Upswing can even work amid the complexities of a

metropolitan school system.

However, it requires reasonably sophisticated

management to do so because of the communications and timing required.

Despite

this, the program is still probably far more cost-effective than anything presently available.

RECOMMENDATIONS FOR FURTHER RESEARCH It should be stated at the outset that we believe the essential questions about tutoring on the Upswing model have been answered.

It is not our intention

here to suggest another Upswing evaluation on the scale of that just completed, although hopefully any replications will come up with some method of objectively checking on results with children.

Some recommendations come to mind, however,

that apply to the category in which the Upswing study fallsi.e., evaluation of the effectiveness of an educational "treatment." A double-blind experiment would be the most profitable approach in evaluation of projects like Upswing.

The Upswing evaluation would have provided more conclusive results if all first-grade children in the participating schools had been tested so that there could have been a tutored group and two matched comparison groups--one identified to teachers and one not identified.

That would have permitted us

to check on whether teacher effects were generalized or confined to the identified control children.

It also would have been desirable to test children in com-

parison schools not involved in Upswing (but similar in child population and resources to the Upswing schools).

Comparison of these results to the test 196

206

BEST COPY AVAILABLE results obtained in Upswing schools would have permitted us to verify more clearly the project effect believed to have occurred. Geographic dispersal does not seem essential in a pilot study such as the Upswing evaluation.

The Upswing evaluation could not use either of the foregoing more precise methods because of the cost and burden of testing, which fell on the projects.

However, we believe it would have been a profitable tradeoff to confine the project to one area and use the money saved to hire and train enough graduate student examiners to handle the required testing.

If the evaluator were in

close proximity to the project sites, the evaluator could assume responsibility for testing and test-scoring and therefore ensure greater consistency of examiner training and administration procedures, as well as scoring accuracy. This approach would free project staff for the work of getting a project under way.

In a pilot study such as Upswing, in which there was never any pretense of sampling and extrapolating the results, there would seem to be no necessity, whatever to go to four different regions of the country.

The four chosen are

insufficient to represent the nation, even in a loose way, if that was the intent. There are undoubtedly many areas around the country where one could find large metropolitan inner-city, suburban, rural, and small city schools in reasonable proximity to each other.

The Washington, D.C. area would have afforded such a

The measures of child progress that are used must be selected very carefully. This would seem to be an obvious stricture, but the Upswing evaluation ran into several problems because inappropriate measures yielded useless data. Although the Metropolitan series is widely used around the nation, the reading portions of the Primer and Primary I batteries definitely were not suitable for the Upswing evaluation; the test did the children an injustice.

As an aside, we

suspect it is not uncommon for school systems to put themselves in a bad light in periodic large-scale self-evaluations because of the tests used. In the first year of Upswing, no objective measure of self-esteem or self-concept was obtained because use of an inappropriate measure was prescribed.

197

207

BEST COPY AVAILABLE The problems with interpreting the results of the Cegelka Academic Readiness Evaluation illustrate the importance of scoring mechanisms designed for the analytical purpose for which the instrument is being used.

Further research seem warranted by several of the secondary findings of the Upswing evaluation.

It seems very important to explore the reasons underlying decisions to retain children in school and to follow children who have been retained at various grade levels to attempt to identify the effects of retention on them. The Upswing data also suggest that the education system could gain financially by an analysis of the cost-effectivenesc, of remedial reading

versus tutoring or other alternatives at various grade levels.

It also seems

worthwhile to look into reasons underlying decision to place a child in a remedial reading program.

The question of how children are evaluated in school involves both of the foregoing areas of inquiry. its impact, have been explored.

Stereotyping in student evaluation, and

Yet, we seem to be far from identifying the

subjective criteria applied by teachers in deciding on their students' capabilities and needs.

The Upswing data suggest that teachers tended to interpret

one or two highly visible problems as indicative of more subtle problems whose effects would have generalized impact.

The Upswing data did not provide

evidence to support such an interpretation. Finally, it is clear that much more work needs to be done on identification and objective measurement of the affective components of learning. The crude and mostly subjective measures now in use to explore such factors as self-esteem aud motivation to achieve are far from adequate.

In a broad

sense, this gets into the question of what makes some people "good learners" (of what society considers it desirable to learn) and other people poor learners; also, what makes some people better tutors or teachers than others.

There is

good evidence that the answers to these questions have more to do with affective

characteristics of people and their relationships than with instructional techniques.

198

208

BEST COPY AVAILABLE

EDITORIAL CLOSING COMMENT

Mark Harris said in reviewing Rosenthal's research on the effects of expectancies, that "teaching is a form of loving ... teachers [and tutors] see what they expect to see, and the pupil sees what the teacher sees." (Psychology Today, September 1973.)

And the pupil does what he thinks he

can do and what is made worthwhile by the response of others.

The effect

of expectancy was evident in several aspects of the Upswing data. it works on tutors and teachers as well as children.

Moreover,

By reverberation, what

the tutors and teachers feel about their ability to teach affects the children too.

We are led to two conclusions.

First, the training given anyone who

is to work with children should emphasize how to look at children carefully; how to interpret their: academic and nonacademic behavior; how to expect good

of them individually without applying too much pressure; the importance of being specific and sincere in recognizing children's successes--in short, training should emphasize the how-to of good learning relationships.

Train-

ing also should help teachers to, (pardon the expression) a gut belief in the importance of relationships.

Too many of us think we have faith, but

when a child does not seem able to grasp the concept of long and short vowels we get desperate, defeated, or even hostile.

As Bruno Bettelheim says,

"love is not enough"; we believe that is true, but we believe love is essential.

The second point we wish to make here has to do with the school environment.

The long-established Hawthorne effect has shown that environmental

changes like those brought about in Upswing can be perceived as a form of attention or recognition by teacher, child, and volunteer. tion there is usually motivation to "look good." activity and improved attitudes.

Under such atten-

This results in positive

For example, teachers apparently became more

perceptive about the children with learning difficulties; they tended to give these children more positive attention.

Shades of Rosenthal, positive attention

seems to improve people's morale and make them work better; it is a way of saying you are imporL.nt--special effort for you is worthwhile. words, environmental change is a stimulant. was in that category of phenomena.

In other

It appears that Project Upswing

Teachers did not always like Upswing, but

it brought them into contact with other people, other ideas. 199

209

They interacted

BEST COPY oto' on a personal basis with university staff. sense special.

-1 r2

Involvement meant they were in a

Tutors generally looked up to them; tutors and project staff

alike valued their opinions and recognized the difficulty of the teacher's job.

Involvement in Upswing may have stimulated competitive spirit in some

teachers.

We cannot pinpoint all of the possibilities of the situation.

Yet,

it seems clear that Upswing created a different kind of environment that was

beneficial to teachers and to children as well.

It seems like a good idea

for school administrators to keep in mind the value in making school a special place, not just for children but for the adults there too.

200

APPENDIX A BIBLIOGRAPHY

American Institutes for Research in the Behavioral Sciences (AIR), 1971. Model Programs: Reading. Programmed Tutorial Reading Project, Indianapolis, Indiana. Palo Alto, California: AIR. Birch, Herbert G and Joan Dye Gussow, 1970. Nutrition, and School Failure. New York:

Disadvantaged Children: Health, Harcourt, Brace & World, Inc.

Bradshaw, Charles I. "Remedial Reading Instruction by Student Tutors in Inner City Schools." Paper presented at the California Educational Research Association Annual Meeting, San Diego, April 1971. Dreyer, Hal B. "Rx for Pupil Tutoring Programs," in The Reading Teacher, Vol. 26, No. 8, May 1973.

Ellison, D.C., Philip Harris, and Larry Barber. "A Field Test of Programmed and Directed Tutoring," in Reading Research Quarterly, Vol. 3, Winter 1969. Harris, Philip L., 1967. "Experimental Comparison of Two Methods of Tutoring: Programmed vs. Directed." Unpublished Ed.D. dissertation, Indiana University, Bloomington, Indiana.

Harrison, Grant V and Vern Brimky. "The Use of Structured Tutoring Techniques in Teaching Low-Achieving Six-Year-Olds to Read." Paper presented at American Educational Research Association Annual Meeting, New York, April 1971. Harrison, Grant V. et al. "The Use of a Structured Tutorial Reading Program in Teaching Nonreading Second Graders in Title I Schools to Read." Paper presented at American Educational Research Association Annual Meeting, Chicago, April 1972.

A-1

211

Keele, Reba and Grant V. Harrison. "A Comparison of the Effectiveness of Structured Tutoring Techniques as Used by Parents and Paid Student Tutors in Teaching Basic Reading Skills." Paper presented to California Educational Research Association, San Diego, April 1971. Liette, Eileen E., 1971. Tutoring: Its Effects on Reading Achievement, Standard-Setting, and Affect-Mediating Self-Evaluation for Black Male Under-Achicoers in Reading. Doctoral thesis in Philosophy. Cleveland, Ohio: Case' Western Reserve University. Mohan, Madan, 1972. Peer Tutoring as a Technique for Teaching the Unmotivated. Fredonia, New York: Teacher Education Research Center, State University College.

Nichols, William J., 1968. "A Study of the Effects of Tutoring on the Self Concept, Reading Achievement, and Selected Attitudes of Culturally Dis advantaged Children." Unpublished Ed.D. dissertation, Ball State University, Muncie, Indilna. Noce, James S., 1968. "An Evaluation of Volunteer Tutorial Programs in the District of -,olumbia Public School System." Unpublished Ph.D. dissertation, The Catholic 'Jniversity of America, Washington, D.C.

Pellegrin, Robert J. and Robert A. Hicks. "Prophecy Effects and Tutorial Instruction for the Di3advantaged Child," in American Educational Research Journal, Vol. 9, No. 3, Summer 1972. Robertson, Douglas J. and Vicki Friedman Sharp, 1971. The Effect of Fifth grade Student Tutors on the Sight Word Vocabulary Attainment of First graders. Northridge, California: San Fernando Valley State College. Roethlisberger, F.J. and William J.. Dickson, 1940. Management and the Worker. Cambridge, Massachusetts: Harvard University Press.

Rosenthal, Robert and L. Jacobson, 1968. York: Holt, Rinehart, & Winston.

Pygmalion in the Classroom.

New

Shaver, James P. and Dee Nuhn. "The Effectiveness of Tutoring Underachievers in Reading and Writing," in The Journal of Educational Research, Vol. 65, No. 3, November 1971. Towson, Shelagh m.J., 1972. Tutor Role Enactment in the Peer Teaching Dyad: the Effects of Tutor-Initiated Tutee Evaluation and Reward. Madison, Wisconsin: University of Wisconsin, Madison Research and Development Center for Cognitive Learning.

A-2

212

APPENDIX B

DATA COLLECTION FORMS USED IN PROJECT UPSWING

OMB No. 51-S73018

Expiration Date: 6/30/73

VOLUNTEER FINAL IMPRESSIONS ROJECT UPSWING

Name of Tutor Child's Name. Teacher's Name.

School:

City INFORMATION FOR PROJECT UPSWING USE ONLY

The information requested on this form is to help evaluate Project Upswing and plan ways of using volunteers effectively in education programs. Please do not Omit any answers. Individual names will not be used when the results are printed. Space for comments is provided to give you the opportunity to express your individual ideas and feelings. Comments are optional.

************************* ********************************* I

.

What is your general opinion about the Upswing group training meetings you attended this year? (Cheek one) a.

I did not attend enough training meetings to have an opinion

b.

I did not need training

c.

Training is not necessary for tutors

d.

Training generally was presented poorly

e.

It was useful but not essential

f.

It was very important to me in my work as a tutor

i

[

I

Comments:

'Li checked a, b. or e in answering question I, skip to question 3. If you found the training sessions generally were useful, please indicate your major reasons. (Cheek the three most important) a.

They gave me confidence

b.

They gave me knowledge of teaching techniques

c.

They helped me know what to expect from my pupil

d.

They helped me handle behavior problems

I

I

e.

They acquainted me with a variety of materials and their uses

I

I

f.

They helped me diagnose my pupil's specific learning needs

1

1

g.

They helped me tutor appropriately to meet my pupil's specific learning needs

1

1

I

I I

B-3

21 4

h.

They helped me to evaluate my pupil's progress

t.

They helped me to have a better relationship with my pupil

j.

They helped me have a better relationship with my pupil's teacher

k.

Other (please specify)

What is your general opinion about individual help you received from Project Upswing staff?

3.

(Cheek one) a.

Individual help was not available to me

b.

I did not need individual help

c.

The Upswing staff generally did not give the kind of individual help I needed

d.

It was useful but not essential

e.

It was very important to me in my work as a tutor

I

I

I

I

Comments:

If 4.

5.

checked a or b in answering question 3, skip to question 5. If you found individual help generally was useful, please indicate your major reasons. (Check the three most important) I

a.

It gave me confidence

b.

It gave me knowledge of teaching techniques

I

I

c.

It helped me know what to expect from my pupil

1

1

d.

It helped me handle behavior problems

1

I

e.

It acquainted me with a variety of materials and their uses

I

1

f.

It helped me diagnose my pupil's specific learning needs

I

I

g.

It helped me tutor appropriately to meet my pupil's specific learning needs

I

1

h.

It helped me to evaluate my pupil's progress

I

1

i.

It helped me to have a better relationship with my pul.il

I

1

j.

It helped me have a better relationship with my pupil's teacher

I

I

k.

Other (please specify)

I

Which approach (group training or individual help) was most useful to you in tutoring? (Cheek one) a.

Group

I

I

b.

individual

I

I

c.

I found the two equally useful

I

I

d.

Neither was useful

1

8-4

215

Please rate the volunteer training materials used by Project Upswing (for example, sample tutoring materials. handouts describing games and teaching techniques for use in tutoring, handouts about sot:Lint. leaining problems). (Check onel

7.

a.

In general, the materials were inappropriate (useless for my needs. confusing, etc.)

h.

Some of the materials were helpful, but on the whole they were of little value in my. opinion

I

I

c.

In general the materials were good

I

I

d.

In general the materials were outstanding

I

1

e.

There were very few or no training materials as far as I know

I

I

Please list the tutoring materials that you found most valuable in working with your child. List no more than three.

To what extent did you work on activities to build your pupil's visual-motor integration skills (for example, cutting out pictures. tracing, handwriting. coloring, puzzles, manipulative games. etc.)? (Cheek onel a.

Rarely or not at all

b.

Sometimes

c.

Frequently

Comments:

Did you receive enough support from Project Upswing staff during the year?

10.

a.

Yes

h.

No

Could you have used other types of group training sessions? a.

Yes

b.

No

If "Yes," describe:

11.

Could you have used more individual counseling? a.

Yes

h.

No

If "Yes." describe:

B-5

216

12.

Could you have used more tutoring materials? a.

Yes

b.

No

If "Yes." describe

13.

What other types of aid could you have used?

14.

Did you find the following aspects of Upswing meetings generally favorable (F) or generally

unfavorable (U)? (Circle F or U for each item a d) Favorable a.

15.

Schedule (day and time meetings were held)

b.

Location(s) of meetings

c.

Physical environment of meetings (for example temperature of room, amount of space, lighting, comfort of chairs, etc.)

d.

Social atmosphere of meetings (for example friendliness, opportunity to express your ideas, etc.)

F

Unfavorable

U

Did any of the aspects of Upswing meetings listed in question 14 cause you to miss one or more meetings? a.

Yes

b.

No

If "Yes." please specify which caused you to miss meetings:

lb.

Please show your satisfaction with communications in Project Upswing by using the scale below. Strongly Dissatisfied

Dissatisfied

Satisfied

Strongly Satisfied

SD

D

S

SS

Circle Satisfaction Rating a.

Communications between you and Upswing staff

SD

D

S

SS

h.

Communications between you and your pupil

SD

D

S

SS

c.

Communications between you and your pupil's teacher

SD

D

S

SS

Communications between you and other volunteer tutors

SD

D

S

SS

SD

D

S

SS

d.

e.

Other (please specify)

Comments:

I 7.

Which types of communication listed in question 16 do you think are most important to being an effective tutor? (Cheek two) b.

a.

d.

c.

Coinments:

Is.

How do you feel about teacher guidance? (Check one) a.

I would have preferred more guidance from my pupil's teacher

b.

The teacher has given me adequate guidance

e.

I would have preferred less guidance from the teacher

d.

I do not need any guidance from the teacher

Comments:

B-7

218

19.

flow do you feel about the overall effect of tutoring on your pupil? Please consider all types of possible effects development of academic skills, self-confidence, ability of child to control his behavior, motor coordination, etc. (Check one) u.

I believe tutoring may have been bad for this child

I

I

b.

I do not know if tutoring has contribly ed to the child's development

I

I

c.

I believe tutoring has made a small contribution to the child's development

I

I

d.

I believe tutoring has made an important contribution to the child's development

I

I

Co!

O.

rents:

Please check the item that best describes how you feel about your experience with Project Upswing. (Check one) a.

Strongly satisfied

b.

Satisfied

c.

Neutral ("take it or leave it")

d.

Dissatisfied

e.

Strongly dissatisfied

Comments:

Thank you for your cooperation in filling out this questionnaire. Please also complete the attached "Student Profile," which has questions about the child you tutored. If you tutored more than one child in Upswing, complete a profile for each child. The space below is provided for any additional comments you may wish to make about Project Upswing and your role in it.

88

21.9

STUDENT PROFILE

Child's Name: The JOIlowing items pertain to the child you tutor. For each underlined topic, check ONE statement that generally applies to the child. 1.

Ease of conversation: a.

Conversation with this child is not possible at this time

b.

Conversation with this child requires considerable effort on my part

c.

Child seems timid. but is able to engage in conversation

d.

Child is able to hold conversation in a relaxed, spontaneous manner

i

I

i

I

Initiation of conversation:

3.

4.

5.

a.

So far. child rarely or never initiates conversation with me

b.

Child occasionally initiates conversation

c.

Child frequently initiates conversation

d.

Child does not converse, but rather wants to talk constantly, cannot handle the "give and take" of conversation

I

[

]

[

i

Speech characteristics: a.

Child speaks at a reasonable level (volume) and can be understood

h.

Child speaks at a very low level (volume). but can be heard and understood

c.

Child speaks very softly, and/or "mumbles," is difficult to understand

d.

Child speaks very loudly and aggressively

Eye contact: a

Child generally looks at the floor, ceiling, or elsewhere when we talk

b.

Child generally "looks me in the eye" when we talk

c.

Child stares at me in a blank or frightened way

Approach to new tasks: a.

Child generally seems interested and confident in undertaking a task he/she has never done before

b.

Child generally seems hesitant, but will try new tasks

c.

Child generally is unwilling to try a task he/she has never done before

B-9

220

()

:ieneral cotidence .1.

This child appears to have little or no self-confidence

b.

This child appears to be reasonably self-confident

c.

Tins .11ld appears to be highly self-confident

**

4.

4.

!: * * * * * * * * * *

************************* ***************

The space below is provided for an observations about the child that you feel might be of interest.

g 10

221

OMB No. 51-S73018

Expiration Date: 6/30/73

TEACHER FINAL IMPRESSIONS ROJECT UPSWING

Name.

City:

School:

INFORMATION FOR PROJECT' UPSWING USE ONLY form is to help evaluate Project Upswing and plan effective ways of The information requested on t using volunteers effectively in education programs. Please do not omit any answers. Individual names will not be used when the t esults are tabulated. Space for comments is provided to give you the opportunity to express your individual ideas and feelings. Comments are optional.

****************** 1.

************************ **************

Project Upswing asked you to fill out a readiness inventory the CARE at the beginning of the school year for each child yciu referred as a tutoring candidate. You have been, or will be asked to fill out the CARE for each one again at the end of the school year. Please check under "Yes" or "No" to answer the following questions about the inventory.

itt

No

a.

The CARE is easy to fill out:

1

1

[

1

b.

The CARE is useful to teachers in identifying children with learning problems or potential learning problems:

[

1

[

1

The CARE covers the appropriate behaviors for first-grade children:

[

1

[

1

The information from the CARE is useful to help volunteer tutors know what to expect from their pupils and what they need to work on:

1

1

[

[

1

1

c.

d.

e.

The CARE is useful in evaluating children's progress:

1

Comments

Did you have enough information about the teaching approaches and materials used by the Upswing volunteer(s) who tutored your pupil(s)? (Check one) a.

Yes

b.

No

Confluents:

222

3

In general. were the Upswing meetings for teachers useful in helping you and the volunteer tutor(s) work well toget het ? (Check one) a.

They were a hindrance

h.

They had no effect

c.

The were somewhat useful

d.

They were very useful

c.

Did not attend

Comments:

4.

Do ,ou think teacher meetings are important for a smoothly running volunteer tutoring program in the schools' (Check (ne) a

No

h.

They are helpful, but not essential They are vet!, impoi taw

COM n lents:

5.

Were you satisfied with the frequency of Upswing teacher meetings? (('heek one) .1.

Yes

I

h.

No, the meetings should have been held more often

I

No. there were more meetings than were needed d.

I

There were no meetings as far as I know

I

I

Comments:

Were you satisfied with the amount of contact between you and Project Upswing staff other than

in meetings? (Check on a.

Yes

h.

Na, I thought more it formal contact was needed

c.

No, I thought Project s all contacted me more than necessary

Continents:

B-12

223

7.

Did you find the following aspects of Upswing meetings generally favorable (F) or generally unfavoreach item a d)

able (Ur ICircie F or U

Favorable

a.

Schedule (day and time) of meetings

b.

Location(s) of meetings

c.

Physical environment of meetings (for example, temperature of room, amount of

Unfavorable

space, lighting, comfort of chairs, etc.) d.

Social atmosphere of meetings (for example, friendliness, opportunity to express your ideas. etc.)

e.

I here were nu meetings as far as I know 1

I did nut go to the meetings g. K.

I have nu opinion

1

1

1

Did any of the aspects of Upswing meetings listed in question 7 cause you to miss 'me or more meetings? (Cheek one) a.

Yes

b.

No

1

1

1

If yes. please specify which caused you to miss meetings.

9.

yoki think teachers generally would be willing to attend meetings as part of their involvement in a project like Upswing if they could not be paid? (Check one) a.

Yes

h.

No

1

1

Comments:

10.

What is your opinion of the individual support given to Upswing volunteers by the project staff? (Check one)

a.

It is a key reason for Upswing's effectiveness

h.

It is useful but not essential

c.

1

1

I

1

It is not needed

I

1

d.

It is detrimental

I

e.

I believe individual support by someone other than school personnel is needed, but the support given by Project Upswing to its volunteers this year was not effective.

f.

I don't know

Comments:

1

1

1

11.

Please rank the following kinds of volunteer-teacher communication. (Give the most desirable a I and the least desirable a 6. Use each ranking number, 1 through 6, only once.) a.

Informal talks initiated by the teacher

b.

Informal talks initiated by the volunteer

c.

Regular meeting of teacher with all of her volunteers together

d.

____ Regular meeting of teacher and single volunteer

e.

Regular school-wide meetings of teacher and volunteers

f.

Regular city-wide meetings of Upswing teachers and volunteers

Comments.

12.

In general, do you believe that volunteer tutors are effective in working with first-grade children who have learning problems? (Check one) a.

No

b.

Yes

Comments.

13.

Was your participation in Project Upswing beneficial to you as a professional educator? (Check one) a.

No

b.

Yes

Comments:

14.

If your answer to Question 13 is "Yes," please indicate how Upswing was professionally helpful to you. (Check all that apply) a.

Acquainted me with a greater variety of instructional materials

I

1

I

1

b.

Acquainted me with additional teaching techniques

c.

Gave me additional understanding about how to diagnose specific learning needs

I

1

d.

Gave me greater insight into how to individualize instruction for my pupils

I

1

e.

Increased my ability to use helping personnel effectively

I

1

I

1

f. g

Motivated me to further my education Other (please specify)

Comments.

B-14

225

15.

Please check the item that best describes how you feel about your experience with Project Upswing. (Check one)

a.

Strongly satisfied

[

1

b.

Satisfied

[

1

c.

Neutral ("take it or leave it")

[

1

d.

Dissatisfied

[

1

e.

Strongly dissatisfied

[

1

Comments:

Thank you for your cooperation in filling out this questionnaire. Please be sure to complete the attached "Student Profile" for each of the pupils in your class who received Upswing tutoring at least up to March 1,1973. Also complete a profile for each of your pupils who was in the Upswing control group.

INDIVIDUAL STUDENT PROFILE Child's Name:

Child's Sex:

Teacher's Name: School:

City.

************************* ************************* ******** 1 .

Did :his child attend kindergarten? (Check One) a.

Yes

h.

No

c.

I don't know

How does this child communicate with his peers? (Check one)

3.

4.

a.

He/she does not communicate with peers at all

I

I

b.

Communication is primarily nonverbal (i.e., play, hitting, signs, etc.)

I

I

c.

Communication is about equal amounts of verbal and nonverbal

I

I

d.

Communication is primarily verbal

I

I

e.

I don't know

I

I

How often does he communicate with peers? (Check one) a.

Frequently. more than most other children; initiates major'ty of the coriimi.nicat ion .

b.

Seems to be average in terms of communication with p,:ers; initiate3:ibf.ut half of communications

I

I

c.

Below average: shows tendency to avoid communication

I

I

d.

I don't know

I

I

I

I

.

I

Please show how readily this child forms relationships. Rank frrrn 1 4. For example, if the child forms relationships most easily with adults, put "1" by "adults': If forms relationships next most easily with children his age, put "2" by :'rat answer choice. If he relates third most easily with older children, put "3" by that aliswer cho.:e. and "4" by the remaining choice "Younger children." Assign each number only once. If you cannot rank all choices. put a check by "Don't know." a.

Adults

b.

Older children

c.

Children his own age

d.

Younger children

e.

I don't know

I

B-17

227

I

11,qk doe, this add approach play time /Cheek

h.

.1.

Prefers to play in large groups of children

I

b.

Prefers to play in small groups

I

c.

Prefers to play by himself

I

I

Li.

I dou'i know

I

I

I

I

I

tioA readily does this child participate in class discussions and activities? (Check one) a.

Child frequently olunteers information. volunteers for tasks, volunteers to answer quest ions, etc.

h.

Child sometimes volunteers, responds readily when called upon to participate Child ticker %olunteers. scenic shy when called upon to participate

d.

Child is unable to participate in class discussions or activities (for example, seems confused, overwhelmed by embarrassment, frightened when called upon to participate, is too easily distracted. cannot carry out a task, etc.)

Hom, does this child relate to you? (Check one) a.

Child seeks attention from me almost constantly

h.

Child is friendly and responsive but able to function independently

c.

Child appears timid when relating to me

d.

Child appears hostile when relating to me

I

f;ir each ,1 the fidleiwilts; underlined topics. please check the statement that generally applies to this child. 8.

Approach to new tasks: .i.

Child generally seems interested and confident in undertaking a task he/she has never done before

h.

Child generally appears hesitant or unwilling to try a task he/she has never done before

Eye contact during conversation:

10.

a.

Child usually looks me in the eye when we talk

I

I

b.

('held usually looks at the floor, ceiling, or elsewhere when we talk

I

I

Disputes with other children: a.

Child generally avoids or withdraws from disputes with other children

h.

Child generally holds his/he, own in disputes with other children

c.

Child frequently initiates conflict. seems to seek out conflict

B-18

218

11.

General confid_ence: a.

Child appears to have little or no self-confidence

b. c.

I

I

Child appears to be reasonably self-confident

I

I

Child appears to be highly self-confident

[

I

**************************************************** The space below is provided for any observations about this child that you fed might be of interest.

****

OMB No. SI-S73018

Expiration Date: 6/30/73

PRINCIPAL'S OPINIONS ABOUT UPSWING

PROJECT UPSWING

Principal's Name:

City:

School:

INFORMATION FOR PROJECT UPSWING USE ONLY

The information requested on this form is to help evaluate Project Upswits and plan ways of using volunteers effectively in education programs. Please do not omit any answers. Individual names will not be used when the results are printed. Space for comments is provided to give you the opportunity to express your individual ideas and feelings. Comments are optional. ************************* ************************* * ****** 1.

2.

How long have you been associated with Project Upswing? (Check one) a.

This school year only

b.

This and the last school year

Did you have a clear understanding of the goals of Project Upswing? a.

Yes. I believe so

b.

No

Comments

3.(a) Please indicate your overall opinion about having volunteer tutors in general working in your school. (Cheek one) a.

I welcome them

b.

Because of space problems I would prefer not to have them

c.

They cause a hardship to my staff that overbalances the good they do

1

d

They disrupt classrooms

1

e.

I do not believe they are effective in helping children overcome learning difficulties

1

I

1

1

Comments

3.(b) Please indicate your overall opinion about having Upswing volunteer tutors working in your school. (Check one) a.

I welcome them

b.

Because of space problems I would prefer not to have them

c.

They cause a hardship to my staff that overbalances the good they do

d.

They disrupt classrooms

e.

I do not believe they are effective in helping children overcome learning difficulties

Comments

B-21

230

I

I

1

1

4.(a ) What is your opinion about the need, in general, for training volunteer tutors? (Check one) A

-pie orientation and guidance from the teacher are sufficient

b.

Training is useful but not essential

c.

Training is essential for most volunteers

Comments

4.(b) What is your opinion about the need for training 112swing volunteer tutors? (Check one) a.

A simple orientation and guidance from the teacher are sufficient

b.

Training is useful but not essential Training is essential for most volunteers

Comments

5

What is your opinion about the individual support given to Project Upswing tutors by the project staff? (Check one) a.

It is a key reason for Upswing's effectiveness

b.

It is useful but nest essential

c.

It is not useful

d.

It is detrimental

e.

I believe individual support by someone other than school personnel is needed, but the support given by Project Upswing to its volunteers this year was not effective

f.

I have no definite opinion

Comments

6.

How did most of the teachers on your staff seem to feel about Project Upswing? (Check one) a.

Very negative

b.

Somewhat negative

c.

They seemed to have a "take it or leave it" attitude

d.

Positive

e.

Very positive

Comments

7.

Please indicate your opinion about one-to-one tutoring (as given in Project Upswing) in the first grade as a way of helping children overcome learning difficulties. (Check one) a.

I .2onsider it an approach that has great potential

b.

I consider it an approach that has moderate potential

c.

I am undecided about the value of this approach

d.

I consider it a low priority approach

1

e.

1 consider it an ineffective approach

1

I

1

[

Comments

H.

Do you believe a tutoring program like Upswing could be operated successfully by an individual school') (That is, the school would recruit its own volunteer tutors and provide them with the necessary training, supervision, and materials.) (Check one) a.

No

b.

Yes, if funds were available

c.

Yes, without any budget adjustment

Comments

9.

To your knowledge, were other elementary school principals in the school system interested in having Upswing volunteers work in their schools? (Check one) a.

Yes

b.

No

c.

Don't know

Comments

10.

Do you think the public school system would be willing to institute a tutoring program like Project Upswing as part of a volunteer program in the schools? (That is, the school system would recruit its own volunteer tutors and provide them with the necessary training, supervision, and materials.) (Check one) a.

Yes, if funds were available

b.

Yes, without a budget adjustment

c.

No

d.

Don't know

Comments

1

1

1

.

it you were associated with Project Upswing both in its first and second years. what differences, if any. have you noticed about the project? ICneck ono a.

I have only been associated with Project Upswing this school year

b.

I have not noriced ny differences I have noticed the following differences

Thank you for your cooperation in filling out this questionnaire. The space below is provided for any additional comments you may wish to make about Project Upswing and your role in it.

B-24

233

PROJECT UPSWING ATTRITION REPORT (Check One)

Child

Teacher Name

School

City Date Upswing Participation Ended (approximate)

Reason for Leaving Program (if known)

Report Filled Out by:

B-25

234

Volunteer

PROJECT UPSWING COST ANALYSIS FORM

CITY NAME

PURPOSE

The purpose of this form is to gather information concerning how the Upswing budget dollars have been or are expected to be expended by the end of the school year. This cost information will then be used in conjunction with measurement of the effectiveness of the results of the Upswing project. It is important that the dollar amounts provided are accurate; however, reasonable estimates will suffice when actual data are unavailable.

B-27

235

BEST COPY AVAILABLE

A.1

.

PROJECT UPSWING STAFF EXPENSES FOR THE 1971-72 YEAR ExperHos (d Mars)

-

St t::

M ::):::

Overhead Paid

Totat Salary R.Iceivod For V..lar From Up-..ving

Travel

Othor

to University

Total

Time Spent on lIpsxinej Number of Weoks r Hours pm:Week For Academic Ye.r ' Spont o::Ups'Aing

Pro; ?cr. .Lr.:.:tor A.; i; .. .: itt

:-'):-..:1

,

S::::-:. oiti _._ ___________ . :

..

I

.

r.

--t-

__ . __

.1:

r

..---

:.. ....,

:

__ , ::

1

'..

I

'.

.._

!

.

._

_.

r. ; _

.

_... ._

.

--4

it.2. ESTIMATE OF PROJECT UPSWING STAFF 5ES1 COPY PMARLE EXPENSES FOR THE 1972-73 YEAR BEST COPY AvhILABLE Expenr.os (dollars) T

it Sllary !Iceived

Time Spent on Upswing

Overhead Paid

:or Ye.ir From Upswing

Travel

Other

to University

..-

r

..

I

B-30

237

Total

Number of Weeks For Academic Year

:lours por Week Spent on Upswing

B.

PERCENT OC' TIME ALLOCATED TO VARIOUS PROJECT TASKS DURING THE 1971-72 YEAR AND AN ESTIMATE FOR THE 1972-73 YEAR

(Total Time = 100%) BEST COPY MAILABLE Managenlvnt and

Orientation of

nutln

11:.traine.d Volunt.-t:rs

Training

Operations of Project

Supervision and

Trained

11- 12/2 -73

t r

--- -

71-72 72-73

71-72 /1 -73 71-72 72-73

Total

A3S15tance to Volunteers

Other

Untrained

(100%)

Trained Untrained

71-7/72-73 71-72 72-73 71-7)/ 12-r

7,-741 71-72 7. -73

-

.1

i1:

.P1'

ti

atom f.

r10.11 Int, Ain't th.t

nurt0),..r r.:f

B-31

238

asli3tants in ,pacn provided.

BEST COPY AVAILABLE

C.

Training Program Costs

(other than staff salaries)

Estimates of Year Costs 1971-72

1972-73

Materials used in training (books, film, etc.)

$

$

Room used in training

$

$

$

$

$

$

$

$

$

$

$

$

Other expenses incurred for training (specify)

Total

_ ..._.

BEST COPY. AVAiliktiti

Estimate of Year Costs D.

Miscellaneous Expenses

1971-72

1972-73

Expenditures for volunteer recruitment not otherwise

listed (e.g., spot announcements, flyers, printing, etc.)

$

$

$

$

$

$

$

$

$

$

$

$

$

$

Cost of instructional materials used by tutors (DISTAR, Peabody, Kits,

etc.) Other costs incurred that are not included in the questions above (please itemize) Item

Total ....

.....

BUT COPY AVAILABLE

E.

Adequacy of Funds (answer for only the 1971-72 year) 1. Do you feel that sufficient funds were

available for project staffing? Yes

No

Please explain.

2. Do you feel that sufficient funds were available for materials? Yes

No

Explain.

3. Do you feel sufficient funds were available to conduct training? Yes

No

Explain.

B-34

241

BEST COPY AVAILABLE

4. Do you feel there were items for which actual costs incurred were significantly higher or lower than had been anticipated in the original budget? Yes

No

Explain.

5. Did the Upswing project overrun its costs? Yes

No

Explain.

6. a. What was the amount of the additional funds received? $

b. What were these funds used for? 7. Suggestions and recommendations for future budgeting. (Please make any suggestions or comments relating to budgeting which you feel might be beneficial to other Upswing Directors.)

BE,r3T COPY 4VAIUME

Total Student Load

F.

What is the total number of students that could be supported in the 71-72 and 72-73 models of Upswing within the budget allowances for those years? Year

71-72 72-73

Present Number

Maximum

1973-12
ERIC Archive, Educational Research, Grade Repetition, Grade 1, Learning Problems, Primary Education, Program Evaluation, Reading Improvement, Reading Instruction, Reading Programs, Remedial Reading, Self Concept, Tutorial Programs, Tutoring, Volunteers, Paramore, B., And Others
English