ERIC ED349492: Project CAST Evaluation Report. OREA Report

DOCUMENT RESUME CG 024 499 ED 349 492 TITLE INSTITUTION PUB DATE NOTE PUB TYPE EDRS PRICE DESCRIPTORS IDENTIFIERS Project CAST Evaluation Report. ...

0 Downloads 8 Views
DOCUMENT RESUME CG 024 499

ED 349 492 TITLE INSTITUTION

PUB DATE NOTE PUB TYPE EDRS PRICE DESCRIPTORS

IDENTIFIERS

Project CAST Evaluation Report. OREA Report. New York City Board of Education, Brooklyn, NY. Office of Research, Evaluation, and Assessment. Dec 91 19p.

Reports

Evaluative/Feasibility (142)

MF01/PC01 Plus Postage. Elementary Secondary Education; *Inservice Teacher Education; *Prevention; Professional Training; Program Effectiveness; *School Personnel; *Staff Development; *Substance Abuse; Teachers *Project CAST NY

ABSTRACT The Comprehensive Approach to Substance Abuse Prevention Training Program (Project CAST) was a federally funded pilot project designed to establish, expand, and enhance staff development for educational personnel in substance abuse prevention education. This was to be accomplished by establishing a training program for both instructional and non-instructional staff, as well as developing a guide for integrating prevention techniques and strategies into the everyday responsibilities of staff. Trainees (N=134) included teachers, social workers, paraprofessionals, secretaries, principals, and assistant principals, all of whom took part in a series of training workshops conducted by a team of specialists from Bank Street College. Feedback from participants indicated that the workshops were very useful, and pre- and post-test results showed a marked increase in knowledge. The following two changes were recommended for future implementation of Project CAST. First, a more active recruitment of training participants, particularly non-instructional staff is needed to enlist the targeted number of participants. Second, a greater collaboration between the project director and the curriculum development team is needed during the curriculum design process. This can help to ensure that the end product meets the goals of the project--to develop a staff training curriculum for both instructional and non-instructional staff. (ABL)

*****************************************1::.**;:************************

AeproaucLions supplied by EURs are tne best tnat can be made from the original document. ***********************************************************************

PROJECT CAST EVALUATION REPORT December, 1991

NEW YORK CITY BOARD OF EDUCATION H. Carl McCall President Irene H. Impellizzeri Vice President Carol A. Gresser Westina L. Matthews Michael J. Petrides Luis 0. Reyes Ninfa Segarra Members

Joseph A. Fernandez Chancellor

DIVISION OF STRATEGIC PLANNING/RESEARCH & DEVELOPMENT Robin Winner Executive Director

It is the policy of the New York City Board of Education not to discriminate on the basis of race, color, creed, religion. national on gin, age, handicapping condition, marital status, sexual orientation, or sex in its educational and employment policies, as required by law. Inquiries regarding compliance with appropriate lawsprograms, activities, may be directed to Mercedes A. Nesfield, Director, Office of Equal Opportunity, 110 Livingston Street, Brooklyn, New York 11201; or to Director, Office for Civil Rights, Department of Education, 26 Federal Plaza, Room 33-10, New York. New York 10278. 7[26/91

EXECUTIVE SUMMARY The Comprehensive Approach to Substance Abuse Prevention Training Program (Project CAST) was a federally-funded pilot project designed to establish, expand, and enhance staff development for educational personnel in substance abuse prevention education. This was to be accomplished by establishing a training program for both instructional and noninstructional staff, as well as developing a guide for integrating prevention techniques and strategies into the everyday responsibilities of staff. Project CAST was piloted in four schools in district 5, with a team of specialists from Bank Street. College conducting a fourpart series of training workshops. Trainees included teachers, social workers, paraprofessionals, secretaries, principals, and Feedback from participants assistant principals, among others. indicated that the workshops were very useful, and pre- and posttest results show a marked increase in knowledge of substance abuse issues by workshop participants. The Project CAST staff training was highly effective and well received. However, the r mber of participants was less than originally intended, especially in terms of non-instructional Further, a separate curriculum designed specifically for staff. non-instructional staff was not developed as proposed, but instead a preexisting curriculum for classroom teachers was adapted. The following changes were recommended for future implementation of Project CAST: (1) A more active recruitment of training participants, particularly non-instructional staff is needed to enlist the targeted number of participants. Communication and cooperation among the project director, district staff, and school principals in publicizing and supporting the training program is important in promoting the attendance of a large and varied group of school staff. (2) Greater collaboration between the project director and the curriculum development team is needed during the curriculum design process. This can help to ensure that the end product meets the goals of the project -- to develop a staff training curriculum for both instructional and non-instructional staff.

ACKNOWLEDGEMENTS This report has been prepared by the Office of Research, Evaluation, and Assessment (O.R.E.A) of the Division of Strategic Planning/Research, and Development. Those who generated it included Jody Imbimbo, who managed the project, Lynne Manzo, who analyzed the data and wrote the report, and Nina Gottlieb, who provided editorial assistance. Additional copies of this report may be obtained from:

Mabel Payne OREA Room 507 New York City Public Schools 110 Livingston Street Brooklyn; NY 11201 (718) 935-5242

.6

TABLE OF CONTENTS PAGE

EXECUTIVE SUMMARY ii

ACKNOWLEDGEMENTS I. INTRODUCTION

1

Background

1

An Overview of Project CAST

2

Methods

4

II. MEETING THE OBJECTIVES

5

Staff Training

5

Curriculum Development

8

III. CONCLUSIONS AND RECOMMENDATIONS

10

IV. REFERENCES

12

7

I. INTRODUCTION

Background

Alcohol and drug abuse are serious and pervasive problems in our society.

Particularly alarming are the large numbers of

young people who use alcohol and illegal drugs.

A national

survey of high school seniors revealed that more than half (54%)

had used an illicit drug at some time in their lives, and almost all (92%) had used alcohol.

Among the users, more than half

indicated that their experiences with drugs and alcohol began before the 10th grade (Johnston, O'Malley, & Bachman, 1989).

In

New York State, junior and senior high school students reported similar rates of drug and alcohol usage.

Approximately 84% of

the students surveyed reported that they used alcohol and about one third indicated that they used an illicit drug (Kandel, 1989) .

Given these sobering figures, it is evident that schools have a responsibility for educating students about the nature of drugs and alcohol and ways to avoid using these substances.

In

fact, the declines observed in substance abuse in New York State over the past 12 years may be attributable to widespread prevention efforts, particularly by schools (NYS Division of Substance Abuse Services, 1990). far from over.

However, the "war on drugs" is

The still staggering figures on substance abuse

indicate the critical role that teachers and school staff, given their substantial impact on students, must play in implementing drug education.

Because many school-based personnel do not have

prior training in substance abuse prevention, staff development

has become an increasingly crucial component of substance abuse prevention education. It is important that substance abuse prevention training and knowledge reach all school staff.

To be

most effective, staff development needs to include practical

methods that teachers can integrate in their classrooms, and other staff can incorporate into their regular duties (see OREA Research Brief /4: Making Staff Development Pay Off,in the Classroom).

An Overview of Project CAST

The Comprehensive Approach to Substance Abuse Prevention Training Program (Project CAST), was a federally-funded pilot project that was developed to establish, expand, and enhance staff development for educational personnel, recognizing their daily impact on students and the influence they can have in substance abuse prevention programs.

Overall, the aim of the project was to increase the

knowledge and awareness of school staff regarding substance abuse This goal was to be

in order to better serve students. accomplished by:

(1) enhancing already existing teacher training,

paying particular attention to the needs of early childhood

teachers; (2) establishing an innovative training program for non-instructional staff; (3) developing and disseminating a guide

of prevention strategies and practices for replication/adaptation in other school districts (which would include curriculum

materials for training school staff); and (4) increasing the level of awareness of all participants regarding the harmful 2

9

effects of alcohol and drug abuse.

The training was to involve

teachers, paraprofessionals, and non-instructional personnel from elementary schools, and administrators from both the school and district level.

Separate workshops were p' ;nned for

instructional and non-instructional staff.

The intention of the

workshops was to provide information on alcohol and illegal drugs; current research on substance abuse prevention; assistance in recognizing abusers and children of substance abusers; and current approaches to substance abuse education, prevention, and intervention.

Project CAST was expected to result in several outcomes.

School-based Substance Abuse Prevention and Intervention Specialists (SAPIS) were to meet with participants of the training throughout the year to reinforce newly learned skills.

The participants were expected to incorporate their knowledge of substance abuse prevention techniques into their usual responsibilities.

Students whose teachers participated in this

program were expected to demonstrate an increased awareness of the dangers of substance abuse, as well as an increased ability to employ coping and resistance skills.

Finally, materials

which were developed by the project would serve as a prototype for future staff training.

This was to be accomplished by

developing a replication guide for distribution throughout the New York City Public Schools. Project CAST was piloted in District 5, which encompasses Central Harlem in Manhattan.

This district was chosen because of 3

10

its need for staff development; the director of the district Substance Abuse prevention and intervention program indicated that there was no formal staff training component in place.

The

director, a school principal, and a SAPIS all agreed that staff development was a critical aspect of a successful substance abuse prevention program.

District 5 is also

characterized by

extensive visibility of drugs and alcohol in the community.

Research data and a review of treatment program records within the community indicated a high enrollment of economically disadvantaged minority students, and large numbers of homeless families and children of alcoholics and substance abusers.

Methods

The evaluation of Project CAST was conducted by the Office of Research, Evaluation, and Assessment (OREA) of the New York City Public Schools, in cooperation with the Office of Substance Abuse Prevention Programs (OSAPP).

An OREA researcher conducted

a site visit in District 5 and interviewed the director of the district substance abuse prevention program, a school principal, and a school-based SAPIS.

The site visit also included

observations of two workshops in the training project.

Pre- and

post-tests were given in the workshops in two of the four schools by OSAPP representatives, and feedback forms were distributed to

The materials used in the

participants in every workshop.

training were collected from the Bank Street College training staff.

4

II. MEETING THE OBJECTIVES

Staff Training

School staff training was planned as a series of monthly after-school workshops for teachers and paraprofessionals.

Workshops for non-instructional staff were originally planned to be held during the day, since these staff members coold be released from their usual duties at no additional cost to the school.

Project CAST proposed to train 284 school staff,

including a majority of non-instructional staff.

However, only

134 school staff actually attended the training workshops, with some participants not attending all four workshops provided.

Training sessions were conducted in four schools in District 5 (P.S. 123, P.S. 36, I.S. 195, and P.S. 197) over a period of

two months, from April 18 to June 11, 1991. Each school held four after-school workshops lasting one and a half hours each.

Slightly more than half of the participants in the training were teachers; others included principals, assistant principals, family workers, social workers, paraprofessionals, SAPIS, school psychologists, and secretaries.

Although separate workshops had

been planned for instructional and non-instructional staff, the relatively small number and diversity of the non-instructional staff precluded conducting separate workshops.

Training sessions were designed to allow for a questionanswer period, small group discussions, and the use of handouts to help generate discussion.

Each of the workshops had a 5

12

distinct focus: the first session dealt with self-assessment, including abuse.

personal experience with and views about substance

The second workshop focused on child development issues,

and the third concentrated on integrating the Project CAST curriculum into the teachers' ongoing curriculum.

The final

session emphasized referrals and links with other organizations.

Pre- and post-tests were given to participants in two schools immediately before and after the four-session training to determine how much they had learned.

Comparisons of pre-test and

post-test scores indicated a marked increase in knowledge and awareness of substance abuse issues by the participants.

The

average pre-test scores were 13 and 15 for each school out of a possible score of 25, while the average post-test score in both schools was 24.

Differences in pre-test knowledge (with scores

between 5 and 22) were substantially decreased at the post-test, with almost all participants scoring between 23 and 25.

The

principal interviewed in one of the schools corroborated these

test results, noting that the staff were more aware of substance abuse prevention issues as a result of the training.

She also

indicated that the staff were better able to identify "at-risk" students and recommended additional staff training. Participants of each of the four training workshops at all four schools were asked to evaluate each workshop.

They were

asked to rate such aspects as the level of knowledge and effectiveness of the presenter, the organization of the workshop, and the clarity of the objectives.

6

Al 3

Responses from these feedback

I

forms indicated that the workshops were very well received.

The

majority of participants rated the workshops as "excellent." Sixty-five percent of all participants (in all workshops, in all schools) rated the clarity of the workshops as excellent; fifty-

nine percent reported that the workshops were excellent in meeting their objectives.

The organization and effectiveness of

the workshops was rated excellent by 67 percent of the

participants, and 69 percent thought the presenters' level of knowledge was excellent.

Most participants reported that the strongest feature of the workshop was either the knowledgeability and dynamism of the presenter, or the group work and the opportunity to share experiences and feelings.

Only a small number of participants

noted weaknesses in the workshops; those who did most frequently mentioned a lack of time.

Additional comments about the

workshops were very positive, with many people noting how useful they found the presentations:

"Again, another worthwhile, highly

motivated, well-prepared workshop presentation."

Observations of the training workshops also indicated that the trainers were engaging and skilled.

The presenter discussed

with the participants a variety of prevention concepts and techniques for infusing them into classroom lessons.

The

observer noted:

The trainers were very skillful at getting the trainees to open up and share experiences, frustrations and concerns regarding the special problems and needs of at-risk students. The discussions and sharing were very lively and, at times, emotional. The trainer made sure that all participants had a chance to share how they would use the 7

materials that were distributed. Based on the participation of the trainees, I would say that this session was a success.

Curriculum Development The New York City Public Schools' OSAPP contracted a team of specialists from Bank Street College to develop the training materials and workshops for Project CAST.

The training that was

developed for Project CAST, "Staff Development Training Plans for Educational Personnel" was based on Bank Street College's preexisting Project Healthy Choices curriculum.

This training

curriculum focuses on the harmful effects of drug use and its impact on children, but is geared for use in the classroom.

Therefore, the format of the Project Healthy Choices curriculum was expanded to include staff development for non-instructional personnel.

An OREA researcher reviewed the curriculum adapted by Bank Street College specialists in order to evaluate its appropriateness for the various roles of the school staff who received the training.

The "Training Plans"

use a small group

design, which allows for discussions and a high level of interaction among participants.

Unfortunately, the curriculum

that was developed was vague; the various components of the "Training Plans" were not very distinct, so that it was difficult to ascertain where one component ended and another began.

In

addition, the curriculum did not focus on how different staff

members could implement knowledge of substance abuse issues into their specific roles.

Also, it was unclear whether the training 8

plans correspond to the activities in the actual training sessions held.

The curriculum development process was intended to result in a replication guide that would be disseminated by the Project CAST director to all school districts in New York City.

The

replication guide would have included curriculum and supplementary materials, feedback from participants about what they found most useful, and specific instructions as to how other districts could undertake the same staff training.

To date, this

replication guide has not been developed.

The "Staff Development Training Plans" that were submitted to the project director by Bank Street College had serious deficiencies which precluded their dissemination. materials were poorly produced.

First, the

Second, they did not provide

actual guidelines for how school staff, and particularly noninstructional personnel, could infuse their increased knowledge into their regular duties.

Third, the manual did not clearly

explain how the training could be replicated.

Finally, the

supporting materials for the training (i.e., puppets, videos)

were from Bank Street College's "Project Healthy Choices", and no other materials were provided.

9

III. CONCLUSIONS AND RECOMMENDATIONS The primary goal of Project CAST was to develop a training program for both instructional and non-instructional educational personnel in order to enhance their roles in substance abuse prevention.

This training was intended to accomplish two tasks:

(1) to implement staff development in the pilot district (District 5) and increase the ability of school personnel to

handle issues regarding substance abuse; and (2) to develop and disseminate a staff training guide for replication and adaptation in other school districts.

Project CAST seems to have succeeded in the first task.

The

training for educational personnel was well-received by participants, who improved (at least on a short-term basis) their knowledge of substance abuse issues.

The trainers from Bank

Street College were highly rated both by the workshop participants and by an OREA observer.

Due to the fact that the

training was given quite late in the academic year, we cannot ascertain what, if any, effects the staff training has had or will have on students in the participating schools.

However,

the number of participants in the training was less than originally intended, particularly for non-instructional personnel.

The training did not result in a newly developed curriculum for non-instructional staff; an existing staff development program was adapted, but still focused primarily on classroom teachers.

Clearly, Project CAST was not able to fulfill its goal 10

17

of disseminating curriculum materials and a replication guide for staff development in substance abuse prevention to other districts.

Future implementation of Project CAST should include a more active recruitment of school-personnel, particularly noninstructional staff, to participate in the training.

Cooperation

among the project director, district staff, and school principals in publicizing and supporting the training program is important in ensuring the participation of a large and diverse group of school staff.

The problems encountered in this implementation of Project CAST with the development of pertinent curriculum materials point to the need for greater collaboration between the project director and the curriculum development team.

This will help to

ensure that the materials developed meet the needs of both teachers and non-instructional staff, and include a clear replication guide for implementation in other school districts. While any effort in educating school staff about substance abuse issues is important, the ability to disseminate quality training materials throughout the city will pave the way for more widespread assistance in dealing with the impacts of substance abuse on students.

1 c) i

0

IV. REFERENCES Drug (1989). Johnston, L. D., O'Malley, P. M., & Bachman, J. G. use, drinking, and smoking: National survey results from high school, college, and young adult populations. Rockville, MD: National Institute on Drug Abuse.

Kandel, D. B. (1989). Drug school students in New Second Annual Research Mental Health, Albany,

use among junior and senior high York State. Paper presented at Conference, New York State Office of NY.

(1990). New York State Division of Substance Abuse Services. Drug and other substance use among school children in New York State: Selected findings. Albany, NY: Author.

4

4

A

12

9