ERIC ED280918: Project PROBE, 1985-1986. OEA Evaluation Report

DOCUMENT RESUME ED 280 918. TITLE INSTITUTION SPONS AGENCY PUB DATE GRANT NOTE UD 025 420 Project PROBE, 1985-1986. OEA Evaluation Report. New Yor...

0 Downloads 2 Views
DOCUMENT RESUME ED 280 918.

TITLE INSTITUTION

SPONS AGENCY PUB DATE GRANT NOTE

UD 025 420

Project PROBE, 1985-1986. OEA Evaluation Report. New York City Board of Education, Brooklyn. Office of Educational Assessment. Department of Education, Washington, DC. [86]

CO08202293 33p.; Prepared by the O.E.A. Bilingual Education Evaluation Unit. For 1984-1985 report, see ED 272 603.

AVAILABLE FROM

Office of Educational Assessment, New York City Board of Education, 110 Livingston St., Brooklyn, NY

PUB TYPE

Reports

EDRS PRICE DESCRIPTORS

MF01/PCO2 Plus Postage. Attendance; *Bilingual Education Programs; Career Awareness; Chinese; Computer Literacy; *English (Second Language); *Nigh Schools; *Native Language Instruction; Parent Participation; Program Effectiveness; Spanish Speaking New York City Board of Education

11201.

IDENTIFIERS

Evaluativc/Feasibility (142)

ABSTRACT In its second year of operation, Project PROBE (Professions Oriented Bilingual Education) experienced difficulty in meeting some of its instructional objectives. The project had sought to provide instructional and supportive services to 200 Spanish-speaking students from Latin America at Louis D. Brandeis High School (Manhattan, New York) and to 100 Vietnamese-born, Chinese-speaking students at Sheepshead Bay High School (Erooklyn, New York). PROBE's goal was to increase career awareness among students of limited English proficiency (LEP) through English as a second language (ESL) and native language arts instruction and through bilingual instruction in content-area subjects. This assessment reports the following findings: (1) students were successful in achieving the priposed English-language achievement objectives; (2) objective mac .ery in native language arts in Chinese and Spanish could not be as:essed as proposed; (3) the project could not achieve its content-area passing rate objective; (4) the attendance rate of project students was significantly higher than the schoolwide rate; (5) the career awareness component was not being implemented at either site; (6) staff development and curriculum development activities were few; and (7) parental involvement in school functions could not be assessed because of missing data. (PS)

*********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ****************************************t******************************

"PERMISSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY

U.S. DEPARTMENT DF EDUCATIDN Office GI Educational Research and anprovement

EDUCATIONAL RESOURCES INFORMATION CENTER IERIC)

his document has been e p r od uc e d as teCeived trom the person or organization

originating it r Minor changes have been nre de In anprOve reproduction Quality

TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC)."

Points ot view ur opinions stated in this doCu men1 do nol necessarily represent official OERI position or policy

O.E.A. Evaluation Section Report Robert Tobias, Administrator of Evaluation Judith S. Torres, Senior Manager Grant Number:

GOO-820-2293

Project Director: Angelo Gatto

PROJECT PROBE 1985-1986

Prepared by the O.E.A. Bilingual Education Evaluation Unit

Armando Cotayo, Evaluation Manager Jose J. Villegas, Evaluation/Planning Specialist Shelley M. Fischer, Evaluation/Planning Specialist Margaret H. Scorza, Editorial/Production Coordinator Dora Collazo-Levy, Evaluation Consultant Eulalia Cabrera, Evaluation Consultant

New York City Public Schools Office of Educational Assessment Richard Guttenberg, Director

3

Arr,

)9t(rfq 97)1U(2.'

It is the policy of the Board of Education not to discriminate on the basis of race, creed, national origin, age,

handicapping condition, sexual orientation, or sex, in its educational programs, activities, and employment policies, as required by law. Any person who believes he or she has been dIsctlminats lnst should contact: Carole Guerra, Local Equal Opportunity Coordinator, Office of Educational Assessment, 110 Lvingston Street, Room 743, Brooklyn, New York 11201. Inquiries regarding compliance with appropriate laws may also be directed to: Mercedes A. Nesfleid, Director, Office of Equal Opportunity, 110 Livingston Street, Room 601, Brooklyn, New York; or the Director, Office of CMI Rights, U.S. Department of Education, 26 Federal Plaza, Room 33-130, New York, New York 10278.

A SUMMARY OF THE REPORT

The Professions Oriented Bilingual Educeion (PROBE) project, in its second year of Title VI funding, sought to provide instructional and supportive assistance to two different populations of recent immigrants: 200 Spanish-speaking students at Louis D. Brandeis High School in Manhattan; and 100 Vietnamese-born Chinese-speaking students at Sheepshead Bay High School in Brooklyn. The proposed Title VII staff included a project director and a coordinator located at the central office, a resource teacher and a paraprofessional at each school, and hourly staff for the parental involvement and the curriculum development components of the project. PROBE students at both sites received English as a second language (E.S.L.) and native language arts instruction. Students at Brandeis also received bilingual content-area instruction through that school's taxlevy program. At Sheepshead Bay, PROBE students received content-area At both sites, instruction in English in the school's mainstream program. PROBE staff members conducted regular tutoring sessions for students who needed it. Staff development activities were limited to Brandeis High School and included attendance at monthly meetings of the foreign language department, a Board of Education conference for personnel involved in centrally-run Title VII bilingual programs, and a university course in computer science. Other non-instructional activities included the development of computer software for remedial math, and the recruitment of parents for E.S.L. classes at Brandeis and for participation in parents' advisory groups at both schools. Project objectives were assessed in English language development (Criterion Referenced English Syntax Test [CREST]); mastery of the native language (teacher-made tests); mathematics, science, and social studies (teacher-made examinations); and attendance (school and program records). Quantitative analysis of student achievement data indicates that: - - Students achieved the program objective for English language achievement by mastering an average of 1.8 CREST objectives per month in the fall and 2.1 CREST objectives per month in the spring. -

Data were not provided to assess students' gains on La Prueba de Lectura. However, students attending Spanish language courses attained high passing rates (89 percent in the fall and 80 percent in the spring) compared to standards of other Title VII high school programs.

-- The vroposed objective for students taking ChinEse language courses could not be assessed as proposed. However, the reported passing rates in Mandarin language classes for these students were found to be very high (100 percent both semesters).

-- Students' passing rates in content-area courses did not attain the 75 percent passing rate set as an objective; hence the program objective was not met. This might have resulted from the use of a passing criterion much higher than that used in other New York City Title VII high school programs. -- The attendance rate of program students at each site was significantly (p.<.01) higher than the schoolwide attendance rate, thus meeting the objective. The following recommendations are aimed at improving the overall effectiveness of the program: -- developing the project's computer literacy component; -- revising the project's content-area objective by setting a realistic criterion based on the performances of E.S.L. high school students citywide; - - reformulating the project's native language arts objective to conform to the standard passing score of 65 percent as the criterion for test performance; - - expanding staff and curriculum development activities to all sites; and

-- fully implementing the project's career awareness component.

ACKNOWLEDGEMENTS

The production of this report, as of all Office of Educational Assessment/Bilingual Education Evaluation Unit reports, is the result of a cooperative effort of regular staff and consultants.

In addition to those

whose names appear on the cover, Arthur Lopatin has edited the reports following the O.E.A. style guide and has written report summaries.

Joseph

Rivera, Marcia Gilbert, Bruce Roach, Gladys Rnsa, and Martin Zurla have worked intensively as word processors to produce and correct reports. Without their able and faithful participation, the unit could not have handled such a large volume of work and still produced quality evaluation reports.

iii

TABLE OF CONTENTS

PAGE

I.

II.

III. IV. V.

OVERVIEW

1

STUDENT CHARACTERISTICS

4

FINDINGS

9

CONCLUSIONS AND RECOMMENDATIONS

20

APPENDICES

22

iv

LIST OF TABLES

PAGE

TABLE 1:

TABLE 2:

Number and Percent of Program Students by Language and Country of Birth.

5

Number and Percent of Program Students by Sex and Grade.

6

TABLE 3:

Number of Program Students by Age and Grade.

7

TABLE 4:

Number of Students Leaving the Program.

8

TABLE 5:

TABLE 6:

Results of the Criterion Referenced English Syntax Test.

12

Passing Rates in Content-Area Courses.

16

PROJECT PROBE

Central OFfice Location:

1171 65th Street Brooklyn, New York

Participating Schools:

11219

Louis D. Brandeis High School 145 W. 84th Street New York, New York 10024 ShPepshead Bay High School 3000 Avenue X Brooklyn, New York 11235

Year of Operation:

1985-86, Second Year of Funding

Target Languages:

Spanish and Chinese

Number of Participants:

260 students reported

Project Director:

Angelo Gatto

I.

OVERVIEW

In its second year of funding, Project PROBE proposed to provide instructional and supportive services to 200 Spanish-speaking students from Latin Americr at Louis D. Brandeis High School, in Manhattan, and to 100 Vietnamese-born, Chinese-speaking students at Sheepshead Bay High School, in Brooklyn.

The project's basic goal was to increase career awareness among students of limited English proficiency (LEP) through English as a second language (E.S.L.) and native language arts instruction, and through bilingual instruction in content-area subjects.

These courses were to have been

supplemented by career-oriented enrichment activities and services, including a resource/computer center in each school (containing computers,

software, and audiovisual materials on content-area subjects and career education) and personal guidance and career counseling.

The project also

proposed E.S.L./Americanization classes for students' parents.

10

Classroom

instruction was to be provided by each school, and the enrichment activities were to be provided by PROBE.

The program was administered by a project director and coordinator, who were also in charge of several other Title VII multi-site programs and were located at a central office.

PROBE's proposed on-site staff consisted

of a resource teacher and a paraprofessional at each of the two participating schools, and hourly staff for the project's parental involvement and curriculum development components.

During its first year of funding, the project had not been implemented at Sheepshead Bay, primarily because of hiring difficulties.

The E.S.L./

foreign language department at this site identified potential participants, while the project director actively recruited Chinese-speaking personnel to start work at the beginning of the 1985-86 school year.

Accordi,

,

to the

project director, 150 students were identified for possible program participation in the fall of 1985.

At Brandeis, PROBE functioned as a supplement to the tax-levy bilingual program for Spanish-speaking LEP students.

The project's resource

teacher and paraprofessional, hired in February 1985, worked with 200 LEP students, the majority of whom were from the Dominican Republic.

The

project's activities during its first year consisted of lunchtime seminars. The project also sponsored a trip to La Guardia Community College and held a meeting for parents to discuss college costs and entrance requirements. During PROBE's first year, computers were purchased and each school made plans for the location and design of the project's computer resource room.

Classes for parents had not begun at either school.

2

11_

In the project's second year, Chinese-speaking personnel were hired for Sheepshead Bay, and a new resource teacher was hired for Brandeis (the original teacher died over the summer).

Students at both sites received E.S.L. and native language arts instruction.

PROBE students at Brandeis participated in bilingual content-

area classes with the school's other Spanish-speaking students.

PROBE

students at Sheepshead Bay received content-area instruction in English in the school's mainstream.

3

12

II.

STUDENT CHARACTERISTICS

During PROBE's second year, data were provided for 190 Spanishspeaking LEP students participating in the project at Brandeis; 70 LEP students were reported at Sheepshead Bay.

Of the students for whom

information was provided, 76 percent were from Latin America (see Table 1). All the PROBE students were recent immigrants and spoke their native languages at home. for free lunch.

All were reading below grade level and were eligible

The majority of the students were female (see Table 2).

Fifty-three percent of PROBE students were overage for their grade placement: many of the students came from countries where the educational systems do not parallel our own, or where educational opportunities were limited (see Table 3).

At the end of the academic year, 55 students left the program: most were twelfth graders who graduated or were mainstreamed (see Table 4).

4

13

TABLE 1

Number and Percent of Program Students by Language and Country of Birth

Language

Country of Birth

Spanish

Dominican Republic Ecuador Puerto Rico Central and South America (country unspecified) United States

Cantonese

Mandarin

Other Chinese

Number

132 14 6

37 1

Percent

53 6 2

15

less than 1

People's Republic of China Hong Kong Other Asian

23 17

9 6

3

1

People's Republic of China Taiwan Other Asian

10 6

4 2

People's Republic of China

1

less than 1

1

less than 1

251*

TOTAL

100

*Data were missing for nine students.

The majority (76 percent) of project students were native speakers of Spanish. The largest number of students in this group were born in the Dominican Republic; the second-largest number were from various Central and South American countries. Sixteen percent of the project's students were Cantonese speakers. They were born in the People's Republic of China, Hong Kong, and other Asian countries. Approximately six percent of the students were native Mandarin speakers. Most of the students in this group were born in the People's Republic of China or Taiwan.

5

14

TABLE 2

Number and Percent of Program Students by Sex and Grade

\ Louis D. Brandeis

Sheepshead Bay

Grade

Male

Female

Male

Female

9

11

12

14

6

25

18

43

10

30

55

12

7

42

62

104

11

17

31

6

13

23

44

67

12

13

21

4

8

17

29

46

Number

71

119

36

34

107

153

260

Percent By Sex

37

63

51

49

41

59

All Students Male

Female

Total

Total

(6

Fifty-nine percent of the project students were female.

o

The majority of students were in the tenth and eleventh grades.

6

15

TABLE 3 Number of Program Students by Age and Grade

Age

Grade 9

Grade 10

Grade 11

Grade 12

0

0

1

0

0

9

Total

1

13

1

0

14

15

22

16

10

17

5

62

18

4

62

19

3

20

0

0

3

10

21

0

1

2

7

41

105

66

255

TOTAL

52

30

Overage Pr gram Studen s

Number

22

59

34

19

134

Percent

54

56

52

44

53

Brandeis High School

Number

12

52

27

17

108

Percent

52

60

57

50

57

Sheepshead Bay High Sch ol

Number

10

7

7

2

26

Percent

56

37

37

22

40

Note: Shaded boxes indicate expected age range for grade. Fifty-three percent of the project's students were overage for their grade. o

The highest percentage of overage students occurred in grade ten (56 percent). The higher percentage of overage students occurred at Brandeis (57 percent).

TABLE 4

Number of Students Leaving the Program

Reason For Leaving

Grade 9

Grade 10

Grade 11

Grade 12

Total

Fully Mainstreamed

0

3

8

25

36

Graduated

0

0

0

7

7

Returned to native country

0

1

0

0

1

Dropped out

3

4

4

0

11

TOTAL

3

8

12

32

55

Thirty-six students leaving the program (65 percent) were fully mainstreamed. An additional seven students (13 percent) graduated, percent) dropped out.

8

17

.nd eleven (20

III.

FINDINGS

The findings in this section are presented by objective according to the requirements of the evaluation design that was included in the Project PROBE proposal.

They include the results of students' performance on

standardized and teacher-made tests, of an examination of program materials and records, of on-site visits, and of interviews with project personnel.

OBJECTIVE 1 -- As a result of participating in the program, students will demonstrate an appropriate increase in English-language proficiency. PROBE students at both sites received E.S.L. instruction with the schools' other LEP students.

PROBE staff members conducted regularly

scheduled tutoring sessions on a pull-out basis for students in need. A member of the evaluation team observed an E.S.L. level

1 class at

Sheepshead Bay, and two E.S.L. classes (levels 3 and 4) at Brandeis.

The

19 students present in the level 1 class were of Chinese, Korean, Haitian, Hispanic, and Turkish backgrounds.

The lesson, on the names of kitchen and

bathroom items, was conducted entirely in English.

A paraprofessional was

present to provide individualized assistance.

In the level 3 class, observed at Brandeis, the 27 students who were present were learning to write persuasive essays like those required by the Regents Competency Test.

This class was also conducted entirely in English

with a paraprofessional present to provide assistance when needed.

Al-

though it was a large group, all the students were attentive to the teacher's explanations and eagerly began their assignment: to write a composition telling their parent or guardian why they should be allowed to buy a stereo system.

9

18

Sixteen students were present in the E.S.L. 4 class, which was studying the use of prepositions.

The number of students who participated

was high, and they quickly and frequently volunteered aGswers to their teacher's questions.

English was used throughout tne lesson.

No parapro-

fessional was present.

Student Achievement in E.S.L..

The Criterion Referenced English_Syntax Test (CREST)* was used to measure the growth of students' knowled a of English syntax.

Mean differ-

ences between pretest and posttest scores were computed to represent the gain score.

The number of objectives mastered per month was computed by

dividing the gain by the months of treatment.

The CREST was administered at the beginning and at the end of the fall and the spring semesters.

Table 5 presents the test results for students

who were pretested and posttested at the same test level during each semester.

Complete data were available for 190 students in the fall and

177 students in the spring.

(A breakdown of CREST performance by school is

presented in Appendix A.)

The program objective calls for an "appropriate" increase in Englishlanguage proficiency.

Because Chapter I programs consider an average of

*The Criterion Referenced English Syntax Test (CREST) was developed by the Board of Education of the City of New York to measure mastery of instructional objectives of the E.S.L. curricula, and thus was constructed to maximize content validity. The test contains four items per curricular objective, and mastery of an objective is achieved when three of these items are answered correctly. The test measures mastery of 25 objectives at Levels 1 and 2, and 15 objectives at Level 3. The Kuder-Richardson reliability estimates for pretest and posttest administrations of the three levels of the CREST are: Level 1 -- pretest (.91)/posttest (.96) Level 2 -- pretest (.94)/posttest (.95) Level 3 -- pretest (.91)/posttest (.91). 10

19

one CREST objective per month as an appropriate level of achievement, this criterion was used to measure the achievement of Project PROBE students. Examinatiun of Table 5 reveals that the objective was met.

Students

mastered an average of 1.8 CREST objectives per month in the fall and an average of 2.1 CREST objectives per month in the spring. the students made gains.

Eighty percent of

Students tested at Level 3 had lower performance

rates than those tested at lower levels, partly because fewer skills are ::.ested at that level, and partly because mastery of objectives takes longer at that level.

11

2

TABLE 5

Results of the Criterion Referenced English Syntax Test

PRETEST Test Level

Number of Students

Mean

S D.

S.D.

Mean

S.D.

Mean Mastery Per Month

POSTTEST Mean

MASTERY

FALL 1

52

9.4

6.4

15.4

6.6

6.0

4.1

2.1

2

83

11.4

6.1

18.0

5.5

6.6

4.1

2.3

3

56

9.9

3.1

12.1

2.6

2.2

2.0

0.8

190

1n.4

5.5

15.6

5.7

5.2

4.1

1.8

TOTAL

SPRING 1

23

12.6

6.9

17.4

6.4

4.8

6.4

2.2

2

80

11.2

6.3

16.3

5.7

5.1

5.7

2.8

3

74

10.4

3.0

13.0

2.1

2.6

2.1

1.3

177

11.0

5.3

15.1

5.0

4.1

5.0

2.1

TOTAL

Overall, students mastered an average of 1.8 and 2.1 CREST objectives per month in the fall and spring, thus meeting the project's objective.

12

21

OBJECTIVE 2 -- As a result of participating in the program, 80 percent of the students will demonstrate a significant increase in native language achievement. The Spanish language arts sequence at Brandeis ranged from beginning to advanced levels.

All courses were taught by certified teachers and

fulfilled state foreign-language requirements.

A member of the evaluation team observed a Spanish language arts class with 13 students in attendance.

The class was discussing part of the text

Tres Horas, in particular the life of San Martin and his crusade to liberate Chile and Peru.

Students participated in the lesson actively

making many comments on the material under discussion.

All teacher-student

exchanges were in Spanish.

At Sheepshead Bay, PROBE students attended one native language arts class in Mandarin each day.

According to the N.L.A. teacher, most of the

Vietnamese students had good backgrounds in Chinese, having spent time in the People's Republic of China, Taiwan, or Hong Kong.

Her aims were to

reinforce what the students had already learned and to help them further their understanding and appreciation of the language.

In her classes, she

taught both the regular Chinese characters (used in Taiwan) and the simplified characters (used in the People's Republic) to enable the students to read all Chinese publications.

Twenty-four students were present in the class that was observed.

The

students were learning new vocabulary words, and a few had been asked to write sentences using the new words on the blackboard.

All the students

participated in the lesson, volunteering to read each other's responses, and making corrections when necessary.

During the lesson, the paraprofes-

sional helped a few students who were seated in the back of the room.

13

22

Student Achievement in Native Language Arts The proposed objective called for a teacher-made instrument to measure whether students made significant gains in Chinese-language achievement. The data analysis procedure prescribed to determine significance is a correlated t-test, which requires pre and post measures.

Since teacher-

made tests are usually administered on a posttest basis, the objective could not be assessed as proposed.

A widely-used alternative for deter-

mining achievement in native language arts courses is to propose that 70 percent of the students will score at or above the 65 percent passing criterion.

The passing rates for the 16 students reported enrolled in

Mandarin and Cantonese courses were 100 percent both semesters. The proposed objective called for La Prueba de Lectura* to be used to measure students' gains in Spanish. evaluate the objective.

However, no such data were provided to

Instead, the passing rates of students enrolled in

Spanish-language courses were examined and were found to be 89 percent in the fall and 80 percent in the spring.

This is a high achievement rate in

comparison with students in other high school bilingual programs.

*The Interamerican Series, La Prueba de Lectura is part of the Interamerican Series of Tests published by the Guidance Testing Associates. The purpose of this series is to measure reading achievement in Spanish for Spanish-speaking students from the western hemisphere. The norms for the test were based on a sample of students from Puerto Rico. However, as these norms may not be appropriate for students studying in other countries, the publishers recommend that local norms be developed for the test. As of this date, local norms are not available. The test has alternate form reliability coefficients ranging from .87 to .90, and validity studies indicate that the test has correlations greater than .80 with other standardized tests of reading, and correlations of about .50 with teacher grades, thus indicating construct validity.

14

23

OBJECTIVE 3 -- Eighty percent of all target students will demonstrate an improvement in attitude toward their cultural heritage as indicated by results on an appropriate Cultural Heritage Scale.

No data were provided to evaluate the proposed objective in this area.

OBJECTIVE 4 -- As a result of participating in the program, 80 percent of the students will achieve a passing rate of 75 percent or better in the areas of mathematics, science, social studies, business, health, and careers. PROBE students at Brandeis received bilingual instruction in mathematics, science, and social studies in the school's tax-levy bilingual program.

PROBE staff members were available for tutoring in these subject

areas in the project office for three periods per day.

At Sheepshead Bay, PROBE students received content-area instruction in English in the school's mainstream.

Student Achievement in Content-Area Sub'ects Table 6 presents the number OT students attending content-area courses and the percentage achieving grades of at least 75 percent.

It also pre-

sents the number of students achieving the standard passing score of 65 percent.

Examination of Table 6 reveals that the objective was not met.

The proportion of students achieving a grade of 75 percent ranged from 35 to 58; however the proportion who passed these courses with a 65 percent ranged from 72 to 88.

The high passing rates at the standard passing score

of 65 percent indicate that the objective might be unrealistic and should perhaps be changed to reflect the 65 percent criterion generally used by Title VII high school programs.

15

24

TABLE 6

Passing Rates in Content-Area Courses

FALL

Number of Students

Content Area

Percent Passing at 75%

SPRING

Percent Passing at 65%

Number of Students

Percent Passing at 75%

Percent Passing at 65%

Math

228

46

75

216

50

78

Science

203

45

87

176

35

72

220

54

83

232

39

72

83

58

88

135

54

78

Social

Studies

Business/ Vocational

The program objective of 80 percent of the students achieving a 75 percent grade was not met either semester.

16

25

OBJECTIVE 5 -- Students will demonstrate an improvement in attitude toward school and professional career plans as indicated by results on a five-point inventory scale. As part of its career education component, the project proposed to maintain a bilingual, computerized, professions-oriented resource room at each high school.

Although the resource rooms had been set up and the

computers installed, as of the evaluation visits in May, no formal program for student use of this equipment had been implemented at either site. Reportedly, this was because software in the students' native languages was unavailable and staff members needed to be trained.

To help address these problems, th, Brandeis resource teacher had enrolled in a computer course at a lc

A university and had developed

computer software in remedial math.

No data were provided to evaluate Le proposed objective in this area.

OBJECTIVE 6 -- As a-result of participation in the program, the attendance rate of program students will be greater than that of mainstream students. The statistical significance of the difference between program attendance rates and overall attendance rates at the two schools was determined through the use of a z-test for the significance of a proportion.*

This procedure tests whether the difference between two comparable

proportions is greater than can be expected from chance variation. The overall attendance rate for program students at Brandeis was 92 percent, approximately 13 percentage points above the schoolwide attendance rate (79 percent).

The z-test results (z=4.27) indicate that the differ-

ence in attendance rates is significant.

The attendance rate for program

*Bruning, J.L. and Kintz, B.L.; Computational Handbook of Statistics; Scott, Foresman and Company, 1968. 17

26

students at Sheepshead Bay was 95 percent, or 12 percentage points above the schoolwide attendance rate (83 percent).

The z-test results (z=2.66)

indicate that the difference in attendance rates is also significant in this case.

Thus the program objective was met.

OBJECTIVE 7 -- As a result of participation in the program, 85 percent of the staff will demonstrate an increase in awareness of pupil needs, problems, and professional goals as indicated by a five-point inventory of problems and needs. Data were not available to assess this objective.

OBJECTIVE 8 -- Ninety percent of program staff will demonstrate professional growth by completing courses of study as indicated by college transcripts and certificates. In the 1985-86 academic year, staff development activities were limited to Brandeis High School.

The PROBE staff at this site attended the

monthly meetings of the foreign language department as well as a Board of Education conference for personnel involved with centrally-run Title VII bilingual programs.

It was also reported that one teacher at this site was

taking a computer course at a local university.

Because this falls below

the proposed 90 percent criterion, the objective in this area was not met.

OBJECTIVE 91-- As a result of the program, the parents in the program will demonstrate more parental involvement than parents of mainstream students, by demonstrating a higher attendance at school functions. Project PROBE proposed to offsr E.S.L./Americanization classes for students' parents.

The E.S.L. class was organized at Brandeis in April,

but staff members were still recruiting parents for participation at the During that same period, project

time of the evaluation visit in May.

staff at Brandeis also organized an advisory group of 18 parents of PROBE

18

27

students.

According to the resource teacher, eight parents attended the

first meeting held that semester.

At Sheepshead Bay, PROBE staff were able to recruit 11 parents to join the advisory group at that site.

However, since attendance data at school functions were not provided for either the program or mainstream group, the achievement of proposed objective could not be assessed.

SECOND-YEAR OBJECTIVE -- Curricula will be developed in the following areas: applied E.S.L. (three levels); native language arts skills; career-related experiences in literature, social studies, and science. New York City Board of Education curricula will be translated into Spanish and Chinese in the following areas: business/secretarial science Finally, computer programs will also skills, and computer science. be developed in targeted subject areas.

According to the resource teacher at Brandeis, computer software was developed in remedial math.

No other curriculum development activities

were reported; the proposed objective was thus not achieved.

19

28

V.

CONCLUSIONS AND RECOMMENDATIONS

In its second year of operation, Project PROBE appears to be experiencing some difficulty in meeting some of its instructional objectives. Although students were successful in achieving the proposed objective in English-language achievement, objective for mastery in native language arts in Chinese and Spanish could not be assessed as proposed.

However,

students' passing rates in Chinese courses were found to be 100 percent both semesters and in Spanish-language courses, passing rates (89 percent

in the fall and 80 percent in the spring) were high compared to ,cywide standards.

The lack of data to assess the objectives as proposed suggests

that they should be reformulated to reflect the generally-used passing score of 65 percent in coursework as the criterion for performance. In addition, the project could not achieve its 75 percent content-area passing rate objective. However, students demonstrated high passing rates at the standard passing criterion of 65 percent.

These results suggest

that the criterion set by the objective is unrnalistic and, thus, might be revised.

Finally, the attendance rate of project students was signif-

icantly higher than the schoolwide rate. In the non-instructional area, the project proposal's unique feature,

its career awareness component, was not being implemented at either site. Staff development and curriculum development activities were few and limited to Brandeis High School.

Parental involvement in school functions

could not be assessed because of missing data.

It is recommended that, for the project's final year of operation, its computer literacy/career awareness component be implemented without delay. Support and direction in this area should be provided by the central 20

29

office.

A structured program for the use of the computer equipment and

software should be part of the project's curriculum at each school.

The

resource teacher at Brandeis appears to have expertise in this area that might be tapped to benefit staff members at both sites.

Finally, the

coordinator must more closely supervise all data-gathering to ensure that objectives can be adequately evaluated.

\

21

30

V.

APPENDICES

31

APPENDIX A

Results of the Criterion Referenced English Syntax Test at Louis O. Brandeis High School

PRETEST Test Level

Number of Students

Mean

S.D.

POSTTEST Mean

MASTERY

S.D.

Mean

S.D.

Mean Mastery Per Month

FALL 1

43

9.3

5.1

15.9

5.7

6.6

4.0

2.4

2

68

10.3

5.7

17.3

5.5

7.0

4.2

2.4

3

39

10.2

2.8

12.1

2.6

1.9

1.8

0.7

150

10.0

4.9

15.5

5.4

5.5

4.2

1.9

TOTAL

SPRING 1

11

12.5

5.0

18.3

4.5

5.8

2.9

3.4

2

71

10.2

5.7

15.6

5.5

5.4

3.3

3.2

3

50

9.5

2.9

12.5

2.3

3.0

1.9

1.9

132

10.2

4.8

14.6

4.8

4.4

3.0

1.8

TOTAL

Results of the Criterion Referenced English Syntax Test at Sheepshead Bay High School

PRETEST

Test Level

Number of Students

Mean

S.D.

Mean

MASTERY

S.D.

Mean

S.D.

Mean Mastery Per Month

POSTTEST

FALL 1

8

9.8

11.4

13.1

10.2

3.3

3.9

1.2

2

15

16.1

5.8

21.6

3.4

5.5

3.4

2.0

3

17

9.2

3.9

12.4

2.5

3.2

2.3

1.2

TOTAL

40

11.9

7.2

16.0

6.7

4.1

3.2

1.5

SPRING 1

12

12.6

8.5

16.7

8.0

4.1

2.6

1.5

2

9

19.1

5.7

22.1

3.6

3.0

2.8

1.1

3

24

12.2

2.3

13.9

1.4

1.7

1.8

0.6

TOTAL

45

13.7

5.8

16.3

5.4

2.6

2.4

1.0

24

33