ASM Tech Report 051711

Description

Urban Claymation. MUNTU African Dance and Percussion. West African Djembe Drumming. Year 3. Sports 37. Songwriting and Producing. Digital Photography ...

Sujets

• Sport

Informations

• Publié par: Sportif
• Nombre de lectures: 28
• Langue: Français
• Poids de l'ouvrage: 2 Mo

Extrait

After-School Programs for High School Students

An Evaluation of After School Matters

Technical Report

Barton J. Hirsch
Larry V. Hedges
Northwestern University
JulieAnn Stawicki
University of Wisconsin – Extension
Megan A. Mekinda
Northwestern University









































Cover photo courtesy of Gloria Hafer


June 2011


© 2011 by B. Hirsch, L. Hedges, J. Stawicki, & M. Mekinda
TABLE OF CONTENTS

Lists of Exhibits ................................................................................................................. ii
Acknowledgements .......... iii
Executive Summary ...........1

Chapter 1 Overview ...........................................................................................................8
Background ......................8
After School Matters ......13
Research Questions ........18

Chapter 2 Research Methods ..........................................................................................24
Sample Selection ............................................24
Participants .....................28
Data Collection/Research Procedures ............................................................................31
Measures ........................................................32
Data Analysis Overview ................................38

Chapter 3 Implementation ..............................................................41
High/Scope .....................................................................................41
Program Design Features ...............................42
Social Climate ................................................43
Implementation and Outcome Measure Correlations ....................43
Qualitative Observations ................................................................................................44
Participation in Alternative Activities ...........46

Chapter 4 Impact Findings: Intent-to-Treat .47

Chapter 5 Impact Findings: Treatment-on-the-Treated ..............................................54

Chapter 6 Discussion .......................................................................59
Conclusions and Future Directions ................................................66

Appendix A: Design Features Instrument .....73

References .........................................................................................................................74

Tables ................................83
i
LIST OF EXHIBITS

Tables
2.1 Data Collected for the Evaluation
2.2 Apprenticeships Included in Evaluation
2.3 Treatment Attrition
2.4 Study Participation Rates
2.5 Demographics: Entire Sample
2.6 Demographics: Treatment-on-the-Treated
2.7 Treatment-on-the-Treated/Dropout Pretest Demographics
2.8 Control Group Extracurricular Activities

3.1 Implementation Measure: High/Scope
3.2 Implementation Measure: Design Features
3.3 Implementation Measure: Social Climate
3.4 Implementation and Outcome Measure Correlations

4.1 Positive Youth Development Outcomes: Intent-to-Treat
4.2 Marketable Job Skills Outcomes: Intent-to-Treat
4.3 Academic Performance Outcomes: Intent-to-Treat
4.4 School Attendance with Prior Year Attendance Control: Intent-to-Treat
4.5 School Attendance Entire Sample: Intent-to-Treat
4.6 Attitudes about School: Intent-to-Treat
4.7 Problem Behavior Outcomes: Intent-to-Treat

5.1 Positive Youth Development Outcomes: Treatment-on-the-Treated
5.2 Marketable Job Skills Outcomes: Treatment-on-the-Treated
5.3 Academic Performance Outcomes: Treatment-on-the-Treated
5.4 School Attendance with Prior Year Attendance Control: Treatment-on-the-Treated
5.5 School Attendance Entire Sample: Treatment-on-the-Treated
5.6 Attitudes about School: Treatment-on-the-Treated
5.7 Problem Behavior Outcomes: Treatment-on-the-Treated

6.1 Average Effect Size by Outcome Domain

ii
ACKNOWLEDGEMENTS

This report was a result of the efforts of many people who have contributed and
collaborated with us during the evaluation.

We are grateful to all the staff of After School Matters, including David Sinski, Executive
Director, and Rachel Klein, Raymond Legler, Natasha Smith, and Myrna Torres. We appreciate
the support of Maggie Daley, Chair of the Board of Directors. Thanks to all the Regional
Directors and Specialists who worked hard and assisted in recruitment and planning, and to the
apprenticeship instructors who generously allowed access to their programs. Of course the study
would not have been possible without the many young people who graciously participated in the
research.

The study has benefited from the cooperation of Chicago Public Schools principals, staff,
and teachers.

We could not have completed this study without the help of Kathadin Cook, Jaime
Platzer, Deborah Puntenney, Christy Serrano, and Oseela Thomas who observed apprenticeships
and conducted interviews with participants and after-school instructional staff. Special thanks to
Deborah Puntenney who helped get the study off the ground in the first year.

We thank the many human resource interviewers who volunteered their time to conduct
mock job interviews in the three years of data collection. We are especially grateful to Cheryl
Berrington and Wilbert Williams whose contributions were invaluable in the design of the Mock
Job Interview.

The study’s scientific advisory board provided valuable insights on the evaluation’s
design and data analysis and in the preparation of this report. We thank Jacquelynne Eccles,
Greg Duncan, Denise Gottfredson, Robert Halpern, Stephen Hamilton, Reed Larson, Jeylan
Mortimer, and Elizabeth Reisner for their expertise and guidance. Spyros Konstantopoulis
provided statistical and methodological guidance in the earliest stages of the research. Joseph
Durlak helped us think through some of the issues raised in the discussion section.

Finally, we thank the William T. Grant Foundation, Wallace Foundation, and Searle Fund
for funding this evaluation. We also thank After School Matters for providing a planning grant
in the development stage of the study.




iii
EXECUTIVE SUMMARY

After-school programs have attracted a strong and growing constituency among both
academic theorists and policy makers. Yet participation in after-school programs is extremely
sparse at the high school level. Clearly there is a need to develop more attractive after-school
programs for HS youth, which makes After School Matters (ASM), one of the country’s largest
programs for this group, an important program to study.
After School Matters offers paid apprenticeship-type experiences in a wide array of areas,
such as technology, arts, and sports. Each apprenticeship involves work in the designated area,
learning and making use of relevant skills to accomplish a task. Instructors are present to
provide information, guidance, and feedback, and to introduce students to the standards,
language, and culture of that line of work. The experience presumably also helps students begin
to appreciate and adapt to the culture of the workplace and improve the “soft skills” increasingly
demanded by employers. The instructors have expertise in – and in many instances earn their
livelihood through – the activity that is the focus of the apprenticeship. Most instructors are not
teachers. Apprentices were paid a stipend equivalent to $5/hour during our study.
After-school programs that have an apprenticeship orientation, such as ASM, have the
potential to provide the benefits of successful part-time work experience at a lower cost than
many workforce development programs. Moreover, as an after-school program, they have the
latitude to focus more broadly on positive youth development than might be the case with
programs targeted exclusively at workforce development. Prior research on ASM suggested that
their apprenticeships could provide such an environment.
This report presents results from a three-year, random assignment evaluation of After
School Matters. The major questions addressed by this evaluation are whether assignment to
ASM apprenticeships results in improvements in positive youth development, marketable job
skills, academic performance, and problem behavior.

1

Methods
The evaluation involved a randomized controlled trial. Randomization is the best
procedure for guarding against selection effects in which youth more likely to improve over time
are disproportionately located in the treatment group. In that event, it is impossible to sort out
whether effects are due to selection (who got into the program) or the program itself (the
experiences of youth while in the program). This is why randomized controlled trials are
typically considered the