兰德-Evaluation-of-the-Regional-Choice-Initiative_112页_1mb

Summary of the Regional Choice Initiative Evaluation

Core Content

The Regional Choice Initiative (RCI) was launched in Beaver County, Pennsylvania, in 2007 by the Beaver Valley Intermediate Unit (BVIU) to expand school choice and improve student outcomes, particularly for those in low-performing districts. The initiative aimed to provide students with access to diverse academic opportunities, including high-performing environments, through four distinct programs: Open Seats, Dual Enrollment, Cyber Learning, and Academies for Success. The RCI was funded by the U.S. Department of Education's Voluntary Public School Choice (VPSC) grant and evaluated by the RAND Corporation to assess its performance and impact.

Main Purpose and Structure

The purpose of this report is to evaluate the implementation and impact of the RCI. The evaluation includes both formative and summative components, with the formative evaluation covering all six years of the program (2007–2008 to 2012–2013) and the summative evaluation focusing on the first four years. The report addresses three key research questions:

1. To what extent did each RCI program achieve its performance and objective measures?
2. What factors enabled or hindered the achievement of performance measures?
3. To what extent did the overall RCI affect student engagement and achievement outcomes?

Key Findings

• Overall Participation in RCI Programs Was Low and Inconsistent

  • 4,357 students participated in RCI programs over six years.
  • Participation increased over time, from 165 in the first year to 2,063 in the final two years.
  • In the first four years, participation was less than 10% of total enrolled students in grades 7–12.
  • Most participants (86%) engaged in the RCI for only one year.
  • Over 90% of participants used only one RCI program, with the largest share in Dual Enrollment (45%), followed by Cyber Learning (25%) and Open Seats (20%).

• RCI Met Most Objective Measures but Failed to Meet Performance Measures

  • By the end of the funding period, the RCI achieved 8 out of 21 measures, with 5 being objective measures.
  • Only 3 performance measures were met, including a reduction in dropout rates in the Academies for Success program and the creation of core curriculum courses.
  • No performance measure related to student achievement (e.g., passing rates in RCI courses) was met.

• Limited Data Restricted the Ability to Assess Impact

  • Data for impact analysis were limited due to a lack of student-level data from participating districts and the short duration of participation for many students.
  • The analysis found no discernible relationship between RCI participation and student engagement or achievement outcomes.

• Factors Contributing to Low Participation

  • Lack of clear indicators for eligible students.
  • Concerns from school stakeholders about program impacts on students, staff, and school operations.
  • Incompatible schedules across schools.
  • Students' reluctance to use virtual learning technologies.
  • Logistical challenges of physical transfers between schools.

• Positive Stakeholder Perceptions

  • Stakeholders viewed the RCI as having a positive impact on participating districts and students.
  • The initiative improved communication and collaboration between districts and transformed the climate and culture of the county.
  • Most districts decided to sustain three of the four RCI programs in the future without external funding, indicating strong support for the initiative.

Recommendations

1. Conduct Feasibility Studies

   • Test the feasibility of program features and implementation plans before launch.
   • Assess demand, readiness, and capacity of participating organizations to implement the initiative.

2. Implement a Multitiered Communication Strategy

   • Develop a strong communication plan to inform students and parents about the RCI.
   • Use experts to create effective recruitment campaigns.

3. Utilize a Dashboard System

   • Use a dashboard from the beginning of the program to monitor implementation and provide feedback to stakeholders.
   • Ensure the dashboard is designed to support program evaluation and improvement.

Key Information

• The RCI was implemented over six years (2007–2008 to 2012–2013), with a no-cost extension in 2012–2013.
• The initiative aimed to improve student achievement, engagement, and access to advanced courses.
• The evaluation used a mixed-methods, quasi-experimental approach, including interviews, surveys, and statistical analyses.
• Propensity score matching and regression analyses were used to assess the impact of RCI on student outcomes.
• The evaluation found that while the RCI did not significantly improve student achievement or engagement, it had a positive effect on district collaboration and school culture.

Conclusion

The Regional Choice Initiative was a comprehensive effort to expand school choice in Beaver County, but it faced significant challenges in achieving its performance goals. Despite low and inconsistent participation, the initiative was perceived positively by stakeholders and contributed to improved communication and collaboration between districts. The evaluation provides important insights for future school-choice programs, emphasizing the need for better communication, feasibility testing, and monitoring systems.