兰德-AEP-Data-Note-Technical-Appendix-2_10页_226kb

AEP Data Note Technical Appendix Summary

Core Content

This document is a technical appendix for the RAND Corporation's American Educator Panels (AEP) Data Notes, focusing on the methodology and data analysis strategies used for the May 2018 Measurement Learning and Improvement (MLI) Survey. It provides detailed information about the sample composition, weighting procedures, and statistical techniques employed to ensure accurate and representative findings.

Key Information

• AEP Overview:
  The AEP consists of two panels:

  • American Teacher Panel (ATP): Includes over 25,000 teachers.
  • American School Leader Panel (ASLP): Includes over 12,000 school leaders.
    Both panels are nationally representative samples of K-12 public school educators.

• Survey Administration:

  • The MLI Survey was administered in May 2018 to the full ATP and ASLP samples.
  • The survey was developed by The Bill & Melinda Gates Foundation (BMGF) in collaboration with RAND.
  • The survey covers topics such as educator preparation, working conditions, curriculum use, data use, and student interventions.

• Response Rates:

  • ATP: 54% response rate (15,719 complete responses out of 28,955 invitations).
  • ASLP: 27% response rate (3,530 complete responses out of 12,954 invitations).

• Weighting and Standard Errors:

  • Calibration Weights: 81 weighting variables are used to ensure the sample reflects the national population of teachers and school leaders.
  • Main Weight: Calculated by modeling response probabilities across various characteristics (individual and school-level).
  • Replicate Weights: 80 replicate weights are used to calculate jackknife standard errors.
  • Standard Errors: Calculated using both jackknife and state-level clustered standard errors to ensure robustness.

• Sample Characteristics:

  • School Characteristics:
    • School level (elementary, middle, high school, other).
    • Student enrollment, percentage of students by race/ethnicity, percentage of students eligible for free or reduced-price lunch, Title I eligibility, and urbanicity.
  • Educator Characteristics:
    • Total years in the role, gender, and race/ethnicity.
    • Missing data rates: ~2% for teachers, ~5% for principals.

Main Estimation Strategy

• The primary estimation method involves simple weighted averages of survey responses across the full sample or subgroups of interest.
• Weighted Means: Used to calculate national averages.
• Jackknife Standard Errors: Derived from replicate weights.
• Subgroup Comparisons:
  • Regression models are used to compare responses across subgroups.
  • The model is specified as:
    $$
    Y_{is} = \beta_0 + \beta_1 X_{is} + \varepsilon_{is}
    $$
    Where $Y_{is}$ is the survey response, $X_{is}$ is an indicator for subgroup membership, and $\beta_1$ represents the differential response.
  • If the difference between $\beta_1$ and $\gamma_1$ (from a supplemental model) exceeds 25%, both estimates are reported.

Supplemental Analyses

• Supplemental Model:
  • Includes additional covariates and a vector of state fixed effects.
  • School-level variables: school level, student enrollment, race/ethnicity distribution, poverty level, Title I eligibility, and urbanicity.
  • Individual-level variables: years in the role, gender, and race/ethnicity.
• State Fixed Effects: Used in some analyses to account for unobserved state-level factors.
• Aggregation for Small Sample States:
  • In states not oversampled, jackknife standard errors were aggregated by census region to maintain robustness.

Limitations

• Correlational Analysis: All findings are based on correlational data and should not be interpreted as causal.
• Self-Report Bias: Educator self-reports may be influenced by social desirability bias, leading to over- or under-reporting of certain activities or resources.
• Confounding Factors: Subgroup differences may be influenced by school or individual characteristics, though all reported relationships were statistically significant in both models.

Conclusion

The AEP Data Notes use a well-defined sampling and weighting strategy to generate national-level estimates of teacher and school leader perspectives. The May 2018 MLI Survey data is representative of the U.S. education system and includes detailed demographic and contextual information. The analysis methods are robust, and the results provide insights into educators' working conditions and views on student and school-related issues. However, the findings should be interpreted with caution due to the limitations of self-reported data and the potential for confounding variables.