The Impact of Air Reading on Student Literacy Growth

September 22nd, 2023 | Research Study

Executive Summary

Crandall Independent School District (Crandall ISD) partnered with Air Reading, an online development platform specializing in early literacy. At one of its low-performing schools, Hollis T. Dietz Elementary School, 41% of third-grade students engaged in 30 sessions of live virtual intervention through Air Reading. Each session, lasting 40 minutes, involved an Air Reading tutor working with a small group of up to four students, conducted four times a week during regular reading blocks.

The first external study conducted for Air Reading by the University of Maryland, College of Education, investigates the relationship between Air Reading participation and student reading proficiency during the implementation period from January 2023 to March 2023. Our findings indicate that students who took part in Air Reading demonstrated significantly accelerated progress in their literacy skills compared to those who did not participate in the program but were similar in demographics and prior achievement, with an effect size of a .26 standard deviation. It is important to acknowledge certain limitations, including the relatively brief implementation window, a restricted sample size, and the non-randomized nature of enrollment.

Introduction

Crandall Independent School District (Crandall ISD) stands as one of the most rapidly expanding educational systems in the United States. As of 2022, it serves 6,100 students, demonstrating a remarkable growth in enrollment by 35% since 2020. Projections indicate that by 2030, the district is poised to nearly double in size, with an anticipated enrollment surpassing 12,000 students. A primary objective for Crandall ISD this year is to lay the foundations for future-ready schools, with a pivotal focus on nurturing early literacy among students.

Crandall ISD has partnered with Air Reading to actualize this vision, facilitating live virtual interventions for 3rd-grade students. Within this framework, students engage with Air Reading tutors during regular school hours 'push-in' model. Anchored in the Science of Reading, the Air Reading curriculum encompasses over 900 meticulously designed reading lessons tailored for Pre-K through 6th grade. This program explicitly addresses five fundamental facets of reading: phonemic awareness, phonics, vocabulary, fluency, and reading comprehension.

The envisioned implementation of Air Reading within Crandall ISD closely mirrors the research-backed structure of high-impact tutoring. A cadre of dedicated Air Reading tutors consistently engages with students, meeting them 3-5 times weekly. Rigorous and ongoing assessment and progress monitoring are integral to this process. This study delves into the effectiveness of the Air Reading 'push-in' model for a cohort of 41 3rd-grade students, with each student receiving 40 sessions of instruction at Hollis T. Dietz Elementary School from January 2023 to March 2023. The evaluation utilizes student achievement data from NWEA’s MAP Growth reading assessments that were administered on December 7th, 2022 (Mid-Year) and March 28th, 2023 (End-of-Year).

Given these data, the ensuing report addresses the pivotal research question:

(IMPACT) Did students engaged in push-in Air Reading exhibit greater literacy skills compared to their peers in the same classroom who did not partake in Air Reading?

Data and Analysis

PARTICIPANTS

Crandall Independent School District (CISD) encompasses 6,003 students, with 61.95% benefiting from free or reduced lunch programs and 18.27% identified as English language learners. Notably, in 2021, 28% of elementary school students read below their designated grade levels.

Within this context, 101 third-grade students volunteered for the Air Reading study, with 41 engaging in Air Reading sessions while the remaining 60 did not. Importantly, all participating students were new to the Air Reading program.

NWEA MAP GROWTH

Academic progress in reading was evaluated using the NWEA MAP Reading Growth assessment. The MAP assessment suite comprises computer-based adaptive tests tailored for students from Pre-K to 12th grade. This assessment is administered three times annually: in August, December, and March. For this study, the December 2022 MAP tests served as the pretest that measures student baseline reading achievement, while the March 2023 MAP tests were employed as the posttest that is utilized to evaluate the impact of Air Reading.

Each MAP test yields an RIT score, employing a Rasch Unit scale to gauge student performance, irrespective of age or grade level. RIT scale scores conventionally span from 150 to 300, with higher scores indicating greater subject proficiency. Additionally, MAP math and reading assessments provide RIT scores in distinct goal areas, offering insights into students' relative strengths and areas of concern.

SIMPLE COMPARISONS

Without considering the differences between participants and non-participants, we find that Air Reading demonstrated a positive and statistically significant association with student performance in the NWEA MAP Growth assessment. Figure 1 visually represents the MAP RIT scores in December and March 2023 for students who engaged with Air Reading and those who did not. Figure 2 visually represents the MAP RIT score differences on average in December and March 2023 for students who engaged with Air Reading and those who did not.

Specifically, students who participated in Air Reading improved their reading scores by 6.29 points. While students who didn’t participate in Air Reading also saw some growth in their reading scores, the growth is much smaller, which is only 1.65 points. Comparing the different levels of growth, Air Reading students on average gained 4.64 more points than non-Air Reading students. This is remarkable given how short a period the intervention was conducted.

REGRESSION ANALYSIS

As mentioned above, simple comparisons might not capture the true impact of Air Reading because students are not randomly assigned to Air Reading. To complement the analyses above, we employed linear regression models to account for differences between participants and non-participants. These included students’ baseline achievement measured in December, gender, race/ethnicity, whether a student is economically disadvantaged, special education status, and teacher background. The baseline achievement is especially important as it ensures we compare students who are otherwise similar academically in literacy.

Specifically, we estimated the ordinary least squares (OLS) regression as follows:

We find that students who participated in Air Reading scored higher than students who didn’t participate in Air Reading by 4.11 points (p<0.05), which is only slightly smaller than the adjusted number (4.64). This is about 26% of the standard deviation of non-participants’ baseline reading achievement. This finding provides further evidence that Air Reading fulfills its promise and is very effective in increasing student reading achievement.

Conclusion

In this study, implementing Air Reading over a concise period (from January 2023 to March 2023, comprising 40 sessions within two months) yielded noteworthy results. Despite the brevity of the intervention, data analysis revealed a substantial and positive impact on students' academic proficiency in reading. Specifically, compared to peers who are otherwise similar in prior achievement and demographics, Air Reading students exhibited an increased reading achievement by 0.26 standard deviation from December 2022 to March 2023.

Directions for Future Research: There are several avenues for expanding this research. Firstly, employing a larger sample size with randomized student assignments could provide more robust insights. Secondly, extending the implementation period would afford students more time for skill development and enhance the likelihood of achieving anticipated levels of participation. A more extensive and protracted study would offer a clearer understanding of Air Reading's full impact.