WHITEPAPER DOWNLOAD
Thank you for your interest in JFYNetWorks‘ Whitepaper.
JFYNetWorks:
Raising Achievement
Closing Gaps
An Equity-Focused Approach to
Recovering Learning Loss
For more information about JFYNetWorks and its programs, please send an inquiry to Gary@jfynet.org.
VIEW WHITEPAPER ONLINE, see below...
JFYNetWorks: Raising Achievement Closing Gaps
An Equity-Focused Approach to Recovering Learning Loss
The Spring 2022 MCAS results, the first full results with a critical mass of students participating since the COVID-19 pandemic, were released with mixed findings. Statewide, 10th grade Math scores improved very slightly (.4 points) over 2021, while English Language Arts (ELA) scores declined 4.3 points. Less than half of 10th graders met or exceeded expectations in Math, while 58% met or exceeded expectations in ELA. In both subjects, scores remained below pre-pandemic levels. Commissioner Jeff Riley has predicted that it will take five years for students to recover academically from the COVID-19 pandemic.
Source: Massachusetts Department of Elem 1 Patterns of learning loss are also confirmed by national data from fourth and eighth grade NAEP scores. Massachusetts students were in good company with most other states who saw scores decline from 2019 to 20222. Massachusetts students’ average scores fell at higher rates than the nation overall from 2019 to 2022, with 23% of students scoring below basic levels in 2022. |
In addition to overall declines in scores, equity gaps among student groups in Massachusetts persist. Gaps in MCAS scores are largest for students with disabilities compared to students without disabilities. This gap has grown over time, and pandemic learning loss increased it further. In 2022, only 20% of students with disabilities met or exceeded expectations in ELA and 15% in Math, while 67% and 58% of students without disabilities met or exceed expectations in ELA and Math respectively3. As illustrated below, gaps of approximately 30 percentage points persist between Hispanic/Latino and white students, and similar gaps exist between low-income and non-low-income students.
1 DESE defines MCAS achievement levels (not meeting, partially meeting, meeting, or exceeding expectations) based on state curriculum standards and whether a student is on track to meet grade-level standards in their current grade in the tested subject. MCAS raw point scores are translated into these achievement levels. Students meeting or exceeding expectations are seen as on track to succeed. Please see here for more explanation: https://profiles.doe.mass.edu/help/data.aspx?section=assess 2 https://www.nationsreportcard.gov/reading/states/scores/?grade=8 3 https://profiles.doe.mass.edu/statereport/nextgenmcas.aspx |
JFYNetWorks (JFY) is a Boston-based nonprofit provider of blended online academic support programs. Its mission is grounded in achieving more equitable outcomes for underserved students by facilitating access to individualized, scaffolded instructional supports to increase student learning. For more than four decades, JFY has partnered with over 120 high schools, middle schools, and non-profit organizations to support underserved students with online and in-person student engagement strategies.
JFY works with partner schools to address equity gaps in student learning by providing focused blended learning programs in middle and high schools to help students build the grade-level academic skills needed to successfully graduate high school and go on to continued success in college and in the workforce. JFY’s blended learning approach builds student skills by integrating supplemental online instruction into the daily schedule as part of regular classes or in study periods. Students can also access their JFY curricula in out-of-school time. This sustainable approach to supplemental instruction is statistically correlated with learning growth and with higher MCAS scores in Math and ELA.
East Boston High School and Northeast Metropolitan Regional Vocational High School are two JFY partners who have been consistent in their use of the JFY blended learning model. Their work shows strong evidence of success, especially when viewed through an equity lens. Disaggregated data from the 2022 MCAS show both ELA and Math scores from these two JFY partners increased at high rates for Hispanic/Latino students, low-income students, EL students, and students with disabilities5. Each of these student groups also met or surpassed pre-pandemic scores in ELA and, with one exception, Math, eliminating the learning loss entirely. Students with disabilities in particular met or surpassed pre-pandemic scores in both ELA and Math at both schools, a marker that neither their peer schools nor the state achieved. 2022 MCAS results at both the state and comparison group levels are mixed, but JFY partner schools saw consistent gains and a nearly complete recovery from learning loss. As discussed below, not only are the one-year gains and the cumulative gains from 2019 impressive, but the sustained efforts of the schools achieved real equity for all student groups.
________________________________
5 Black or African American students do not appear as a subgroup in this analysis. DESE does not publish MCAS results for student subgroups with fewer than 10 students. Neither EBHS nor NEMT met this threshold for Black or African American students in the three years of Next Generation MCAS 2.0 (2019, 2021 and 2022; there was no MCAS in 2020). However, Black or African American scores and Hispanic/Latino scores statewide and in Boston are virtually identical.
The student body of East Boston High School, a traditional open enrollment Boston Public High School, is over 81% Hispanic/Latino, double the proportion of similar Boston Public high schools overall and nearly quadruple the state level. East Boston High School has partnered with JFY since 2015 and has steadily implemented the JFY model in all grades, including the recently added 7th and 8th grades.
Proportion of 10th Grade Hispanic/Latino Students Meeting or Exceeding Expectations
With these sustained efforts and the use of JFY resources integrated into Math and English instruction, EBHS Hispanic/Latino students outperformed their BPS and state peers in the proportion of students meeting or exceeding expectations on the 2022 Math MCAS. The %M/E in 2022 in both Math and ELA increased 14 and 9 points respectively. These increases also meant that EBHS Hispanic/Latino students topped the state by six points in Math and tied in ELA.
EBHS Hispanic/Latino students bucked the trends seen at similar BPS schools and at the state level with these significant increases from 2021 to 2022. The %M/E in Math was more than double that of similar BPS schools in 2022, and close to double the BPS rate in ELA.
Even more significantly, rates of students meeting or exceeding expectations for Spring 2022 MCAS scores equaled or surpassed pre-pandemic rates, eliminating learning loss at EBHS.
Similarly to Hispanic/Latino students, EBHS students with disabilities bucked trends and saw significant gains in %M/E from 2021 to 2022, well surpassing both the state and their district peers. The %M/E significantly passed pre-pandemic levels for this group – another instance of eliminating learning loss. At EBHS, the %M/E for students with disabilities was more than five times that of their BPS peers in Math, and more than eight times in ELA.
Proportion of 10th Grade Students with Disabilities Meeting or Exceeding Expectations
Trends in EBHS student success can also be seen for EL and former EL students. In 2019, EBHS’s %M/E in Math for this group was lower than both BPS peers and the state. In 2022, EBHS outperformed both BPS peers and the state. As with other groups, EL and former EL students reduced the pandemic learning gap. The 22% ELA rate in 2022 topped 2019 by one point, while the 7-point gain in Math from 2021 to 2022 came within 2 points of the pre-Covid mark.
Proportion of 10th Grade EL or Former EL Students Meeting or Exceeding Expectations
Similar robust growth can be seen for low-Income students. A 16-percentage point increase in Math over 2021 outpaced both BPS peers and the state. The Math %M/E was nearly three times higher than BPS peer schools. The 16-point gain came within 4 points of the pre-pandemic 2019 Math score. In ELA, low-income students saw continued growth throughout the pandemic, and the 2022 ELA rate of 34% M/E exceeded 2019 by 10 percentage points.
Proportion of 10th Grade Low-Income Students Meeting or Exceeding Expectations
Each of these these critical student subgroups achieved significant gains in both ELA and Math from 2021 to 2022 and all eliminated pandemic learning loss in ELA. Three of the four groups closed the pandemic gap in Math. EL came within two percentage points of pandemic closure on the strength of a 13 point one-year gain.
All four of these groups at EBHS -- Hispanic/Latino students, students with disabilities, EL students, and low-income students –outscored their BPS peers and the state in Math in 2022. In ELA, three of the four topped the peer group and the state while low-income students outscored their BPS peer group and came within 6 points of the state. All four groups closed the pandemic gap in ELA, while EL and low-income students came within 2 points in Math. These significant gains demonstrate the equity impact of the JFY approach. Sustained focus on individualized, supplemental tutoring, organized by JFY and implemented consistently by classroom teachers and support staff, produces learning growth for all students and closes equity gaps.
Though these results have been part of sustained, consistent efforts at EBHS, other schools have seen substantial student learning gains in their initial year of partnership with JFY.
JFY first partnered with Northeast Metropolitan Regional Vocational Technical High School (NEMT) in 2019. Through the pandemic and since returning to the classroom, NEMT has integrated JFY’s blended learning model into their class schedule, on average about twice per week during academic sessions. With this ongoing focus on individual student needs, the %M/E drastically increased in both Math and ELA. Not only did scores increase from 2021 to 2022, but all student subgroups eliminated pandemic learning loss in both Math and ELA, meeting or surpassing 2019 levels of %M/E. One-year increases ranged from 2 percentage points to 22 percentage points.
About 38% of NEMT students are Hispanic/Latino, compared to 23% of the state, and higher than comparable regional vocational high schools. In 2019, NEMT Hispanic/Latino students scored well below the state in %M/E, and also well below a group of similar regional vocational technical school peers6. By 2022, after working with JFY on individualized, intensive tutoring and instruction, NEMT Hispanic/Latino students far exceeded their pre-pandemic %M/E and also scored above their state peers. The gap between NEMT and peer regional vocational schools also substantially closed, especially in ELA where NEMT saw a 22-percentage point one-year increase in %M/E
Proportion of 10th Grade Hispanic/Latino Students Meeting or Exceeding Expectations
Students with disabilities make up 25% of NEMT compared to only 19% across the state. These students also saw substantial gains from 2021 to 2022, with a five-percentage point increase in %M/E in Math and a nine-percentage point increase in ELA. They narrowed the gaps with the state and the peer regional vocational comparison group.
Proportion of 10th Grade Students with Disabilities Meeting or Exceeding Expectations
Partnering with JFY has also been effective for EL and former EL students. With only minimal declines from 2019 to 2021, EL students saw a significant one-year increase in %M/E in both Math and ELA in 2022. From 2021 to 2022, the %M/E for EL students increased 9 percentage points in Math, outperforming state peers and matching regional vocational peers. In ELA, the %M/E for EL students increased 22 percentage points in one-year, outperforming peers and the state.
Proportion of 10th Grade EL or Former EL Students Meeting or Exceeding Expectations
Low-income students also saw substantial one-year gains in both Math and ELA. The %M/E for low-income students increased by 13 percentage points in Math and 22 percentage points in ELA. In Math, this meant that NEMT narrowed the gap with their peers from 11 percentage points to only 3. In ELA, the %M/E was 5 percentage points higher than their state peers.
Proportion of 10th Grade Low-Income Students Meeting or Exceeding Expectations
As the results above show, one year of consistent and sustained effort with JFY is correlated with substantial gains in the proportion of students meeting or exceeding expectations. Additionally, for each group of students, in both Math and ELA, all pandemic learning loss gaps were closed - 2022 scores were at or above 2019 levels. Gains of this magnitude, especially in the wake of the devastating pandemic learning loss, demonstrate the equity focus that JFY brings to partner schools. Sustained efforts of individualized, supplemental tutoring, delivered and supervised by teachers, organized and supported by JFY, facilitate learning growth and the closing of equity gaps.
________________________________
6 See Appendix A for more information on peer comparison group selection.
JFYNetWorks: A Model of Success
JFY has been engaged in state-approved programming to support student learning growth and MCAS preparedness, particularly for underserved populations, for more than two decades. (Its first two decades of operation were focused on job training.) Using a blended learning model, JFY partners with teachers to support and supplement live classroom instruction with individualized online tutoring supervised by teachers. Students build skills aligned to grade-level standards in an online environment that includes formats and questions that mirror MCAS.
The elements of the JFY blended learning model are condensed in the acronym AIMS: Assess, Instruct, Measure, Support.
- Assessment of each student’s skills uses online measurement tools or existing school data to diagnose learning needs. Disaggregation of available data pinpoints where students are strong and where additional support is required.
- Instruction with high-quality software offers classroom teachers flexibility in the utilization of a wealth of online materials. Standards-aligned instruction can be smoothly integrated into course curricula and district benchmarking. Individualized, self-paced supplemental resources stimulate student engagement in learning and enrich the learning experience while reinforcing and expanding core curriculum.
- Measurement of student activity and progress is embedded in online software systems. Formative assessment reports enable students and teachers to monitor learning growth in real time.
- Support for teachers and students is provided by JFY Learning Specialists via email, text, phone, videoconference or in-person consultation. Using real-time student data and teacher feedback, JFY and teachers make instructional adjustments and recommendations to keep students on track and maximize learning and acquisition of grade-level skills.
JFY’s expert staff work closely with teachers and department heads to create and adapt instructional units that respond to the immediate needs of the class and individual students – supporting both generalized and individualized curriculum development. With these tools and JFY support, supplemental work can be geared toward grade-level mastery, credit recovery, MCAS preparation, or reinforcement of course content to stimulate and challenge students. As demonstrated by the findings in this report, these tools include responsive adaptations and scaffolded instruction that are effective with all students, including those most at risk.
JFY’s resources and platforms are aligned to state curriculum standards, from which MCAS questions are drawn. Sustained focus on grade-level skills produces higher MCAS performance.
In addition to the significant improvements seen at the two high schools described above, statistical analyses also demonstrate the impact that JFY’s resources and model have on student learning and on MCAS scores.
JFY currently utilizes two software platforms for supplemental tutoring in Math and ELA. As previously noted, these platforms track student activity, learning and growth, yielding data that demonstrate the measured impact of the intervention. For ELA, those gains are measured and reported in Lexiles7, while math uses grade-level standards for appropriate assessment. Correlation analyses demonstrate that time spent in these software systems was positively correlated with learning growth and positively correlated with MCAS scores. The more time spent with JFY software, the higher the learning gains, and the higher the MCAS scores.
Lexile growth had a moderate, positive relationship with the total time spent in the ELA platform (R = 0.38) and the number of activities completed in the software (R = 0.41). Final grade-level assessment scores had a moderate, positive relationship with the number of tasks attempted (R = 0.31), the number of questions answered (R = 0.42), and total time spent in the Math software platform (R = 0.51).
The more time spent or more activities students completed, the higher students’ overall learning growth. That learning growth is also positively correlated with 10th grade MCAS scores. Correlation analyses demonstrate that individual student ELA MCAS scores are strongly correlated with work completed in the software. Student MCAS scores are strongly, positively correlated with students’ Lexile scores (R = 0.72), and moderately correlated with the number of activities completed in the ELA software (R = 0.41) and the average time spent in the ELA software (R = 0.39). Individual Math MCAS scores are moderately, positively correlated to work in JFY’s Math software platform. The platform’s final grade-level assessment is moderately correlated with MCAS score (R = 0.53), and average scores on software tasks are also moderately correlated with MCAS scores (R = 0.48), showing that this software platform is appropriate to prepare for and assess Math MCAS readiness. Additionally, time spent in the software is moderately, positively correlated with Math MCAS scores (R = 0.31). The more time spent and number of activities completed, the higher students’ MCAS scores.
The JFY model of individualized tutoring combined with effective classroom instruction produced the following summary results in the 2021-2022 academic year:
Math:
- 84% of students improved on grade-level math assessments
- 50% of students gained at least 20 percentage points in grade-level math assessments by the end of the year
ELA:
- Three-quarters of students achieved Lexile gains
- Average increase of nearly 100 Lexiles per grade
- 20% decrease in the proportion of students far below grade reading level
- Increase in the proportion of students meeting or exceeding grade reading level by about 50%
The examples from EBHS and NEMT, corroborated by statistical analyses, document the results that JFY has seen for twenty-three years: sustained effort over time with JFY’s instructional and tutoring resources produces measurable learning gains and higher MCAS scores for all students.
The Mission of JFY is to create opportunities for youth who have historically been marginalized and underserved by the Massachusetts public education system. The results outlined above demonstrate that JFY’s stated commitment to equity translates into concrete action that supports student success in academic measures such as grade level skills and MCAS performance, leading to college and career readiness.
For further information about JFYNetWorks and its programs, please send an inquiry to Gary@jfynet.org.
____________________
7 The Lexile is a nationally recognized unit of measurement that describes an individual’s level of reading ability. The higher the Lexile level, the more advanced the reader. Lexile levels correlate with grade-level standards of reading ability.
