There had to be a mistake.
Dylan Jones knew in his gut that something was amiss the second that he saw the Sunflower County Consolidated School District’s 2023-24 English test scores.
He was so shaken by what he saw that he not only called SCCSD Superintendent Dr. Miskia Davis at 9 o’clock that night, he drove from Indianola to her house in Greenville to show her what appeared to be dismal test scores from across the county.
“I have a program, and it will alert me to a drastic change in trends,” Jones, who is the head over federal programs at the district told The Enterprise-Tocsin this week. “When I started uploading the English data into the system, I was getting alerts for all grades, every school. The performance didn’t match any of the models that we had built out, using almost eight years’ worth of data at this point.”
Jones, who has been tracking all data for the district since the 2017-18 school year, wasn’t the only one who saw drastic shifts in student performance.
When he attended a federal programs conference that week, it was the talk of the room.
Multiple districts were reporting what looked to be erroneous English scores on the Mississippi Academic Assessment Program test.
Students who had been tracked for years as proficient were now low basic, Jones said.
But when Mississippi Department of Education Chief Accountability Officer Paula Vanderford emailed the state’s school districts on June 21, she assured them the data were correct.
The E-T obtained copies of multiple email communications between MDE and the districts from this summer regarding test scores. We reached out to Vanderford by email for comment on this story, but we had not received a response by press time.
“The MAAP tests are custom built each year by vendors with expertise in statewide assessments across the country,” the June 21 email from Vanderford read in part. “All MAAP 2024 spring assessments were constructed using the same approved psychometric methods and parameters as in prior years. The MDE did not change the established cut scores in the current administration.”
It turns out that Oregon-based Northwest Evaluation Association, the vendor which has been scoring tests for MDE for the last eight years, according to a report from Mississippi Today, sent erroneous data to the state department that was then sent to the districts.
Six days after that initial email, Vanderford sent out a follow-up communication, acknowledging problems with the initial data.
“The MDE identified areas of concern with the preliminary assessment data files released on June 17, 2024 by NWEA and has asked NWEA to conduct an investigation into these concerns,” the email read in part. “The MDE is asking that districts pause any review of preliminary data until MDE receives the results of NWEA’s investigation and reviews the results with our independent advisor.”
On July 18, MDE cut ties with NWEA, according to Mississippi Today.
This came weeks after SCCSD leaders began meeting with principals and other school leaders about what the district thought was a massive decrease in student performance.
“We called principals on their vacations,” Jones said.
Davis said principals in the district were in disbelief, many brought to tears over the results.
“Principals were crying, I’m crying,” Davis said. “And I’m saying, ‘What has happened?’ We knew something was wrong.”
When Davis took over the district in the middle of the 2016-17 school year, she pushed to hire Jones in the central office to oversee data.
Over the last seven years, the district has risen from an F rating to a B rating.
Davis credits Jones’ data, which she says has consistently been accurate when it comes to predicting student performance, for that success.
Jones provided The E-T with examples of his projections of how the district would score year-over-year, based on frequent assessments given to students throughout the school year.
The projections appear to have been fairly accurate.
“We know the system that we have works,” Davis said. “We’re going to continue to use our system. We’re focused more on making sure that our children are educated than a number that comes from MDE, because we don’t know if it’s true or not.”
Even though MDE sent updated data files on July 12, Davis and Jones are skeptical of the accuracy of that data, and they fear the fallout from the scoring debacle might last for months, given that district letter grades will be released later in the fall.
“They’ve admitted that they scored the test wrong,” Davis said. “We have third graders who can’t pass to the next grade if they don’t pass this third-grade gate. How do we know that this is not wrong? We have students who may not have graduated. How do we know that wasn’t scored wrong? Is that not terrifying?”
Davis and Jones said that their skepticism is grounded in the fact that the initial bad data passed the smell test at MDE, and the department defended the data in its first email in spite of the fact that multiple districts had raised concerns.
Jones said that he is glad there was some accountability with the vendor, but he also said that it’s not fair to place all of the blame on that one firm.
“When you have a system like that, you have to make sure that before anything is sent that it is correct,” Jones said. “They kept putting it back on the vendor, but they admitted that the vendor sent them the files for them to review, so why did no one catch it?”
Jones said SCCSD contracts with a testing vendor itself, and he analyzes assessment data vigorously before he releases it to district leadership and teachers.
“We use an assessment vendor,” Jones said. “When I get the data, I spend a lot of time going back and forth with the vendor before giving data to the superintendent.”
Davis said that those fears are heightened by the fact that some of the so-called corrected data still don’t match up with Jones’ typically-accurate projections.
“Even now, when we look at the data, we still have some outliers that haven’t been accounted for,” Davis said. “We’re not comfortable making projections, because we don’t know if we’re going to get another set of scores, because we’re still seeing stuff that’s just not in alignment with what we’ve been tracking… We’re still seeing some areas that don’t make sense, so we’re hesitant to put our stamp on it until we get the final files from the state, because we don’t know if something else is going to happen.”
Davis said that if another round of corrections from MDE does not come about, the district will have to accept whatever letter grade is given to it, but she advises folks to take that with a grain of salt.
Meanwhile, Jones projects the district to be a high B or low A for the 2023-24 school year.
