Opinion: Mary Willingham’s data offers limited view

F or months, the national media has had a field day with the claims made by Mary Willingham that 60 percent of UNC’s athletes read below an eighth grade level. There have been articles, exposes and even segments on The Colbert Report devoted to it.

After all this, the University has finally released the reports conducted by three independent, external experts of literacy and higher education. All three of these experts found that Willingham’s conclusions were largely unfounded and the data, as well as the test used, were flawed .

And yet, there have been cries that the reports are useless as the reviewers did not consult Willingham or additional data .

However, this notion is groundless as it does not speak to the true purpose and intent of these reports.

Each university-commissioned report was guided by a specific set of questions that focused on claims made by Willingham in January. Therefore, reviewers inquired solely about the validity of the Scholastic Abilities Test for Adults used by Willingham, the notion of defining reading levels by grade and the integrity of the data and test scores on which she based her claims .

In this function, the reviewers succeeded. A cursory look at each report makes clear the flawed nature of Willingham’s conclusions and the grounds on which they were based.

Each expert found problems with the SATA test used to assess athletes and the level of reading skill assigned based on their results. Critiques of the test ranged from its age to its reliability to the low-stakes setting in which the test was administered to athletes. Furthermore, the experts found the idea of grade-level equivalents to be an outdated concept, one the SATA manual specifically recommends against .

The reports also found flaws in the data and the conclusions Willingham drew from it. The sample of athletes studied included far too many from revenue sports, though this might have been Willingham’s intention given her public focus on football and basketball players. Athletes from revenue sports represent a total of 18 percent of all UNC athletes, but were 81 percent of the test sample . This was largely disregarded by the national media, whose headlines accused the whole of athletes of low literacy.

Finally, as many times as each expert reviewed the test scores, they could not come up with the 60 percent figure that Willingham purported, instead finding a 6 percent figure to be more statistically sound — if one were to even use the flawed grade-level equivalents.

These findings were clearly based on hard statistics and facts. As stated by one the experts, the process of a peer review is one of independence and is concerned solely with the evidence — not testimony . Experts, in general, must be wary of others’ opinions and only concern themselves with the story provided by the data . It is hard to see how consulting her or her research partner, Lyn Johnson, would change these facts.

Despite the fact that she has had a large amount of hands-on experience with UNC student-athletes and was sure her data was “100 percent correct,” all of this makes it hard to believe Willingham’s assessment of the general state of athletes’ academics .

However, it would be just as flawed to use these reports, also based on limited data, as a sweeping confirmation that UNC’s athletic program is free of academic shortcomings. The fact is, there were problems with no-show classes and fraudulent grades, and while the administration seems to be working to address these, it would be foolish to use these reports as evidence of success.

Furthermore, one of these reports suggested potential tools that the University could use to assess the progress of these academic reforms. UNC would do well to make use of these tools, whether it continues to work with Willingham or not.

As of now, the narrative pushed by the national media has run its course but has left the impression that UNC athletes can’t read at a college level, regardless of whether it is true or not. No reports refuting Willingham’s claims will drastically affect that notion — there will be no CNN exposes calling her a fraud. While the University should continue to work to mend its image, the most important task is ensuring that the reform it has promised actually works.