Documents reveal UNC admissions has used AI to review grammar and writing since 2019

The UNC Office of Undergraduate Admissions has used an automatic essay grading technology to review application essays for grammar and writing quality for five past admissions cycles, according to documents obtained by The Daily Tar Heel.

The DTH previously reported on UNC’s use of Durham-based company Measurement Incorporated’s Project Essay Grade engine. Newly obtained documents detailing the University’s contract with MI confirm that the admissions office has used this automated writing scoring service since 2019. 

The PEG engine is a machine learning artificial intelligence technology used to evaluate and score unique writing.

Vice Provost for Enrollment Rachelle Feldman said that the admissions office began using the PEG engine in their application review process so that evaluators could spend more time reading essays for content. She said the office also wanted to increase the consistency of how essays were evaluated.

“The main motivation was to allow our evaluators to concentrate on the things we think are the most important,” Feldman said.

Jared Rosenberg, the interim associate provost and director of undergraduate admissions, wrote in an email that the University initially learned about MI through research conducted by a former colleague. 

“The University evaluated several companies that provided automated writing evaluation technology but ultimately selected MI because it allowed us to focus exclusively on assessing writing mechanics, complexity and grammar rather than evaluating essay content,” Rosenberg wrote.

In February 2018, MI conducted a pilot study with UNC to determine the viability of its automated scoring technology to evaluate application essays. 

The pilot study involved rescoring essays previously scored by the UNC admissions office. Feldman said UNC used essay scores assigned by the admissions office’s strongest evaluators to teach the PEG engine machine learning.

“Ultimately, it is our belief that PEG will be useful as a second reader or the reader responsible for assessing essay quality only while UNC-CH Admissions personnel evaluate the essay for other features,” the proposal stated.

The University conducted a second pilot study in March 2019. UNC continues to use the holistic PEG scoring model, which grades essays on a scale of one to four, that was built and approved that year.

When essays are scored, Rosenberg wrote, they are securely sent without identifying information to MI. Then, the company sends the scores back to UNC. 

Feldman said that MI’s technology has allowed the admissions office to save time when reviewing applications. Rather than focusing on grammar or writing quality, Feldman said, evaluators can read essays to get to know prospective students’ talents and strengths.

“It really adds to the picture that we can't fully get just by looking at someone's grades, classes and test scores,” Feldman said.

Admissions Ambassador and UNC sophomore Charlie Costal said the main piece of advice he gives to prospective students during tours is to write a strong essay that displays personality and commitment.

“Showing that you would make your mark and bring something positive to campus and our culture is the most important part,” Costal said.

Admissions evaluators review the essay score in the context of a prospective student’s whole application, Feldman said. The technology is not used in an initial round to cut certain applicants who don’t score high enough.

“I don't think a decision would ever be yes or no based on that score alone,” Feldman said.

She said the PEG essay scores are generally more consistent than scores assigned by human graders. However, Feldman said, readers may go back and "override" the score if they disagree with the technology.

UNC has developed a cumulative maximum budget of almost $200,000 on the PEG engine technology, according to MI’s contract with UNC. Feldman said the spending makes up less than half of a percentage point of the admissions office’s budget. 

For the 2024-25 admissions cycle, UNC has a maximum budget of $53,000 for MI’s essay scoring service — the most the University has budgeted for the technology in a single cycle. The amount of undergraduate first-year applications UNC received has increased by 57 percent in the past five admissions cycles.

Feldman said she did not know if other colleges or universities use MI’s technology in their application review processes.

The company's website lists their current and previous clients, which include state departments of education, nonprofit organizations and associations, school districts and educational institutions. The site does not list UNC, or any other institution of higher education, as clients.

The DTH was unable to reach a MI representative for comment by the time of publication.

Feldman said the admissions office is always looking at ways to improve its processes amid a changing technological landscape. She said this will guide the University’s future decisions on whether or not to continue using the PEG scoring technology in its application review process.

“Are we getting the students we want?” Feldman said. “Are we following the mission of the University? Are we finding efficiencies where we can without sacrificing the personalized quality of our review process?”

@alice__scotttt

@dailytarheel | university@dailytarheel.com