The Pangram backlash unfolding on college campuses
In recent months, a tool called Pangram has emerged as the gold standard for AI detection. But in higher education, the most commonly used tool is made by Turnitin, which offers AI detection as an add-on to its popular plagiarism checker. About 1,400 colleges and universities in North America have purchased Turnitin’s detection tool, which can automatically scan students’ work when they submit it, Annie Chechitelli, the company’s chief product officer, told me.
In the past academic year, she said, Turnitin detected at least some AI writing in nearly half of all student submissions across U.S. universities. Since its release in 2023, the tool has drawn scrutiny for its propensity to make mistakes. The company claims that its detector has a “false positive” rate—that is, mistaking human writing for AI—of less than 1%. Its “false negative” rate, meanwhile—mistaking AI writing for human work—is about 15%. Like some other AI detectors, Turnitin’s tool is intentionally tuned to give students the benefit of the doubt.
Those numbers might sound low. But as Vanderbilt University noted when it disabled Turnitin’s AI checker in 2023, even a 1% false-positive rate can cause trouble. If Vanderbilt runs some 75,000 student papers a year through Turnitin, that could equate to as many as 750 being mistakenly flagged.
If the main objection to AI detectors is that they make too many mistakes, Pangram might seem like the answer. The startup claims a false-positive rate of one in 25,000 (0.004%) in internal testing; last year, a working paper from researchers at the University of Chicago found that Pangram essentially never mistook long passages of human text for AI. (It occasionally errs on shorter snippets.) Like Turnitin, Pangram can connect to course-management systems such as Canvas. “Education has become a significant part of our business,” Alex Roitman, Pangram’s head of growth, told me.
