The teacher did not respond and took Ostovitz’s grade away.
Ostovitz’s mother, Stephanie Rizk, says her daughter is a high-achieving student who cares about doing well in school and was alarmed when the teacher jumped to conclusions about Ostovitz’s work so early in the school year.
“Know your skill level and then maybe your AI detector will be useful,” Rizk says.
Rizk told NPR that she met with the teacher in mid-November and the teacher said they never saw her daughter’s message.
The school district, Prince George’s County Public Schools, made clear in a statement that Ostovitz’s teacher used an AI detection tool on his own and that the district does not pay for this software.
“During staff training, we advise educators not to rely on such tools, as multiple sources have documented their potential inaccuracies and inconsistencies,” the statement said.
PGCPS declined to make Ostovitz’s teacher available for an interview. Rizk told NPR that after their meeting, the teacher no longer believed Ostovitz used AI.
But what happened to Ostovitz is not surprising.
More than 40% of sixth through twelfth grade teachers surveyed used AI detection tools during the last school year, according to a nationally representative survey by the Center for Democracy and Technology, a nonprofit organization that advocates for civil rights and liberties in the digital age.
that is despite it numerous investigation studies proving that AI detection tools are far from reliable.
“It’s now pretty well established in the field of academic integrity that these tools are not fit for purpose,” says Mike Perkins, a leading researcher on academic integrity and artificial intelligence at the British University of Vietnam.
Perkins found that some of the most popular AI detectors, including Turnitin, GPTZero, and Copyleaks, marked some things as AI that were not, and vice versa. Their accuracy rates dropped even further when the AI text was manipulated to make it look more human.
“We saw some really concerning issues with some of the most prolific AI text detection tools,” he says.
Despite those problems, NPR found that school districts from Utah to Ohio and Alabama They are spending thousands of dollars on these tools.
Why one of the largest districts in the country is using AI screening software
Near Miami, Broward County Public Schools is spending more than $550,000 on a three-year contract with Turnitin. The long-standing edtech company has historically provided schools with plagiarism detection software; In 2023, it introduced an AI detection feature. When educators submit student work to this tool, it generates a percentage, which reflects the amount of text the software determines was likely generated by AI. A warning: According to the companyscores of 20% or less are less reliable.
“The Turnitin tool is something that helps us facilitate conversation and feedback, not grading,” says Sherri Wilson, director of innovative learning for the Broward school district, which enrolls more than 230,000 students and is one of the largest school districts in the country.
Wilson says the district is “fully aware” of research showing that AI screening tools, including Turnitin, are not 100% accurate or reliable.
Turnitin also recognizes this: On the company websitesays, “our AI handwriting detection may not always be accurate…so it should not be used as the sole basis for adverse actions against a student.”
Turnitin wrote in a statement to NPR that it is more important to avoid falsely accusing students of cheating than it is to capture all of AI’s writings.
Wilson says the Turnitin tool remains valuable because it saves teachers time by quickly scanning student work for potential AI use.
Another reason Broward teachers have access to the tool, Wilson says, is that the district participates in academic programs, such as the International Baccalaureate, or IB, in which teachers must authenticate students’ work before submitting it for external review.
Both programs offered by Broward, IB and International Education at Cambridge, told NPR that schools are not required to use AI screening software as part of the authentication process. However, Broward told NPR in a statement, “we have chosen to provide our teachers [Turnitin] as one of the tools to meet the requirements.”
But Wilson says teachers are the ultimate authority on whether a student’s work is theirs, not the AI detection tool.
“They’re using these tools as feedback to then have those teachable moments with the students,” he says.
Why a teacher uses AI detection tools
Language arts professor John Grady says that, for him, AI detection tools provide “a starting point” to start a conversation with a student who may have used AI.
“It’s certainly not infallible,” he says. “But it gives you something to hang your hat on.”
Grady teaches at Shaker Heights High School, part of the Shaker Heights City School District outside Cleveland. The district serves about 4,400 students and is paying GPTZero, another AI screening software company, about $5,600 this year for annual licenses for 27 of the district’s teachers. The tool calculates a percentage probability that a student’s work is generated by AI.
Grady says he submits all student essays to GPTZero; If the tool shows more than a 50% chance that AI was used for the task, Grady goes deeper. That includes using revision history tools to see how much time a student spent on an assignment and how many edits they made during the writing process. If it appears that a student made only a few edits and barely spent any time writing, you will contact that student.
“And I’m like, ‘Hey, this scored. Can you tell me why?’ I would say most of the time, like 75%, if it were AI, they would say, ‘Yeah, I did that.’ And I was like, ‘Okay, now you have to rewrite it with less credit,'” Grady says.
Edward Tian, co-founder and CEO of GPTZero, says this is how educators ought You will use your company’s tool.
“We definitely don’t think this is a punishment tool,” Tian says. “This should be just another tool in the toolkit and not the smoking gun.”
He says it’s important to understand that a GPTZero probability score below 50% means the text is more likely to have been generated by humans than by AI. He says scores above 50% warrant closer examination, as Grady describes.
Tian does not dispute the research showing that GPTZero is not always reliable. But he notes that there are educators, like Grady, who still find it valuable for the information it provides.
He says tools like his offer a “signal about what’s happening in the classroom,” but that teachers should always follow up with students if that sign shows something concerning.
AI Detection Skeptics
Zi Shi, a junior from Shaker Heights, whose first language is Mandarin, says her writing style can sometimes resemble that of the AI ”because of the repetition of the words I use. I feel like it’s because of how limited my vocabulary is.”
Shi, who is not a Grady student, says he is still working on his writing skills and worries that AI detection software may be biased against non-native English speakers like him.
Some educators share this concern, although the research so far is limited and contradictory.
Shi says GPTZero flagged an assignment he completed for his English class earlier this fall as possibly generated by AI. He says his professor suggested that using an online tool called Grammarly could have activated the detection software. Grammarly uses AI to correct grammar and, if requested, generate text. (The teacher confirmed Shi’s account with NPR.)
Shi says he only used Grammarly to clean up his writing and that he wrote the assignment himself. “It was definitely disappointing to see the comment that it had been marked as AI,” Shi says.
Shi believes AI detectors should be considered as a “smoke alarm, that is, a signal or warning. But, you know, sometimes it could be like a false alarm.”
He wonders if the school district should spend thousands of dollars on AI detection software. He says the money could be better spent on professional development for teachers.
Carrie Cofer, a high school English teacher in the Cleveland Metropolitan School District, just a few miles from Shaker Heights, shares that view.
Last year, as an experiment, he uploaded a chapter of his Ph.D. dissertation at GPTZero. “And it turned out that 89% or 91% of it was written by AI, and I was like, ‘Oh, no, I don’t think that’s right, because it was all mine,’” Cofer says.
Cofer is helping his district shape its AI policies and guidelines; She says Cleveland schools do not currently pay for AI screening software and she would advocate against it.
“I don’t think it’s an efficient use of your money,” Cofer says. “The kids are going to figure it out one way or another.”
Some workarounds that students could turn to include using AI detection software, unmarked workshop assignments, and using “AI humanizing” programsthat aim to make AI-generated writing seem more human.
Ultimately, he says, teachers will have to adapt to AI by changing the way they teach and assess student learning.
Back in Maryland, high school student Ailsa Ostovitz is also adjusting. It now runs all its tasks through multiple AI detection tools before handing them off.
The writing is his, he says, but he rewrites sentences that the software identifies as possibly generated by AI, an extra step that adds about half an hour to each task.
“I think I’ve definitely become more careful about presenting my work as my own and not AI,” she explains.
She doesn’t want to take any risks.
This report was supported by a grant from the Tarbell Center for AI Journalism.
