You against Turnitin

Isla Black, a senior at Gloucester High School, takes writing seriously.  She has been writing for The Gilnetter for two years and has taken two AP English classes. Recently, she finished an essay for AP Literature that she spent hours drafting and redrafting. Just to be safe, she decided to run it through an online AI checker after she watched a friend get falsely flagged for using ChatGPT.

The essay came out as 40% AI-generated.

“I was so baffled,” Black said.

But with virtually no way to prove that her typed essay was authentically written, “I went back and I dumbed down so many of my words and changed a lot of phrasing,” Black said.

This included painstakingly deleting all of the sophisticated vocabulary words and smooth transitions that so clearly conveyed her knack for writing essays. Turning in the dumbed-down essay, Black accepted that it was not her best work anymore– but what could she do?

“I would rather sound less professional than get a zero on my assignment,” Black said.

As AI use has become a prominent issue in schools, many districts have purchased AI detection software to help teachers determine if writing is authentic or artificially generated. While this may be necessary due to an increase in AI use, not all AI checkers are accurate.

According to Forbes Magazine content specialist  Dianna Mason, running the Declaration of Independence through an AI content detector shows the document being “98.51% AI-generated–despite being written in 1776… a mere 246 years before ChatGPT came along.”

AI detectors rely on “machine learning algorithms” trained to recognize patterns more common in AI-written text than human-written. These patterns include consistent sentence structure and length, repetition and phrasing, and a lack of personal voice.

But, as it’s clear from the false flagging of a 246-year-old document, these tools have no way to know if AI was used– they simply look for suggestions it has been.

And while these patterns- repetitive phrasing and consistent sentence structure – are common to the way generative AI writes, they are also common for students with autism, ADHD, or dyslexia. This is related to the reliance on repeated sentence structure, phrases, terms, and words -a sort of “compositional masking” where neurodivergent individuals learn pattern recognition rather than prose, according to an article from the University of Nebraska. 

In a Stanford University study, 61% non-native English speaker essays were misclassified as AI generated- compared to near-zero scores for native speakers. Similar to neurodivergent individuals, this is likely due to the fact that non-native speakers score lower on perplexity measures such as “syntactic and grammatical complexity,” researcher James Zou said.

Gloucester High AP Literature student Jeferson do Carmo emigrated from Brazil four years ago. He learned English at age 14 after he arrived in the United States. The senior has also struggled with his writing being falsely marked as up to 40% AI generated. He guesses that this may be due to lacking some of these perplexity measures, such as repeating transition phrases in his writing.

“I didn’t know how to write essays when I was in Brazil so when I came here I kind of searched up how to do it,” do Carmo said. “One of the things that showed up is the use of connection words like therefore, however, nevertheless– so I would use them a lot.”

He also points to structural differences in his native language of Portuguese as another factor.

“When you speak another language and when you learn a new language, you’re writing in the way that you speak your native language,” do Carmo said. “A normal person would write a sentence in a more informal way, but Portuguese is more formal. The syntax is different from native english speakers and I think that can make it look like AI.”

Cheating in general has become easier since students began using chromebooks, noted Gloucester High English Program Leader Michael Telles.

“Before Chromebooks, plagiarism was pretty cut and dry and it was pretty easy to catch,” Telles said. “The minute I noticed a syntax change or a vocabulary or diction change in a student’s essay, often I would just enter a section of the essay into google and the plagiarized essay would pop right up.”

But AI has blurred the lines.

“Typically, like a month into school, you have an idea of the voice in which each student writes,” GHS English teacher Rory Gentile said. “I feel like with AI becoming more popular that’s becoming more difficult. I’ll read something, even from one of my high achieving students, and it’ll cross my mind: is it their voice, or not?”

Like many high schools around the country, Gloucester High subscribes annually to a plagiarism detection service called Turnitin. The company’s website boasts false positive rates for use of AI to be under 1%– but it also wrote that “it should not be used as the sole basis for adverse actions against a student.”

For GHS teachers, the goal of introducing Turnitin was to streamline checking for AI after use of platforms like ChatGPT became widespread. But for Gentile, percentage-based checkers are ultimately “problematic” in his classroom for a few reasons.

“Grading takes longer because when you suspect something’s wrong, you need to go through like five different AI checkers to figure it out so that you have a leg to stand on,” he said.

And it only gets more awkward after that.

“Your job as a teacher is to create a relationship with a student,” Gentile said. “Anytime you accuse someone of cheating, that relationship erodes inevitably. Even if you’re just saying I don’t think you did this but it popped and I need you to rewrite it, it sounds like you’re saying that they did something wrong.”

For Gentile, the worst part of these conversations may be the fact that “You never really know if they’ve used it,” Gentile said. “I’ve had situations where I’ve come out on the other side thinking that maybe the tool was wrong, after meeting with students’ parents and learning more about their background.”

Black recalls a time when she copied and pasted her own writing from another document and was accused of using AI because a paragraph was shown as being written in just a few seconds.

“I had to explain after getting this long lecture that it was all my own work, and I just copied and pasted my thesis from one document to the next,” Black said. “My relationship has definitely eased up with that teacher, but for the first few weeks it was a little awkward…It was just kind of weird to get completely reprimanded without being able to explain.”

Black noted the anxiety that comes up now before she turns in papers – ones that are her own,  authentic writing.

“I feel the need to run all of my essays through a checker, and it just adds an extra layer of stress to my writing,” she said.

Black’s experience isn’t unique. Gloucester High Librarian Samantha Teixeira remembers a similar situation when she was working with an AP history class on thesis creation.

“I watched the students write their introduction paragraph and then they became very anxious because they went to plug it back into an AI checker and it said it was AI, even though I watched them write it,” she said. “It was really interesting that they became anxious and freaking out about it even though they knew that they wrote it.”

Though no students ended up getting in trouble for plagiarism, Texeira noticed how this AI-related stress impacted them – and their learning.

“It was this disconnect from what they were doing–which was highly engaging, a lot of students were interested in it,” she said. “But the minute that they thought that it might be AI – even though it wasn’t – they just became really disconnected from the project. It was a major focus on ‘I don’t wanna get caught, it’s saying 50% AI.’ So I think that’s an issue.”

For GHS teachers, situations related to this leave them worried about how AI will affect the future of education – and some simply burnt out.

“Ultimately, I just walk away from situations [around cheating and AI] frustrated and exhausted because I’ve accomplished nothing,” Gentile said. “It’s just a lose-lose for everybody right now.”