Why Teachers Are Worried About AI

For academics concerned about the impact of artificial intelligence (AI) on academic integrity, it has been a very busy week.

The reason is that OpenAI, the creators of the GPT line of AI products, opened the doors to their latest iteration, ChatGPT. The new version marks a significant advancement over their GPT3 AI, which was already known for being both extremely capable and easy to use. To make matters worse, ChatGPT is free to use during a “feedback” period, giving everyone access to it.

This has led many instructors to test out ChatGPT to see how well it can tackle essay assignments. The results have many in education worried.

In an article Alex Hern for the Guardian, Dan Gillmor, a journalism professor at Arizona State University, asked the AI to complete one of his assignments. He said he would have given the paper a “good grade” and that, “Academia has some very serious issues to confront.”

An article by Stephen Marche at The Atlantic takes it to the next level, exclaiming that “The college essay is dead.” There, UK professor Mike Sharples did something similar to Gillmor, and not only determined that the output was graduate-level, but would have likely received a B+ or higher.

The article also notes that some students have already confessed to using AI to write their papers, with some liking it to tools like Grammarly. 

All this raises a simple question: Is this concern warranted and, if so, what can instructors do about the problem?

An Ongoing Problem

The main fear with AI is that students will not have to write their own essays, making them functionally useless as an assessment tool.

However, if you look at it from that perspective, the problem AI creates has been a growing one in academia for decades. 

Essay mills, in many ways, function just like an AI, allowing students to provide a  prompt and then getting a custom-written essay back. Over the past 10 years, essay mills have gotten more plentiful in number, cheaper to use and faster. 

I highlighted this in 2020 by paying a writer $10 to write an essay on the history of copyright, which they did in two days time with a reasonable amount of quality. Though most contract cheating still takes place in person, students that want to skip writing an essay already have plenty of options online and are likely being bombarded by advertising for them already.

AI simply replaces the human author with a computer. What this means is that the problem of AI writing isn’t a new issue, but rather, a continuation, deepening or worsening of one that present since before the internet.

The same as the internet made mail-based essay mills obsolete, AI may make the online essay mill equally outdated. Obviously, this will have major impacts, but there is some hope that the tools developed to combat contract cheating will also work here.

New Tools, New Problems

With the increase in contract cheating, tools that detect plagiarism have attempted to adapt. In 2018 Turnitin launched their Authorship Investigation Tool. Around the same time, Unicheck was working on a similar tool dubbed Emma and, earlier this year, Copyleaks secured funding to AI-powered plagiarism detection service.

These tools all work functionally the same way, by detecting the “voice” of a particular author, usually by analyzing a body of work known to be written by them, and comparing it to new works to detect any differences.

While the results of these tools are often impressive, they come with a serious limitation: The results are often not actionable.

Basically, such tools can point out when they suspect a student’s writing has changed and when a piece was likely written by someone else. However, it can’t prove it as there’s not enough research into baselines to be confident enough to take action.

In short, the tools can point out when there is a suspicion of cheating, but rarely provide enough evidence to act on that suspicion. Many instructors find themselves in this position already with their own intuition, noting that students have changed writing style significantly, but there is no direct proof of plagiarism.

As these tools turn their focus to AI, this will likely continue to be a problem. One of the best tools for detecting AI writing has been using those same AIs to find it. However, confidence in those findings is often low, once again, making it difficult to take action.

In short, detecting AI writing with enough confidence and enough evidence to act is going to be a huge problem moving forward. However, the tools that detect AI writing are, out of necessity, held to a higher standard than those that generate AI text, making it unlikely that they will ever truly catch up.

Shifting the Focus

This challenge has long been understood and, given the difficulties that academia has had responding to a rise in contract cheating, it’s unlikely that it’s prepared for what happens when the floodgates open on AI-written essays (if they aren’t open already).

This is where the focus on changing assessment strategies comes from. 

Previously, much of the focus has been on crafting plagiarism-resistant assignments, giving assignments that are impossible to simply search for and would even be difficult for a human ghostwriter to tackle if they weren’t in the same class.

However, tips the scale on this approach. Students can simply revise the writing prompt and regenerate the essay if the first attempt doesn’t work. Narrow writing prompts or other tactics to frustrate Google searches and human ghostwriters won’t work as well with an AI that can simply try again.

However, requiring more in-class portions, having students submit multiple drafts and other tactics that require more effort from students can still help. The key is to make it impossible for students to simply generate the final draft of their essay, and instead require other elements that an AI can not create.

But this creates another problem. Teachers are already often overworked, and these approaches not only call for more work from the students, but them as well. 

In short, limiting the bots will likely require a good amount of human effort. However, the time and resources to put in that effort that may simply not be available.

Bottom Line

In the end, I don’t think that the college essay is dead. However, it is rightfully being challenged. In fact, the SAT made its essay portion optional in 2016 and that essay was written in pencil under strict supervision, no AI or contract cheating challenges.

However, it’s important to remember that all assessment techniques face challenges with cheating. The pandemic exposed a lot of those issues with the rushed transition to remote learning. 

Despite the challenges, there are still times when an essay is an appropriate assessment tool. Even if it ceases being the default or the gold standard, the essay will likely remain as a tool instructors use to assess student’s grasp of the material.

AI won’t be the death of the essay, but it may change it. It may change the prompts that are used, the receivables that need to be graded, and the general approach to the concept.

However, assessment should always be changing and should always be evolving. The question now for academia is whether the threat of AI be ignored, or seized as an opportunity to evolve not just the essay, but assessment methods in general.