Know

News in a Number: How Long Does It Take AI To Write a College Essay With a Passing Grade?

3 min read
Feb 25, 2021
  • Post on Twitter
  • Share on Facebook
  • Post on LinkedIn
  • Post on Reddit
  • Copy link to clipboard
    Link copied to clipboard

20

That’s the maximum number of minutes it took artificial intelligence (AI) to write a college essay with a passing grade.

Don’t look away now. AI and natural language processing (NLP) are very close to being able to imitate human writing in a more meaningful way than ever before. The development will dramatically change our relationships with software, algorithms, and machines going forward.

In a recent study by EduRef, a website with resources for students and teachers, researchers found that it took AI anywhere from three to 20 minutes to complete a college essay. Human students and recent graduates also took part in the experiment. The human writers averaged three days to complete the same assignment.

The AI system used was GPT-3. GPT-3 was created by OpenAI, a research and deployment business whose stated goal is to “ensure that artificial general intelligence benefits all of humanity.” The organization was co-founded by Elon Musk. Released in June 2020, GPT-3 was made to use the structure of the human language to create content better than any current AI system.

Back to those college essays. Here’s how the experiment worked.

Researchers gathered a group of professors to give essay assignments to both the AI and human writers. The topics were creative writing, law, research methods (specifically, COVID-19 vaccine efficacy), and U.S. History, among other topics. The AI failed only one assignment while getting an average of a “C” in four other topics. Its highest grade was a “B-” in U.S. History and law. The professors didn’t know that AI was involved in the experiment.

What’s the most surprising is that the AI and human writers basically received the same feedback from the professors.

"While 49.2% of comments on GPT-3's work were related to grammar and syntax, 26.2% were about focus and details. Voice and organization were also mentioned, but only 12.3% and 10.8% of the time, respectively,” EduRef said in their report on the experiment. “Similarly, our human writers received comments in nearly identical proportions. Almost 50% of comments on the human papers were related to grammar and syntax, with 25.4% related to focus and details. Just over 13% of comments were about the humans' use of voice, while 10.4% were related to organization."

Over the last few years, NLP and AI has developed to the point that it is now often able to create copy that has context and appears to have been written by a human. Still, NLP has problems that have been difficult to overcome. Up until now, AI hasn’t been able to understand its own writing and often comes up with nonsensical combinations of words and sentences. 

If this latest experiment from EduRef is to be believed, the research suggests that AI might have learned how to more accurately recreate human writing in such a way that university professors can’t even tell it’s not from a human. 

That being said, the writing from AI and humans had marked differences.

“We asked our graders what they thought of GPT-3’s paper and the ‘writer’ behind it without letting them in on our AI secret. Despite earning a passing grade in research methods, U.S. History, and Law, the professors didn’t have great things to say about GPT-3,” EduRef’s report stated. “The grader for research methods noted that the ‘student’ had average writing proficiency and needed to work on clarity and attention to the topic. While the professor grading the history assignment said GPT-3’s prose was fluid at the sentence level, they also noted that the ‘writer’ failed to think critically.”

It’s probably not a matter of “if” but “when” AI will develop enough to truly imitate human writing. When that happens, it’ll change writing, communication, and how we interact with machines permanently.