Tuesday, January 30, 2024

New AIdeas: Comparing the Quality of Human vs. ChatGPT Feedback on Students' Writing



By now, you have hopefully seen how ChatGPT can save teachers time by completing a variety of tasks like designing lesson plans and assessments. But how does ChatGPT stack up when it comes to providing feedback to students on their written work? 


Imagine this: you're a teacher with piles of essays to grade, and each student deserves thoughtful feedback to improve their work. But time is precious, and sometimes finding the hours to give detailed feedback feels like scaling Mount Everest in flip-flops.

That's where ChatGPT comes in, like a friendly writing AI sherpa! This tool can generate feedback on student essays, pointing out strengths, suggesting areas for improvement, and even offering tips for specific writing strategies. But how does it stack up against the gold standard: human feedback from experienced teachers?

A recent study explored this very question, comparing the quality of feedback from ChatGPT and human evaluators. Here's the scoop:

  • Overall, human feedback still reigns supreme: In most categories, like clarity, accuracy, and focusing on key areas for improvement, experienced teachers provided better feedback than ChatGPT.
  • But hold on, ChatGPT isn't a total slouch: When it comes to explicitly referencing writing criteria, ChatGPT actually edged out human evaluators. Think of it like having a writing checklist robot always at your side.
  • It's not one-size-fits-all: Both human and AI feedback were better for higher-quality essays compared to lower-quality ones. So, no matter who's giving the feedback, it's more effective when students have a solid foundation to build on.
  • Teamwork makes the dream work: Maybe the future lies in combining the strengths of both humans and AI. Teachers can provide personalized insights and guidance, while ChatGPT can handle the heavy lifting of checking criteria and suggesting basic improvements. This way, teachers can free up time for the things that truly matter, like one-on-one mentorship and celebrating student progress.

Remember, ChatGPT is still under development, and its feedback skills are constantly learning and growing. While it may not be ready to replace your favorite teacher just yet, it can definitely be a helpful tool in the writing toolbox. So, the next time you're feeling buried under essays, give ChatGPT a try and see if it can lend a hand (or rather, a keyboard)!