For some education reformers, the dream is to have machines grade examinations – it’s a cheaper and quicker way of scoring test papers, that also removes human bias. However, this sort of automated scoring only works reliably for bubble tests and assessments based on superficial objective questions. Automated scoring is still unable to reliably grade essay writing.
The biggest problem with automated scoring for essay questions continues to be the algorithm’s inability to distinguish between quality and drivel. In fact, it is easy to manipulate the algorithm once one understands how the automated system works – “if someone is smart enough to pay attention to all the things that an automated system pays attention to, and to incorporate them in their writing, then it becomes good writing”.
Trying to manipulate computer algorithms successfully can also lead to more negative consequences. In American states like Utah and Ohio where automated scoring is used for essay writing, more time is wasted on teaching students how to satisfy the computer algorithm instead of developing their own writing skills and voice to become better communicators.
Another key point to note is that computer algorithms are not completely free from human bias as they are claimed to be. Creating the algorithms in the first place involves a certain level of human bias when creators pick the best model essays and input their own biased perspectives on what a good answer should be.
The bigger issue here is the vulnerability of essay marking to human biases. Undoubtedly, all markers and examiners have our own inherent biases. None of us can say we are 100% unbiased. In that respect, AI and automated scoring systems for essay writing is a good solution to ensure a degree of fairness during marking. However, as evident from the article, we have a long way to go to the day when AI is perfected and completely free from all bias.
If you want to find out more about the research, click the following link:
https://www.forbes.com/sites/petergreene/2018/07/02/automated-essay-scoring-remains-an-empty-dream/#662ce39374b9