I learned via this report (https://www.nbcboston.com/investigations/ai-grading-massachusetts-mcas/3807392/) that the essays in the MCAS are being graded with AI, and that it has graded at least 1,400 essays incorrectly. And that is only the grading that was obviously wrong and discoverable. There are likely more undiscovered critical mistakes.
It is unsettling that my kid’s essays and other kids’ essay are being evaluated by models that have no real understanding of writing. They use statistical methods to find correlations between sequences of symbols and past scores, without understanding anything. This punishes individuality in addition to just plain not working.
It is the wrong tool for the job. As part of my work, I audit AI evaluations of human code to make sure that no erroneous evaluations get out, and I know well the limitations of AI evaluations. I would never, ever allow AI to judge children’s work.
If an AI model said that your child’s essay deserved a score of zero, would you trust it? If you then were able to check it and found that it was wrong, would you just say, “Oh, well, AI models operate stochastically and don’t have a real understanding of writing, so this is bound to happen sometimes, but the efficiency gains and cost savings for the state’s favored contractor make it all worth it!”
Even worse than the gross misapplication of technology here is the disrespect to the students. We ask our students to do their best and use their brains when they write and read. We would not find it acceptable for kids to write an essay with ChatGPT and turn it in or to feed assigned reading to ChatGPT and turn in a summary provided by it. Yet, DESE (the Department of Elementary and Seconday Education) somehow finds it OK to feed student essays to a model and only actually look at 10% of the students’ work.
And it isn’t DESE’s fate to which the MCAS outcomes are tied. DESE has little skin in this game. Rather, it’s the students and schools that are harmed. In many school districts, MCAS decides whether or not students graduate from high school, and the state can take punitive action against districts and schools that do not score well.
Finally, the state has paid Cognia $36.5 million (https://www.mass.gov/info-details/department-of-elementary-and-secondary-education-appendix-a) last year to run MCAS and has signed a five-year, $150 million extension. For $150 million, they’re feeding students’ essays to a model and spitting back whatever comes back, checking only 10% of the scored essays. Absolutely immoral.
If MCAS essays can no longer be graded by people (and I don’t know why this would have changed, other than a hot business trend colliding with a lack of critical thinking), then we shouldn’t have them, and we should reconsider MCAS as a whole. However, if we are going to continue MCAS testing, and we want to make it valid, then legislators and administrators must ensure they are graded by people rather than a probabilistic process. Otherwise, we are teaching kids — by example — that they must submit to nonsense for which we have no well-founded explanation.
If you live in the state of Massachusetts, please call or write to your state congressperson and senator as well as your school district. The state has done something you might think of as ridiculous, but apparently, they don’t know that, so you have to let them know. Thanks to Tracy Novick for posting about this.