HomeworkGPT and Grades

I've had a few conversations in recent weeks about the impacts of ChatGPT on academic life. One friend shared that if they were in school now they couldn't imagine continuing because ChatGPT would make it feel pointless. My cousin, a philosophy professor, remarked that its existence makes grading difficult (since students can make passable work with little more than a prompt), while also threatening his role in exposing students to concepts (see also On ChatGPT and Explaining).

These reactions surprised me. I've been making and toying with various language models for over 10 years, and I'm no stranger to their magic. In recent months I've even tried to get ChatGPT and GPT4 to write like me and found them disappointingly unable. I've also received feedback that my writing "doesn't seem like it was written by ChatGPT". (Though I wonder how long until this is no longer true.)

So then why is assessment difficult? Alternatively, how do we create academic environments where assignments don't feel pointless?

Perhaps these two questions are related, and the observations from my friend and my cousin share a common root. If academic environments are oriented around external performance–from the production of essays to calculations written on paper–then they will be gamed and, later, automated.

My first essay for 10th grade English was graded a C. (Oh the things that stick with you for decades...) So I learned how to write better essays. I studied the assigned reading in depth, took notes, and was eventually getting consistent A's. Two years later I had the same teacher for AP English, but by this point a general malaise (read: depression) led me to not do any of the reading. Despite this I discovered that I could flip the book to random pages, pull quotations, and construct arguments which looked quite similar to the kinds of essays I was writing before by leveraging the literary and philosophical tools I'd already learned.* These essays, too, received A's.

I could try to say that such graded assignments were pointless. But if I hadn't gotten that initial C I'd never have learned how to write those later essays. The real failure, it seems to me, is that the grading system was no longer giving me feedback towards improving.

But what is the nature of that feedback and what, I wonder, was I improving at? Is the teacher attempting simulating the world, judging pieces of writing as publishable or not? Or are grades a proxy for the aesthetics of college admissions essays, or the AP English exam, or the SAT? Or are the grades they hand out simply a reflection of their own preferences? Or, perhaps, are they a reflection of the teacher's perception of the student's potential–a mirror for the student to see themselves in their learning journey?

I imagine those questions depend strongly on the particular teacher, but they lead me to wonder why we have grades in the first place. A hundred years ago the A-F letter grade system wasn't even norm in the United States. (How did our great-grandparents learn?) Now grades have numeric values attached, and you need an undergrad GPA over 3.5 to apply to most grad schools. Which is to say that grades have become an inter-institutional communication protocol. Did you know that prestigious universities have higher grade inflation than elsewhere? And half of all grades at Harvard are A's. Causality gets confusing (they do get to pick students who are good at getting grades), but it's not a leap to surmise that elite institutions are incentivized to give their students the highest GPAs they can without losing credibility, as those high GPAs help their alumni to be over-represented in elite professions.

Personally, I'd much rather have a teacher who is willing to grade harshly and give me real signal on the things I could be improving at. In such an environment I wouldn't write bullshit essays on books I didn't read, nor have ChatGPT write them for me. I'm not convinced that this is difficult because ChatGPT can write passable essays, but rather because grades have become so soft as to not require originality of thought. Let 90% of students get a B or lower if we must. If universities are to have any value at all they'd do well to recognize that it isn't by handing out diplomas to humans on the ChatGPT shell. It doesn't seem to matter whether they do this to retain credibility in inter-institutional communication, or because they no longer provide value for their students (since grades as an improvement signal fail to elevate them beyond the machine).

With any luck this won't just make assessment more feasible, but also contribute to learning environments which feel more meaningful for the students inside of them.

* See also the quotation "every mathematician has only a few tricks" (examples)