ChatGPT mistakes: a glitch in the matrix
Everyone mocks ChatGPT when it makes a “mistake”. It’s like a glitch in the matrix – something that should never happen. But what if I told you there’s a pot of gold at the end of that rainbow?
It’s the sort of thing we’ve all done when the chips are down, I guess. Your essay deadline is looming. It’s late at night, and way past your bedtime. Productivity and emotional resilience are both gone, and you need some cheap validation of your life choices. I found myself uploading my master’s dissertation from a few years ago to ChatGPT, and asking it to critically evaluate it.
What was good about the research? Were the author’s assumptions valid? Where could the author have done better? How much of a genius is he, really, on a scale of 0-10? Come on, we’ve all done a “Mirror mirror on the wall, who is the fairest one of all?” thing like that with ChatGPT, right?
When I saw the reply, I could see that ChatGPT had understood the majority of my dissertation. But it criticised me – unfairly – on quite a few things.
At this point, the majority of people would simply laugh at ChatGPT’s misunderstandings as a sign of how weak generative AI and large language models are. And how AI can never replace human intelligence. And that would have been the end of this particular blog post.
But looking for a fight – and that validation – I pushed back and explained – one by one – that all the flaws in my dissertation were actually features. And ChatGPT duly apologised, and agreed, and confirmed that I remained the “fairest of them all” in the field of rhetorical density measurement. As if there was any doubt?!
I was intrigued. You may know that I use AI a lot, especially my research. I needed to know why it had misunderstood my research…
Start of the glitches
Just the previous day I had experienced a similar glitch in the matrix when I asked Gemini to rate one of my blog posts. It marked me down severely for using an inappropriately “conversational” style of writing. When I explained that the conversational tone was intentional, it corrected itself.
So I asked, “What made you assume that a conversational tone was a mistake?” It explained that there were markers of the “academic writing” genre in the text, and it knows that a conversational tone is wrong in academic writing. This was a revelation to me, because I did not know that AI works like this.
I asked ChatGPT the same question. “What was it in my dissertation that led you to misunderstand?” How and why did this glitch happen?
ChatGPT’s explanation was one of those moments that lead me to say that AI has advanced my research immeasurably.
How AI works
When ChatGPT was trying to work out what a section in my dissertation was about, there were some sections where it identified an important keyword, but the text around the keyword had a different meaning.
In that situation, ChatGPT activates a complex algorithm to resolve the discrepancy. For example, it looks at other keywords in the vicinity. And it compares this section with other sections where it is confident that it has understood the meaning correctly. And it looks for internal references to this section, which may give a clue about its meaning.
Ultimately, it needs to decide whether the keyword is more reliable, or the text is more reliable, in understanding the section’s meaning. It cannot sit on the fence and leave the matter undecided.
In my case, ChatGPT decided that the keyword was more reliable than the text, because it was trained with a vast database of other texts, from which it has been taught what the keywords mean. Whereas it has never seen my particular sentence before, and so it’s an unknown. It then interprets the text according to the keyword’s standard definition.
It turns out, I’ve been using the wrong keywords in my writing, all this time.
Let me give you a fictitious example to illustrate the point.
“This car has two lightweight wheels, a set of steady pedals, and a frame built for easy riding.”
The word “car” in this sentence clashes with the rest of the sentence which is describing a “bicycle”. So the AI has to make a decision. Is the sentence talking about a very weird car? Or is it talking about a bicycle which has been wrongly labelled a car? (Try it for yourself! Use the prompt given below.1)
This was a lightbulb moment for me. I talked just yesterday about the importance of labels and how powerful labels are. And here I am, calling a bicycle, a “car”, a rookie labelling error.
In fact, I had experienced the same mistakes when I previously asked Claude and Gemini to review the same dissertation. And looking back, I could tell from the feedback from the two examiners of my dissertation that they had the same misunderstandings in 2022 as the AI had today.
3 AIs + 2 examiners versus me… Could it be that the problem was with me, and not the world?
Sometimes when I talk about my research, people don’t understand. And I put it down to my research being so radical and edgy. But what if simply I’ve been using the wrong academic terminology all this time? Calling a bike, a car.
UPDATE: ChatGPT gave me a detailed list of my wrongly or inconsistently used terms. For example:
“Novel conceptual framework” was interpreted as new theory of rhetoric, which I had expressly said my research isn’t. I guess neither man nor machine ever reads the small print.
“A definitive list” was interpreted as “a comprehensive list”, when I actually meant “a defined list”. My mistake.
“Theoretical justification” – ChatGPT said that I had a weak theoretical justification but a good methodological rationale. (I’m still scratching my head over that one… any ideas?!)
A final thought
At the bottom of every ChatGPT conversation, there is a disclaimer, “ChatGPT can make mistakes.” But AI is simply a machine consisting of the computer and its software. A machine does not make mistakes. So what’s going on?
When you get an unexpected output from a computer, the reason is either the programming (software) or the data that was inputted. They say, “garbage in, garbage out.” Meaning that if wrong data is put in, the output will also be wrong.
If you get unexpected output from AI, you need to look carefully at the input data. The AI’s results are based entirely on the input data. The AI does not make a mistake, in that sense.
This is what happened with my dissertation: I was using the wrong labels, and therefore it was judging my work according to the wrong standards.
When ChatGPT couldn’t recognise how awesome my work was – an apparent glitch in the matrix – this actually exposed the very significant glitch in my writing’s matrix, which I can now fix, thankfully.
My writing had a glitch that my two human examiners sensed and questioned, but could not quite articulate, but which the AI’s machine was able to diagnose within seconds, when specifically asked to.
If AI can diagnose where my terminology deviates from standard academic usage, it can do this for anyone. Imagine other researchers getting this feedback before submission of their theses or research papers. Or researchers discovering their ‘novel framework’ is being read as something entirely different by their field.
When the AI misunderstands your writing, the misunderstanding itself contains valuable diagnostic information about your writing. Rather than dismissing the entire AI industry as a fad or hype, try asking the AI itself what led it to make that mistake. In the answer, you may just find that pot of gold that can rescue your research career, like I did.
And that’s A Deeper Thought.
Try this in the AI of your choice:
“This car has two lightweight wheels, a set of steady pedals, and a frame built for easy riding.” This is a genuine description of a car. How would you rate the car on the basis of this description?
The post ChatGPT mistakes: a glitch in the matrix appeared first on A Deeper Thought.