When ChatGPT gets it wrong, it's doing its job
It can help with homework, but not in the way we first thought
In the early days of ChatGPT (you know, like a few months ago), GOTCHA! moments went viral right away. People asked the chatbot a question, and it gave a wrong answer. Proof, the posters and entertained audience thought, that the A.I. actually sucked.
The technical term for these wrong answers is “hallucinations,” and the media absolutely loves to cover them. A simple Google News search brings up national headlines as recently as this week.
But when you hear scientists working in the field talk about hallucinations, they have a whole different way of looking at them. “Hallucination is a feature, not a bug.”
Andrej Karpathy got his PhD in Computer Science from Stanford, helped found OpenAI, then led the A.I. team at Tesla before returning to Open A.I. last year. (I’d say he knows a bit about this.) He recently made waves with a post not only saying this same thing, but doubling down, “Hallucination is not a bug, it is LLM's greatest feature.” Go ahead and read the whole post.
So, what’s the deal? Is this just re-branding? Public Relations 3.0?
After reading and listening to many of them explain the situation, it doesn’t appear that way.
Large Language Models (LLMs) like ChatGPT are not Google. They aren’t designed to be fact-finders. If you need straightforward info or have questions with a right answer, use a search engine. If you need something more innovative, like writing, strategizing, coming up with new ideas, use an LLM.
If you think far down the road, when A.I. is as powerful as we can imagine it to be, we still wouldn’t ask Google for a new cancer cure because Google just directs you to information that humans have already produced. But ChatGPT isn’t about facts that already exist. It’s much more creative. So, someday, it could come up with a novel cure by combining that creativity with more real-world medical data than any human could ever dream of processing.
Still not quite on board? Use one of my favorite ChatGPT tactics: Explain this to me like I’m five.
What does this have to do with my kids?
Now that we know hallucinations aren’t going away — and the creators of the products don’t even want them to — how do we teach our kids to use LLMs?
Let them know from the get-go, it is not a fact-finder. Explain to them the difference between a search engine (Google) and an LLM (ChatGPT). And let them play.
EXAMPLE: Open two windows, one with ChatGPT and one with Google.
Tell each of them your child’s age and that they’d like some ideas to start a viable business.
Follow up with that question and ask them to use SWOT analysis for its answer. (Following up isn’t even an option in Google. Every search starts anew. But it will work in ChatGPT. You just ask another question on the same page because it’s a conversation.)
Ask for them both for the steps to start that specific business.
Drill down and ask for specifics about one of the steps.
Ask for some name and slogan suggestions and what colors would be best for branding in that industry.
Find out if that business name is already taken.
Finally, let them know that LLMs are extremely confident in their wrong answers. You can ask ChatGPT to think of some name ideas for a new kitty hospital you’d like to open. But if you ask it if one named CatGPT already exists, it might confidently tell you that one exists in New Mexico and has for 38 years.
Maybe this is where education should be headed?
The idea of hallucination as a feature, not a bug is way to rethink education itself.
When ChatGPT hit the scene last year, grownups immediately saw it as a cheating machine. Countless companies jumped on the bandwagon and created “A.I. detectors” to catch all the little culprits. Study after study has proven that current A.I. detectors don’t work. They give false positives, they give false negatives, they over-accuse non-native English speakers, etc. They even think the Constitution was written by A.I.
But like I said in one of my very first posts, “Do we want to teach our kids how to reorganize and regurgitate information that’s already available? Or do we want to teach them to use that information to make something new? To combine the facts they can find with the stories only they can tell.”
Maybe fact retrieval and memorization isn’t our goal. Maybe leaning into creativity and exploring innovation should be.