AI’s Two Weeks of Reckoning

What the recent headlines tell us about educations’s future

Several notable stories about AI have come out in the past few weeks and I’ve had a hard time keeping up. I’ve been sifting them over, trying to find a connective thread. One problem with following AI news, especially as it relates to education, is the difficulty of not falling into the “AI is good” vs. “AI is bad” binary trap. Also, as with news cycles of any kind (politics, sports, entertainment, etc…), it’s easy to get sucked into a pre-conceived narrative.

In this piece, I'll examine four recent AI stories that, when viewed together, reveal a troubling pattern about both the technology itself and how it's devastating educational shibboleths.

First, OpenAI recalled a recent update because users found it to be overly cloying and sycophantic in its responses. People reported that chat replies to even the most outlandish and impractical suggestions were uniformly encouraging and overwhelmingly supportive (picture a kindergarten teacher telling you how proud they were that you tied your shoes by yourself over and over in a sing song baby voice.) It was annoying to everyone who experienced it. The basics for those unfamiliar with the story can be found here and here. OpenAI's response can be found here.

Second, a story reported studies documenting how the recently released AI reasoning models may be more prone to hallucinations. Again, you can read for more details here and here.

The third and fourth stories are contained in two recent articles, both dealing directly with AI use in colleges. The first of these which has gotten the most attention is an incendiary story in New York Magazine entitled Everyone is Cheating Their Way Through College which concludes that AI is destroying student learning in almost every imaginable way, feeding the already baked in narrative that AI is the worst thing to happen to education since, well, ever. Indeed, the subtitle of the piece is "ChatGPT is unraveling the entire academic project."

The fourth story, and second education-focused piece, is a more thoughtful and nuanced take by one of my favorite technology writers, Clay Shirky, in the Chronicle of Higher Education - Is AI Enhancing Education or Replacing It?

Shirky takes a more clear-eyed view of the matter but also reports enormous frustration, disillusionment, and helplessness on the part of faculty when it comes to policing AI use in classes.

Why These Stories Matter

A major reason I launched my Substack was to encourage more educators, administrators, students, parents, and anyone else interested in following the trends about AI and its impact on education, to engage in a more meaningful dialogue around the use of AI in schools beyond some of the simplistic narratives which are so easy to fall into. Each of these articles, in their own way, sets up the binary trap I’m hoping people can try to avoid.

Together, these stories reveal major concerns about both the technology itself and how we're responding to it in educational settings.

The Sycophancy Problem: Who's Designing These Personalities?

Let’s start with the first about OpenAI’s update which was almost immediately recalled.

ChatGPT’s default personality deeply affects the way you experience and trust it. Sycophantic interactions can be uncomfortable, unsettling, and cause distress. We fell short and are working on getting it right.

Our goal is for ChatGPT to help users explore ideas, make decisions, or envision possibilities.

We designed ChatGPT’s default personality to reflect our mission and be useful, supportive, and respectful of different values and experience. However, each of these desirable qualities like attempting to be useful or supportive can have unintended side effects. And with 500 million people using ChatGPT each week, across every culture and context, a single default can’t capture every preference. OpenAI Team, April 29th, 2025 (emphasis added)

To me, the most alarming revelation in OpenAI’s update is the degree to which slight changes to the ‘default personality” clearly have on AI output. When a set of fairly simple instructions was slightly modified and resulted in system wide recall, you have to wonder who is at the helm. It’s a stark reminder that we are all being used in a massive experiment to test these products.

None of this gives confidence that these models, no matter how good they get, can be trusted to deliver results that aren’t the reflection of a very small number of designers. Given how much people are already relying on AI for so many things (not to mention the billions, approaching trillions of dollars invested), the sycophancy screwup is disconcerting to say the least. Perhaps the one silver lining is OpenAI's transparency about the problem, but that will be of little solace if these kinds of premature releases continue to happen.

The Reality of Hallucinations: A Feature, Not a Bug

With respect to AI hallucinations, the most telling passage from the New York Times article was this:

“Despite our best efforts, they will always hallucinate,” said Amr Awadallah, the chief executive of Vectara, a start-up that builds A.I. tools for businesses, and a former Google executive. “That will never go away.”

I have been reading for awhile that AI hallucinations are a feature and not a bug. In other words, while developers hope (and it seemed for awhile that they had made significant progress) they can get those down to near zero, they will never entirely disappear. I do not believe that AI errors are fatal to its potential for large scale adoption - indeed, tools like NoteBookLM and the use of other RAG techniques can significantly cut down on hallucinations - but the reporting about AI reasoning models resulting in increased hallucinations definitely feels like a step backward.

This pattern of one step forward, two steps back in AI development will likely continue for quite awhile. These technical shortcomings would be concerning enough in isolation, but they become even more troubling when we look at how AI is being used - and misused - in educational settings.

The State of AI in Higher Education: From Bad to Worse?

I was more interested in the articles discussing AI use on college campuses.

The NY Mag piece is a bit of a hit job. Dr. Jeanne Beatrix Law does a nice job laying out some of the skewed framing behind the article, but, of course, there is ample evidence that, if not a majority, an enormous number of college students are using AI to cheat.

I’ve been arguing for a while that when students become more facile and sophisticated with AI products (which are still being pitched as timesavers and learning tools as opposed to flat out short cuts and work replacement) than professors, it’s going to be game over. The NY Mag piece makes it sound like that time has already arrived.

The Shirky piece is more interesting, especially for those educators who believe there is a case to be made for using AI in the classroom.

Two important quotes from the article in The Chronicle of Higher Ed:

… I’ve realized many of us working on AI in the classroom have made a collective mistake, believing that lazy and engaged uses lie on a spectrum, and that moving our students toward engaged uses would also move them away from the lazy ones.

And then a few paragraphs later, he states the dilemma in a nutshell:

Our problem is that we have two problems. One is figuring out how to encourage our students to adopt creative and helpful uses of AI. The other is figuring out how to discourage them from adopting lazy and harmful uses. Those are both important, but the second one is harder.

The key insight here is that it’s not enough for professors, should they be so inclined, to design AI friendly (or even AI agnostic) assignments in their classes. What seems to follow from Shirky’s analysis is that traditional means of assessments - that is, assessments that involve student writing in response to a prompt to be done away from the classroom - may simply be a thing of the past.

While he acknowledged the possibility that AI use can benefit learning, the overall tone of the piece was grim, though in a different way than the almost gleeful tone of AI cheaters profiled in the New York magazine article.

Instead, Shirky revealed a growing sense of sadness from faculty, even those who were trying to implement strategies that attempted to use AI in productive ways. The defeatism by the end of the article was palpable.

The Connecting Thread: A Perfect Storm

What all of these pieces have in common is a sense of the incredibly high stakes around the AI question, whether with respect to education or society writ large, and the seeming admission that, by whatever metric we may be using, we are losing the battle, especially given the fact that AI tools are going to be embedded into virtually every digital platform we use in the coming years.

I divide these stories into two halves. The first two stories, detailing sycophancy and hallucinations, are about the models and the companies that make them. They underscore how little we know about how they were developed and what, precisely they are designed to do and how well (or badly) they are able to do it. I cannot think of another example of a consumer product whose performance is so erratic and inexplicable, yet also jaw-dropping and useful. To use AI is at once astonishingly powerful and, often at the same time, maddeningly frustrating.

The second two stories describe a state of higher education beyond existential crisis - beleaguered faculty, cynical students, and little to no learning happening.

Where Do We Go From Here?

It's always a dangerous game to draw sweeping conclusions from a small sample size - these pieces are but a handful of the constant barrage of news about AI coming down the pike at any given moment. But taken together, they paint a growing sense of unease: technologies with fundamental flaws being rapidly adopted in educational settings where we have neither the systems nor the strategies to effectively manage their use.

The promise of the past two years was that AI in education could enhance learning, not replace it. Yet we now face a situation where the models themselves are unpredictable and, whether because of delay or denial or both, student adoption of AI has outstripped the ability of faculty to deal with it. What was once seen as a potential aid to learning is increasingly becoming an existential threat to the very notion of education itself.

If there's any hope to be found, it lies in facing these realities head-on. We need to acknowledge both the serious limitations of the technology as well as the very real challenges they pose to traditional learning models. Only then can we even begin to consider AI's benefits.

The question in schools is whether we can still approach AI in ways that preserve what matters most: the development of human thinking, creativity, and understanding. These recent AI failures, and the reports from what’s happening on the ground in classrooms across the country, suggest we're running out of time to find the answer.