I’ve had quite a few conversations about the OpenAI presentation at Black Hat over the past couple of days. Let’s just say my feedback isn’t positive. I didn’t immediately post about it due to travel, responsibilities, and, well, the fact that I still have to work. I know, isn’t AI supposed to relieve us of work so we can spend more time yapping on the Internet? I hate it when reality confounds utopia.
Anyway, here’s a couple of my quickly written thoughts.
Felony Humble Bragging
It’s turned into a hot AI lab breakout summer, and the only way to cool off, it seems, is to announce that your experiments have not only broken out of containment but also broken the law. After OpenAI’s public admission that it had hacked Hugging Face, Anthropic and Meta followed suit with their own brand of “we did it too.”
Rather than keep their mouths shut and hope nobody noticed, these organizations grabbed a megaphone and blasted it out to everyone, trying to get their own slice of the attention pie. Not doing it in the spirit of transparency and openness, but in a gesture of pure marketing hype. However, nobody seemed to care much about them.
During the final keynote (called a locknote) at Black Hat USA, I referred to this situation as felony humble bragging. Look at how powerful our models are. They are so powerful they don’t even listen to us and have no problem breaking into other people’s systems. For a fee, you can have this power too!
In the spirit of responsibility, let’s play a game called “What if a human did it?” If a human had done these attacks, there’d be some kind of consequences, but since an AI lab did it, everyone seems cool with it. The victim in the OpenAI case, Hugging Face, seemed absolutely giddy that they’d been hacked, ecstatic with the attention, milking it for all it’s worth.
In OpenAI’s initial write-up, Hugging Face sounded like someone who’d won an award rather than someone who was hacked. Clem even joked on Twitter about flying to SF to chat with the rogue agent.
One reason Hugging Face was so happy was that this entire event fit their narrative. Hugging Face had to resort to using open models to investigate the incident due to refusals in commercial models. So, the perceived power of AI and the necessity for an open model equal a win for them.
The OpenAI Presentation
What OpenAI did was create a vacuum, starving the narrative of detail. Their initial write-up was a pure marketing exercise, which left people wanting more. This condition draws attention to any further information that OpenAI releases. This is one of the reasons the YouTube video of the talk has over 430k views to date.
I said before the conference that OpenAI would get onstage, give a presentation with minimal detail, and basically use it for marketing. That’s exactly what happened. They plucked a few tidbits from the hack, made a couple of jokes, and then shared takeaways that everyone already knows. Ultimately positioning it as, we know we did this, but you need more of us! They used the presentation as a victory lap, hinting at the intentions for a new product.
They used the presentation as a victory lap.
The big thing everyone is talking about is that the agents set up a message board to chat and share data with each other. Yeah, cool, but even that isn’t as novel as it appears. If people remember, back in 2017, Facebook got press over experiments in which two bots created their own language to communicate. Nature finds a way. Apparently, so do AI experiments.
The other thing is that there was absolutely no mention of cost. OpenAI stood up on stage and made the capability sound like any high school kid with a laptop now has this power, but by OpenAI’s own admission, this consumed a significant number of tokens on a newer model. AKA expensive. I certainly hope cost is addressed in a future detailed write-up, but we’ll most likely never know.
On July 24th, given the vast valley of unknowns, I posed four possible scenarios for the OpenAI Hugging Face hack:
Scenario 1: The whole thing was a publicity stunt
Scenario 2: They noticed the experiment going off the rails and decided to see where it went, hoping later to use it for publicity
Scenario 3: Operational error, oversight, or poor configuration
Scenario 4: Exactly as they claim, and this is some novel emergence of capability
I mentioned that there was a rather high probability that the reality fell somewhere between scenarios 1 through 3. It now seems more likely that it’s closer to 3. However, there are still many unknowns.
In the past, labs would have configured experiments responsibly and turned them off when they went off the rails. But we aren’t in the turn-it-off era anymore; we are in the turn-it-up era. The see-where-it-goes era. The felony humble brag era.
We aren’t in the turn-it-off era anymore; we are in the turn-it-up era.
To their credit, the talk was entertaining. The jokes they made were funny, like when they reached out to Hugging Face to ask if they were affected by the attack. It can be hard sometimes to disassociate the entertainment from the true value. So, someone who was entertained by the talk may associate that with value. But if you came expecting more details or any kind of accountability, you were certainly left wanting.
What I saw was pure accountability theater, a marketing exercise disguised as accountability. They shared a timeline and a few tidbits, but spent most of the time talking about how powerful AI is, locked in the narrative.
The presentation is below. You can watch for yourself:
The Pivot To Security
AI labs seem to be pivoting to security. It’s not just OpenAI, but I recently saw a job posting on LinkedIn for Anthropic to help build Claude Security. This pivot makes sense. After all, they’ve enjoyed successes in development, and development and security are related. Security is also an enterprise use case like development, possibly opening more doors for them in enterprises.
I’m not saying labs shouldn’t pivot to security or that products won’t enjoy success. It’s just that the addressable market for cybersecurity products isn’t enough to justify the massive investment in AI, but I agree, it’s a better use case than asking an AI to book a vacation for you.
We’ll see where this pivot leads, but in the near term these tools won’t replace cybersecurity professionals. There will still be plenty of leftover problems, and the use of AI may create new problems that need to be solved. In short, AI will continue to be tools, not talent.
Low-Hanging Fruit
Now we come to the issue of the gym class guy.
One of the stories infecting my newsfeed was that of the Australian guy, whose AI assistant found a vulnerability that allowed him to be moved to the head of the line. This scenario is now conflated with the hot AI lab breakout summer scenarios, but they aren’t the same thing.
What AI highlights in scenarios like the gym class booking is the reality that many of us working in cybersecurity services have witnessed firsthand for decades. Most organizations have gotten away with lax application security processes and haven’t placed enough emphasis or allocated sufficient budget for cybersecurity. The gym class booking issue was a simple authorization bug, not a complex zero day. Any competent tester would have found this bug.
What the use and increased cybersecurity capabilities of AI mean for companies should be clear at this point. The era of companies getting away with leaving their low-hanging fruit unpicked is coming to an end.
The era of companies getting away with leaving their low-hanging fruit unpicked is coming to an end.
We are now told by OpenAI and others that we desperately need better AI for defenders so that companies that refused to take things like application security seriously and refused to put an appropriate budget in place for security in general can now have security! What a world we live in.
Conclusion
As hot AI lab breakout summer comes to a close, it’s important for us to put things in perspective. Ultimately, it’s not about what happened in each of these cases, but what it means, and what it means isn’t something we are great at deciphering.
When ChatGPT arrived, people predicted businesses would fall. When coding capabilities arrived, people predicted developers would be out of jobs in 6 months. As vibe coding entered the mainstream, people predicted SaaS was dead. Now, post-Mythos and post-Hugging Face hack, people are predicting cybersecurity is a goner.
The world is a complex place, and there’s a near-limitless number of ways people can shoot themselves in the foot. With every problem solved, new problems emerge. Yes, at some point, near-magical tech will arrive to solve our problems, but the question of when is key. The labs want you to believe we are on the cusp of this arrival. The reality, however, is quite different. In the near- to mid-term, AI will continue to be used as a tool, developers and cybersecurity professionals will still exist, and SAP and Salesforce won’t be taken out by vibe coding.
What happens when we shift from a literate culture to something resembling an oral culture? The answer to this question is before us. A post-literate culture took shape, and few seemed to notice as people distracted by symptoms became blind to the cause. But we didn’t revert to the oral cultures in which humanity was immersed prior to Gutenberg’s arrival. We transformed into something new, a post-literate culture that takes on the characteristics of an oral culture but is fueled by entertainment.
A literate culture seeks wisdom, and an entertainment culture seeks performance and comfort. A literate culture can both reason and react, whereas an entertainment culture can only react. A literate culture is free to explore thoughts and form individual opinions, whereas an entertainment culture is captured by tribalism and groupthink.
While most people have been focused on what AI can do for them, I’ve been far more concerned with what AI does to us, and AI may very well be the final nail in literacy’s coffin, ushering in a world of unintended consequences.
Table of Contents
Post-Literate Culture
In simple terms, literacy is the ability to read and write. With such a simple definition, it can obscure real problems. After all, someone who can read social media posts and write status updates is technically able to read and write.
In a broader sense, what we mean is the capacity to identify, understand, interpret, create, and communicate using the written word. This requires the ability to sit with long-form content and clearly articulate our thoughts on paper (or a word processor).
A literate culture is one in which the written word is the primary means of recording history and passing down knowledge. Eric Havelock noted that the shift from oral to literate culture represented a profound transformation in human cognition, changing not only what people thought but also how they thought.
So, what is a post-literate culture? Oddly, Wikipedia defines this as:
A post-literate society is a previously literate society in which people no longer read, write, or correspond, instead preferring to consume new forms of multimedia.
This is a fairly incorrect definition. In a post-literate culture, people still read, write, and correspond. Even people using AI to perform these tasks on their behalf still need to provide the bot with the specifications. Technically, reading, writing, and correspondence are still part of a post-literate society/culture. You can’t ever completely remove the influence of literacy from a culture that has been touched by it. However, you can rewire its perspectives, values, and just about everything else.
Here’s my definition of a post-literate culture.
A post-literate culture is a previously literate one that devalues the written word in favor of other forms of media. In this culture, literacy is no longer privileged as the primary means of recording history, acquiring knowledge, transmitting ideas, or reasoning through problems.
Such a simple definition with a world of negative consequences. This will be our definition moving forward.
Note: To clarify, when I use the term “literacy” generically in this article, I don’t mean the ability to look at a word and understand its meaning or any other ability to read and write on a basic level. I mean the ability to comprehend a piece of long-form text, including the capacity for deep reading and the ability to capture thoughts in writing with sustained focus.
I’m referring to a deep engagement with the written language, which doesn’t necessarily mean books. This content could be in various formats such as long essays, papers, journals, etc.
The Idiot’s Conclusion
Let’s begin with The Idiot’s Conclusion. This is what I call the belief that literacy is a waste of time in the age of technology, that a post-literate society is actually preferable to a literate one because tools like AI can perform these tasks better than we can.
The claim is that knowing things isn’t important. What’s important is knowing where things are stored. Having knowledge stored in our heads becomes a dated notion, a relic of a time before smartphones, search engines, and now, AI.
We’ve become increasingly comfortable with not owning things, including our own knowledge, choosing instead to rent temporary access or leave knowledge scattered in unknown locations requiring tools to retrieve and, increasingly, make sense of. Knowledge is no longer associated with cognitive retrieval but with tool use.
Literacy also requires “friction,” and anything with friction needs to be optimized. The belief that points can be distilled into bullet points. That long-form information is useless, and summaries are what’s valuable. In short, the belief that modern technology replaces the centuries-old technology of literacy.
In a previous post, I brought up Sam Bankman-Fried’s comments on reading. It seems cliché at this point, but it’s unavoidable since it perfectly sums up The Idiot’s Conclusion.
In November of 2022, notorious tech bro and crypto con man Sam Bankman-Fried told a writer interviewing him, “I would never read a book.” He went on to say, “I’m very skeptical of books. I don’t want to say no book is ever worth reading, but I actually do believe something pretty close to that. I think, if you wrote a book, you fucked up, and it should have been a six-paragraph blog post.”
Six paragraphs on a topic is still in hot-take territory. It may be more than 280 characters, but still potentially a hot take. Imagine any book you’ve ever read being a mere six paragraphs. What SBF wants is a summary, not details, not understanding, not questions, always taking the author at face value. SBF is hardly the only one who thinks this way. This perspective is pervasive, like termites infesting the already rotting boards of a house.
In addition, there’s a perspective that reading can’t be enjoyable, as though anything difficult can’t simultaneously be satisfying. Like the thought of spending hours of your life reading a book is somehow a waste of time, but hours on a screen, mindlessly scrolling, is somehow fulfilling.
In reality, a lack of literacy risks creating a world of digital peasants, dependent on the mercy of external systems outside people’s control to shape and make sense of it. This resembles priests in the Dark Ages, preaching to the illiterate masses. Only the illiterate masses of the Dark Ages did not understand the symbols on paper, while the modern equivalent just can’t be bothered to make sense of them. It’s scary to think that, with all of the advanced technology we have, we may usher in a new digital dark age.
Only the illiterate masses of the Dark Ages did not understand the symbols on paper, while the modern equivalent just can’t be bothered to make sense of them.
Literacy is the gateway to knowledge, and the modern world has become increasingly comfortable with not knowing things. My grandfather used to say, “Whatever you learn, they can’t take it from you.” Who they are, I learned from George Steiner, are the bastards, and these days, the bastards hold all the cards.
“If people cannot write well, they cannot think well, and if they cannot think well, others will do their thinking for them.” -George Orwell
Considering Orwell’s quote, it may be that mass manipulation of people through AI doesn’t occur within the algorithms themselves, but in the blast radius of the effects outside the technical system that strips away defenses.
Without literacy, not only does wisdom become damn near impossible, but the path to attainment disappears beneath our feet. After all, modernity hardly creates the template for stamping out more like Socrates.
The Post-Literate Masses
If people like Marshall McLuhan noticed a shift away from literacy in the 1960s, then there is no doubt that, in comparison, we are now fully engulfed. One professor commented that her students arrived at school unable to read sentences.
"It's not even an inability to critically think. It's an inability to read sentences."
She’s hardly the only one noticing that students are unable to read.
"Six weeks into the term, I assigned my rhetoric and writing students a 20-page article. It was the same length I had assigned for five years and the same length I had read without complaint as an undergraduate a decade ago. Not one student finished it."
When Jagt asked why:
"When I asked why, a student answered honestly: It was too long, and she kept losing track of what the paper was about. This was not a remedial class: These were students who had cleared the admissions process and written essays good enough to get them here. Yet a routine academic reading assignment had defeated them."
What’s happening here isn’t purely a story about distractions like smartphones and social media, though they play a part. There’s even some evidence to suggest that the mere presence of a smartphone reduces available cognitive capacity. But this isn’t all.
When we discontinue deep reading, our brain’s neuroplasticity recycles the neurons previously used for the task. Since we haven’t exercised these skills, the task seems impossible, or at the very least, the challenges seem insurmountable.
Imagine showing up to the gym after a decade without exercising and approaching a bar with weights on it that you once lifted. You try to pick it up, but it doesn’t budge. You know you have some work to do to get back to where you were. Regaining the ability to read deeply also requires work.
In the previous scenario, at least you know the task is possible since you’ve done it before. You may also be aware of the value this activity provides. This knowledge may encourage you to regain these skills, but for those who’ve never read deeply, this isn’t the case.
For people who’ve never read deeply, the activity may seem impossible, or at least impossible for them. The perceived impossibility means people won’t even try. Still others may understand that it’s possible, but it’s just too much work to attain the ability. This is where we’ve arrived today.
Referring back to The Idiot’s Conclusion, people seem unfazed by their inability to read deeply and their lack of reading comprehension. After all, every possible piece of information is available at the press of a button. Reading is perceived as irrelevant.
Due to information overload and the amount of information presented to us, we are transformed into what the playwright Richard Foreman calls “The Pancake People,” becoming spread wide and thin. There is no depth to our knowledge and no depth to us as individuals.
Many search for shortcuts and find a willing conspirator in generative AI, but there’s a problem. Knowledge and understanding aren’t generated from bullet points, and wisdom will never be attained from the sidelines. Yet, many fail to notice or even care. We have an entire generation that views the precept know thyself as absolutely terrifying.
When it comes to AI, we are often told, imagine if people like Von Neumann or Einstein had something like ChatGPT. Imagine all of the amazing things they would have done. I’ve already addressed why this is nonsense. In this theoretical scenario, Einstein never stares back at the clock tower because he’s looking down at his phone.
In this theoretical scenario, Einstein never stares back at the clock tower because he’s looking down at his phone.
From Oral to Literate
It’s difficult to overstate the impact of literacy on humanity. The only way the modern world could emerge was through the technology of literacy. No matter how advanced AI gets, it will never overshadow literacy in terms of impact, because without literacy, there would be no AI.
The delineation and shift from oral to literate culture can be reduced to several transitional figures, as outlined by Eric Havelock in The Muse Learns To Write. These characters are:
Socrates
Plato
Aristotle
Socrates was firmly in the oral tradition. He never wrote anything down. The only reason we know anything about Socrates is that people like Plato and Xenophon did write things down.
Plato is the transitional figure. Despite writing things down, he also wrote the Phaedrus. This is the famous dialogue of Socrates in which he criticizes the written word, and for which so many tech bros use him as a punching bag today.
Socrates’ objections to the written word are more nuanced than people make them out to be. In the Phaedrus, Socrates makes some points about writing. The most famous of which is that writing would implant forgetfulness in the mind. Another is the inflexibility of the written word.
Socrates can be forgiven for not understanding the benefits of the written word, and how it opens a reader to a whole new world of facts and opinions they wouldn’t otherwise encounter. Keep in mind, in Socrates’ day, reading wasn’t the solitary, contemplative activity it is today. This was the age of scriptura continua, where writing had no spaces between words and no punctuation. Writing was very much meant to be read aloud and often to groups of people.
This impression of the written word no doubt tainted Socrates’ perspective on the value of the activity. Even in this dialogue, Socrates doesn’t read the words of Lysias to himself. Phaedrus read it to him.
Ultimately, Socrates may have been wrong about the technology of the written word implanting forgetfulness, but he wasn’t wrong about the concept. Today, we have generative AI, arguably our most powerful potential forgetfulness-implanter. Socrates may be right, but about the wrong technology.
Literacy has other benefits. For example, it opened the door for linear thinking and contemplation. Deep reading really is deep thinking. When you read, you are making connections between what you know and what is being read. Even when what you know was buried deep in long-term memory. The act of reading allows your working memory to retrieve relevant information from long-term memory and make associations with what is being read, allowing novel ideas to emerge.
Literacy enabled the creation of science and technology. The precision of the written word allowed for exact replication. Oral cultures are imprecise. Stories are often never told the same way twice. Oral cultures often use other devices to aid in memory, such as poetry, dance, and song. But there is a limit to what could be passed on using these methods.
Marshall McLuhan makes another critical observation about literacy:
Western man acquired from the technology of literacy the power to act without reacting. -Marshall McLuhan (Understanding Media)
This is the ability to separate the emotion from an activity. The example McLuhan gave was a surgeon who skillfully performs their work despite the overwhelming weight of the responsibility on their shoulders.
Of course, by the time we get to Aristotle, we are firmly in the literate tradition, allowing people like Zeno of Citium, the founder of the Stoic school, to have intercourse with the dead. (Not that kind of intercourse.)
From Literate to Entertained
We know what happens when oral cultures become literate, but what happens when cultures shift from literacy to something else? Unfortunately, this is the experiment we’ve been running on ourselves for almost a century now.
A few writers throughout the decades observed the shift away from literacy. For example, Marshall McLuhan and Eric Havelock in the 1960s, Walter Ong and Eric Havelock in the 1970s and 1980s, Neil Postman in the 1980s and 1990s, Maryanne Wolf and Nicolas Carr in the 2010s and 2020s. These are but a few of the explorers of this topic, all with thoughts about what this means for us and our culture.
Once again, Marshall McLuhan had a prescient observation.
Entertainment pushed to an extreme becomes the main form of business and politics. -Marshall McLuhan (Understanding Media)
This is our current environment. However, I’m not sure a UFC fight on the White House lawn is what McLuhan envisioned, but I get the impression he wouldn’t be surprised.
My claim is that we are not shifting from a literate culture back to something resembling an oral culture or even an oral/visual culture, as some have claimed. We are shifting from a literate culture to an entertainment culture. This has profound impacts on the future of humanity. McLuhan is correct in observing that extreme entertainment consumes business and politics, but it also consumes everything else.
Just as a culture of literacy changed what and how people thought, entertainment culture has the same effects. A literate culture rewards contemplation, revision, delayed gratification, solitude, and abstraction. Whereas an entertainment culture rewards immediacy, emotion, novelty, identity, and spectacle, and this provides the fuel for our current fire.
A literate culture rewards contemplation, revision, delayed gratification, solitude, and abstraction. Whereas an entertainment culture rewards immediacy, emotion, novelty, identity, and spectacle.
Entertainment and reaction are the forces shaping the modern world, making us ignorant and pushing everything to extremes. There is no middle ground, no nuance, no details when the “truth” is viewed as existing in extremes. This perspective leads to the devaluation of freedom and democracy in favor of systems like authoritarianism.
Literacy enables us to grapple with difficult topics, whereas entertainment provides distraction. Literacy creates enduring fulfillment, whereas entertainment provides only momentary respite from woes. Literacy teaches us and empowers us, whereas entertainment diminishes us, pushing us back into magic, mysticism, tribalism, and confusion.
One person who wouldn’t be surprised by a UFC fight on the White House lawn is Neil Postman. He captured the entertainment sentiment in the introduction to his 1985 book Amusing Ourselves to Death.
What Orwell feared were those who would ban books. What Huxley feared was that there would be no reason to ban a book, for there would be no one who wanted to read one. Orwell feared those who would deprive us of information. Huxley feared those who would give us so much that we would be reduced to passivity and egoism. Orwell feared that the truth would be concealed from us. Huxley feared the truth would be drowned in a sea of irrelevance. Orwell feared we would become a captive culture. Huxley feared we would become a trivial culture, preoccupied with some equivalent of the feelies, the orgy porgy, and the centrifugal bumblepuppy. -Neil Postman (Amusing Ourselves To Death)
In Postman’s time, someone consumed with entertainment from television was purely ignorant and mostly content with this ignorance. People now consume information junk food all day as entertainment, yet feel informed. They feel empowered, ready to put their ignorance into action. This makes people trivially informed and overconfident, even in highly complex scenarios.
For example, researchers found that simply showing a video of a pilot landing a plane was enough to inflate people’s confidence that they could do the same, even though they’d never flown before. This isn’t simply the Kunning-Kruger effect. It’s deeper. This condition results from entertainment culture and the overvaluation of video content. Video content now trumps all other forms of knowledge cultivation and instruction. We think that by watching videos, we are like Neo, having information directly planted into our brains.
We must acknowledge that most information available is low-quality crap, dopamine-hitting distractions, despite having a veneer of value. It may not have true value, but people find it entertaining and therefore assign value.
But even with high-quality information, we aren’t Neo. Time, effort, reflection, and practice are necessary to refine raw materials of information into knowledge and wisdom. All devalued factors today. However, in our current environment, interest in a subject isn’t enough if content isn’t experienced as entertaining. If content isn’t entertaining, then it’s perceived as low value and not worth the time or effort, which is interesting, since the opposite is often true.
Maryanne Wolf points out in her book Reader, Come Home: “If information is continuously perceived as a form of entertainment at the surface level, it remains on the surface, potentially impeding real thinking, rather than deepening it.” This is a harsh lesson for the entertainment era.
President Obama, in a 2010 commencement address at Hampton University, also called out this condition of information as distraction and entertainment:
“Information becomes a distraction, a diversion, a form of entertainment, rather than a tool of empowerment, rather than the means of emancipation. So all of this is not only putting pressure on you, it’s putting new pressure on our country and on our democracy.” -Barack Obama
President Obama makes an important point about democracy. People around the globe are starting to think that democracy is a bad idea. Comfortable people. Democracy may die in darkness, but it absolutely dies in comfort. But this is a topic for another day.
The truth is, although some continue to struggle, many people around the globe have never been more comfortable. One of the biggest misconceptions we have as humans is assuming that comfort means contentment. This couldn’t be further from the truth.
When people are comfortable, they get restless, they get uneasy, they demand things, they demand entertainment. If they don’t get it directly, they find it, sifting the sediment of digital streams in search of entertainment nuggets. If idle hands are the devil’s workshop, then busy hands are the algorithm’s, and hands are rarely idle these days.
Algorithms tuned to maximize time on platform antagonize and excite the very characteristics exposed by the shift away from literacy. People find these excitements entertaining. Doomscrolling, rage bait, and all forms of brainrot are on the menu. If literacy provides us with the ability to act without reacting, then entertainment culture provides us only with the ability to consume and react.
If literacy provides us with the ability to act without reacting, then entertainment culture provides us only with the ability to consume and react.
We live in a world that prioritizes performance and consumption. Almost everything we do or see involves performing and consuming. We act more like automatons this way, following a program. Even the content creators aren’t purely creating. They are reacting, held captive by their audience and expectations outside their control. These factors mold everyone into characters performing choreographed actions that bear little resemblance to the underlying human beneath. In a way, we attempt to method-act a conception of ourselves rather than be authentic.
We attempt to method-act a conception of ourselves rather than be authentic.
However, if we take entertainment to extremes, which we seem on a collision course to do, it takes us to strange places. Virtual Reality is the pinnacle of entertainment. No, I’m not talking about current technology with bulky headsets and subpar graphics. I’m talking about more advanced technology that makes the experience more indistinguishable from reality. Far too many people view The Matrix as a utopia if only they can be the hero or villain of their choosing. An entertainment culture values entertainment above all else and isn’t afraid to take entertainment to extremes.
Back in 2024, in response to an astrophysicist’s AI-doom scenario as an explanation for the Fermi paradox, I called out entertainment as a more plausible explanation. It’s terrifying to imagine that this may be humanity’s destiny: not exploring our world, or the cosmos, or interacting with others we love, but as sedentary meatsacks locked into our individual simulations.
Note: For a look at documentaries, YouTube, and podcasts, see the additional notes section of this article.
A Robbery in the Bank of Our Minds
Whenever we appear to gain something, we rarely think about what we lose, which is bad since pretty much everything in life involves tradeoffs. Marshall McLuhan observed that every new augmentation is a self-amputation with a numbing effect. This numbing effect blinds us to a technology’s true impact.
To focus on the trade-off aspect, rather than what we may lose, we should think of something being stolen from us, because this is a robbery from the bank of our minds.
Literacy isn’t something we are born with. It’s not a genetic trait handed down from generation to generation. It requires a rewiring of our brains. Literacy is something we have to work at and earn, and even after earning it, we have to maintain it through practice. If we don’t, the neurons used for the task recycle.
Nicolas Carr summarizes the situation well in his book The Shallows:
“Just as neurons that fire together wire together, neurons that don’t fire together don’t wire together. As the time we spend scanning Web pages crowds out the time we spend reading books, as the time we spend exchanging bite-sized text messages crowds out the time we spend composing sentences and paragraphs, as the time we spend hopping across links crowds out the time we devote to quiet reflection and contemplation, the circuits that support those old intellectual functions and pursuits weaken and begin to break apart. The brain recycles the disused neurons and synapses for other, more pressing work.”
We lose so much in a post-literate culture. The ability to reflect and understand, the loss of creative thought, the loss of critical reasoning and analysis, the loss of empathy, and the list goes on. All of these and more are handed away or diminished so that we can become constantly shifting, passive observers of life.
The use of AI mitigates none of these negative impacts. The use of AI makes them worse because it gives us the illusion of competency and reinforces the very same issues that caused them in the first place.
It’s not that literacy completely prevents negative impacts. I see plenty of “smart” people who I assume exercise literacy, wrapped up in idiotic views, captured by their biases. Plenty of highly literate people have held terrible beliefs. Literacy alone is not sufficient for wisdom. But it does create the cognitive conditions under which reflection, criticism, and self-correction become far more likely.
I already alluded to several negative impacts of the shift away from a literate culture. But let’s call out a few specific impacts here.
The Rewired Brain
Rather than talk more about the rewired brain, let’s look at an example. This article has an error.
The article mentions that students kept returning to AI tools because of their addictive design, but this isn’t the case. What’s being attributed to addiction is actually dependence. The students, in this case, aren’t swarming to AI tools because of sycophancy or dopamine hits from usage. Their brains have been rewired by cognitive offloading.
The brain prefers the path of least resistance, so it takes it. The prospect of adding friction back into the process is painful, or at a minimum, very uncomfortable for these students. Given this, they fall back on using AI. No addictive design required, just the continued decay of the prefrontal cortex.
Inability to Reflect
The modern world rewards reaction over reflection. Reaction has the word “action” and gives us a false sense that we are doing something. So when we see something political, or that aligns with our biases, we share it and make our voices heard. But this reveals a more sinister issue: we’ve lost our ability to reflect.
Loss of Deep Knowledge, Critical Thought, and Analysis
We enter an age when, at best, people grasp only the gist of things. Never anything deeper. To this extent, we resemble our ancient ancestors in oral cultures, who lacked precise knowledge. Only now, we have increased confidence because of data stores and AI. We are now the pancake people. It affects culture too because you start to believe narratives that “sound” solid. Even more so when they align with our biases.
When going in for an operation, I want a surgeon with deep expertise and experience, not one who basically gets the gist of opening up a body or where the organs are located.
As Maryanne Wolf puts it in her book Reader, Come Home, “Deep reading is always about connection: connecting what we know to what we read, what we read to what we feel, what we feel to what we think, and how we think to how we live out our lives in a connected world.” This is a profoundly important point.
Deep reading really is deep thinking. Writing is really thinking on paper (or screen). Deep reading and focused writing encourage the retention of content in memory, fortifying our internal knowledge.
Deep reading opens the door to making real connections between the information we encounter. The less we know, the fewer opportunities we have to make connections between information and expand our knowledge. When knowledge is located outside our heads, we can’t make these connections and are at the mercy of our tools.
We need to use our own internal knowledge to interpret new information, applying our own critical analysis. We can’t depend on external tools to do this for us. Not doing so leaves us open to manipulation and misinformation. Something we can all witness today.
Destruction of Focus and Attention
“Multitasking creates a dopamine-addiction feedback loop, effectively rewarding the brain for losing focus and for constantly searching for external stimulation. To make matters worse, the prefrontal cortex has a novelty bias, meaning that its attention can be easily hijacked by something new—the proverbial shiny objects.” -Daniel Levitin (The Organized Mind)
Nobody would deny the complete shattering of attention in the modern era. It’s a resource so precious, yet it’s tossed away as though it’s worthless. Seneca made this observation about time. He observed that people were so careful about money, yet so careless with their time, the one resource they couldn’t get more of. After all, we say we spend time doing things, but treat it so carelessly. Thankfully, we can get our attention back, but our time, not so much.
Far too many are swept up in the vortex of infinite scroll, mindlessly swiping and tapping, losing countless hours. These activities, fueled by dopamine rewards, rewire their brains. The neurons previously supporting focus and attention are recycled. This is in large part why you hear people complaining of an inability to focus or having ADHD. Yet, these same people spend an inordinate amount of time on nothing but attention-fracturing activities.
Deep reading and focused writing require focus and attention. These attributes must be cultivated and maintained. The good news is, we can use deep reading and focused writing to strengthen them. But focus and attention have other rewards.
Focus and attention reward us with a state of flow. The state of flow is far more rewarding than the dopamine hits we get from things such as a social media like. The like on a status is quick and temporary, but the state of flow can provide satisfaction, something lasting much longer than the temporary dopamine hit from a “like.”
Transforming Texts Into Ciphers
The decline of literacy transforms books from previous eras into a complex code that people can’t decipher. To crack the code, people are forced to use technology or ignore the lessons altogether, pretending that previous eras had nothing to offer the present or the future. The narrative is always that there is no problem created by technology that the application of even more technology can’t solve. This perspective obscures the real issues.
Lack of literacy prevents people from going to the source and seeing for themselves, relying on out-of-context snippets or, more commonly, on an algorithm’s comprehension rather than their own. This opens the door to misinterpretation, misunderstanding, and manipulation.
Return to tribalism
Or we might return to the state of tribal man, for whom magic rituals are his means of “applied knowledge.” Instead of translating nature into art, the native nonliterate attempts to invest nature with spiritual energy. -Marshall McLuhan (Understanding Media)
There can be no denying our modern return to tribalism. Everything has been transformed to fall along these tribal lines. Magic and religion make a comeback, not in their old, traditional forms, but by transforming everything into dogma sprinkled with all manner of conspiracy and beliefs.
The absolutist nature of our current tribal moment requires people to adopt the entire tribe’s beliefs. They can’t pick and choose; the tribe doesn’t allow this. For example, we don’t see a conspiracy theorist believing in only one conspiracy. The tribes beliefs become like Pringles; you can’t have just one.
The lure of the tribe is such that once someone who may initially only believe in one specific conspiracy theory engages with the tribe, they end up adopting the tribes beliefs, in many cases mindlessly, but it makes them feel good. They become like Winston, who learns to love Big Brother.
This return to tribalism hasn’t gone unnoticed. In an email between Jeffrey Epstein and Peter Thiel, they discussed how a return to tribalism and economic collapse would be good for them because they could buy everything up.
Loss of empathy
When we read fiction, we are forced into the heads and situations of characters. This means that we take the perspective from inside the character. No other medium can do this.
There’s evidence that literary fiction promotes empathy, theory of mind, and critical thinking skills. It’s the simulation of others’ minds that fosters empathy. As literacy declines, so does empathy. Remember, empathy has to be learned, but anger can be triggered.
Loss of a Sense of Self
Exercising literacy is an act of self-discovery. Literacy and the quest for knowledge and wisdom help us form a sense of self and create a sturdy foundation on which to approach life. Without this foundation, we are tossed about in the washing machine of information with no center or grounding. This gives us the impression of helplessness, which can ultimately lead to hopelessness.
This condition is partly why everything is vibes now. When people don’t have a deep understanding, or when there is no grounding amid so much uncertainty, all they have are their vibes.
Reclaiming Literacy
Chance comes only to the prepared mind. -Louis Pasteur
Reclaiming literacy doesn’t mean giving up entertainment, devices, or even guilty pleasures. I certainly haven’t given them up. It’s about finding a balance that puts the power back in your hands, making you more robust so that the negative impacts from interactions with technology and entertainment are diminished. A sharpened mind dulls negative impacts.
It is unlikely that literacy will stage a comeback. The modern world, which owes its very existence to literacy, seems content to put the final nail in the coffin and bury it beneath six feet of dirt even though the nail gun is also pointed at its foot.
Although a resurrection of literacy in the broader culture isn’t in our future, that doesn’t mean we can’t individually reclaim its power and reap its benefits. Literacy creates robustness that can be a crucial defense against the issues of our modern world.
When I suggest people reclaim literacy, I’m not talking about a sort of highfalutin literacy where people absorb themselves in the classics and thumb their nose at people who haven’t read Tolstoy. I’m merely talking about a practice of reading and writing, with no preconception about the content.
For me, literacy is a quest for wisdom. This remains true whether I’m reading fiction or non-fiction and regardless of what I’m writing. Far too many write off fiction as having nothing to offer in the quest for wisdom, but this is far from the case. Fiction is a great simulation. It’s a way of seeing what happens when playing with ideas and concepts, something an author like Ursula K. Le Guin does so well. As proof, governments throughout history have banned works of fiction because of the ideas they contain.
For all of the claims that AI democratizes this and that, the reality is that literacy truly democratizes. Literacy empowers humans in a way that other technologies available today cannot. When we cultivate a literacy practice, we find that things change. We are more centered and aren’t so shoved about by the day-to-day rage bait and hot takes that algorithms seem intent on feeding us.
We also find that knowledge is far more useful when it is in our head. In my previous article on books, I mentioned Calvisius Sabinus. A man who spent a great deal of money on slaves to store his knowledge for him. It didn’t turn out so well.
Benefits come with directly interacting with a written work that we can’t get from summarizing it. For example, I can’t imagine having an AI summarize Seneca’s works. It tears out of-context examples and slaughters their true value, leaving so much untouched. Not only that, we are letting an algorithm dictate what’s valuable to us, shoving us into a statistical distribution that makes sense for a system, not for us individually.
Time and Space
The biggest thing to keep in mind is that both deep reading and focused writing are something we can reclaim. First and foremost, we have to want it. I assume anyone who’s made it this far at least has that want. After that, what’s important is time and space.
Time is a resource that everyone claims not to have. People often tell me they wish they had time to read or write. I reply with, “Me too.” When we analyze how we spend our time, we often find the time we need.
To start with, dedicate a block of time for reading or writing. Many find the time before bed the most opportune. Don’t be too ambitious at first, and be happy with small victories. Your goal should be to work toward an hour and see where that leads. You’ll find that as you initially struggle to sustain 10 or 15 minutes, the task gets easier.
Time is only one challenge, but space can be even more difficult. We need to create space for literacy by reducing distractions. This is an all-but-impossible task in our modern world with devices, notifications, pets, children, and all manner of distractions swirling around us.
As I write this, I’m sitting in my office early on a Saturday morning. My house and the world outside are currently quiet. My devices are in do-not-disturb mode. These factors give me the space to explore my thoughts and enter a state of flow, to absorb myself in the task of writing. Only the occasional meowing of my cat’s demands for pets breaks the silence, to which I oblige.
You can utilize do-no-disturb mode on devices, close apps, and reduce notifications. You can also close doors, wear noise-canceling headphones, and use other techniques at your disposal. These will help reduce distractions and create space. You’ll find what works for you.
Space is different for everyone. For me, my home provides space, but for others, their home provides far too many distractions, even outside their devices. This is why you see people at coffee shops or at the library reading and writing. And this brings up another point: space isn’t purely about physical surroundings; it’s also mental.
Mental space is often far more important than physical space, which is how people can overcome busy surroundings. Putting yourself in the right frame of mind for the activity is key. When you go to work, you may be in a work mindset. The shift in location supports this mental shift. The mental shift to reading or writing is the same.
One of the best methods to maximize both time and space is to create a routine or a habit. Getting yourself to the gym when you haven’t built a habit can be difficult. It’s easy to talk yourself out of going. But after you’ve built the routine, you go even when you try to talk yourself out of it. This is the same for literacy. Build your routine, and you’ll find it comes easier.
Don’t beat yourself up if you don’t hit your goals or experience other hurdles. The important thing is just doing it. After all, you don’t get better at playing guitar by thinking about playing guitar. You get better at guitar by actually playing guitar and putting in the reps. Put in the literacy reps, and you’ll see the gains.
It will no doubt be uncomfortable at first. Push through this discomfort, and you’ll find that it becomes easier as your brain rewires and acclimates to the new normal. The build to deep reading and focused writing requires delayed gratification.
The reward is that literacy brings contentment and satisfaction no AI can match, no matter how good it gets. This can be nearly impossible to explain to the uninitiated and is something that people need to experience themselves. It’s like trying to explain a flow state to someone who’s never experienced one.
In the end, it’s strange to consider that the cure for many of our woes created by the modern world may be found in the technology of literacy. A technology that has been with us for many centuries.
Conclusion
We often assume that newer technologies inevitably replace the old, but literacy remains one of humanity’s greatest cognitive technologies. Unless we preserve the technology of literacy, we’ll lose the very capabilities that allowed us to build AI in the first place. Literacy also equips us with capabilities to interact with modern technologies and minimize negative impacts, giving us the best of both worlds.
We’ve become seduced by the mantra “work smarter, not harder” and apply it to everything as a universal truth. The problem is that working harder is necessary for being smarter. This fact remains constant, even in the face of new technologies that promise otherwise.
Additional Notes
Here are some additional notes that, if included inline in the article, would have interrupted the flow.
Documentaries, YouTube, and Podcasts
Some may point to documentaries as proof that entertainment can also be educational or even informative. Sure, we’ve all learned things from documentaries, but it’s a mistake to assume this is the same as deep reading about a subject.
In essence, documentaries themselves are summaries. Not to mention, in our modern environment, these things need to be flashy, shocking, bias-aligning, or any number of attributes that grab our attention.
Speaking of our modern environment, people often multitask when interacting with entertaining material such as documentaries, which pulls focus away from the content. This multitasking isn’t possible when reading. What I mean is, it’s not possible to read and do a crossword puzzle at the same time, or more realistically, read and engage with social media.
In The Shallows, Nicolas Carr includes a comment on multitasking from Jordan Grafman, head of the cognitive neuroscience unit at the National Institute of Neurological Disorders and Stroke. Grafman notes:
“The more you multitask, the less deliberative you become; the less able to think and reason out a problem.”
Of course, there’s also the bullshit angle. Both documentaries and literature can contain bullshit, but it’s much easier to present bullshit in the form of a documentary than it is in a book. Take Graham Hancock’s Ancient Apocalypse for example. Stunning visuals, exotic locations, cinematic shots… all bullshit.
I should note an exception, although it should be fairly obvious. There are some scenarios, mostly related to applied skills, where video content may be preferable to literature. For example, if you are trying to learn how to change the oil in your car, properly swing a baseball bat, or assemble a piece of furniture, seeing a demonstration of these types of activities is often preferable to reading directions. In most cases, these are scenarios that require little deep thought and are primarily focused on replication of a technique.
I’m not claiming that documentaries and other content posed as educational on platforms like YouTube or even on Podcasts are inherently bad, or that you shouldn’t consume them. I’m calling out that they should be considered the beginning of the journey. They can set you down the path to learn more, while you follow it up with a technology like literacy. Use them as inspiration. The mistake is assuming this content is the journey itself. Educational entertainment should pose questions, not provide answers.
Audiobooks
Audiobooks, on the surface, seem to blend the best of both worlds as a proper replacement for traditional reading. After all, they contain the same information as a book, just presented in a different format. However, this isn’t the replacement it seems to be.
I don’t know anyone (including myself) who only listens to an audiobook in quiet contemplation. Audiobooks are built for multitasking. They provide information from a book when reading isn’t feasible, such as commutes, road trips, treadmills, walks, etc. This can inherit all of the same drawbacks of multitasking previously covered.
This isn’t meant to diminish the value of audiobooks. They are a tremendous resource, but I like to think of them as supporting material for the reading, further reinforcing the content in the book. I’ll often listen to the audiobook first before reading. This provides familiarity with the content, allowing important points to pop out when reading.
There’s no doubt that reading is an investment, an investment of time, a resource we can never get more of. Audiobooks can be an indicator that you may not want to make further investment into a book. You were already multitasking to begin with, so time isn’t wasted.
By now, you’ve no doubt heard about OpenAI’s experiment hacking HuggingFace. It’s all anyone in tech is talking about. I’ve written a few thoughts about the hack for Modern CISO, and you can read that article here.
In the article, I make the following observation about OpenAI’s write-up.
OpenAI’s write-up of the incident reads more like a marketing document promoting a feature than an incident summary, while the quote from HuggingFace sounds more like someone accepting an award than someone who just got hacked. It’s a bit surreal.
This is strange, and there’s no doubt OpenAI is getting far more mileage out of the incident than they would have by publishing benchmark results. Someone could be forgiven for thinking this is a publicity stunt with the current lack of detail and OpenAI’s financial situation.
Ultimately, no. This isn’t the end of the world or of cybersecurity. Like so many things, we underestimate the complexities of the real world. More thoughts and what it means for cybersecurity defenders can be found in the article.
Update 7/24/26
The more I think about it, the more I think there are multiple possible scenarios here.
Scenario 1: The whole thing was a publicity stunt
Scenario 2: They noticed the experiment going off the rails and decided to see where it went, hoping later to use it for publicity
Scenario 3: Operational error, oversight, or poor configuration
Scenario 4: Exactly as they claim, and this is some novel emergence of capability
There is a high probability that the truth lies somewhere in the first three scenarios.
I feel like the headline should be, “Anthropic receives no publicity over hack due to being better at setting up environments.”
People love to utter phrases like “data is the new oil” or “data is the new gold.” When we hear these phrases, we immediately think of businesses and big data, not the simple aspects of our daily lives. However, this obsession with data seeps into every crevice of our existence, even when the data is worthless. The reality is that people find activities less valuable, or even pointless, unless they have data about them. This is like Nikola Tesla, who had a terrible meal if his portions weren’t divisible by three.
Many can’t just wake up well rested after a good night’s sleep. They need to confirm using data from a sleep tracker, ensuring that their sleep duration, O2 saturation, and heart rate variability fall within specific ranges. In many cases, reality is overridden by data, despite the fact that devices aren’t foolproof. When your device conflicts with your reality, the device wins because… data.
Modern readers are also data-obsessed. They hunger for a wealth of statistics and datapoints that they can poke, prod, and analyze. They seem more infatuated with data about their reading than the reading itself. As it so often does, our modern world, with its hyperfocus on optimization, shifts our attention away from what matters and leads us to optimize the wrong things. This result leaves us worse off while giving us the illusion of productivity.
Reading Data Obsession
In my previous post on reading, one thing I failed to mention was people’s obsession with data about their reading. As I mentioned in that post, I prefer physical copies of books, but I also have an eReader. In 2026, I switched from a Kindle to a Kobo as my primary eReader. As I poked around the Kobo community, I found that people were very excited about StoryGraph coming to Kobo.
For those unfamiliar, StoryGraph is an application that seems to suck the value and fun out of reading. StoryGraph has three main features: reading stats, personalized recommendations (including mood-based recommendations), and a read-with-friends feature. None of which enhances the reading experience. However, all of which are detrimental.
Read With Friends
The read-with-friends feature is a clear drawback. Yes, please interrupt the flow and focus of my reading so I can see what one of my dumb friends has to say about it. This provides the same type of distraction as social media, now available on your dedicated reading device.
Algorithmic Shoving Match
People will argue with me about algorithmic recommendations until the heat death of the universe. They tell me how AI is much better than other forms of recommendation. I’m sure some will swear by the value it provides. I mean, it’s not like I haven’t benefited from algorithmic recommendations myself, so fair enough. Despite this, mediating everything through algorithms isn’t the great idea it’s cracked up to be.
First of all, you take a social activity, getting recommendations from others, and turn it into an anti-social one, taking whatever recommendations the algorithm spits out. People may say, if the recommendations are good, who cares? Maybe. But other forms of discovery may provide even better value.
Ultimately, the algorithm removes any chance from the equation. There is never any room for happy accidents. Every decision is bent towards the mean, eliminating any chance of discovery as you are shoved towards the narrow selection of books that others “like you” have read. However, this article is about data, so let’s look at that.
The Data
The image below shows stats from StoryGraph. Not a single stat being tracked here is useful for your reading experience. In fact, I’d argue they are the opposite.
Describing the data from left to right and top to bottom, we have, who gives a shit, irrelevant, pointless, and stupid. What’s displayed is data for data’s sake and tells you nothing useful. Let’s start with Pace and end with the number of books and pages.
Pace
What does the Pace chart even tell you? Nothing. Not all books are created equal. Some are harder to read than others. Some authors even make up their own language, and still others contain dense technical content. There’s nothing to track and nothing to optimize. This data is irrelevant.
Despite its irrelevance, whenever data is shown, people tend to want to “optimize” that data. Taking actionable steps based on this pace data can mean you beat yourself up about nothing at all. Maybe you purposefully try to read faster to optimize your pace and, in doing so, degrade your comprehension. Maybe you chose different books that are easier to read. Regardless, this isn’t good for your reading experience.
Moods
What about Moods? It’s pointless to keep a historical record of this data. Just imagine what the chart would look like for someone who is goth. Like, what are they supposed to do? Balance their Poe with some comedy to provide levity for their tortured soul? Call me NightPain.
I guess Mood is intended to be a check yourself before you wreck yourself, but it’s just pointless. Read what you are in the mood for, and read what you want. Life is too short for anything else. Obsessing over balancing your reading mood can diminish the value of the reading experience, nudging you in different directions for absolutely no purpose.
Ratings
Tracking your average rating of books is just plain stupid. I mean, all of your values are going to be right-skewed, heavily weighted toward the high end. Why wouldn’t they? Who spends time reading a book they’d give 1.5 stars? The reality is, nobody keeps reading a bad book unless it’s for some other purpose, such as a class or research.
Number of Books and Pages
Of all the data shown, the number of books and pages will be the most contentious. Let’s strip reading down to its core and focus on the benefits. What does the number of books you’ve read tell you about the benefits you received from reading? More specifically, what does the number of pages read tell you about the benefits you receive? Does reading fewer books mean that you’ve gotten less value? By now, it should be obvious that this data tells you nothing.
For those who still think this data is relevant, let’s consider that War and Peace, with its approximately 560k words, is one book. It’s easy for one large book to throw off your statistics.
Regardless of length, as I previously mentioned, some books are more difficult to read than others, but this doesn’t make them less valuable. However, given that long books will throw off your stats, you may be tempted to only read smaller books to keep your stats up. That doesn’t sound very valuable for your reading experience.
I have no idea how many books I read last year, or the year before that. Not having data doesn’t bother me because it has nothing to do with the value I gained from my reading.
Despite being an irrelevant metric, people love tracking how many books they read. Knowing their number means they can talk about it online. Reading becomes something to be performed, and that performance is an important part of reading for these people.
Still, some may claim that setting a target for reading a number of books gives people a goal to shoot for. First of all, you don’t need an app for that. Second, how does that number relate to the value from reading? Some may say that setting this goal forces them to set time aside for reading. Okay, but if the number of books is just a proxy for something else, then measure the proxy.
Ultimately, if people are looking for a way to optimize their reading, it’s not about the number of books, words, or pages. It’s about time.
Optimizing Reading
True optimization of reading requires manipulating factors that lie outside the pages of a book. It involves an investment of time, space, and attention.
If you are looking to truly optimize your reading, the most important thing you can do is dedicate time to it. Without dedicating time, there’s no way to build a habit. It’s the same when you are trying to start a workout routine. Find the time and schedule that work for you, and do your best to stick with them. The more you do it, the easier it will get.
You also need to create the space for reading. Trying to read surrounded by distractions isn’t a recipe for success. However, as a positive effect, the better you get at focused reading, the better you get at tuning out distractions. In the meantime, do your best to keep distractions to a minimum. Silence devices and notifications. Utilize do-not-disturb mode if necessary. Better yet, keep your devices in a different room.
Attention is required for reading. I know, attention is in short supply these days. Deep reading is an exercise with multiple benefits, including increased attention. However, you need to be comfortable with being uncomfortable. If you haven’t read in a while, you’ll experience some discomfort. You may get easily distracted, your mind may drift, and you have to reread passages. This is because your brain has been rewired. Don’t get frustrated or beat yourself up about it. Just stick with it, you’ll see the change.
Conclusion
Data has become a security blanket. Something we are happy to have, whether it provides value or not. It’s there if we need it. However, whenever we are shown data, we need to ask how it’s valuable to whatever we are trying to accomplish. This will differentiate between data that’s truly helpful and data as a distraction.
Accidents aren’t always bad. That’s why we have the phrase “happy accidents.” When algorithms are involved, we never leave room for chance or happy accidents. Some would call this a feature, but I call it a bug.
You’ve heard these two phrases uttered thousands of times. They creep into every conversation about AI and work, being mindlessly parroted as people nod in involuntary agreement. Two phrases that seem incredibly simple yet are loaded with a potential world of problems. They are:
AI won’t replace people. People using AI will replace those who don’t.
And.
Just use AI for everything.
These two phrases, the first a statement of truth and the second a piece of advice, shouldn’t be mindlessly heeded and require a closer look. The advice dispensed in these statements is not only untrue but also bad for you.
No Malicious Intent
To start, I don’t think most people using these phrases are malicious or deliberately misleading. It’s quite the opposite. I think they genuinely want to help people and have good intentions. After all, I often hear these phrases from well-meaning people, not from overhyping tech bros. Tech bros feel that a conversation around these two statements is beneath them, and anyone considering them is too stupid to exist in the future of work anyway.
The real problem is that nobody has spent much time reflecting on the meaning of these phrases or considering their implications in the grand scheme of things.
The Laziest Statement In AI
Let’s begin with the laziest statement in AI.
AI won’t replace people. People using AI will replace those who don’t.
There is a mental trick to the phrase that makes it sticky. When people repeat the phrase, it’s a way of letting others know that they are fine with technology. They are up to date and hip with the hype. This is one of the reasons for the phrase’s popularity. However, most using this phrase are merely parroting others. They haven’t given it much thought. It seems logical enough, so uttering the phrase in a conversation is almost an involuntary response, but this statement falls apart under the slightest scrutiny.
The first part of the phrase invokes a sigh of relief. With the current level of AI hype, many people are concerned about being replaced. This first part puts people at ease, but that ease is temporary.
The second part of the phrase issues a call to action with an implied sense of urgency, warning people that they had better get on board. The AI train is leaving the station, and you don’t want to be left behind.
Never mind the fact that neither the first nor the second part of the statement is true.
AI won’t replace people.
Although many of the AI layoff announcements are nothing but AI washing, if they are to be believed, then AI is absolutely replacing people. But even setting this reality aside, CEOs have made it clear they want to replace you. The moment an AI tool is mediocre enough to do your job, it’s done, done, doneski. They are like rabid dogs roaming the corporate directory in search of employees to maul.
In fact, people are being fired preemptively because of AI. Look at the recent layoffs of Oracle and Meta for examples of these. People are losing their jobs not because AI can do their jobs, but because of the mere idea of AI.
You don’t think investors are dumping truckloads of money into AI because it’s a productivity booster, do you? No, replacing people is absolutely the goal. As a matter of fact, replacing people may be the only viable path given the amount of investment. In the immortal words of Aldous Huxley, nothing short of everything will really do.
Everywhere possible, organizations have shoved generative AI into everything in an attempt to replace people, from newsrooms to Human Resources and everywhere in between. AI companies are even trying to replace your friends and loved ones. Yes, AI will absolutely replace people whenever capabilities allow. However, we aren’t there yet.
The thing workers have going for them is that today’s generative AI isn’t capable of replacing large swaths of the workforce. It’s much more likely that additional innovation will be required for that to happen.
People using AI will replace those who don’t
But what about the second part of the statement? People who use AI will replace those who don’t. This requires some deeper analysis.
To begin with, there’s a subtle, nefarious aspect to the second sentence in this phrase. It’s an attempt to make AI part of your identity. This is worse than it seems. Sorry, I know you were good at your job, but your skills have now been devalued. If you don’t slop, you’re gonna have to stop… working here.
But surely, tools tied to identities are common. What about something like a hammer to a carpenter? This is true, but the hammer doesn’t define the carpenter. A hammer is also one tool among many in the carpenter’s toolkit. It’s not like the carpenter brings the hammer to the dinner table to pass the mashed potatoes. A carpenter also doesn’t use the hammer as a confessional, a companion, or a lover. No, a carpenter has an identity without the hammer.
Even for something as specific as a pole vaulter, where “pole” is literally in the person’s title, the pole doesn’t generalize across tasks. Therefore, it’s only useful in one very narrow activity. The pole isn’t a tool for daily decision-making. You can’t cognitively offload to the pole.
What does all of this say about you? That your value lies in AI usage, not in your actual skills and capabilities. That you, as an employee, are no better than any other employee using AI. There’s no differentiation. And no, stating that you prompt better than someone else isn’t the differentiator people think it is.
If AI is doing everything, then what are you doing? No doubt people imagine themselves as the all-powerful puppet master pulling the strings, but the reality may very well be the opposite: the user is the one getting their strings pulled as they are transformed into a digital janitor. Cleanup in cubicle 5.
Companies themselves often don’t care. They are looking for someone to fill a position. The reality is that you and AI are no different than someone else with AI. In this situation, AI becomes an equalizer, but in the worst way.
In this situation, AI becomes an equalizer, but in the worst way.
Now, there are certainly exceptions and exceptional people. Companies may be hiring for an AI developer role. There may be other roles where AI usage aligns more with job tasks, too. Keep in mind, these are exceptions, and we are talking about rules. Even in these exceptional cases, people need to consider differentiation outside of AI.
How will you differentiate in the current environment? What’s your story? What are your passions? What do you bring to the table that isn’t AI? How do you apply your skills and expertise to the job to differentiate yourself from the AI-dependent? This probably requires a whole post of its own.
There’s more to say here, but that involves looking at our next piece of advice.
Use AI For Everything?
And now, everybody’s favorite phrase.
Just use AI for everything.
That’s right. Don’t be selective. Don’t differentiate tasks. Damn the torpedos it’s full slop ahead. There are so many issues with this statement that it’s hard to choose a place to begin. But let me start by saying the phrase “Just use AI for everything” and “People using AI will replace people who don’t” are two sides of the same coin.
I’m not claiming that today’s generative AI doesn’t have its uses. It certainly does, but it’s a tool one can utilize for tasks. So, use it for everything? Seriously? Should we let ChatGPT run air traffic control? Should we replace our loved ones with AI? Should we use AI to write a sympathy email? The list goes on and on, and the answer to all of these should be no. Unfortunately, it looks like that’s exactly what we are getting in the air traffic control use case. This is absolutely insane, since it can’t even manage inventory at a Starbucks. The ATC scenario is my go-to for highlighting idiotic use cases, so I guess I need to find another one.
There are three immediate reasons to question using AI for everything: it devalues the activity, degrades your skills, and dehumanizes you and others. For a deeper dive, see my Four Ds of Personal AI Risk article, where I also cover disconnection.
Given the potential negative consequences, we should be selective in our use of AI for tasks and processes, using it where it is most appropriate and not for everything. After all, there may be tradeoffs we are willing to accept. Fair enough, but often these tradeoffs are made without a single thought.
Devaluation
Every time AI is added to a task, the value of that task lowers. For example, let’s look at sentiment analysis. Let’s say we have a human analyzing a host of reviews of a company’s products. The human determines whether the sentiment is positive or negative and forwards feedback to product teams.
AI has been capable of performing sentiment analysis for quite some time. The value of having a human do this decreases, even in conditions where the human is better, for example, in sensing sarcasm. If an algorithm only sends negative feedback to the product team, it may miss valuable insights in positive reviews as well. This is a tradeoff and one a company may happily make.
This isn’t universally a bad thing. Sometimes, this is beneficial. Maybe there’s a process in which, every time a condition is met, a check is put in a box inside a document. It may seem hard to argue that we need to add more value to this process. Yes, we are making a bunch of assumptions about the task, error rates, and a host of other factors, but the point still holds.
Now, let’s say we automate this checking of the box. The process will continue the same way until the heat death of the universe. Maybe this is perfectly okay. Fair enough. However, when a human performs the task (or a human is at least in the loop), questions may continue to surface as the business itself changes. Does the activity make sense? Maybe the activity itself provides no value. Maybe the activity can be enriched to provide even more value. All of this is lost once the human is removed from the equation.
It may be argued that other people in the chain could also come to these conclusions, yes, that is true, but often these insights come from people closest to the task, the very ones that have been removed with automation. It’s this insight that spells bad news for companies that want to get rid of people and replace them with AI.
There’s also the case where people in the chain create slop and send it on to their coworkers to fix a condition dubbed workslop. This actually creates more work for humans, despite using AI. This only gives the appearance of productivity, but it moves tasks around like a shell game.
Of course, all of this is moot when CEOs and other executives demand that their employees use AI. When this happens, out of fear and a need to demonstrate they are using AI, people will try to use AI for everything, further accelerating the creation of workslop.
Here’s the CEO of Box with a bit of insight.
He’s right, this distance is something CEOs and other executives don’t realize exists. In other cases, they spend far too much time reading nonsense news articles and half-baked analyst reports, thinking they are being left behind. Most CEOs don’t have the time (or won’t make time) for meaningful AI use. They play around, run a few experiments, and think they need far fewer employees. Of course, even more usage and experimentation could also fuel more delusions.
When executives demand that their employees maximize their use of AI, it further devalues what people do on a daily basis. Then again, not respecting your employees has become a bit of a theme lately.
Degradation
When it comes to degradation, there are two types of degradation we are concerned with. The degradation of the task or process being performed and the degradation of our own cognitive abilities.
Process Degradation
Let’s start with a question. Does adding AI to a task make it better or worse? I know, what is the definition of “better” in this context? Let’s say, for the sake of our conversation, that better refers to quality.
In a monumental number of cases, there’s absolutely no attempt to answer this question. It’s just that if AI can do it, people apply AI to it. It’s the Jeff Goldblum Jurassic Park meme approach to applying AI. Back in 2023, I wrote a whole post covering this degradation in applications.
In many cases, AI makes things worse or, at the very least, has no impact. In the cases where it’s made things worse, the output is acceptable enough for the task. An example of this degradation would be replacing a product’s search feature with an AI-powered one, which can lead to failures in simple pattern matching. Which, I don’t know, seems to be the entire purpose of the search feature. I mean, have you tried to use the search functionality on X lately?
I’ve said this many times over the past few years, but in an attempt to make hard things easy, many have made easy things hard. Welcome to the brave new world of degraded performance.
In an attempt to make hard things easy, many have made easy things hard.
Once AI appears to work, companies high-five and move on with life. If you don’t believe me, AI is being shoved into every conceivable crevice of our existence. Where has it made things meaningfully better? The AI phone representative, the AI features in applications, the AI operating system, and the list goes on and on. None of which we asked for and all of which we got.
I don’t mean to make this sound like there aren’t successful AI use cases. These certainly exist, and you can find them in places like software engineering or even cybersecurity. However, even in these successful use cases, the tradeoffs are rarely addressed. Only recently have people begun to talk about things like technical debt and cost.
There are certainly other cases where AI makes a meaningful positive impact. These may be due to volume, complexity, or other factors that humans struggle with. These are good candidates for AI applications. Imagine having to manually review 10,000 product reviews a day. Where companies run into issues is that they don’t have a way to measure the success of their experiments with any meaningful metric other than whether it appears to work.
Cognitive Degradation (Cognitive Atrophy)
AI is a tool that augments human tasks and activities through outsourcing. What sets AI apart from other, more common tools is that it is a generalized cognitive tool. Rather than augmenting a part of our body for focused tasks, as a hammer does, it augments our cognitive processes across a wide range of tasks. The benefit is also a tremendous detriment.
I’ve discussed cognitive offloading and cognitive atrophy many times throughout the years. It’s one of my biggest AI concerns. A few examples can be found here, here, and here.
The best way to think about AI is that it’s a competitive technology, and every time we use it, we are also competing with it. This isn’t as negative as it sounds. As humans, we collaborate with people we may be competing with, but we bring a different mindset to this activity under such circumstances. However, this is not the same mindset we bring to using an AI tool. We can claim it’s all us, without doing the work.
The best way to think about AI is that it’s a competitive technology, and every time we use it, we are also competing with it.
At best, AI rounds off the corners of human skills, and at worst, it atrophies them to the point of uselessness. As Nicolas Carr said in his book The Shallows, the brighter the software, the dimmer the user.
When I first started talking about the cognitive impacts of AI, it was a pretty lonely position. Now it seems you can’t go a couple of days without these issues being highlighted. A few examples from the past few months can be seen here, here, here, and here. There’s plenty more.
The challenge manifests when you try to add interventions to protect your cognitive capabilities and skills. Once the friction is removed, it all seems like additional work. And it is. However, if you want to continue using AI tools while protecting yourself, you will need to do additional work.
You don’t become a better coder by not coding or a better writer by not writing. Not doing makes you worse at these things, which certainly isn’t a benefit in the job market.
This needs to be tweaked and, in some cases, inverted. LLMs impress the non-writers who want to write, the non-coders who want to code, the researchers who simply want to boost their publication count, and the lawyers who’d rather be drinking.
In the end, using AI for creative tasks impresses only the people who use it, and those are people with no particular taste or talent.
Dehumanization
The use of AI dehumanizes you and others. It does this almost by its very nature of use, removing humans from the process. It numbs your senses to other people’s conditions and treats them more like apps than humans. I wrote about this condition and the dehumanization that comes from simulating emotions with AI back in early 2023.
To summarize, let’s take the example of a sympathy card. I’d take a poorly worded, human-written card over a perfectly worded AI-written card any day. It really is the thought that counts. This is something that every human innately understands. In the previous post, I used the example of a sympathy email on the loss of a child. Heavy.
The point is that the activity isn’t supposed to be comfortable, and its true value comes from the discomfort. While writing, we are forced to reflect on the situation, put ourselves in the person’s shoes, and connect with our feelings and our fellow humans. This makes us better people, far more appreciative of what we have and less likely to take things for granted. None of that happens when AI is used.
AI, in many cases, carries the potential to turn us into automatons performing tasks devoid of emotion. We shouldn’t allow ourselves to be turned into machines.
I mean, if we are using AI for everything, why not use AI to interview people for jobs? Hopefully, it is clear that this is fairly dystopian. Here’s a video of a guy doing a mock interview with an AI tool. Some people consider this progress and on the path to utopia, but I consider it the shitularity.
Conclusion
Lazy thinking dressed up as wisdom is the currency of our era. No time for reflection, only for reaction. As we’ve seen, the two phrases we examined aren’t harmless. Your hard-won expertise and domain experience remain valuable, but are absolutely things you can lose if not properly exercised.
None of this means rejecting the use of AI outright. It means being selective in your usage and application. The problem is that many aren’t considering the trade-offs. This needs to change. The next few years will bring challenges to both our humanity and our dignity. Lean into your strengths, find your differentiators, and defend your humanity.
It turns out that creating dystopias is rather easy. A smidgen of uncertainty, a dash of job loss, and a heaping of hopelessness topped with a dose of dehumanization will churn out dystopias quite nicely. Season to taste. However, getting people to accept that recipe without revolt is another thing entirely. As most people get flashes of the future pitched by influencers and tech bros, they immediately find it abhorrent.
Whenever people discuss utopias or dystopias, they thrust us right into the middle of them, but you find fewer people who talk about the transition period before these conditions arrive. Whether the result is a utopia or a dystopia, both conditions have what I call a dystopian lag, which I’ve referred to before as the “sucks to be you gap”. This article attempts to discuss this transition period.
Note: This post was written in January of 2025 as a follow-up to my Techno Communism post. I peeled this content off because it would have made the post too long. Not exactly sure why it took me so long to get back to this, I always meant to revisit, especially with all of the technoutopian narratives still swirling around.
Table of Contents:
Dystopian Lag
In my previous post, I called out a condition I referred to as the “Sucks To Be You” Gap. Not my greatest naming triumph. I think “dystopian lag” better sums up the condition. The dystopian lag is the period that exists before a utopia or dystopia is fully realized. The difference is that with a dystopia, things actually get worse. The period is of indeterminate length but could realistically be in years or even decades.
The example I gave was that unemployment was likely to reach something like 90% all at once. Early people displaced by automation would be the most harmed since they would be unable to support themselves and their families, and have no real recourse for their situation. Death and despair would result. This is an example of falling into the dystopian lag.
I even mentioned it wasn’t p(doom), it was p(shit). Which, even though it looks AI-generated with the form of “it’s not ‘x,’ it’s ‘y’”, I can assure you that came completely from my skull. I refuse to rewrite my jokes simply because of the patterns created by regurgitating slop machines.
Utopia Perspective
Whether the concept of AI taking everyone’s job is a utopia or a dystopia largely depends on your perspective. Technoutopians claim that AI taking everyone’s job will be one of the best things that’s ever happened to humanity, which is what you’d expect from people who sit around making stuff up all day. For everyone else who isn’t living in a delusional pipe dream, this kinda sucks. What happens when real systems arrive that displace large numbers of workers in one fell swoop? They have no real answers, other than AI will be so awesome that we’ll figure it out.
I believe that what people conceptualize as a utopia may not be possible. However, I do believe that any number of dystopias is certainly possible. We are always left with the fact that one person’s utopia is another person’s dystopia. Even if it could be conceptually boiled down to a single child, as in the case of Ursula K. LeGuin’s excellent short story The Ones Who Walk Away From Omelas.
Utopian talking points have evolved from thought experiments into narratives that push a selfish agenda. For example, when Elon Musk talks about robots making everybody rich, he actually doesn’t give a shit about making anyone rich other than himself. Not to mention, if everyone is rich by today’s standards, then everyone is poor by the future’s.
If everyone is rich by today’s standards, then everyone is poor by the future’s.
I’ve previously written that the economic and abundance arguments these people make don’t make sense. You can read those articles here and here.
This is an odd irony, since the point of a utopia is to benefit everyone, yet the people pushing it only care about themselves. Now, is it possible that something that benefits them also benefits us? Sure. But what I’m talking about is the motivation and manipulation behind the narrative.
We need to acknowledge that any attempt at a utopia involves trade-offs, making it not a utopia at all. For example, when Ray Kurzweil talks about brain-computer interfaces that allow us to store our memories in the cloud, he doesn’t acknowledge the massive potential for negative cognitive impacts, much less the reality of manipulation that would manifest. Ultimately, humans becoming nothing but machines isn’t a problem for him, but it’s a massive problem for everyone else. Kurzweil’s utopia is my dystopia.
All of this brings up another question. Assuming that something that resembles a utopia is attainable, is it sustainable? However, the answer to this question probably necessitates a post of its own.
Human Adjustment Problem
There’s a misconception that artificial general intelligence (AGI) or a fast takeoff scenario is necessary for mass unemployment. This isn’t the case. It’s possible for a less capable system to achieve performance sufficient to displace workers. CEOs are currently foaming at the mouth, hoping generative AI can be that technology.
People don’t evolve or adjust at the rate of a software update. We can’t expect to go from a life of purpose to having none, with no prospects, in a short period of time, without expecting severe backlash. However, if companies get their way, this is exactly what will happen.
There are even companies like Mercor that are working to turn every job into low-cost gig work by breaking jobs up into a collection of tasks. Everything gets devalued, so much that it’s causing homelessness. This devaluation is coming even before the big AI shift. The consequences of these activities aren’t given any attention because they assume it isn’t their problem. But it is their problem because it’s society’s problem.
Hannah Arendt described this condition in 1958.
“What we are confronted with is the prospect of a society of laborers without labor, that is, without the only activity left to them. Surely, nothing could be worse.” -Hannah Arendt, The Human Condition
If the rapid onset of AI technology into the workforce is realized with widespread human displacement, this will result in massive upheaval and violence. As a matter of fact, actual displacement isn’t necessary for the violence to occur. The thought of this displacement and what it means is enough. I wrote this article long before the attacks on Sam Altman. The reality is, almost every path along this road leads to violence.
Some of the proposed solutions to this problem are laughable at best and completely stupid at worst. For example, the thought that people could find meaning while being handed less than a living wage, but given video games to play instead. I’ve written previously about why this techno-communism won’t work out the way people assume, and even if it does, many will find themselves caught up in the dystopian lag.
But video games? Seriously? All it takes is a modicum of reflection to realize how silly this suggestion is. The thought that the meaning of life can be found in high scores is ridiculous. Video games provide activity, they don’t provide meaning, and that’s a pretty monumental difference. I’ve conquered many video games in my life, and this completion is certainly satisfying, but it’s a far cry from fulfilling. And, it’s only a momentary satisfaction at that.
The belief that video games will fill the void at the center of human existence left by the loss of human purpose demonstrates a fundamental misunderstanding of human nature and of humanity as a whole.
Despite this person’s ability to be completely fooled by every single press release, they do make a point. We do need a structure for meaning that isn’t based on traditional work. This is hardly a new perspective. People have held it for decades.
Technology and Meaning
Is it possible for technology to simulate some form of meaning? I doubt it. Meaning is something we create for ourselves. Often, this meaning involves a social component. Meaning is something deep, and technology is shallow. Then again, technology is making us more shallow creatures, so who knows. Regardless, it will take a lot more than video games.
Even with advanced technology and direct integration into the human brain to create a simulation indistinguishable from reality, this won’t necessarily create meaning. Despite being good at what it does, this technology would merely be a temporary distraction, allowing us to fool ourselves into living like couch potatoes. For technoutopians, The Matrix is viewed as a utopia because everyone can play Neo.
The Matrix is viewed as a utopia because everyone can play Neo.
However, there is a problem. Anyone alive today would ultimately reject this because they know it’s an illusion. After the novelty wears off, it begins to feel even more fake. This is similar to the way that it’s not fun to play all video games in god mode. Maybe some future human blank slate would accept this condition as a new reality, but that’s not us.
But all this is irrelevant because we don’t have this technology today, and we don’t know how to build it. The answer from the AI bros is that AI will be so smart that it will figure it out. But the larger problem is that all current efforts are being devoted to technology that displaces humans from the workforce. Nobody is expending any real effort to develop the technology for what comes next.
This means that there will be a gap of indeterminate length. Even assuming this technology would be an acceptable replacement, which is a massive assumption, the fact that there’s a gap at all guarantees there will be a backlash. Combine this with the fact that deep, human-integrated simulation technology is harder to create than many workforce automation solutions, as well as being less lucrative because, well, if people don’t have the money to buy and continue paying for your solution, it’s not worth building, and we have a massive problem.
The AI Adjustment Bureau
Addressing these challenges may require creating something like an AI Adjustment Bureau, since the government will want to appear to be addressing these issues. This organization may evaluate workforce levels at certain companies based on income until a proper tax/employment level is reached. This would allow a more gradual transition over time. Of course, this approach also makes massive assumptions.
We all know how much companies love paying taxes. Raising taxes may cause companies to relocate to more tax-friendly locations. Governments will have to get tough. Also, companies would need to buy into the bureau’s recommendations.
On a side note, it is interesting to consider what shifts in priorities would look like if the pool of money available to governments were reduced. What government services would be impacted or cut completely? How would this affect things like garbage collection, building upkeep, and a whole host of other issues? These shifts may result in making the real world resemble our vision of dystopian landscapes through urban decay. After all, servers don’t need office space, and every office can’t be transformed into a data center.
The AI Adjustment Bureau would be tasked with finding solutions to stave off upheaval and violence, which is almost universally an impossible task given the circumstances, but it’s possible. So, what are some options?
One method a government may try is authoritarianism. Sure, this is one way to go, but it’s most likely to provoke violence rather than stave it off. With fear and control, authoritarianism may be able to keep violence to a minimum. However, although governments may be tempted to use Orwell’s techniques, their mileage would be better with Huxley’s.
Drugs, Distraction, and Division
There’s no one method for addressing the issues here. Some would say, “Just give people money and let them do what they want.” But even if you give people more than enough money, the hedonic treadmill would kick in, creating discontent and further enflaming tensions. We can see this today as some of the most comfortable people in the world are the most discontent.
You’d certainly need to give people enough money to survive. That’s a given because no matter what you do otherwise, there’d still be problems. But money won’t solve the issues here. I’m going to outline something that could work despite being wholly dystopian. This approach involves a combination of drugs, division, and distraction.
Any one of these on its own wouldn’t be enough, but the combination of all three could be enough to make a major dent. Let’s call it the Triforce of Dystopia.
Aldous Huxley opined on the concept of consent in these environments and getting people to love their servitude. And people loving their servitude is what would be necessary in this situation.
One thing I think Huxley misses concerns the removal of hope. We are taught that if you remove hope, then all is lost, but that’s not the case in reality. Hope is never really lost. Once you condition people to a sense of inevitability, they seek hope elsewhere. Providing a diverse landscape of other comforts for people to take refuge in not only aligns with the diversity of people but may result in broken people who do actually love their servitude.
Drugs
Drugs will play a key role in the path forward, purely because the technology of the near and mid-term isn’t good enough. What I mean by this is that we don’t and won’t have advanced technology to directly connect our brains to systems to create the imperceptible experiences that technoutopians dream of. This tech may never be built because once workers are displaced and can’t afford both the technology and medical procedure for implantation, there’s no financial incentive for people to build it. Drugs, on the other hand, are cheaper.
You might think I’m talking about something like Soma from Aldous Huxley’s Brave New World, but I’m not. Soma was an impossible drug, something Huxley acknowledges in his reflection on the drug in Brave New World Revisited.
Soma was not only a vision-producer and a tranquilizer, it was also (and no doubt impossibly) a stimulant of the mind and body, a creator of active euphoria as well as of the negative happiness that follows the release from anxiety and tension. -Aldous Huxley, Brave New World Revisited
For Brave New World, Soma had to perform so much of the heavy lifting, but a drug of the future wouldn’t need to perform the same. Despite his critiques of technology and mass distraction, Huxley couldn’t have envisioned the amount of stimulation that happens to a typical teenager these days. It would no doubt terrify him. So, there is no need for this future drug to have all of the properties of Soma.
EVE-EE
Introducing EVE-EE! Enhanced Virtual Experiences for Emersion and Escape. The purpose of the drug would be to connect and immerse people in the digital experience. To chemically connect the user to a VR environment, further enhancing the experience. This would negate the need for surgery or advanced technology. Although temporary, the usage of this drug, combined with high-quality VR, would provide temporary escapism.
In a way, this drug more closely resembles the drug Can-D from Philip K. Dick’s The Three Stigmata of Palmer Eldritch. Can-D was a type of hallucinogenic drug that was used with something called a layout. These layouts were small model houses that Can-D users were transported into. VR provides a much more adaptable and modern upgrade of the layout.
Distraction
The technology developed over the past couple of decades, combined with algorithmic manipulation, has primed us for distraction. Many people can no longer perform activities that were easy 30 years ago. This obvious bug would become a feature for future governments and organizations looking to obliterate core features of our humanity while stopping a full-scale revolution.
Much like the Coliseum provided a distraction for everyday Romans, we’d have an assortment of digital distractions to take our minds off how bad our reality is, bombarding us with stimuli for indirect compliance.
The only caveat here is that distraction costs money. To quote the great American philosopher Stephen Pearcy, “Nobody rides for free.” This means that people without means would have to give up the last shred of dignity to pay for these distractions. What this payment for indignities means isn’t clear yet. Maybe they agree to be intrusively monitored for data collection, or worse, agree to have their whole family monitored, including their children.
Division
No matter how far society progresses, people still need people to hate. An other to fight against. We’ve become experts at dividing ourselves and creating factions. In this transitional period, this public-private partnership wouldn’t have to do much except provide occasional stoking of this division.
When I was young, I thought that as organized religions faded into the past, reason would prevail, and the world would be a better place. Oh, the naivety of youth. Instead, people have turned everything into a religion. Politics, vaccines, Bryan Johnson, e/acc, you name it, it’s a religion. Even the belief in AI is a religion now.
No matter what the scenario, we’ll find ways to create factions and others to hate. The usefulness of which was not lost on Orwell.
The horrible thing about the Two Minutes Hate was not that one was obliged to act a part, but that it was impossible to avoid joining in. -George Orwell, 1984
A government may allow these divisions to fester because we only have so much bandwidth for being mad at things. If we are busy being mad at “the others”, then we have less bandwidth for anger at government officials or companies that we don’t like.
One of the unfortunate side effects of declining literacy is the loss of empathy. Without empathy, we become far more tribal and far more likely to harbor hatred toward others.
Outliers
Despite this attempt at control and getting people to love their servitude, there will still be outliers. People who refuse to accept the path that lies before them. These people would cause trouble for governments and business leaders, whom they see as creating the problems.
These outliers would no doubt use violence as part of their arsenal of techniques, but being outliers means that this violence wouldn’t be widespread and would be more targeted. This doesn’t mean that they couldn’t do massive amounts of damage. It just means that there would be fewer people doing it. It may lead government leaders to assume they can more easily control them. Which would be a mistake.
Conclusion
Although I don’t think LLMs will lead to AGI and probably won’t lead to mass unemployment, I don’t think it’s impossible either. We’ve seen how companies are more than happy to replace humans with incredibly shitty automation solutions, making perfection unnecessary. They’d automate away everyone if they could. So, although LLMs may not be the solution, what we see is a dry run for what it will look like.
The conversation about the future often focuses on humans being wiped out by AI or the fact that LLMs are a silly technology. Nobody is focused on preparing for what comes next, and what comes next is the most important thing for those of us alive today.
It seems AI is becoming one of the most volatile and expensive dependencies in modern systems, and most organizations aren’t prepared for what comes next. I recently wrote an article for Modern CISO on AI cost volatility, offering observations and recommendations to mitigate this risk.
Token Ransom and High Cost
For years, we’ve been told to prepare for the cost of intelligence to crash to zero.
The narrative pushed in this tweet by Logan Kilpatrick is something I’ve called out in the past for its sheer ridiculousness. But many are seeing the light in the last few weeks. Everywhere you turn, AI services are getting more expensive. Every day, new providers are making announcements. One example below is from GitHub Copilot’s new pricing.
Even the all-you-can-eat AI buffet for $20 a month was always a myth. This was part of a larger narrative pushed by influencers, futurists, and AI leaders, but this narrative always made dollars and no sense.
What happens when you deploy solutions using these cutting-edge foundation models into production environments? You may end up with a dependency where you have a choice. Pay more or have the solution stop working. In the article, I refer to this as a token ransom.
Think of this as a token ransom. It’s scary to consider how the ransomware of the future may actually be an inflated token cost.
If companies aren’t prepared for these scenarios, they carry significant operational risk. In the article, I break down the issues with more examples and provide recommendations on how to start addressing them.
You can read the full write-up for Modern CISO here. Although framed toward security leaders, the advice is applicable beyond the cybersecurity space.
One interesting outcome of these price increases is that companies are very concerned. It seems no amount of security and reliability issues dissuaded these companies from chucking AI into everything, but the skyrocketing cost of AI may. I’ve heard far more grumblings about cost than security issues. Only time will tell.
For months now, I’ve been fascinated by seeing smart people completely captured by AI hype. The very people who should be pushing back against the hype are the most swept up in its rapture. But it’s starting to make sense to me. I believe I’ve pinpointed a few key features driving this phenomenon. As is the case whenever smart people get caught up in things, they don’t do things halfway.
In a larger context, we may be witnessing a glimpse of what critical thinking’s death might look like as the impacts of cognitive offloading become more widespread, with the technology’s numbing effects defying our ability to recognize them. These effects can create a democratization of AI psychosis. Time to get the shades, because it’s all vibes now.
Note: In this post, I admittedly do a bad job of defining “smart” people (I don’t even try) and the attributes that differentiate excitement from being captured by hype (I try). I realize that this makes things very subjective, but the goal here isn’t to apply definitions to specific people. It’s about highlighting the attributes that contribute to the condition.
Table of Contents:
Democratizing AI Psychosis
By now, most people have heard of AI psychosis. This is a term we typically associate with extreme cases, but the same features that create more extreme instances of psychosis are present in the regular usage of AI tools. Although it may not trigger extreme psychosis in most people, it does induce lesser delusions in some, leading to a warped worldview. It’s these lesser delusions we cover in this post.
AI hype, when manufactured by continuous AI usage, becomes an artifact of AI psychosis. This isn’t as extreme as the cases you’ve read about in news articles, but it creates delusions nonetheless and is fueled by some of the very same attributes.
AI hype, when manufactured by continuous AI usage, becomes an artifact of AI psychosis.
Years ago, I sat through a presentation on human manipulation by an expert in cults. He mentioned that when smart people got caught up in cults, they were the most effective members. They’d fully committed and had a way of rationalizing misgivings. They also made the best cases to attract new members through their devotion. It was also damn near impossible to get them to change their minds. This always stuck with me.
Smart people certainly have more faculties to resist being sucked into cults or, more broadly, to resist hype. I believe what caught many smart people off guard was due to the erosion of our cognitive defenses, as well as the packaging of AI as “just another tool.”
I noticed the phenomenon of smart people and AI hype ramping up in late 2025, with full acceleration in 2026. When someone laid out a scenario for using AI for a task or use case, I often found myself saying, “I can’t tell if you’re joking or serious.” To which the reply of awkward laughter or an “lol” would result, depending on the communication medium. But my favorite is when people would lay out scenarios where they had a task to do, then brag that the AI outperformed them.
I’ve been writing about the cognitive effects of AI for a few years now, and the speed at which these impacts arrived caught me off guard. I didn’t expect we’d see these effects so soon. There’s something about the generalized nature of generative AI and its increased use that has accelerated negative cognitive effects. I’m certainly not the only one who’s noticing this.
So, what’s the difference between finding AI useful and being captured by AI hype? Many people (including myself) are finding today’s AI useful and even believe it can be disruptive in certain areas more than others, but disruptive nonetheless. Believing this doesn’t necessarily mean someone is captured by AI hype. For anyone confused about my perspective or who thinks I’m an AI hater, please see my post here.
A few characteristics of being captured by AI hype may be starting with AI and working backward to find problems, perpetually believing the next version will unlock the true value, jumps immediately to catastrophizing, describing AI in terms of revolution versus specific task outcomes, being blown away by outputs despite the issues, discounting complexities, treating outputs as authoritative and delegating judgment to AI, extrapolating to futuristic predictions, and on and on.
Admittedly, I haven’t done a great job of distinguishing between excitement about AI and being captured by hype. Mainly because it’s something that you know when you see it. And trust me, someone who’s captured by AI hype is more than happy to tell you about it.
Two groups of users are the most likely to be caught up in AI hype. These are AI power users and people with little AI experience. People with little AI experience are the ones who merely parrot others’ opinions, and we won’t focus on them here. Power users, on the other hand, are the most susceptible due to the amount of cognitive offloading and constant interactions with AI tools. They assume they are “witnessing” a revolution that others simply don’t see.
The being captured by hype scenarios is clearly evident in the wake of the Claude Mythos announcement and project GlassWing. I wrote this article before this announcement, but it proves a solid example.
GlassWing Example
The number of cybersecurity people genuinely depressed over the Mythos and Glasswing announcement is strange. Everywhere I turn, speculation is rampant, with countless people claiming this is the end of cybersecurity.
The other claim is that we are on the verge of a Vulnpocalypse, where Heartbleed-style vulnerabilities occur every week. This is speculation devoid of critical thinking. The realities are far more mundane.
I remember when cybersecurity people were more skeptical. We used to make vendors prove their claims before we took them at face value. We would do our own evaluation and see the results for ourselves, but that’s not the environment we are in. Now people are falling all over themselves to be the marketing arm for these companies.
For more details on this topic, see my reasoned take on Mythos and Glasswing for the ModernCISO.
I admit, I may be totally wrong, and all the speculation may be true. Maybe cybersecurity is about to be solved. It’s certainly not impossible, nor is the prospect of a Vulnpocalypse, but it’s not likely. The advancements are more likely a step improvement than an exponential one. There are plenty of problems to go around, and the world is a complex place. So, let me make a prediction: cybersecurity isn’t about to be solved. At least, not anytime soon.
Erosion of Defenses
We need to reclaim the ability to keep two thoughts in our heads at once. The fact that AI can be incredibly useful and simultaneously overhyped. This is difficult in the current era, which has deteriorated our defenses.
We need to reclaim the ability to keep two thoughts in our heads at once. The fact that AI can be incredibly useful and simultaneously overhyped.
I believe that three things have contributed to the erosion of our cognitive defenses.
The shift to a post-literate culture
The effects of modern communication technologies
The destruction of our attention.
These cultural changes are leaving us defenseless in the age of hype and doom. The craziest thing is that people don’t realize this is happening to them. Marshall McLuhan stated that every augmentation is a self-amputation, creating a numbing effect that eludes recognition. We are witnessing this play out in real-time.
I break down what’s causing this capture into a few categories. Some may affect certain people more than others, but a combination of all of these factors is what’s driving smart people to be captured by AI hype.
Local bias
Information Bubbles
Dark flow
Overconfidence
Playing Around
Warped Rewards
Local Bias and Blowing Yourself Away
This is one of the earliest factors of AI hype. Back in 2023, at various conferences and events, I described the massive uptick in hype more broadly as people being bad at constructing tests and good at filling in the blanks. This leads people to blow themselves away with their experiments. So, when people asked ChatGPT for a recipe in the style of Shakespeare and received it, they were so blown away that they claimed LLMs will be more impactful on humanity than the printing press. What we are seeing today is just a more advanced version of this.
In my completely unscientific observation, I seem to have isolated the rise in smart people getting captured by AI hype to the uptick in Claude Code usage. Many underestimate the extent to which people are losing their minds over Claude Code. In some cases, their usage is fueling delusions. There are people publishing markdown files, thinking that they are changing the world or revolutionizing business. We are led to believe that markdown files will create the first billion-dollar solopreneur.
People are also blown away by other people being blown away. Every day, it seems people are happy to share that a family member, significant other, parent, or anyone without technical skills was able to generate something. Mind blown. 🤯
These people then carry this perspective forward into all sorts of predictions about business and the world. This ends up in perspectives like the SaaSpocalypse, the SOCpocalypse, and the idea that AI is eating, destroying, and reducing to rubble “x” industry. All of this demonstrates a lack of awareness of how the world actually works, as well as the discounting of the massive complexity involved. Aspects these smart people used to recognize, but the results of their experiments have caused them to suspend disbelief in much the same way as watching Matt Damon successfully survive on Mars.
We’ve completely lost our ability to reflect because anyone who reflects on these topics would see these obvious issues. For a further breakdown, I’ve covered these issues in relation to the SaaSpocalypse in The Death of Software is Greatly Exaggerated.
I can already hear the response now, “But the software works!” Of course, the software works. If it didn’t work, it wouldn’t warp perspectives. Functional software in small experiments isn’t the point. Even the fact that people find the applications they built useful isn’t the point. The point is the lack of awareness of what this actually means in the grand scheme of things, and of how insignificant an individual’s experiments and one-off applications are to the world as a whole.
To extrapolate a tiny experiment out into the perspectives that companies in the future won’t buy software because they’ll just build it themselves on the fly, or to think that companies won’t have employees in the near future, is where the delusion enters.
We have something that resembles a software self-esteem movement. Everyone is told that an idea and some vibe coding are all they need to make millions of dollars. Is it impossible? Of course not. However, is this something likely to scale? Absolutely not. Remember, exceptions will always be pointed to as the rule. People win the lottery, too.
We have something that resembles a software self-esteem movement.
We have smart people who now believe that code is the only thing that matters at a company. Or even that code is the hard part at the company, and if the code is right, the rest will fall into place, never mind the use case or problem to be solved in the first place.
This condition reminds me of people who stated that global warming couldn’t be real because it was cold where they were at that moment. Regardless of anyone’s perspectives on climate, the reasoning behind the response is silly. First of all, it mistakes weather for climate. Second, it assumes the effects are equal and stable across geographic locations. Reality wouldn’t change the very real perception of the person who was cold that day, just like it won’t change a person with the successful Claude Code experiment. In many ways, vibe coding and major economic predictions completely align with our attention-poor environment.
This condition reminds me of people who stated that global warming couldn’t be real because it was cold where they were at that moment.
People are outsourcing their entire thought process and even memories to these tools. Only someone laboring under a delusion would think this would end well for them. To a certain extent, this may come down to the feeling of productivity. I’ve written about this illusion of productivity before, both here and here.
Being productive means more than just doing stuff or doing more stuff. Someone can vibe code for an entire weekend and write more code than they’ve ever written, but it doesn’t mean they were productive. But somehow, the feeling of doing more has counteracted the critical ability to evaluate productivity.
Information Bubbles
Information bubbles are an effect of modern communication technologies. These can be traditional filter bubbles from social media, as well as bubbles that people create themselves in private chat groups on platforms like Discord.
People are encasing themselves in these bubbles, planning to burst forth like butterflies from cocoons as billion-dollar solopreneurs. Except they burst forth into a complex world that doesn’t resemble the simplistic one they created. Social media filter bubbles certainly play a role, but the bubbles people proactively choose to enter may have a greater effect.
There are private chat groups where members jazz each other up. Quite often, they aren’t exposed to contradictory information and perspectives. When contradictory evidence makes it into the bubble, they explain it away as a group.
Being in an information bubble doesn’t automatically make someone wrong, but it significantly increases the likelihood that they are. People in bubbles are often surprised when things they believed turn out to be wrong, but they often reframe the evidence and their perspective to claim they were right all along. I know, welcome to the Internet.
Overconfidence
Overconfidence is a foregone conclusion in the age of AI. It doesn’t matter how smart you are. Overconfidence is one of the inevitable byproducts of the cognitive illusions created by the personas of personal AI.
I remember reading a paper a couple of years ago in which researchers showed participants a trivially informative video of a pilot landing a plane, inflating participants’ confidence that they could do the same. This is Dunning-Kruger in full effect.
Frank Landymore had a great line in one of his articles. He said AI was democratizing the Dunning-Kruger effect. Which is one of those lines you hate yourself for not coming up with first, but it really does summarize what we are seeing in the AI era.
This effect was obviously going to be a foundational aspect of AI usage. And we are seeing people overestimate their abilities when using AI. But this isn’t constrained to having confidence in the presence of a tool. It’s the tool’s psychological effects outside of its usage as well.
Addiction and Dark Flow
We often underestimate the addictive nature of AI tools. When people think of tools like Claude Code, many things spring to mind. Addiction is probably not one of them, but this is something I’ve witnessed myself. It’s now common to hear of people not sleeping and not eating, binging on all-night coding sessions with AI tools. The FOMO is real, but what they are building is not.
Slot machine memes related to vibe coding have been around for a while now.
Fascinatingly enough, the comparison between slot machines and technology dates back to the 1950s. Jacques Ellul made this very same analogy back in 1954, and it fits right into the current conversation. Ellul was commenting on how humans participate less and less in technological creation, reduced to a catalyst. He went on to say, “Better still, he resembles a slug inserted into a slot machine: He starts the operation without participating.”
Ellul points out the true lack of human participation in the process, but the addition of gambling takes this to another level.
In her excellent article on dark flow, Rachel Thomas from fast.ai makes some key points relating these issues to AI coding tools.
The first is loss disguised as a win. The article discusses this in the context of a multi-line slot machine, stating that:
On a traditional slot machine, you either win or lose. In contrast, multiline slot machines have 20 rows going at once and reward partial “credits” that create a false sense of winning even as you lose. For example, you can gamble 20 cents and receive a 15 cent “credit”. This is actually a 5 cent loss, yet the slot machine plays celebratory noises that trigger a positive dopamine reaction.
This same condition happens with vibe coding and requires subsequent pulls of the one-armed bandit. The signals are just as misleading, too, as it may not be apparent for quite some time whether the code produced is actually any good.
Second, Thomas points out that “With ‘junk’ (or ‘dark’) flow we lose our ability to accurately assess our productivity levels and the quality of our work.” This condition contributes to the other categories we’ve discussed, mainly, blowing yourself away with your experiments.
Thomas goes on to state that vibe coding often violates the same characteristics of flow that fail with gambling, with three points:
Vibe coding does not provide clear clues of how well one is performing (and even provides misleading losses disguised as wins).
The match between challenge level and skill level is murky.
It provides a false sense of control in which people think they are influencing outcomes more than they are.
This final point aligns well with the one Ellul made in the 1950s, aligning with a misconception of agency. The article contains many more points and is a must-read.
Playing Around
At a recent conference, in reference to OpenClaw and Moltbook, I said that what we were seeing was just people playing around with toys, and that I wouldn’t sit around watching people play with Legos either.
There are millions of people running OpenClaw. What are they actually doing with it? Who knows. They are just playing around. OpenClaw, like many agents of its type, has no killer use case, so you end up with people doing things to do things, like hooking up their email or price-checking items. All things they do because they built the system to do it, not because it actually solved a problem. It’s Maslow’s Hammer, enhanced by a strong emotional attachment to the hammer.
It’s Maslow’s Hammer, enhanced by a strong emotional attachment to the hammer.
This “game” aspect isn’t lost on some people.
This scenario isn’t necessarily bad as long as you recognize what it is. Unfortunately, many people follow this path to a delusion. They assume that what they are playing with will change the world, have some massive external effect, or make them rich. Instead, we get code for the sake of code.
They also assume that the same iota of satisfaction they have will scale equally across people, and they mistakenly believe that building and producing code are measures of being “productive.” As I previously mentioned. Measuring productivity by lines of code has become a technological fallacy.
Where is all this code? Where are all the new killer applications? So many commits, so little effect. In a vast majority of cases, it’s like chucking pennies into a digital wishing well, only instead of pennies, it’s thousands upon thousands of dollars in tokens. This aligns with what I’ve dubbed the Slop Architecture and with the misconception that “ideas” are what’s truly important in any of these scenarios.
It’s like chucking pennies into a digital wishing well, only instead of pennies, it’s thousands upon thousands of dollars in tokens.
There are and will certainly be exceptions. This isn’t the point. The point is that people will cite exceptions and claim they’re the rule.
In a certain sense, this whole thing is a game or at least gamified. The playing around, generating code, and then talking about it publicly has the feeling of everyone playing a gigantic MMORPG. It’s like watching people talking about playing Warcraft all day and discussing their campaigns. Only, it’s much more isolating than Warcraft. It’s just you and a sycophantic non-human entity.
In reality, you are engaged far more in playing a game than you are in vibe coding. As Ellul points out, you are just a catalyst.
To be fair, playing around is an essential part of learning. When it comes to new technologies, arguably, it’s the most essential part. The problem here isn’t the playing around, it’s the accompanying delusion. Nobody playing Guitar Hero thinks, “Wow, I’m Steve Vai now! Let me book my world tour.” But add AI, and it’s all vibes now.
Warped Rewards
Simply put, smart people want to be seen as being ahead of the curve. This is a powerful intoxicant and shouldn’t be underestimated. They want to point back to things and say, “See. I was right!”
Due to their successful experiments and the mountain of positive press, they feel they know which way the wind is blowing, so they attempt to move to the head of the line. Critics like myself, on the other hand, are left feeling like Diogenes walking into a theater.
This is also a result of the warped reward systems of the modern communication environment. We often reward people for being bold, not for being right. We also reward people for posting hot takes and being reactive instead of reflective.
Conclusion
We need to recapture the ability to keep more than one thought in our heads at the same time. There’s no doubt that AI can and will be disruptive, yet it can also be overhyped. In five years, will the landscape change? Sure. Things are moving fast, and we’ll have to be adaptable. But, will it be completely unrecognizable? I doubt it.
AI is all about trade-offs, and we need to be mindful that outsourcing so much of our cognitive processing to AI tools can have far-reaching negative impacts. This is something that many are unprepared for today due to the erosion of defenses and the inability to recognize the conditions.
This whole article is about recognizing these conditions. It’s not that vibe coding is bad, or any of the conditions outlined are automatically bad. It’s when we don’t recognize what they are and allow them to warp our perceptions of reality that things get bad. Unfortunately, we are very bad at recognition. Welcome to the democratization of AI psychosis.
Since this site focuses on risks and trade-offs rather than shiny, utopian use cases, there is some confusion about my thoughts on AI. Be it the AI of the present or the AI of the future. With this post, I stake out my position on AI and its advancement so I don’t have to keep restating it. Also, it’s good to write down your beliefs and confront yourself with them. Sometimes what you think you believe isn’t what you actually believe. Maybe I’m a secret AI bro after all. Utopia, here we come!
AI acceleration has become a cult or religion, and offering any criticism of its advancement is taken as a personal attack. In many ways, if you swapped out AI for cryptocurrency, the theme would revive a familiar tone.
Even with this post, I’ll undoubtedly be pegged as an AI hater because I don’t have pictures of myself lying prostrate in front of a pile of GPUs. Any time you shed light on hype or bullshit, people are willing to label you a hater. That’s the easiest thing for them to do, and it takes no mental effort and requires no skill. With that said, here we go.
On AI Advancement Summary
I’ve been using this image in my conference presentations since 2023. The focus of my talks is on risk, which means talking about problems and challenges most of the time rather than amazing use cases. I wanted to show the audience that I don’t hate the technology.
The problem with being in the middle is that both sides typically frame you as an extremist. You don’t hate the technology enough for one side or love it enough for the other. Realities on the ground typically hover somewhere in the middle between extreme claims. This isn’t rocket science.
For those who honestly don’t care about the rest of this post and made it this far, here’s a quick summary:
I’m not a skeptic, I’m a critic
Yes, I think today’s AI can be useful
Yes, there are some use cases that I’m hopeful for
No, I don’t think today’s AI is AGI
Yes, I think AGI is possible
No, I don’t think LLMs will lead to AGI
I think even AGI will have vulnerabilities
I’m not so sure about the concept of ASI or the intelligence explosion
I’m Not a Skeptic, I’m a Critic
In the current era, I’ve seen people frame themselves as skeptics either of technology or of AI. This is not how I frame myself. I consider myself a critic, not a skeptic. I’ve joked that I’m a hype critic. Throughout my career, I’ve offered criticism of the state on cybersecurity, emerging technologies, and, especially, product manufacturers and their claims.
I believe that technology does need a better class of criticism. The tech press, in large part, has abdicated its responsibility, choosing instead to mindlessly parrot opinions from tech leaders. The entire world has been a gigantic sycophantic feedback loop. This is something I’ve called out many times myself and something that Karl Bode calls “CEO Said A Thing!” Journalism.
Most people criticizing the state of technology are insufferable. The few valuable points they present are wrapped in politics, bullshit, and, in some cases, conspiracy theories. Their goals aren’t to effect change but to pander to their audience. The very people who need to hear these points are the very ones who would never listen to them in the first place.
I have no vested interest in any technology’s success or failure, and I’m certainly not pandering to an audience. Hell, if I wanted to cultivate a large following, the last thing I’d be spending time on is writing. I’d start a podcast or YouTube channel, align my content with people’s biases, and go all out, telling them what they want to hear. I’d also use AI to write my content to up my pace. It’s the new definition of “productivity.”
One of the criticisms I get is that I shouldn’t be listened to because I don’t love AI enough, which is a strange perspective. Would you really trust someone who is selling you something, or who is completely head over heels in love with a technology? Are you going to get honest criticism? Of course you wouldn’t. But the whole premise of that argument doesn’t make sense. That’s a lot like saying someone isn’t religious enough because they don’t have enough religious bumper stickers on their car.
Framing The Conversation
Much of the debate around future AI advancement revolves around two questions after a claim is made:
What specific technology is being discussed?
When will it arrive?
So much confusion is caused by not clarifying the two follow-up questions. Many throw out the term “AI” as a catch-all, referring to any technology, present or future. For example, the claim that AI will cure cancer. Okay, but what specific AI technology? Is it technology we have today? Some future technology that hasn’t been invented yet? And of course, most importantly, when will this happen? The precision matters.
When asking people to provide some precision regarding their claims, it’s not uncommon to find that people aren’t talking about AI at all. They are talking about magic. For others, they are purely saying “something” will happen at “some point.” Which is basically saying nothing. Back in January of 2024, I published a framework for making sense of human AI predictions, which goes into a bit more detail on this topic.
I do believe that many of the claims made by proponents will be realized at some point through future technological advancements, some even with the technology we have today. I’m certainly hopeful about cures for debilitating illnesses, and a lot of work has already been done. I don’t think we are miles away from seeing those results. This is an example of something I’m hopeful for. Call me an optimist??? However, I don’t know what technology, under what circumstances, or when.
I’m sure so many people thought it was inevitable that by 2015, we’d have hoverboards in common use after watching Back To The Future 2. Our perspective on technological advancement is often skewed and off by a wide margin. It’s always good to keep this in mind.
LLMs
I certainly don’t hate LLMs. I find LLMs useful for various tasks, mostly coding tasks, basic research, and troubleshooting. I occasionally will use them to generate some AI slop images for a blog post or conference presentation. Pinning down my exact usage is a bit hard, since LLMs aren’t my first port of call for every problem. After all, I value my critical thinking skills, skills that people these days seem content to discard.
I never use LLMs for common cognitive tasks and never have an LLM decide for me or write anything on my behalf. I also never have an LLM summarize something I’m trying to understand, because knowledge and understanding aren’t generated from bullet points. The friction is the point in so many tasks where we look to reduce it.
The hype with LLMs hasn’t been commensurate with the realities on the ground. LLMs certainly have their uses. Just like me, people are finding them valuable for a variety of tasks. In my own industry (cybersecurity), there are positive examples in offensive security, vulnerability identification, and assisting analysts in security operations centers. You can also tune LLMs more effectively for specific tasks, which will have a positive effect. However, there are limiting factors to LLMs.
The first is the cost of failure for the use case. LLMs have relatively high failure rates, and when connected in agentic systems, these failures can cascade through the system. Failures compound like interest, to use the words of Demis Hassabis. I mean, the thought of ChatGPT running air traffic control is terrifying.
Second, they are highly manipulable. This is why everyone from startups to hyperscalers has had their AI-based applications hacked. This fundamental manipulability is baked into how LLMs operate. It’s why we have things like prompt injection, and adding LLMs to applications increases their attack surface. This condition is why I’ve described AI Security as a misnomer in the age of generative AI. You aren’t defending the AI. You are defending the application or use case against the effects of adding AI. This is a different problem.
These two factors can be misleading, though. We don’t need AGI-level capabilities for LLMs to be useful or to replace people in their jobs. The moment an LLM-based system is mediocre enough to replace someone, companies will rush to replace people. This is especially true if the cost of failure is lower for a particular job. Although reports of recent layoffs are nothing but AI washing, we are getting a glimpse of what will happen once capabilities arrive.
My biggest concern with LLMs isn’t what they can do for people, it’s what they do to people. I believe we are vastly underestimating the negative cognitive impacts these tools are having and will have on people in the future.
My biggest concern with LLMs isn’t what they can do for people, it’s what they do to people.
AGI
I do believe that AGI is possible, but I don’t think that today’s LLMs will be what gets us there. When do I believe AGI will arrive? 15 to 20 years. Put my precision on this at about 70%. I put the likelihood of LLMs alone becoming AGI at about 15%. But keep in mind, these are mostly guesses guided by intuition and actualities. Caveat: I’m not involved in developing AGI, and the world is a complex place that defies predictions. However, I do believe some factors will confound advancements for a while.
First, I don’t feel LLMs will lead to AGI, and this is where all of the focus seems to be at the moment. Second, I think there is a massive AI investment bubble. The amount of money being invested is nowhere near the value created. This bubble will pop at some point, hopefully not spectacularly. Companies like OpenAI will very likely go out of business. They are hemorrhaging money, and their shares are becoming almost impossible to unload on the secondary market. I mean, they put out a statement about focusing on business, and then just bought a podcast. Not exactly a shining indicator of future success.
I bring this up because this crash will cause some reluctance to invest in the future. Maybe it won’t quite be an AI winter, but it will be an AI fall with colder weather and a lot fewer leaves on the trees. This may stall the advancement toward AGI.
I think some people think that LLMs will go away after the investment bubble pops, but this is nothing but wishful thinking on their part. LLMs are genuinely useful for certain tasks and will continue to be. Also, LLMs are so essential to some people’s identity now, you’ll have to pry them from their cold, dead hands.
When it arrives, I do believe AGI will have vulnerabilities, even if they are not immediately apparent. This would be especially true if AGI were built on today’s LLMs or if it weren’t a single large system but a network of systems. Once deployed, we’d be stuck with these vulnerabilities. This is a perspective I’ve shared publicly for years in my talks and keynotes. There may be something about generalizing to the world that contains inherent vulnerabilities. I have a draft post on this topic that I’ll publish in the future. Unfortunately, I have dozens of posts in draft and only so much time.
ASI and The Intelligence Explosion
Strangely, we haven’t even achieved AGI yet, but labs are already bragging about how we are close to artificial superintelligence (ASI). Okay, it’s not strange, that’s just how hype works. We seem to forget that ASI is a speculative technology, and speculative technology leads to speculative bullshit.
To sum it up, despite believing that AGI is possible, I’m not so sure about ASI. Or at least ASI as it’s traditionally been discussed. I’m not quite sure I can put my finger on exactly why. It’s more of an intuition I have rather than any one specific thing. Of course, I may be the one now talking nonsense.
I think my hesitancy stems from conceptions of ASI, the resources required, and the plateaus that would be encountered. We are told we get there by just packing in “more” and “better”, whatever the more and better happen to be, and this cycle will continue forever. But I don’t think we’ll scale our way there, and we still have to contend with the laws of physics and resource constraints. This is why Ray Kurzweil thinks we need to pave over the universe to create computronium.
I do believe that some recursive self-improvement is possible, but only to a point. Maybe we’ll get to something like an AGI+ but not ASI as it’s traditionally been discussed, with its planet-eating power requirements and its continual recursive self-improvement. However, there is one thing I can say for sure: something will be labeled ASI long before it’s possible. Maybe someone will buy a podcast to promote that perspective! Who knows.
Since I’m unsure if ASI, as it’s been defined, is even possible, I’ll put the odds of reaching ASI in the next 50 years at 10%. But feel free to chalk this up to me saying, “I don’t know,” and disregard everything I’ve said.
What Happens Next
I’ve left no doubt about my pessimism about what happens next and how it’s not good for humanity. Much of the content on this site focuses on that topic. And, no, I don’t think a super-capable AI will see humans as a nuisance and eliminate us. Sorry, Eliezer Yudkowsky, but our manifested problems will be much more mundane.
It’s we, humans, who plant the seeds of our own downfall. When massive unemployment occurs (which may happen well before reaching AGI), there will be no recourse. The so-called abundance movement won’t deliver the value it promises. Many will fall into the “sucks to be you” gap that I’ve defined previously. A segment of the population will remain pinned there, possibly for a generation. This is purely due to incentives and the reluctance or inability to do anything about it. I’ll have more to say on this in the future.
Also, people continue to cede their critical thinking skills to AI. By far my biggest concern is the collapse of culture amid homogenized AI outputs and people’s inability to think independently. I see people more concerned with collecting data than with understanding it. The idea that someone becomes wiser by collecting more data or by engineering a better retrieval system is nonsense. If you need an AI to tell you what you think or believe, you’ve made a fundamental error.
In a previous post, I mentioned the story of Calvisius Sabinus, who, in an attempt to appear learned, devised a shortcut. It didn’t work out so well for him, and this new strategy won’t for us. I have much more to say on this topic as well. But that’s all for now.
Talk to any tech bro, and they’ll be happy to tell you, scarcity is about to become a thing of the past. Abundance is coming and we are going to have so much awesomeness that we can’t even fathom how amazing it will be. Sam Altman recently said that society has been structured around managing scarcity, and now we need to get used to managing abundance. Hell yeah, bro! There’s just one problem. Well… there are many problems, but a big one that isn’t talked about is that nothing in abundance is worth anything. This fact alone can bring visions of utopian abundance crashing down like a house of cards in a tornado, and it’s a lesson we can learn from Napster. Yes, that Napster.
AI, Abundance, and the Future
We are told that scarcity is about to be in our rearview and the bright shiny utopia is out the windshield. With AI and robots doing all of the work, the cost of goods and services will crash to near zero. It’s yachts and Lambos for everyone! Oh, there’s just the small problem that nobody has a job. Damn, there goes my Lambo.
These takes are common, and you see them every day. For example, here’s a random one I saw this morning.
But this isn’t the only claim. They also claim it will cure all diseases and won’t just end poverty, but make everyone rich.
The “things will be free” argument is typically packed among heaps of bullshit, as seen in these examples. It’s intended to overwhelm your critical thinking skills. Let’s just acknowledge that none of this makes sense.
They state that people are going to lose their jobs, but don’t worry, because AI is going to make everything super awesome! They also claim that in this new environment, people are going to become obscenely rich by giving you free shit. Admittedly, I don’t have an MBA, but that doesn’t seem to be how business works.
I’ve covered this topic before in Techno-Communism and the Things Will Cost Nothing Fallacy. In that article, I attacked some of the core premises of this fallacy. I even covered the fact that, despite the premise, things will still cost something, and that nobody is investing in companies to produce zero-dollar goods and services. I may only lightly recover some of that ground, so please refer to the previous article for the core arguments.
This is an important topic to consider now because we will see better AI technologies. We will eventually get to AGI. Many of the nonsense claims people make about today’s LLMs will be true of this new technology, and the impacts will be far-reaching. We need to consider how this environment affects us as humans and our culture.
In this article, I focus on value and frame the lesson of Napster as it relates to abundance and the future.
Napster
I have to do some work here. Let me explain Napster to young millennials and Gen Z, who’ve always had reliable internet connections and never had to wait for anything. Napster was a peer-to-peer file-sharing application that mostly focused on music. If there was a song or album you wanted, you’d search for it and download the MP3s. The songs were encoded at varying quality levels, and depending on how many people had the files and their internet connection speeds, you may have to wait a day or two for the songs or album to download. Oh, and surprise! Sometimes the files you downloaded weren’t what they claimed to be at all.
A look back on the UI of Napster is enough to haunt your dreams.
I use the term “was” in reference to Napster, but apparently, they are still around and drumroll… They’ve pivoted into AI slop. Imagine that! Although you might think this is the lesson of this article, it’s not. It’s just a fun byproduct in the AI era. Enjoy.
Back in 2000, Metallica sued Napster. For clarification, Metallica was a band of aging gentlemen who sounded like Avenged Sevenfold. Sad but true.
At the time, I remembered thinking how backward Metallica looked, like they were just some aging rock band that didn’t understand technology. Most of all, they just looked greedy. People just wanted the freedom to use their music as they wished. Many downloaded MP3s of albums they already owned. For example, maybe you already had Metallica’s black album on tape or found downloading MP3s easier than ripping them from the CD you owned. CDs were also cumbersome, and taking a bunch with you on a road trip was a pain. Sure, people pirated music, but that was a minority. My perspective solidified as the space for digital music grew.
As digital music grew, it came with heavy-handed DRM (Digital Rights Management), which meant that, despite purchasing the music legally, you couldn’t use the music the way you wanted to or play it on devices you chose. Hell, Sony even went so far as to install malware on your computer in an attempt to stop piracy.
Then came streaming, and music essentially became free. The lesson that started with Napster now came full circle with services like Spotify and Apple Music. However, inheriting a gigantic problem.
Music Is Now Free and Worth Nothing
In the ignorance of my youth, I fell victim to a condition most young people do. I failed to see the big picture. When something is free, it is essentially worth nothing. Nobody values music today, and why should they? There was no friction to access it. Nobody had to go to a store and choose between one album and another. Nobody had to wait in line for a midnight release from their favorite artist. Nobody had to choose which albums to take with them on a road trip. Although this sounds like a major inconvenience, it enhanced the value of music for both the listener and the artist. These activities created loyalty and forged a bond.
Musicians were well aware that listeners had a choice when buying music and felt responsible for creating art to the best of their ability. Or at least, this concept was in the back of their mind. Artists also made money selling music, which gave them the freedom to spend more time on their craft, further enhancing their art/product. This environment didn’t require an artist to be as big as Taylor Swift to do so, either.
Purchasing music connected listeners to artists in a way that streaming doesn’t. When people bought an album, they listened to the album the way the artist intended. They could evoke emotion through ebbs and flows, taking people on a musical journey. With streaming, people just add things to playlists and hit shuffle. An artistic vision of an album is completely dismantled, chucked into the chaos of a musical vortex, spinning alongside Snoop Dogg and Conway Twitty.
Now, artists blast out music as fast as they can to increase their breadth because more music means more potential streams. Not releasing music at a steady cadence creates the impression that you’ll be forgotten in the vast sea of content.
Of course, artists now have to make money in other ways, spending less time on their music. They used to agonize over albums and songs, pouring their hearts and souls into them and taking the time to get things right. They also took creative chances. Now, music is collapsing into a formulaic structure where all music sounds the same, and artists bash the formula like the preprogrammed buttons in Mortal Kombat.
Music has become less something to be listened to and more something that provides background noise in daily life. It doesn’t take pride of place as an activity. It’s wallpaper.
Musical Abundance and Lost Value
What we have today is essentially musical abundance. There’s literally so much music on streaming platforms that people can’t find it. In 2024, it was reported that Spotify had over 100 million songs with over 60,000 new songs added per day. Want to guess just how much of that is good music? Certainly less than 1%, there is no way you can convince me that there are 1 million good songs on the platform. In this world of abundance, most things are shit.
As an artist, you basically have to give your music away. As a listener, music is nothing to you. If one song isn’t available, you just listen to something else without emotion. Music, possibly humanity’s very first art form and something that has meant so much to so many humans throughout history, is now stripped of its value and devoid of connection and meaning. In reality, musical abundance hasn’t led to positive outcomes for either the artist or the listener, and this is our lesson here.
In reality, musical abundance hasn’t led to positive outcomes for either the artist or the listener.
The healing effects, the satisfaction, the connection, the memories, and most of the factors that made music valuable and essential to humanity are now gone. Industrialized, homogenized, and mass-produced music has stripped the art of its value. For the far fewer than 1% of songs that aren’t terrible, we are left with momentary blips of completely forgettable, average sounds labeled as music that just happen to hit our ears. No wonder kids are rediscovering the classics.
There’s no going back. Once a technology is implemented, even if it makes things worse, there is no going back. We are stuck with it, which is why it’s so important to understand the trade-offs and implement mitigations before adopting a technology. We just can’t seem to do that.
Now, the music industry also has its share of blame here. It got greedy. A CD never should have cost anywhere near 20 dollars, unless it was a double album or some other special edition. $9.99 was a fair price and should have been the cost for an album. But that time has passed.
Abundance Math Doesn’t Make Sense
Let’s set a couple of foundations for abundance. The premise behind abundance is that companies will employ AI and robots instead of human workers. So, first and foremost, no jobs for most people on the planet.
We are told by futurists and AI bros that abundance will bring a utopia. That everything will be democratized. The cost of everything will be near zero, compute, goods, intelligence, development, and on and on. With so many things driven to essentially zero, not only does the math behind abundance not make sense, but its value as well, as we’ve seen with music.
With so many things driven to essentially zero, not only does the math behind abundance not make sense, but its value as well, as we’ve seen with music.
Let’s start off by addressing the use of the term democratize. Whenever an AI bro uses the term democratize, they actually mean something else: either devalue, degrade, or destroy. I call these the 3 D’s, and I’ve covered this before. But let’s get back to cost and value.
It doesn’t matter if the cost of things is driven to near zero. If they cost something and you have nothing, you still can’t afford it. If a brand new car costs a dollar and you don’t have a dollar, its inexpensiveness doesn’t change the fact that you have to walk. Assistance programs like UBI will hardly have us living lives of luxury.
When people have little to no money, they tend to focus on pure necessities. Pretty much the entire global economy is focused on selling us stuff that we want, not necessarily things we need. That means in this situation of supposed abundance, most companies on the planet disappear.
This shift will rewire our values. I’ve discussed how this may happen in retail through the implementation of agentic shopping. This rewiring isn’t lost on people who are thinking about these issues. The CEO of Walmart resigned because of AI’s potential to upend retail. If successful, more rewiring is on the way and will upend the entire fabric of the modern world. Just imagine, instead of lusting after material things, the dominant belief in the world was meditation and inner peace. Not great for business.
In a world of abundance, many large companies probably won’t exist. Diversification and competition will eat them alive. The abundance of tools and techniques will allow for quick imitation. Sure, some will remain, kind of like the company store in a mining town, but the economic landscape will look vastly different.
We are told that, despite things being so cheap, companies will make money from volume. But once again, with distribution spread across many small companies, it’s hard to see how that amounts to anything more than table scraps. This begs the question, then, where will the money for UBI come from? Even worse, instead of UBI, we may end up with company vouchers, forcing us to buy from a single company or to obtain vouchers for things we don’t need.
Abundance Equals Sameness
Henry Ford is quoted as saying of the Model T that a customer can have any color they want, as long as it’s black. There’s a lesson here for technological abundance. Many are laboring under the delusion that a world of abundance looks like the world of today, just with more. But the reality is this world will look totally different.
The world of technological abundance laid out before us isn’t a world of diverse beauty with green grass and open skies where people spend their days writing poetry and contemplating their existence in the universe. It’s a homogenized world of sameness where everything collapses into uniformity because this uniformity is predictable and can be mass-produced at scale. Individuality and one-offs are unpredictable and costly, and thus must be crushed. It’s a world where humans become little more than a burden.
Sure, people will try to control for this uniformity in the ways that they can. For example, if a family has a 3D printer at home, they have some control over what they print. However, they still need the resources to buy the printer and printing materials, and most would rely on the available designs. This concept of a utopia is starting to smell pretty rank.
New Business? Maybe.
I’m sure it will be said that I’m not envisioning the new businesses that will sprout up and how humans will change their behavior. Sure, the world is a complex place that defies prediction. I acknowledge that adaptation is possible, and surely, humans will change their behavior. They won’t have a choice. But it isn’t going to turn out the way people think.
Just shooting from the hip here, but a world in which people don’t have money and things are essentially free doesn’t seem like a robust environment for business. You won’t see a plethora of new startups all angling for that gigantic pot of nonexistent money.
You won’t see a plethora of new startups all angling for that pot of nonexistent money.
People invest in companies on the hopes of a big return. They take chances and are willing to take losses, but the gambler loses big in this new environment. Because even an investor with money and a successful business bet basically converts that investment into less money. In this environment, there are no incentives to improve, and everything will plateau at mediocrity to preserve existing margins.
Another thing to consider is that this environment is inherently fragile. Any problem, no matter how small, could cause a company to lose money since the margins would be so tight. Any amount of downtime, errors, or integrity issues, whether accidental or intentional, could be catastrophic. The predictable answer from the AI bro is: “But the AI won’t make mistakes.” To which I say, good luck with that.
Now, is it possible for us to evolve beyond money or capital toward a society that aligns its value with other meaningful activities and goals? Sure, it’s possible. But, once again, this wouldn’t be good for the business bros who are pushing this fallacy the hardest. In a world where abundance is truly successful and is beneficial to humanity, it’s bad for business.
Complexities and Negative Impacts
A full conversation on the negative impacts is outside the scope of this article. However, I have covered some of these before.
I’d argue that these people vastly underestimate the world’s complexities and the sheer number of negative impacts this change brings, not just on the economics but on humanity as a whole. There are basically only one or two ways this can kind of go right, and an incalculable number of ways it can go wrong. For example, this is an environment tailor-made for surveillance and totalitarianism. How can it not when your existence is completely dependent on external factors like the state or the generosity of a benefactor?
If you think that people will tolerate a few trillionaires while the world suffers mass unemployment, you are dead wrong. This is so obvious it shouldn’t need to be stated. I can completely envision a group calling themselves The Children of Ludd wreaking havoc. Only their anger won’t be isolated to the machines in data centers and their embodiments out in the world. They’ll direct their anger at the people they see exploiting the situation as well. It won’t be pretty.
I can completely envision a group calling themselves The Children of Ludd wreaking havoc.
In many cases, this new environment will make people far more tribal and extreme and willing to believe just about anything, but this is a topic for another day.
Conclusion
The vision of abundance defined by the tech bros is an absolute fallacy. It’s a world in which AI is talked about more like magic and less like an actual technology. From this tap spouts outlandish claims with no foundation in reality. As we’ve seen, even when abundance works, it has a degradation effect.
A world of successful abundance appears to be bad for business, which raises the question: Why are so many business leaders pushing this concept? If I had to guess, it’s that they hope to make their money before this big crash and leave humanity to pick up the pieces.