One constant throughout the generative AI craze is summarization. Why read a book, listen to a podcast, or YouTube video… Just summarize it! Large swaths of content, distilled into several bullet points with countless hours saved. However, this isn’t the utopia many claim.
We all love a good shortcut. Humans are wired for them. This is why we are so good at cognitive offloading, but the tradeoffs from shortcuts are never recognized or shoved deep into our subconscious. Every shortcut has tradeoffs. With generative AI, tradeoffs are never acknowledged or discussed. However, here’s an inconvenient truth: knowledge and understanding aren’t generated from bullet points.
Fake Optimization
Many of the claims made by influencers, transhumanists, and the e/acc community revolve around fake optimization. Fake optimization claims that something lowers friction for a task or activity while not providing the same value.
So many things in this world require friction for success, especially knowledge and understanding.
These people see everything as a game of lowering friction, but there’s just one problem. So many things require friction for success, especially knowledge and understanding. To go further, there are many activities where the friction of the activity is the point, such as art or meditation. However, telling people that won’t get clicks and someone’s “thought leader” badge may be revoked. So we end up with the environment we have today, with everyone from tech leaders to influencers telling people friction is about to be a thing of the past.
Take this example of promising people they don’t have to put in the work and still gain the benefit. Anyone claiming you can gain the same value from cramming three hours into three minutes demonstrates a fundamental lack of understanding of how knowledge transfer works and a near-religious level of faith in AI.
If we step back, people listen to content like podcasts for two different reasons: entertainment and information. Quite often, it’s a combination of both. So, by summarizing, we’ve removed all of the entertainment factor, immediately reducing the value of an activity. However, before we get too far, let’s examine a scenario that should be obvious to people.
Imagine summarizing a one-hour stand-up comedy performance. “Just tell me the best jokes.” Is that really an hour saved? Of course not. It won’t be funny, and anyone who thinks differently has been sitting behind a computer screen for too long. We instinctively know that comedy is situational and relies on context and delivery. Comedians like Mitch Hedberg prove this point.
The comedy scenario is obvious for most to understand. However, what’s difficult to understand is that a similar value loss also exists for non-entertainment activities. Summarization isn’t the shortcut people think it is. Without the surrounding context, we may not be committing these summaries to memory, where we can take action on them or put them to use.
Thinking Deeply
There’s no thinking deeply about bullet points or summaries. You can’t. This is because the action of summarizing strips away all of the context. For thinking deeply, the context is key. Summaries are just a set of condensed words shoved into a predetermined space. Important bits of information (sometimes the most important bits) are left out. There’s no way they can’t be.
There’s no connection to bullet points and summaries, no deeper meaning, emotion, or content to chew on mentally. Nobody contemplates something deeper or dreams about something bigger with summaries. The same can’t be said about reading a book or other longer-form content. The inherent dehumanization of summaries drives some of this lack of connection.
In summarization tasks like these, we take someone’s uniqueness, including their perspective, delivery, language, and flair, and crush the life out of it to get the resulting bullet points. This act results in a shift. Instead of viewing someone as a person, we view them as data or a product to be manipulated, and summaries strip humanity away, leaving us with several cold sentences generated from the compactor of a black box.
Make no mistake, the dehumanization aspect is a selling point for many. The human aspect is often seen as flawed, whereas the AI aspect appears superior. But this perspective doesn’t serve us well—you know… we humans—especially when it affects our ability to think deeply.
There can be rare exceptions where a quote or simple statement does cause some deep thought. For example, this quote is often attributed to Einstein, even though he never precisely said these words.
“If you can't explain it simply, you don't understand it well enough.”
A statement like this can trigger deeper thoughts about ourselves and our view of knowledge. As a theoretical, let’s pretend Einstein was on a podcast and uttered this statement, making a larger point about knowledge and understanding. Mediated through an AI system in a summarization task, this statement could be transformed into:
“You need to explain things simply.”
The difference between these two examples is stark, and they do not even remotely mean the same thing. There’s certainly nothing to think more deeply about in the second example.
The ability to think deeply about any topic is a skill we are losing fast and for younger generations, possibly never cultivating in the first place. Our modern world, filled with its distractions, is not only pulverizing our ability to ponder, to wonder, and to dream, but also to question.
The act of questioning requires effort and friction. It isn’t purely asking a question to an AI system and getting a response because the act of questioning isn’t easily satisfied. Don’t let people reframe them as equal. We will not be better off for it.
Context, Value, and Illusion
In reality, longer-form content can be bloated. I’ve read books that should have been four chapters and podcasts that could have been reduced to thirty minutes. However, it’s a mistake to consider context as bloat and an even bigger mistake to assume an LLM knows the difference. This is because you often can’t tell the difference until after the fact. Something that seems like bloat at the beginning is context in the end. That pointless story turns into a connection reinforcing a particular point.
It’s a mistake to consider context as bloat and an even bigger mistake to assume an AI knows the difference.
Let’s consider the importance of context for a moment. Consider something larger, such as a slide deck from a presentation. There are not just several but many bullet points along with images and diagrams. If you are already an expert on the topic, it may be possible (but not always) to glean something from the slide deck. However, the real value is the context in which the content was delivered and the commentary around it. Conversely, if you watched the presentation and had the context, the slides are helpful because they can reinforce the content and even jog your memory. This is true for all sorts of content.
You may be convinced (or not) by a set of built points or summaries, whereas hearing the whole argument would have proved otherwise. In life, we say it’s all about context, but context is what we discard when we summarize.
Also, even for general accuracy, the act of summarization strips away all of the supporting or disproving elements, leaving us with a couple of sentences that may or may not be important. Without the context, how do you know if a point is accurate? You have to blindly trust the system.
One of the most commonly encountered bits of summarization is survey results. Most people never dig into the details of surveys or studies, but this is where you find issues. These are problems with the approach, sample size, sample diversity, and many more pieces of context that may cast a shadow over the results, transforming those groundbreaking results into more questions than answers. Summarizing everything leads to many misunderstandings.
We spend little time evaluating the proposed value from summarization. We are told we can spend far less time yet gain a commensurate level of insight from summaries. This value proposition speaks to our modern low-attention-span world, but if we take a step back and consider the realities, it just doesn’t jibe for the reasons outlined in this article.
Much of this disconnection stems from a lack of presence. We need to exercise a certain amount of presence to read a book or join a meeting. However, this is becoming a lost skill. New technology promises we no longer need to be fully present again, but there are consequences in nearly all contexts. This is why the Illusion of Presence is one of my cognitive illusions created by personal AI personas.
Unfortunately, we do end up fooling ourselves. Using an AI to summarize content for knowledge gives us the illusion that we are working smarter and creating more knowledge with less effort, but as we’ve seen, that’s not the case. The reality is a world of summaries creates a world of fools.
A world of summaries creates a world of fools.
Although harsh, if we consider what we’ve already discussed, it makes sense. Not only are we not gaining the promised value from activities, but we also fool ourselves into believing we do.
AI Mediation
AI mediation is both a bug and a feature. What we want out of content may very well be in the dense center of some data blob. However, something must be said about getting all of our information mediated through an AI system. So much of our world is already mediated by algorithms, and we aren’t exactly better off for it. We are pushed and nudged in various directions, making us more predictable, with all of us shoved toward the dense center of a distribution. But what you don’t find there are uniqueness, creativity, or innovation. Sparks, inspiration, and innovation don’t come from bullet points, although you are certainly being sold on the opinion that it can.
Ultimately, we leave it up to an algorithm to determine the main points, the most important things we should pay attention to. A black box plucking data points with some higher purpose that nobody understands. Many times, the points being distilled may very well be the most important, but certainly not always, and without context, it’s impossible to tell.
Ultimately, we need to ask ourselves a question. How many filters do we want between us and reality? Using AI for mediation is yet another filter on top of reality. We should work to remove filters in places where the activities are important to us.
I’m not trying to overplay the dangers here. You certainly won’t be hurt by occasional summarization tasks with an AI system. However, when used often, there is not only a value mismatch, but it can also warp our understanding of reality. So, there are consequences.
Wasted Time, Not Optimization
The funny thing is we don’t even ask ourselves if the time spent is worth it. Let’s say we cut down on reading time to generate summaries instead. This way, we can cover more ground on more topics. Many may consider this a solid strategy. Subconsciously, this also feels right, which makes it a powerful argument. This is why influencers are so fooled by it. However, when we dig deeper, it’s not the benefit it seems.
So, in the three hours to three minutes optimization sale, you lose time. The three minutes are wasted because you never had the content reinforced with the surrounding context. It becomes bullet points scrawled across a mental billboard as you drive past at 120 mph. Of course, this assumes that the content distilled wasn’t so generic to be a waste in the first place.
Say, for instance, that we use AI to summarize Peter Attia’s book Outlive or possibly one of his podcast appearances. One of the summary bullets may be:
Put a larger emphasis on Zone 2 training.
Okay, but why? What is Zone 2 training? How do I do that? Answers to these questions were covered in the surrounding context, but now you spend extra time tracking down the answers.
Multiple people have already joked that we are on the cusp of someone writing something based on bullet points only to have the other person convert it back to bullet points. There’s something rather dystopian about this.
If something is worth learning, then it’s worth spending time on. This was true in the past and will be true in the future.
Conclusion
There are no shortcuts to creating knowledge. Knowledge generation always takes friction, but through this friction comes reward. When we take shortcuts, we deprive ourselves of the reward, leaving us with a hollow task that doesn’t provide the same value. Ultimately, nobody gets smart from bullet points.
I’m not claiming all summarization tasks are bad. They may be helpful and fine for task-based systems and under certain conditions. But they are not for generating knowledge and understanding. It’s becoming increasingly obvious that we must defend our cognitive functions because nobody else will.
In just a few short years, we’ll not only have achieved AGI but live in a world of abundance. Physical goods will be so cheap that they’ll basically be free, and we’ll be able to 3D print anything we’d like. One can only assume with their free 3D printer. We’ll connect our brains to the cloud and have seemingly endless compute, which will also essentially be free. We’ll have cured our illnesses, created replicants, and become immortal. This is but a taste of the nonsense peddled by Ray Kurzweil.
In his new book, The Singularity is Nearer: When We Merge with AI, he pushes a couple of themes. One is that all technological advancements will be universally positive. Second, the only way humans can ever hope to compete is by fully merging with technology. I have issues with both of these themes. This post attempts to address only a tiny amount of the BS in the book.
Ray Kurzweil
If you are unfamiliar with Ray Kurzweil, he’s someone propped up by many as the preeminent futurist. I recently caught one of his appearances, and his ramblings elicited a noticeable grimace from me. I must admit, I wasn’t familiar with his unique brand of absurdity. I knew he’d said some wacky stuff in the past and had a book about the singularity, but I didn’t pay him much attention. After this interview, I purchased his new book, The Singularity Is Nearer: When We Merge with AI.
The strange thing about Kurzweil is the way people treat him during interviews. I haven’t really seen anyone push him on his asininity. When interviewers attempt to question him, he makes up more nonsense and says things like if people can live 20 more years, they’ll be able to live indefinitely or that things will be free in the future, avoiding the question altogether.
The book demonstrates how disconnected and out of touch Kurzweil is from reality. But it also highlights a bigger problem. As long as tech is allowed to be presented as magic, charlatans and hucksters will run rampant. This is the playbook that Kurzweil exploits. Unfortunately, I don’t have the time to address all of the issues with the book, but I will point out a few things that stood out to me.
As long as tech is allowed to be presented as magic, charlatans and hucksters will run rampant.
Before We Start
A few thoughts before we begin. If you read the reviews of this book, they are overwhelmingly positive. I’m sure many people won’t care for this post. Kurzweil is a famous tech personality with multiple books, TV appearances, and impressive credentials. I’m a nobody security researcher. I’ll never be in demand like Kurzweil or sell as many books as him, so I’ll have to cry myself to sleep at night with my integrity intact.
After all, he was just listed on the Time100/AI list, which caused me to laugh out loud. Then again, we live in a performative age, and Kurzweil is a performer.
However, I’ve spent my entire career analyzing risks and envisioning future threats to technology, something Kurzweil is oblivious to or completely ignores. Neither is a good scenario.
It’s also important to know that Kurzweil has been wrong many, many times before. I stumbled upon this old Newsweek article from 2009, which had an amazing quote.
P. Z. Myers, a biologist at the University of Minnesota, Morris, who has used his blog to poke fun at Kurzweil and other armchair futurists who, according to Myers, rely on junk science and don't understand basic biology. "I am completely baffled by Kurzweil's popularity, and in particular the respect he gets in some circles, since his claims simply do not hold up to even casually critical examination," writes Myers. He says Kurzweil's Singularity theories are closer to a deluded religious movement than they are to science. "It's a New Age spiritualism—that's all it is," Myers says. "Even geeks want to find God somewhere, and Kurzweil provides it for them."
The author even made a midlife crisis joke and another person accused him of trying to start a religion. Fifteen years, and not much has changed.
Let me also say that given enough time and technological progress, just about anything is possible. I think this is something that everyone innately knows. However, people like Kurzweil exploit this instinct for their benefit, running up the clock and leveraging the hype. We should be aware of this trick when evaluating claims.
Why Write This?
You might ask, why would I dedicate time to writing this article out of all the other things I could be writing? Indeed, I’d rather be writing something else, but as I was sketching my thoughts for this post, I read an article with the following quote.
“A colleague of mine, without a hint of irony, claimed that because of AI, high school education would be obsolete within five years, and that by 2029 we would live in an egalitarian paradise, free from menial labor. This prediction, inspired by Ray Kurzweil’s forecast of the “AI Singularity,” suggests a future brimming with utopian promises.”
THIS is why I’m writing it. These predictions powered by Kurzweil are fabricated bullshit. Let me go on record and say we won’t have AGI by 2029 or a utopia. Now, I’m not delusional in thinking that I would have nearly the reach needed to make a dent in Kurzweil’s impact, but I’ll reach a few people and get this off my chest. So, let’s dive in and call it like it is.
Kurzweil is a BS Artist
If I had to summarize The Singularity Is Nearer, I’d say it’s the ramblings of an aging gentleman confronted with his mortality, hoping that wishful thinking and vibes are enough to speed the tech he imagines into existence. It’s a book of absurdity wrapped in historical and disconnected examples attempting to give Kurzweil’s bullshit credibility. Even the title of the book is a sleight of hand. Sure, everything is nearer than when his previous book was written, but that doesn’t mean it’s close.
Another obvious fact on display is that if Kurzweil found himself down at the crossroads, we know exactly what he’d sell his soul for. He wants to become a robot so badly that he’s willing to shed every bit of his humanity to get it. Oddly enough, this doesn’t seem to be the bright, wavy red flag it should be. He’s so scared of death that he’s willing to discontinue being human for a small taste of an extended life.
Why Are People Convinced?
So, if my statements are true, then why is the book so convincing and the reviews universally positive? It’s not because people are stupid, but something far more simple. The book doesn’t continually and all at once slap you in the face with his flatulence. The pungent aroma is layered between positive messages (a utopia, immortality, etc.), topics that have nothing to do with this title, and historical examples of technological progress. This layering is a mental sleight of hand that has a reinforcing effect. Let me give you an example.
Imagine I wrote a book claiming that within ten years, humans would be exploring and populating the cosmos outside of our solar system. Rather than go into the specifics of my claim and address the real risks and challenges, I spend most of the pages talking about other things. I discuss at length the history of NASA and the challenges conquered to put humans on the moon, all in the span of a decade. I talk about the potential of solar sails and other propulsion technologies. I even go off on a tangent imagining the impact on humanity of having a working Dyson sphere. Kurzweil employed the same distraction techniques instead of making points or providing supporting evidence in his book.
The book itself has little to do with its title. He dedicates only a small portion of the text to this topic. He really wants you to know that, based on his vibes, utopia, and immortality are just a few years away. Kurzweil claims we’ll have a utopia in the next 20 years. It’s an easy sell since many people reading his book will still be alive. The entire book consists of telling people what they want to hear. He spends no time talking about challenges or issues. He knows that you sell a lot more books telling people what they want to hear rather than confronting hard truths. This sums up so much of our current age.
He knows that you sell a lot more books telling people what they want to hear rather than confronting hard truths.
That said, the book is an informative glimpse into the mindset of a certain type of person. These would be people with the transhumanist, posthumanist, or e/acc mindset. So much of the transhumanist argument is framed around making us better humans, but it’s really about making us into machines. I’m sure Kurzweil believes he describes a utopia. But like so many utopias, it’s just a thin layer of cheap wallpaper over a dystopia.
So much of the transhumanist argument is framed around making us better humans, but it’s really about making us into machines.
Disconnected From Reality
Kurzweil gives some of the most absurd examples in this book, proving that he has no idea how the world works and is disconnected from reality altogether. For example, after connecting our brains to the cloud, he imagines entertainment where we don’t merely watch a movie but feel the actor’s complex and disorganized emotions. Uhm… Can someone please tell him that actors are… well… acting? He doesn’t seem to realize that people acting in movies are expressing emotions, not feeling emotions. When we insist a tortured character in a movie needs to actually be tortured for entertainment, whose utopia are we living in?
Virtual Experiences
The book has an obsession with virtual experiences. He imagines scenarios such as a virtual beach vacation for your family, where you have the sights and smells of an actual beach. Nothing like taking a vacation with your family while, in reality, not taking a vacation. It reminds me of the company Rekal from the Philip K. Dick short story We Can Remember It For You Wholesale, which became the movie Total Recall for those who never read the story. I don’t know what it is with these people who seem to look at a dystopian SciFi and say, “Yes, that’s the technology we need.” These are cheap illusions that don’t have the impact of the real thing, but not to Kurzweil.
He claims simulations will be so good that there will be no point in doing the real thing and uses the example of climbing Mount Everest, which demonstrates he doesn’t understand what stakes are or what the point of doing something challenging is in the first place. In many cases, the point of performing the activity is the friction and difficulty. We just had the Olympics. Imagine telling Simone Biles, “Why put all of that hard work into competition? Soon, you’ll be able to ‘experience’ Olympic competition.” What Kurzweil doesn’t understand is that when experiences become easy, they lose their value.
When experiences become easy, they lose their value.
Confronted With His Aging
In many ways Kurzweil is keenly aware of his aging. This is obvious in his obsession with simple technology like replicants, which are merely trained on your writings. He discusses the replicant he made of his deceased father and rambles on about how fooled he was by his creation. This experiment was supposed to demonstrate the impressive capabilities of today’s technology, but it ended up just being sad.
However, what’s the point of having a chatbot trained on your writings and other material that exists after you pass away? It’s not you. It doesn’t have your identity or your true thoughts, nor does it encapsulate the complexities that make up your true identity. Even if you could create a more exact replicant, what’s the point? It still won’t be you. It could be a perfect copy of you, but it isn’t you. This is the kind of thing a narcissist would want. I don’t want a copy of myself running around, and I’m sure the world thanks me for that.
When you think more deeply about them, replicants have another problem. They are a get-out-of-jail-free card for not doing the right thing. Why spend time with your loved ones when they are alive if you can create a cheap copy to chat with at your convenience after they are gone? More time doing what you want and less time spending with the ones you love.
Things Will Cost Nothing
Not only will things be better in the future, goods will basically cost nothing. Kurzweil says that everything will become information technology, and the cost will go to zero or nearly zero, even basic necessities like food and clothing. He uses this transformation to say people won’t fight over resources anymore and uses a silly example, such as people fighting over a PDF. The whole premise is absurd. Vertical farming won’t drive food costs to zero, and people will fight over information. People get into fights over social media posts all the time.
Speaking of social media, he makes more ridiculous claims about social media and the cost/value tradeoffs. For example, he says it costs companies like Facebook, Google, and TikTok nothing after they’ve built their infrastructure, suspiciously omitting the energy costs and maintenance to run the infrastructure and the veritable army of people these organizations employ. He justifies his claim by stating that there’s no difference in cost between connecting you to a hundred people or a thousand people, as though the connection between people is where the cost is, but that’s not the stupidest part.
He says that if you could make $20 mowing a lawn but choose to spend that time on TikTok instead, then TikTok is worth $20 to you. This is asinine. Not every action you take in life is in service to make money, and not every free moment is a lost opportunity, either. So, in Kurzweil’s logic, if you could make $5 on Fiverr by designing a logo for someone but decide to sleep instead, then sleep is worth $5. You could make that $5 the next morning with no money lost. None of this even considers algorithms, the addictive nature of social media, and humans just wasting time.
Another spit-take moment is his discussion of radical life extension technology, which he states will not be available solely to the wealthy but also to the less fortunate worldwide. To prove this point, he uses the mobile phone as an example. Nope, you read that right.
Kurzweil says that since most people on the planet have a mobile phone, radical life extension technology will be available to them in much the same way due to extremely low cost. However, I think the mobile phone analogy is worth a deeper look. There’s a big difference between the iPhone in my pocket and an adware-riddled cheap cell phone subsidized by some company squeezing every drop of data from a user that it can. Tack onto this subsidized connectivity like Facebook’s Free Basics program meant to provide free internet to users in developing countries, which ultimately traps them in a Facebook hellscape, and you have the blueprint for something fairly dystopian.
Continuing his cost-nothing crusade, Kurzweil states that using robotics, cheap energy, and automation to replace labor outright in the 2030s would make it relatively inexpensive to live at a level considered luxurious. Telling people things will be cheaper, but you won’t be able to afford them because you don’t have a job is a contradiction that apparently didn’t dawn on him when he wrote that passage.
And… I’m not even going to get into his Bitcoin comments.
Jobs and Wages
Kurzweil has odd claims about jobs and wages. For example, he claims that more jobs will be created than lost, but he can’t answer what those jobs will be because they haven’t been invented yet. He uses examples like farming and the textile industry to prove his point. But this doesn’t make sense since AI is a far more generalized technology than a tractor or the power loom and can cross many different industries.
On wage stagnation, he boasts about how stagnated wages can buy more compute. Imagine that conversation with your family when having to skip a meal because you can’t afford food. “I know you are hungry, kids, but just think about how much more compute we have!”
I know you are hungry, kids, but just think about how much more compute we have!
One of Kurzweil’s favorite scare tactics is claiming there won’t be jobs for unenhanced humans and stating that until we fully merge with AI, there will be almost no jobs left. He makes multiple claims throughout the book on this point, saying biological brains cannot keep up with non-biological precision nanoengineering. Whatever the f—k that word salad means. This is another one of Kurzweil’s tactics on display. He knows most people know nothing about nanoengineering, so he bloviates on the topic. For good measure, he also mentions a world where we watch political ads or share personal data to get free nano-manufactured products. Ah, yes. The utopia we were all hoping for.
When it comes to automation replacing and disrupting the job market, he brings up a silver lining. The gig economy. He mentions the gig economy offers people more flexibility, autonomy, and leisure time. Kurzweil is so out of touch he doesn’t realize these aren’t the same thing. Once again, imagine that conversation. Telling someone who delivers for DoorDash, “Sure, you don’t have a regular job that pays well enough or has benefits, but isn’t all that leisure time great?” When you can’t pay your bills, downtime isn’t leisure time.
When you can’t pay your bills, downtime isn’t leisure time.
Being Human
In one part of the book, he questions what being human even means when introducing non-biological components and brain-computer interfaces. This is actually a great question, which, of course, Kurzweil doesn’t answer. Instead of answering, he vomits more of his pontification about inevitability, saying the non-biological component will grow exponentially while our biological intelligence will stay the same, providing a more specific prediction that in the 2030’s our thinking itself will be largely non-biological. Kurzweil has a way of stating questions as though he’ll answer them but never answering them. This is how a con artist operates, appearing to be upfront.
It should be obvious to anyone reading the book that Kurzweil really doesn’t like being human and yearns for the day to transform into something else. It doesn’t even matter to him what he becomes as long as it isn’t human.
For example, it’s uncomfortable (but necessary) to think about how replacing our biological components with synthetic ones may change us, especially when it’s not for the better. Instead of addressing this complicated reality, he makes the point that we remain the same person despite our cells going through a replacement process and our brains being almost completely replaced over the span of a few months. The implication he hopes you draw is that this non-biological replacement shouldn’t bother us. Once again, more absurdity.
Bodily regenerative processes are not the same as a wholesale replacement by synthetic alternatives. This holds true for both physical and cognitive functions. This irritates me to no end, and it’s one of the most obvious flaws in his logic. Kurzweil hopes to smother us with a pillow while he whispers, “Just let the singularity happen.”
No Downsides
One of the most apparent aspects to readers of the book is Kurzweil’s failure to mention nearly any negative aspects or potential adverse outcomes in his book. Either he’s oblivious to them or feels that adverse outcomes don’t align with his message. My guess is it’s a mixture of both.
I’ve discussed many of these downsides already, but one in particular is his presentation of simulation and self-driving cars as though they’re magic. To support this, he mentions the success of companies like Waymo. There is never a mention of Waymo’s issues, such as how these cars have been found driving down the wrong side of the road or mysteriously honking their horns. We don’t have capable Level 5 self-driving cars on the road today, and this problem is not solved. Every company working on self-driving features, from Waymo to Tesla, has issues they cannot solve today.
These are undoubtedly solvable issues, and we will have full driverless technology in the future, possibly even in the near future, but today, these companies can’t solve the problems. It’s undoubtedly disingenuous to talk about driverless cars as though they are a solved problem today.
Okay Not Knowing
Another of Kurzweil’s comfortabilities is agreeableness to not knowing how AI works or comes to its conclusions. He mentions that we may not know or understand even if explanations were provided. It’s odd that he mentions this while talking about the judicial system, an area that’s been plagued with algorithmic issues. Even outside of the judicial system and policing, there have been so many instances where algorithms have unfairly discriminated against people, denying them benefits and even entry into schools. Recently, it was announced that Nevada will use Google’s AI to determine whether people get benefits. People have a right to know why they were denied benefits, and it can’t be, “because the algorithm says so.”
Imagine an air traffic control AI that instructs pilots to fly figure 8’s around the airport before landing. Will we question this or receive it as some sort of hidden knowledge that the AI system has that we can’t fathom? This would be an obvious example that the system has an issue, but countless hidden issues wouldn’t surface in the same way. When we don’t understand how a system came to its conclusions, we set ourselves up for confounders to run rampant.
As I read the section on the judicial system, I wondered how you would ever get a fair trial by jury in the future. When everyone is permanently connected and has access to data that biases them, it may be possible for anyone to get away with a crime purely by spending enough money to taint the data. Or will you be forced to install the JuryBlocker software directly into your cognitive processes? I’m sure Kurzweil would think this thought exercise is silly because the goal is to remove humans from the judicial process altogether, but as we know, we don’t live in a perfect world. Our technology is rarely that good, and humans have a habit of not making the right decisions.
Not The Whole Story
There were so many parts of the book where Kurzweil would bring something up, and I’d be left with the thought, “That’s not the whole story.”
For example, he references things like ChatGPT passing the Bar Exam or AlphaGo beating the best Go player in the world but never tells the whole story. For example, when ChatGPT passed the Bar exam, it also passed other similar standardized tests. Researchers reworded the questions to ask the same question differently, and ChatGPT failed, proving that it had memorized data in its training data. Kurzweil wants you to believe that because of this, a lawyer’s days as a profession are numbered, but his exercise misses the more significant point that lawyers don’t sit around answering Bar exam questions all day.
AlphaGo was a truly amazing accomplishment, but Kurzweil leaves out that even average Go players can beat superhuman Go AIs these days. They can exploit these systems through adversarial policy attacks. These attacks are highly concerning if the technology is deployed in high-risk scenarios outside the game of Go.
In his discussion about disruption from AI, he claims that sometimes there aren’t any losers. For example, a revenue stream from treating a particular illness. He says there are many areas of technological change where losers don’t exist and gives the example of creating a cure for a disease. In this scenario, companies and individuals lose a long-term stream of revenue. This is another one of those sleight-of-hand things Kurzweil does. The cure is indeed more beneficial to society, but that’s certainly not how things play out in practice as large pharmaceutical companies hang on to revenue streams. No matter how cloud-connected your brain is, you won’t be able to compete with large organizations and a mobilized workforce. There may be occasional exceptions, but it’s hardly the rule.
Conclusion
This lengthy post didn’t scratch the surface of the nonsense hawked by Ray Kurzweil in his book. There are so many points I take issue with, most specifically the arguments of inevitability and the aggressive timelines he’s attached. Given all of his bullshit, you might be surprised that prominent people continue to hold him in high regard, but I’m not. Kurzweil’s tech spirituality aligns with their larger goals.
We need to ask more serious questions of people trying to sell us things, even those selling us ideas, because these things have consequences.
Look For Yourself
Before writing this article, I didn’t look for other articles or takedowns of Ray Kurzweil. I didn’t want these pieces to taint my impressions of the claims made in the book. The only exception was the Newsweek article from 2009, which I stumbled upon while looking up a specific piece of information about him. After writing the article, I was curious about what others had to say, and I promised myself I wouldn’t go back and reword anything in this article based on what I had read.
If you are still unconvinced, I’ve highlighted a few articles you can read for yourself below. Some of these are older articles, proving that nothing has changed. These are worth the read.
How Ray Kurzweil Sells His Junk Science – Geoffrey James – June 17, 2010 This is an old article, but it’s worth the read for the rules of selling junk science.
Ray Kurzweil Does Not Understand the Brain – PZ Myers – August 17, 2010 This is PZ Myers smashing Kurzweil for making the claim that by 2020, we’ll have reversed engineered the human brain. Obviously, that didn’t happen.
The singularity is not near: The intellectual fraud of the “Singularitarians” – Corey Pein – May 13, 2018. This article has an amazing quote. “Science begins with doubt. Everything else is sales.” This is something we should all keep in mind as we blindly take as fact the drivel of the AI Hype Bros. For some more notable bangers by Corey Pein, check out Cyborg Soothsayers of the High-Tech Hogwash Emporia. “Ray Kurzweil’s Singularity is an overheated white paper by a zealot for the American dream of luxury and convenience.” There were a whole lot of references to this type of thing in the book.
Anyone digging even mildly beneath the surface will see that Ray Kurzweil is a charlatan and a huckster. He’s not someone to be taken seriously. Despite this, many tech people will continue to genuflect for Kurzweil because he says what they want to hear. I also mentioned in a previous post that in our short attention span existence, we reward people for being bold, not for being accurate. Something that Kurzweil happily exploits. Welcome to the age of post-reality.
You might wonder why AI companies are working on seemingly simple and unimportant advancements in AI when there are much more significant problems to solve. Why would companies trying to create AGI get sidetracked by focusing on potentially already-solved problems? A couple of examples are OpenAI’s voice cloning, Google’s VLOGGER, and Microsoft’s VASA-1.This research, for many, only seems to have use cases for fakes and frauds, but I believe this work signals something much deeper: that we could be near the peak of LLM capabilities. With AGI off the table, it is time to go deep and get very personal.
Peak LLM
Although you can do some cool things with LLMs, and we’ll no doubt see further applicability in other use cases, it’s a far cry from their touted value. You know what I’m talking about, the more impactful than the printing press crowd that still seems to swarm every conversation on the topic. These people talk about 10x, 100x, and even 1000x productivity boosts with LLMs. Compared to bold AGI claims and nonsense productivity levels, a 10% efficiency gain seems inconsequential.
The Wall Street Journal reported that the AI industry spent $50 billion on the Nvidia chips used to train advanced AI models last year but brought in only $3 billion in revenue. Ouch! There is reporting on the dismal outlook for generative AI, and some foresee a new Dotcom crash.
People have become more skeptical of claims (as they should), and it seems that many more people are noticing. You can’t believe the demos you see. Many are highly controlled or manufactured altogether. Even the SORA demo that everyone lost their minds over wasn’t what it purported to be.
LLMs are under-delivering on their overhyped promises.
I don’t know what to think about the economic angle. It’s not my area of expertise. I just now know that LLMs are under-delivering on their overhyped promises. Where leads economically, I don’t know.
Many LLMs, including open-source models like Llama 3, are catching up to GPT-4. Even if they don’t have the exact level of performance, they are close, which should tell us something. We may be hitting peak LLM capabilities. This means GPT-5 won’t be AGI or exponentially better than GPT-4. GPT-5 may be better than GPT-4 in some ways, but it is far from a groundbreaking explosion of capabilities.
This lack of performance isn’t going unnoticed at the companies building the technology either. This is why a new approach is needed by companies looking to monetize AI investments further. There’s about to be a shift away from a focus on AGI (although they’ll still talk about it) and ever more capable models to you. That’s right, you.
You’re Next
Just because we may be hitting peak LLM capabilities doesn’t mean things will stop. When you’ve reached the limit of going wide (general), you go deep (personal). This will be a sleight of hand shifting from purely training larger models on more data, creating more capabilities in a broad sense, to deeper, more personal integration.
These companies will make it all about you, not because you are the most important aspect, but because you are where the data is. With systems that are closer to you and more integrated with your data and activities, these companies are hoping to make the products more sticky, with the beneficial exhaust of having access to all your data.
The hope is that an epiphany will sprout from your screen as you find the same tools you previously could take or leave now indispensable. Or maybe even fool yourself with the tech, as the public launch of ChatGPT showed. ChatGPT became a social contagion not because people found it so indispensable but because we are bad at constructing tests and good at filling in the blanks.
But don’t take my word for it. Sam Altman has already started pivoting in this direction. Here’s what he says about the goal of AI: “A super-competent colleague that knows absolutely everything about my whole life, every email, every conversation I’ve ever had, but doesn’t feel like an extension.” That’s pretty creepy. But there’s more.
You can make the tech more sticky by allowing people to personalize and customize in more advanced ways. Technology like voice cloning and animating faces supports this customization aspect. When you can choose whoever you want to be your assistant’s AI avatar, you can anthropomorphize it more. How would you feel if a random stranger used your face and voice as their personal assistant? What about a family member? Is this creepier still? Oddly enough, it serves no purpose for the individual user. It doesn’t make the tool any smarter or more capable. It only exists to manipulate us or allow us to manipulate ourselves.
In the end, you’ll be blamed for LLMs’ lack of success by not allowing them to plunge deeply enough into your life. There’s a saying that if you don’t pay for something, then you are the product. Well, in the age of generative AI, you can pay for something and still be the product. The future’s so bright 😎
Even Deeper
AI companies are doing their best to make this technology unavoidable. We are getting AI whether we want it or not. It’s being baked into the very foundations of our computing systems, and even your humble mouse hasn’t escaped this integration.
How you deactivate these integrations will be anyone’s guess, as the flood of new integrations infects every application imaginable. A security check will be due soon, but security issues aren’t the only problem. As I’ve said, we are creating a brave new world of degraded performance. In an attempt to make hard things easier, we may make easy things hard.
Applications of narrow AI are cool and can be incredibly useful for certain tasks, but does it warrant hooking everything up to LLMs and hoping for the best? I don’t think so, and this approach is fairly misguided, opening us up to unnecessary risks.
Conclusion
We must be much more selective before blindly accepting deep data access and personal integration for these tools. This can start with a few relatively simple questions. What do we hope to gain from this access? How will this provide a measurable benefit? And, most importantly, are the trade-offs worth it? The answers to these questions will be different for everyone.
In many cases, it appears that for the small price of your soul, you can appear and sometimes feel marginally better in some aspects but be measurably worse in others. Does that sound like a good trade?
Something that may have gone unnoticed in recent months is that there has been a brewing backlash against physical AI-powered devices. The most recent example of this was the crowd attacking the Waymo vehicle in San Francisco, but this is far from the only example.
I started noticing this trend with AI-powered food delivery vehicles. This was a bit odd since they seemed to be pretty good at failing themselves, often found tipped over, going the wrong direction, and even, in one case, committing a hit-and-run.
Although many would write this cheering crowd off, envisioning Enoch in the hands of the destroyers, this would be a mistake. Those of us working in tech can roll our eyes and dismiss outlandish reporting and other nonsense but think about the countless people who have a steady stream of this clogging their newsfeeds. There is an underlying mood to these activities that cuts much deeper than tech hate or the realities of technology innovations from the past. In this post, we explore what’s bubbling beneath the surface.
Quick Update
I haven’t had much time to write content lately. This isn’t for lack of material since there’s been a firehose of things to write about. I’ve been working on something much, much longer than blog posts, so that’s consumed quite a bit of my time. Also, you can check out my post on something I call the AI Solutions Risk Gap on the Modern CISO blog. Here, I break down what really matters with AI Risk and give leaders some topics for consideration.
Why The Backlash?
I believe what we see here is the beginning of something bubbling to the surface. This is the inevitable outcome when hype meets uncertainty. AI hype is putting all of humanity on notice, and humanity notices.
AI hype is putting all of humanity on notice, and humanity notices.
So, you may wonder why attack self-driving cars or delivery robots. It’s because these are the physical manifestations of AI in the real world. After all, it’s kind of hard to punch ChatGPT in the face. These devices represent symbols of a future that doesn’t need humans at all, erasing humanity from the equation. It’s a mistake to attribute this to Luddism or tech hate.
First of all, anyone invoking the Luddites should read Brian Merchant’s book Blood in the Machine. Second, the Luddites’ concerns about tech affected their industry and the social and political environment surrounding it. It’s an entirely different scenario when discussing a large swath of humanity in multiple industry verticals and positions. Unfortunately, this uncertainty is primarily driven by exaggerated news reports and speculation.
Examples like this gem from Yahoo Finance that shows few cuts attributed to AI but speculates that people are lying despite direct quotes to the contrary. There are also framing issues and more to critique in the article, but most people won’t notice any of this. All they’ll see is, once again, AI is coming for your job.
People have a sense that the technology isn’t as good as it claims to be and yet continually see reporting to the contrary. They also see the launch of what I refer to as shitty AI gadgets, like the Humane Pin and the Rabbit, two devices that apparently investors and the media love, but the rest of the world, not so much.
AI is now being shoved down our throats in absolutely everything, whether we want it or not. Even Mozilla is scaling back to focus on adding AI to beloved Firefox, despite the fact that absolutely no Firefox user actually wants it. Microsoft is cramming it into every corner of the Windows operating system. Deep AI integration is bad for both security and privacy and despite this being known, the push continues. Nothing is sacred anymore.
To pile on, people are being told their jobs are in danger, which they are, but not from super capable AI, but from overzealous business leaders who hope that the tech catches up faster than they’ll have to backfill positions or rehire the people they let go. This is despite underwhelming performance when they do demo products or launch experiments.
There is no doubt that, in general, AI technologies will continue to make progress, solve problems, and become more capable. We will even get to AGI. But in the short term, we are being sold a bill of goods before these companies even get the technology working, much less working effectively. It’s like thinking you are buying a Ferrari, but when you take delivery, it’s a wooden go-cart with wet paint and the word Ferrari on the back.
It’s like thinking you are buying a Ferrari, but when you take delivery, it’s a wooden go-cart with wet paint and the word Ferrari on the back.
You even have people like Sam Altman telling people ChatGPT will evolve in uncomfortable ways, wanting to push this technology further into your personal life with far more access to your data. No wonder people protested outside the OpenAI’s office. Give us more of your data so we can replace—I mean, help you. The reality may not be so cut and dry, but that’s what’s in people’s heads, and they don’t like it.
Tech companies hope to employ their standard brute force playbook and steamroll through the problems, but I think it’s far more challenging this time. The AI field, in general, will bring us a lot of advancements. LLMs are undoubtedly useful for some tasks but remain overhyped and won’t get us to AGI. LLMs are the Diet Coke of AGI. Just one calorie is not nearly enough.
Human Manipulation May Win The Day
If all this wasn’t depressing enough, if there’s one thing we know for certain, it’s that humans are easily manipulated. We can reliably reproduce these results. Companies will start to employ more manipulation techniques to avoid larger issues and ease adoption.
These can be subtle and often go unnoticed. For example, have you noticed how the responses are displayed while using ChatGPT? They are completed across the screen as though someone else is typing back to you. This makes it feel more human.
I remember reading an article years ago about home assistant robots that were in development and how people didn’t like them. Then, the developers projected simple facial expressions on the robot’s face, and people warmed up to them. They were the same product that now had a simple face, with no further capabilities added.
To take this further, look at the image I used as the featured image for this post. It may make you feel sorry for the robot despite being imaginary and completely manufactured. The robot never existed, the human never existed, and the scenario didn’t exist. Yet, we still can’t help feeling sorry for the poor robot despite the fact it may have been a homicidal, mass-murdering robot whose sole purpose was to kill as many people as possible.
So, if we apply subtle manipulation to the current situation, imagine the delivery robot having a statement printed on it that says, “If you see me in trouble, please help me.” This is a statement from a piece of technology asking you, the human, for help. Since most people help when asked, people may be likely to stand up a tipped-over device and less likely to kick or destroy a device requesting help.
Or a wilder scenario, projecting a frowny face on the windows when a car is attacked and a voice that says, “Stop, you’re hurting me.” These techniques may reduce the number of incidents by manipulating the humans coming in contact with the technology through techno-social engineering.
Our world is already filled with priming, subtle manipulations, and nudges
Our world is already filled with priming, subtle manipulations, and nudges. Companies building this technology won’t find ways to make the situation more equitable for humans; honestly, that’s not their job. However, they will find ways to manipulate us into believing it’s in our best interest, ease adoption, and minimize backlash. Anthropomorphism and other human manipulation techniques will be employed to serve the company’s goals. On the other hand, this is something we should all be concerned about.
One example of manipulation is this article. Notice the mental tricks Sam Altman employs. By claiming AI is dangerous in this way, he’s creating a humble brag about its incredible capability. Claiming to want regulation makes him appear reasonable and concerned. He gets to play the hero and the victim at the same time. It’s a lot less genuine when you realize this is a push toward regulatory capture. I’m sure there’s an Onion article in here, somewhere like, Man Creating AI Says It’s Dangerous And Wishes There Was A Way to Stop Himself.
Business Leaders
Business leaders play a critical role here and need to be more critical of claimed advances in the AI space. When putting pressure on internal developers, they need to understand that the biggest companies in the world are struggling with operationalizing generative AI. So it’s reasonable to assume you’ll have challenges as well.
Business leaders also need to be far more critical of vendor AI claims. Keep in mind that demos are staged and offer known variables for vendors to present during sales meetings. These situations don’t match your organization and the unique data and challenges you’ll encounter. When evaluating a demo, ensure that it’s evaluated on your data with problems that you encounter. Also, ask the vendor about challenges you’ll have, as well as things that their tooling doesn’t do well. If you don’t get good answers, run as fast as you can in the opposite direction.
Common things I hear are, “Why would make up stuff about their products?” This is typically when I spit my drink out. Dig in and verify claims. Just because a product may work in one environment doesn’t mean it will work in yours.
Conclusion
Although we all love to rage against the machine, the problem is that we are all a part of it. In the near future, we’ll start to see more applications of techno-social engineering. We also need to be far more critical of the news stories we consume. There’s a deluge of junk research and sensational news stories out there. Staying level-headed and asking the right questions can help keep you grounded on where the realities are.
Update
I wrote this article before seeing Brian Merchant’s article. You can read that here: https://www.bloodinthemachine.com/p/torching-the-google-car-why-the-growing He digs a bit deeper into the self-driving vehicle aspect, so we have a similar theme but a different focus. It’s well worth the read. Also, I learned from that article that people were destroying e-scooters as well.
I wanted to take a moment to address an obvious issue. In my last post, I discussed making sense of AI predictions and gave a framework to help, but how do you make sense of the absurd? After writing that post, I’ve seen imaginings of not a 10x productivity boost from AI but 100x and even 1000x. Not with some futuristic, cutting-edge technology yet to be developed, but with what we have today. Now is a good time to remind these people that the “x” after the number means times, although it seems to have morphed into a generic representation of “better.”
There is truth to their statements when they claim humanity isn’t prepared for 100x or 1000x productivity increases. This is true. We also aren’t prepared for a stampede of unicorns frying us with their laser beam eyes. A 100x or 1000x productivity increase is such a large number that it makes us mentally check out, insert the word “better,” and open the door for pointless pontification. Not to mention, this level of performance increase would be pointless in almost every context.
The Reality
Let’s think about this realistically, with some numbers. A widget factory produces a 1000 widgets per day. It then uses some magical AI dust to gain a 100x output boost, that’s 100,000 widgets a day. Now, at 1000x, that’s 1,000,000 widgets per day. In one year at full capacity, they’ll have 365,000,000 widgets vs. the previous output of 365,000. I hope they have a whole lot of storage.
Now, think of the developer who previously wrote 500 lines of code per day and now outputs half a million lines of code daily. Or, take the case of a prolific author who writes a book per year. Now, they are writing a thousand books per year. What should be clear by now is not only the absurdity of all this but also the larger problem: where are your 1000x new customers?
Output on this scale is meaningless without a massive increase in consumption. A 1000x productivity boost is pointless and possibly burdensome without a 1000x increase in consumption, aka customers and demand. Even with a scale-back in production, it’s still burdensome at this rate. I mean, unless you only plan on firing up the factory for a single day a year.
These things remind me of the Philip K. Dick short story Autofac, where automated factories keep replicating themselves and producing goods that nobody wants. Maybe the milk really is pizzled.
Y, Tho?
So, why are people claiming that 100x or 1000x productivity boosts could be on the horizon? Well, I can only assume because those numbers are bigger than 10x 🤷♂️ Even though 10x is already a massive productivity increase, bigger claims, bigger hype. People aren’t going to read your blog posts anymore if you are still 10x’ing. In the hype game, it’s go big or go home.
There is something about spelling out claims like this that makes the obvious flaws shine through. Putting numbers to this demonstrates the absurdity of the claims and should highlight a significant flaw in the logic. Of course, maybe there’s another likely scenario. People using ChatGPT to write their blog posts aren’t confronted with reality since they aren’t actually reasoning through the content they are churning out. They are too busy 10x’ing to realize flaws in their statements. Oh well, that’s a blog post for a different day.
People using ChatGPT to write their blog posts aren’t confronted with reality since they aren’t actually reasoning through the content they are churning out.
Real World Performance Boosts are Happening
There are certainly situations in which organizations can and even are gaining productivity increases using todays AI technology. These may even be doubling, tripling, or, in some rare cases, approaching 10x productivity increases, but these are highly specific, situational, and typically related to tasks and not the entire chain. Many of these areas are creative in nature: copywriting, stock photos, VoiceOver, and even video game design. All of these areas have seen massive productivity boosts from generative AI. Of course, we won’t get into a conversation on quality, but good enough is fine for many of these tasks.
Conclusion
There is no doubt that AI has the potential to transform our world in a wide variety of ways. This is both with the technology we have today as well as the technology we’ve yet to invent. There will be plenty of surprises, advancements, and things we didn’t see coming. However, we have to stop giving oxygen to people making outlandish and absurd claims. Remember, they aren’t making these claims for your benefit.
Happy New Year! 2024 is in full swing, and already, people are coming out with their big, over-the-top AI predictions. This was to be expected, but I think it’s helpful to calibrate the conversation so that people have a point of reference in their attempts to make sense of these predictions. Even though an overwhelming majority of AI predictions are nothing more than nonsense, it’s helpful to have a simple framework to quickly evaluate the claims in a prediction and determine if even very smart people you respect are saying nothing at all.
AI
The term AI has become so generic that it encompasses technology and approaches that have been both invented and yet to be invented. Anything falling under the umbrella of “automation” is now called AI. Given this generic nature, how do you even know what people are referring to when making claims and predictions?
In my presentations at conferences and events in 2023, I told the audience the first step in making sense of AI advances is understanding what people are referring to in the first place. In this post, I hope to add some clarity around this topic.
Published Human AI Predictions
Let me tell you something you may already know: most people’s published AI predictions are nonsense, falling into the categories of pure guesses, wishful thinking, or pointless parroting. These predictions only serve to drum up hype and marketing buzz. Predictions are often highly biased based on who is making them. Even my own 2024 AI predictions are biased based on the fact that I’m a security researcher. Worse, quite a few have absolutely no visibility or exposure to the technology trends they are predicting.
A fun experiment is to look at the person generating these predictions and their position at the company and guess their prediction. This can be done with a high degree of accuracy. Still, there are others who, when you read their thoughts on AI and their predictions, can tell they’ve never used the technology they are discussing. Yes, please give me more of that person’s predictions.
There are perverse incentives all the way around, from people making the predictions to the media organizations pumping them out for clickbait. It’s no wonder people are confused and unable to understand where things are.
Speaking of perverse incentives, there are the AI Influencers. Don’t get me started. Anyone who missed the cryptocurrency craze and wants the full crypto bro experience only needs to subscribe to their content because everything is 10x-ing around there.
So, what about the leaders of the AI companies? They must be safe, right? Well… eh.
It would be best if you took Sam Altman’s or any other AI organization’s leaders’ AI predictions with a grain of salt. Not to be too cynical, but they are never giving an answer in service of your benefit. Even their critical assessments of AI, such as their concern about AI risks, boost the image of how powerful AI is and further fuel hype. It’s like a human extinction, humble brag.
It’s like a human extinction, humble brag.
So, before you take a prediction seriously, you have to consider a couple of things.
Would this person have some insight into the trend?
Do they have an incentive to say or not say certain things?
Now, let’s move on to our framework.
Framework for Making Sense of Predictions
Given that absolutely everyone is making AI predictions, how do we even begin to make sense of these predictions and whether they should be taken seriously? Although there is no hard and fast answer, a simple framework can help reduce the nonsense.
When evaluating AI predictions, look at a couple of factors. What specific technology is being referred to, what is the prediction timeframe, and did they provide a reason? You can filter out most of the absurdity using only technology precision and timeframe.
Technology Precision
Timeframe
ReasonProvided
Weak
Wide
No
Strong
Narrow
Yes
To make sense of this, let’s look at an example below. This was published a couple of days ago by a well-known security professional. Now, this is a person I respect, and I even enjoyed their last book, so I’m not singling them out. It just happened to be the latest example I’ve seen.
“AI changes everything,” PERSON tells MEDIA on a video call. “The AI revolution is going to be bigger than the internet revolution.”
It’s easy to scratch your head and ask, “WTF does this statement even mean?” Well, it means absolutely nothing, but in terms of our framework, this would be classified as Weak, Wide, and No. What AI approach is this person referring to? When will It be bigger than the internet? There are countless examples of similar overly generic declarations, and I’d wager that a vast majority of people making these claims couldn’t answer with any precision or timeframe.
Without qualifying the precision and a designated timeframe, the statements say absolutely nothing. It’s window dressing for drivel. You might as well say, “Cyborgs will change everything, and the cyborg revolution will have a massive impact on humanity.”
I don’t think that many doubt at this point that some AI technology will transform humanity at some point, even in the near future. Science fiction writers have been envisioning this for quite some time. This is why details matter.
Here’s another generic goodie. Yeah, but what and when bro? Remember, people can purposefully wield being vague as a superpower. If you are vague, you can fuel the hype, wiggle into alternate descriptions, and never worry about being called out on anything specific when it doesn’t happen. This is the Nostradamus playbook: be vague so that everything fits. Musk’s statement was made a few days before Grok was launched, so there’s that.
As a general rule, you can toss out anything falling into the Weak Precision and Wide Timeframe category, as it can mean everything and nothing simultaneously. Most AI predictions I see fall into this category.
Let’s look at some more predictions. Here’s another example below.
These predictions weren’t accurate either, but at least they have strong precision and a narrow timeframe, even without reason. I know he’s generically referring to “AI” here, but through another context, he’s referring to Large Language Models. Sometimes, you have to infer a bit of surrounding context with these. None of these came to pass, but it illustrates another important lesson.
We reward people for being bold, not for being right.
In our modern, social media-driven, self-promoting, nonstop content-producing world, we reward people for being bold, not for being right. People making wild, baseless predictions continue gaining followers and directing people to funnels and newsletters that further allow them to monetize content, which seems (and is) entirely backward from what you’d expect. It’s part of what I call the influencer hype circle. It’s possible to outrun your predictions. People have such short attention spans you can make precise predictions, and a few hours later, people won’t remember, much less months or even a year later.
So, why do we get sucked into these things?
There are a couple of psychological tricks at play. We become desensitized to things we see more often, making more outrageous claims seem not so outrageous. We also tend to assume there must be a consensus the more we see something. The more hype-confirming things we see, the more people say hype-confirming things, which leads to us seeing more hype-confirming things, and over and over, building a false consensus. Hype doesn’t have to be pro-AI hype either; existential risk hype or hype around AI-generated misinformation can fall in the same category.
Note: In fairness, you have to cut people some slack. There can be a big difference between what you tell someone in an interview and what gets cut up and printed. Also, media stories try to be brief. They are likely to print predictions without providing the reason. So YMMV.
Reason
We end by evaluating the reason. By looking at the reason, you can determine whether to put stock in the predictions and the person making them.
Does the reason make sense?
Does it apply to what’s being predicted?
Keep in mind the reason can be overly vague as well. “Technology is moving so fast that it’s inevitable.” That’s not a good reason. “Everyone seems to be saying…” Also, not a good reason. We got an early glimpse of this with ChatGPT. So many people made so many claims about how it was going to be more impactful than the printing press. When you’d listen to their reason, it was because it gave them a recipe in the style of Shakespeare or something similar. It’s hard to put a lot of stock into people who are basically fooling themselves.
Reasons are going to vary along with the amount of included details, so use your best judgment in your evaluation.
Conclusion
Hopefully, this post provided some basic ideas for filtering some of the noise. There are truly insightful people who share their thoughts, drowned out by a cacophony of voices saying nothing much. We are also nowhere near peak AI predictions, so using a basic framework can help reduce the noise and get you to what is important faster. Welcome to 2024.
Seems everything is clickbait these days. News sources are struggling for the scarce resource of attention. In this environment, a simple task becomes a revolution, and a mundane story gets a new life as a groundbreaking advancement. These titles and the resulting amplification by AI hustle bros provide fuel for the AI hype train, which continues in a circle like an ouroboros. In this post, we’ll look at an example of one of these and talk about the issues and risks.
Taking Spins
I saw this article on Bloomberg that mentions the US Military taking generative AI for a spin. The mental image, along with the photo they used of military cyber operation, conjures thoughts of autonomous systems duking it out or missiles launching. This is by design. It’s meant to create this image for you, but nothing so sensational happened.
What really happened is they built a chatbot over their documents. Doesn’t sound as exciting when you put it that way. For those involved, I’m sure this approach, compared to looking over 13 different manuals trying to cross-reference data and find the right content, felt fast and effective. It may also be the right approach for the problem they are trying to solve. Generative AI isn’t some all-powerful technology. It’s good for some things and not so good for others. This is also something you don’t see covered in news stories.
The military article is far from the most sensational example out there. There’s this little gem.
I probably could have found an even more sensational example to make my point, but recency bias kicked in, and the military story was top of mind since I’d discussed it on social media.
Takeaways
There are several takeaways from these types of titles and stories. Below, I’ll hit a few highlights. Let me specify that what I’m talking about here is mostly related to LLMs. Generative AI related to images, audio, and even video is a different topic and something I’ve written about previously here and here. Success is a different story in use cases with graphics and image modeling. I may write more about this in a future post, but for now, let’s stick to LLMs.
Overhyping
Overhyping in reporting is the norm and not the exception. Most cases where there’s proof of LLM success in various industries essentially boil down to people creating a chatbot over documents, some knowledge base, or even log files. This can certainly be valuable and a productivity boost, but it also sounds incredibly boring, so you end up with titles like ChatGPT is revolutionizing the financial industry, are bankers now obsolete??? Most people will never read the article, just the headline.
Most cases where there’s proof of LLM success in various industries essentially boil down to people creating a chatbot over documents.
Accuracy and Reliability
Let’s punctuate the knowledge base chatbot approach by mentioning when dealing with chatbots over sources of information, there’s no guarantee that the bot will return the correct information. It’s not like creating embeddings and doing similarity searches is foolproof. For high-impact situations with a high cost of failure, this would need to be done incredibly well to avoid a catastrophe, even with a human in the loop. Extra steps to allow a human to verify the right data and data source, ensure the data is up to date, and other additional steps are key in doing this right.
Overconfidence and Extension
Finally, the real danger is looking at the apparent success of something like a bot over a data source and making the leap that the technology has capabilities it doesn’t have or the ability to do even more impactful things with an even higher cost of failure. More impactful things, such as suggesting whether to launch missiles or to drive a tank. These are extreme cases, but it proves a point.
Edge cases and complexity are AI’s worst enemies. You don’t see the edge cases in small experiments or super simple tasks, there may not be any, but as with many use cases with high impacts for failure, edge cases may be everywhere, lurking in the shadows waiting to strike when you least expect them.
You don’t see the edge cases in small experiments or super simple tasks.
This overconfidence and extension of generative AI into other areas where it’s not well-suited will cause damage. As this tech is put in more and more critical paths, it’s only a matter of time until there’s a catastrophic failure.
Conclusion
There are a lot of people experimenting and a lot of money flowing in the generative AI space, and as with any technological advancement, we should be prepared to be surprised. However, take the reporting on generative AI and any stories hyped up by the AI hustle crowd with a grain of salt. Perverse incentives are everywhere. Generative AI may be a good fit for your use case, but beware, this isn’t without pitfalls. Generative AI is far from some utopian technology, and given critical use cases with a high cost of failure, the only winning move is not to play.
