The Intelligence Illusion

Excerpts from the book

From Artificial General Intelligence and the bird brains of Silicon Valley

Because text and language are the primary ways we experience other people’s reasoning, it’ll be next to impossible to dislodge the notion that these are genuine intelligences. No amount of examples, scientific research, or analysis will convince those who want to maintain a pseudo-religious belief in alien peer intelligences. After all, if you want to believe in aliens, an artificial one made out of supercomputers and wishful thinking feels much more plausible than little grey men from outer space. But that’s what it is: a belief in aliens.

(Read the full chapter online)

From Beware of AI pseudoscience and snake oil

Even the latest and greatest, the absolute best that the AI industry has to offer today, the aforementioned GPT-4 appears to suffer from this issue where its unbelievable performance in exams and benchmarks seems to be mostly down to training data contamination.

When its predecessor, ChatGPT using GPT-3.5, was compared to less advanced but more specialised language models, it performed worse on most, if not all, natural language tasks.

There’s even reason to be sceptical of much of the criticism of AI coming out from the AI industry.

Much of it consists of hand-wringing that their product might be too good to be safe—akin to a manufacturer promoting a car as so powerful it might not be safe on the streets. Many of the AI ‘doomsday’ style of critics are performing what others in the field have been calling “criti-hype”. They are assuming that the products are at least as good as vendors claim, or even better, and extrapolate science-fiction disasters from a marketing fantasy.

The harms that come from these systems don’t require any science-fiction—they don’t even require any further advancement in AI. They are risky enough as they are, with the capabilities they have today.21 Some of those risks come from abuse—the systems lend themselves to both legal and illegal abuses. Some of the risks come using them in contexts that are well beyond their capabilities—where they don’t work as promised.

(Read the full chapter online)

From AI code copilots are backwards-facing tools in a novelty-seeking industry

These two biases combined mean that users of code assistants are extremely likely to accept the first suggestion the tool makes that doesn’t cause errors.

That means autocompletes need to be substantially better than an average coder to avoid having a detrimental effect on overall code quality. Obviously, if programmers are going to be picking the first suggestion that works, that suggestion needs to be at least as good as what they would have written themselves unaided. What’s less obvious is that the lack of familiarity—not having written the generated code by hand themselves—is likely to lead them to miss bugs and integration issues that would have been trivially obvious if they had typed it out themselves. To balance that out, the generated code needs to be better than average, which is a tricky thing to ask of a system that’s specifically trained on mountains of average code.

Unfortunately, that mediocrity seems to be reflected in the output. GitHub Copilot, for example seems to regularly generate vulnerable code with security issues.

From The Elegiac Hindsight of Intelligent Machines, the book’s finale

The intelligence illusion, the conviction that these are artificial minds capable of powerful reasoning, when combined with anthropomorphism supercharges our automation bias. Our first response to even the most inane pablum from a language model chatbot is awe and wonder. It sounds like a real person at your beck and call! The drive to treat it as, not just a person, but an expert is irresistible. For most people, the incoherence, mediocrity, hallucinations, plagiarism, and biases won’t register over their sense of wonder.

This anthropomorphism-induced delusion is the fatal flaw of all AI assistant and copilot systems. It all but guarantees that—even though the outcome you get from using them is likely to be worse than if you’d done it yourself, because of the flaws inherent in these models—you will feel more confident in it, not less.

From the Afterword

Diffusion and language models look like powerful tools. The technology presents us with so many opportunities—such power—that I have to go all the way back to the early days of the web to find its equal.

Machine learning has already delivered many improvements to our work and lives. It’s the driving force behind many of the innovations we’ve come to rely on in our photography. It’s transformed special effects work. The opportunities it creates for working with audio are amazing. Language and diffusion models are intriguing.

But they are being implemented in the worst possible ways.

Buy the ebook for $35 USD

Table of Contents

INTRODUCTION

1. What this book is and isn’t
2. What is Generative AI?
3. Strategies for using Generative AI

THE RECOMMENDATIONS

4. Be aware of the abundance of bullshit and snake oil
5. Only use specialised Generative AI tools that you’ve vetted
6. Strengthen your defences against fraud and abuse
7. Avoid externally-facing chatbots—prefer internal tools
8. Prefer transparent and open generative AI
9. Use it primarily for derivation, conversion, and modification

THE RISKS

10. We don’t know if they’re breaking laws or regulations
11. Copyright protections are diminished
12. Much of the training data is biased, harmful, or unsafe
13. Stochastic plagiarism
14. Hallucinations, or AI don’t do facts
15. Much of the output is biased, harmful, or unsafe
16. Code quality
17. Shortcut reasoning
18. Fear of Missing Out

WHAT NEXT?

19. The Elegiac Hindsight of Intelligent Machines
20. Afterword: a straw to grasp
21. Further reading
22. References