Smarter Than Us: The Rise of Machine Intelligence by Stuart Armstrong
When most people think of AI, they probably think of things like IBM’s Watson or the Terminator. Thoughtful analysis is by and large replaced by cached thoughts like “The AI will feel threatened by us and lash out” or even less helpfully “Maybe the human race doesn’t deserve to be saved.” But an AI will do precisely what you tell it to do which is not necessarily the same as what you want it to do, as anyone with even a modicum of programming experience can attest (“I didn’t mean for that to happen…”).
Commissioned by the Machine Intelligence Research Institute (MIRI), Smarter Than Us is a short primer with the intent of closing the inferential distance on the topics of AI risk and the need for AI safety to avoid catastrophic futures. Imagine you told the AI “Maximize human happiness” and it computed for a second and began to forcibly administer high-potency opiates to the global population. Sorry, you didn’t program the AI with “common sense” and now you are living a life of drug-induced bliss but lack metaphysical fulfillment. It’s not a person you can reason with or that will pick up subtle hints, unless all that is explicitly coded in.
Two main theses are that if an AI reaches a level of about human performance in some field (think of computer chess in the mid-1990s) then it can quickly scale to superhuman levels just by throwing more hardware at the problem or tweaking the algorithms (computer chess now), and that coding in correct human morality with precision is currently beyond our abilities. It’s not enough to have built a moral philosophy understandable by humans with enough fuzziness such that edge cases are debatable; it must be precise and formal (Dan Dennett has the saying “AI makes philosophy honest”). This requires active research, now being conducted by MIRI, the Future of Humanity Institute, and more recently the Centre for the Study of Existential Risk, and the book closes by a call for support for these organizations.
The aim of the book is not to be an exhaustive exploration of these issues, but rather a brief discursis meant to upgrade lay audience understanding from a Hollywood level of AI to “Hey, maybe these issues are actually kind of important.”