So this hold finally came in to my local library and I had a chance to read it. I don’t really buy books anymore when I can get them for free months later. I wouldn’t call this a blockbuster anyway. Overall, the author argues that containment of Artifical General Intelligence (AGI) is both nearly impossible and absolutely essential. By the end of the book, I’d been convinced of neither.
While the ability to control and even shutdown an intelligence greater than our own seems important, I am pessimistic that humanity will get its act together as there is too much financial incentive for nations and corporations to build AGI first. Unlike nuclear weapons, the hardware for AGI will get cheaper each year. Just look at what hobbyists are doing with ChatGPT alternatives.
Overall, I’m in the camp that nearterm problems from disinformation controlling the populace is a bigger problem today than AGI in the future. Finally, I’m more worried about climate chaos leading to serious war and refugee problems over some billionaires with a lot of GPUs.