AI as threat to humanity

Wait But Why Article

Computer Colossus is learning to think. We need to help develop it in a way that’s good. There are plenty of times in life when it’s a good strategy to take a risk in order to give yourself a chance for the best possible outcome, but when the stakes are at their absolute highest, the right move is usually to play it safe. Power is one of those times. That’s why, even though democracy essentially guarantees a certain level of mediocrity, Elon says, “I think you’re hard-pressed to find many people in the United States who, no matter what they think of any given president, would advocate for a dictatorship.”
And since Elon sees AI as the ultimate power, he sees AI development as the ultimate “play it safe” situation. Which is why his strategy for minimizing existential AI risk seems to essentially be that AI power needs to be of the people, by the people, for the people.

For the people

More broadly, a single pioneer’s magic wand would likely have been built to serve that inventor’s own needs and purposes. But by turning the future magic wand industry into a collective effort, a wide variety of needs and purposes will have a wand made for them, making it more likely that the capabilities of the world’s aggregate mass of magic wands will overarchingly represent the needs of the masses.

Of the people

In a future world made up of AI and everyone else, he thinks we have only one good option: To be AI. [It’s like trying to imagine a color you’ve never seen.]

The challenge is the communication bandwidth is extremely slow, particularly output. When you’re outputting on a phone, you’re moving two thumbs very slowly. That’s crazy slow communication. … If the bandwidth is too low, then your integration with AI would be very weak. Given the limits of very low bandwidth, it’s kind of pointless. The AI is just going to go by itself, because it’s too slow to talk to. The faster the communication, the more you’ll be integrated—the slower the communication, the less. And the more separate we are—the more the AI is “other”—the more likely it is to turn on us. If the AIs are all separate, and vastly more intelligent than us, how do you ensure that they don’t have optimization functions that are contrary to the best interests of humanity? … If we achieve tight symbiosis, the AI wouldn’t be “other”—it would be you and with a relationship to your cortex analogous to the relationship your cortex has with your limbic system.

In a world in which millions of people control a little piece of the world’s aggregate AI power—people who can think with AI, can defend themselves with AI, and who fundamentally understand AI because of their own integration with it—humans are less vulnerable…..the AI on the planet will be so widely distributed and varied in its goals.

https://hbr.org/2014/11/with-big-data-comes-big-responsibility