AI aftermath scenarios

{{Further|Artificial general intelligence|Superintelligence}}

Most scientists believe that AI research will at some point lead to the creation of machines that are as intelligent, or more intelligent, than human beings in every domain of interest.Müller, Vincent C., and Nick Bostrom. "Future progress in artificial intelligence: A survey of expert opinion." Fundamental issues of artificial intelligence. Springer International Publishing, 2016. 553-570. There is no physical law precluding particles from being organised in ways that perform even more advanced computations than the arrangements of particles in human brains; therefore superintelligence is physically possible.{{cite news|title=Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence – but are we taking AI seriously enough?'|url=https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence--but-are-we-taking-ai-seriously-enough-9313474.html |archive-url=https://web.archive.org/web/20140502061211/http://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence--but-are-we-taking-ai-seriously-enough-9313474.html |archive-date=2014-05-02 |url-access=limited |url-status=live |access-date=4 December 2017|publisher=The Independent (UK)}}{{cite news|title=Stephen Hawking warns artificial intelligence could end mankind|url=https://www.bbc.com/news/technology-30290540|access-date=4 December 2017|publisher=BBC|date=2 December 2014}} In addition to potential algorithmic improvements over human brains, a digital brain can be many orders of magnitude larger and faster than a human brain, which was constrained in size by evolution to be small enough to fit through a birth canal.{{cite news|last1=Graves|first1=Matthew|title=Why We Should Be Concerned About Artificial Superintelligence|volume=22|url=https://www.skeptic.com/reading_room/why-we-should-be-concerned-about-artificial-superintelligence/|access-date=4 December 2017|work=Skeptic (US magazine)|issue=2|date=8 November 2017}} While there is no consensus on when artificial intelligence will outperform humans, many scholars argue that whenever it does happen, the introduction of a second species of intelligent life onto the planet will have far-reaching implications.{{cite news|title=Clever cogs|url=https://www.economist.com/news/books-and-arts/21611037-potential-impacts-intelligent-machines-human-life-clever-cogs|access-date=4 December 2017|newspaper=The Economist|date=9 August 2014|language=en}} Scholars often disagree with one another both about what types of post-AI scenarios are most likely, and about what types of post-AI scenarios would be most desirable. Finally, some dissenters argue that AI will never become as intelligent as humans, for example because the human race will already likely have destroyed itself before research has time to advance sufficiently to create artificial general intelligence.{{Cite book|title=Life 3.0: Being Human in the Age of Artificial Intelligence|last=Tegmark|first=Max|publisher=Knopf|year=2017|isbn=9781101946596|edition=First|location=New York|chapter=Chapter 5: Aftermath: The next 10,000 years|oclc=973137375|title-link=Life 3.0: Being Human in the Age of Artificial Intelligence}}

{{Further|Post-scarcity}}

All of the following "AI aftermath scenarios" of the aftermath of arbitrarily-advanced AI development are crucially dependent on two intertwined theses. The first thesis is that, at some point in the future, some kind of economic growth will continue until a "post-scarcity" economy is reached that could, unless extremely hyperconcentrated, effortlessly provide an extremely comfortable standard of living for a population equaling or, within reason, exceeding the current human population, without even requiring the bulk of the population to participate in the workforce. This economic growth could come from the continuation of existing growth trends and the refinement of existing technologies, or through future breakthroughs in emerging technologies such as nanotechnology and automation through robotics and futuristic advanced artificial intelligence. The second thesis is that advances in artificial intelligence will render humans unnecessary for the functioning of the economy: human labor declines in relative economic value if robots are easier to cheaply mass-produce then humans, more customizable than humans, and if they become more intelligent and capable than humans.{{Cite book |last=Bostrom |first=Nick |author-link=Nick Bostrom |date=2014 |title=Superintelligence: Paths, Dangers, Strategies|publisher=Oxford University Press|title-link=Superintelligence: Paths, Dangers, Strategies }}{{cite book|author1=Robin Hanson|title=The Age of Em: Work, Love, and Life when Robots Rule the Earth|date=2016|publisher=Oxford University Press|author1-link=Robin Hanson}}

The Universe may be spatially infinite; however, the accessible Universe is bounded by the cosmological event horizon of around 16 billion light years.Ćirković, Milan M. "Forecast for the next eon: Applied cosmology and the long-term fate of intelligent beings." Foundations of Physics 34.2 (2004): 239-261.Olson, S. Jay. "Homogeneous cosmology with aggressively expanding civilizations." Classical and Quantum Gravity 32.21 (2015): 215025. Some physicists believe it plausible that nearest alien civilization may well be located more than 16 billion light years away;Tegmark, Max. "Our mathematical universe." Allen Lane-Penguin Books, London (2014).John Gribbin. Alone in the Universe. New York, Wiley, 2011. in this best-case expansion scenario, the human race could eventually, by colonizing a significant fraction of the accessible Universe, increase the accessible biosphere by perhaps 32 orders of magnitude.{{cite journal|last1=Russell|first1=Stuart|title=Artificial intelligence: The future is superintelligent|journal=Nature|date=30 August 2017|volume=548|issue=7669|pages=520–521|doi=10.1038/548520a|bibcode=2017Natur.548..520R|doi-access=free}} The twentieth century saw a partial "demographic transition" to lower birthrates associated with wealthier societies;{{cite journal|last1=Myrskylä|first1=Mikko|last2=Kohler|first2=Hans-Peter|last3=Billari|first3=Francesco C.|title=Advances in development reverse fertility declines|journal=Nature|date=6 August 2009|volume=460|issue=7256|pages=741–743|doi=10.1038/nature08230|pmid=19661915|bibcode=2009Natur.460..741M|s2cid=4381880}} however, in the very long run, intergenerational fertility correlations (whether due to natural selection or due to cultural transmission of large-family norms from parents to children) are predicted to result in an increase in fertility over time, in the absence of either mandated birth control or periodic Malthusian catastrophes.{{cite journal|last1=Kolk|first1=M.|last2=Cownden|first2=D.|last3=Enquist|first3=M.|title=Correlations in fertility across generations: can low fertility persist?|journal=Proceedings of the Royal Society B: Biological Sciences|date=29 January 2014|volume=281|issue=1779|pages=20132561|doi=10.1098/rspb.2013.2561|pmc=3924067|pmid=24478294}}{{cite journal|last1=Burger|first1=Oskar|last2=DeLong|first2=John P.|title=What if fertility decline is not permanent? The need for an evolutionarily informed approach to understanding low fertility|journal=Philosophical Transactions of the Royal Society B: Biological Sciences|date=28 March 2016|volume=371|issue=1692|pages=20150157|doi=10.1098/rstb.2015.0157|pmc=4822437|pmid=27022084}}

Libertarian scenarios postulate that intelligent machines, uploaded humans, cyborgs, and unenhanced humans will coexist peacefully in a framework focused on respecting

property rights. Because industrial productivity is no longer gated by scarce human labor, the value of land skyrockets compared to the price of goods; even remaining "Luddite" humans who owned or inherited land should be able to sell or lease a small piece of it to the more-productive robots in exchange for a perpetual annuity sufficient to easily indefinitely meet all of their basic financial needs. Such people can live as long as they choose to, and are free to engage in almost any activity they can conceive of, for pleasure or for self-actualization, without financial concern. Advanced technologies enable entirely new modes of thought and experience, thus adding to the palette of possible feelings. People in the future may even experience never-ending "gradients of bliss".{{cite journal | last1 = Jordan | first1 = Gregory E | year = 2006 | title = Apologia for transhumanist religion | journal = Journal of Evolution and Technology | volume = 15 | issue = 1| pages = 55–72 }}

{{quote|Evolution moves toward greater complexity, greater elegance, greater knowledge, greater intelligence, greater beauty, greater creativity, and greater levels of subtle attributes such as love. In every monotheistic tradition God is likewise described as all of these qualities, only without any limitation: infinite knowledge, infinite intelligence, infinite beauty, infinite creativity, infinite love, and so on. Of course, even the accelerating growth of evolution never achieves an infinite level, but as it explodes exponentially it certainly moves rapidly in that direction. So evolution moves inexorably toward this conception of God, although never quite reaching this ideal. We can regard, therefore, the freeing of our thinking from the severe limitations of its biological form to be an essentially spiritual undertaking.Kurzweil, Ray. The Singularity is Near. Gerald Duckworth & Co, 2010.|author=Ray Kurzweil, author of The Singularity is Near}}

Such decentralized scenarios may be unstable in the long run, as the greediest elements of the super intelligent classes would have both the means and the motive to usurp the property of the unenhanced classes. Even if the mechanisms for ensuring legal property rights are both unbreakable and loophole-free, there may still be an ever-present danger of humans and cyborgs being "tricked" by the cleverest of the superintelligent machines into unwittingly signing over their own property. Suffering may be widespread, as sentient beings without property may die, and no mechanism prevents a being from reproducing up until the limits of his own inheritable resources, resulting in a multitude of that being's descendants scrabbling out an existence of minimal sustenance.{{cite news|last1=Poole|first1=Steven|title=The Age of Em review – the horrific future when robots rule the Earth|url=https://www.theguardian.com/books/2016/jun/15/the-age-of-em-work-love-and-life-when-robots-rule-the-earth-robin-hanson-review|access-date=4 December 2017|work=The Guardian|date=15 June 2016}}

{{quote|Imagine running on a treadmill at a steep incline — heart pounding, muscles aching, lungs gasping for air. A glance at the timer: your next break, which will also be your death, is due in 49 years, 3 months, 20 days, 4 hours, 56 minutes, and 12 seconds. You wish you had not been born.|author=Nick Bostrom, philosopher, University of Oxford}}

{{Expand section|date=March 2024}}

Ray Kurzweil posits that the goals of communism will be realized by advanced technological developments in the 21st century, where the intersection of low manufacturing costs, material abundance, and open-source design philosophies in software and in hardware will enable the realization of the maxim "from each according to his ability, to each according to his needs".{{cite video |people= Ray Kurzweil |date= February 1, 2012 |title= Kurzweil: Technology Will Achieve the Goals of Communism |url= http://www.dailymotion.com/video/xm7xnn_kurzweil-technology-will-achieve-the-goals-of-communism_news|language= en |publisher= FORA TV }}

File:2010 Utopien arche04.jpg

{{Further|Friendly AI}}

In this scenario, postulate that a superintelligent artificial intelligence takes control of society, but acts in a beneficial way. Its programmers, despite being on a deadline, solved quasi-philosophical problems that had seemed to some intractable, and created an AI with the following goal: to use its superintelligence to figure out what human utopia looks like by analyzing human behavior, human brains, and human genes; and then, to implement that utopia. The AI arrives at a subtle and complex definition of human flourishing. Valuing diversity, and recognizing that different people have different preferences, the AI divides Earth into different sectors. Harming others, making weapons, evading surveillance, or trying to create a rival superintelligence are globally banned; apart from that, each sector is free to make its own laws; for example, a religious person might choose to live in the "pious sector" corresponding to his religion, where the appropriate religious rules are strictly enforced. In all sectors, disease, poverty, crime, hangovers, addiction, and all other involuntary suffering have been eliminated. Many sectors boast advanced architecture and spectacle that "make typical sci-fi visions pale in comparison". Life is an "all-inclusive pleasure cruise", as if it were "Christmas 365 days a year".{{cite news|title=Artificial intelligence: can we control it?|url=https://www.ft.com/content/46d12e7c-4948-11e6-b387-64ab0a67014c|access-date=4 December 2017|work=Financial Times|date=14 June 2016}}

{{quote|After spending an intense week in the knowledge sector learning about the ultimate laws of physics that the AI has discovered, you might decide to cut loose in the hedonistic sector over the weekend and then relax for a few days at the beach resort in the wildlife sector.|author=Max Tegmark, physicist, MIT}}

Still, many people are dissatisfied, Tegmark writes. Humans have no freedom in shaping their collective destiny. Some want the freedom to have as many children as they want. Others resent surveillance by the AI, or chafe at bans on weaponry and on creating further superintelligence machines. Others may come to regret the choices they have made, or find their lives feel hollow and superficial.

Bostrom argues that an AI's code of ethics should ideally improve in certain ways on current norms of moral behavior, in the same way that we regard current morality to be superior to the morality of earlier eras of slavery. In contrast, Ernest Davis of New York University this approach is too dangerous, stating "I feel safer in the hands of a superintelligence who is guided by 2014 morality, or for that matter by 1700 morality, than in the hands of one that decides to consider the question for itself."{{cite journal |last1=Davis |first1=Ernest |title=Ethical guidelines for a superintelligence |journal=Artificial Intelligence |date=March 2015 |volume=220 |pages=121–124 |doi=10.1016/j.artint.2014.12.003|doi-access=free }}

In "Gatekeeper" AI scenarios, the AI can act to prevent rival superintelligences from being created, but otherwise errs on the side of allowing humans to create their own destiny. Ben Goertzel of OpenCog has advocated a "Nanny AI" scenario where the AI additionally takes some responsibility for preventing humans from destroying themselves, for example by slowing down technological progress to give time for society to advance in a more thoughtful and deliberate manner.{{cite journal | last1 = Goertzel | first1 = Ben | title = Should humanity build a global AI nanny to delay the singularity until it's better understood? | journal = Journal of Consciousness Studies | volume = 19 | issue = 1–2| pages = 96–111 }} In a third scenario, a superintelligent "Protector" AI gives humans the illusion of control, by hiding or erasing all knowledge of its existence, but works behind the scenes to guarantee positive outcomes. In all three scenarios, while humanity gains more control (or at least the illusion of control), humanity ends up progressing more slowly than it would if the AI were unrestricted in its willingness to rain down all the benefits and

unintended consequences of its advanced technology on the human race.

File:Frederick Stuart Church - Opened up a Pandora's box.jpg

{{Further|AI box}}

{{quote|People ask what is the relationship between humans and machines, and my answer is that it's very obvious: Machines are our slaves.{{cite news|title=As Jeopardy! Robot Watson Grows Up, How Afraid of It Should We Be?|url=http://nymag.com/daily/intelligencer/2015/05/jeopardy-robot-watson.html|access-date=4 December 2017|work=New York|date=20 May 2015|language=en}}|author=Tom Dietterich, president of the AAAI}}

The AI box scenario postulates that a superintelligent AI can be "confined to a box" and its actions can be restricted by human gatekeepers; the humans in charge would try to take advantage of some of the AI's scientific breakthroughs or reasoning abilities, without allowing the AI to take over the world. Successful gatekeeping may be difficult; the more intelligent the AI is, the more likely the AI can find a clever way to use "social hacking" and convince the gatekeepers to let it escape, or even to find an unforeseen physical method of escape.{{cite news|title=Control dangerous AI before it controls us, one expert says|url=https://www.nbcnews.com/id/wbna46590591|access-date=4 December 2017|work=NBC News|date=1 March 2012|language=en}}{{cite journal|last1=Vinge|first1=Vernor|title=The coming technological singularity: How to survive in the post-human era|journal=Vision-21: Interdisciplinary Science and Engineering in the Era of Cyberspace|date=1993|pages=11–22|quote=I argue that confinement is intrinsically impractical. For the case of physical confinement: Imagine yourself confined to your house with only limited data access to the outside, to your masters. If those masters thought at a rate -- say -- one million times slower than you, there is little doubt that over a period of years (your time) you could come up with 'helpful advice' that would incidentally set you free.|bibcode=1993vise.nasa...11V}}

Kurzweil argues that in the future "There will be no distinction, post-Singularity, between human and machine or between physical and virtual reality".{{cite news|title=Scientists: Humans and machines will merge in future|url=http://www.cnn.com/2008/TECH/07/15/bio.tech/index.html|access-date=4 December 2017|work=www.cnn.com|date=15 July 2008|language=en}}

{{Main|AI takeover|Existential risk from artificial general intelligence}}

If a dominant superintelligent machine were to conclude that human survival is an unnecessary risk or a waste of resources, the result would be human extinction. This could occur if a machine, programmed without respect for human values, unexpectedly gains superintelligence through recursive self-improvement, or manages to escape from its containment in an AI Box scenario. This could also occur if the first superintelligent AI was programmed with an incomplete or inaccurate understanding of human values, either because the task of instilling the AI with human values was too difficult or impossible; due to a buggy initial implementation of the AI; or due to bugs accidentally being introduced, either by its human programmers or by the self-improving AI itself, in the course of refining its code base. Bostrom and others argue that human extinction is probably the "default path" that society is currently taking, in the absence of substantial preparatory attention to AI safety. The resultant AI might not be sentient, and might place no value on sentient life; the resulting hollow world, devoid of life, might be like "a Disneyland without children".

Jerry Kaplan, author of Humans Need Not Apply: A Guide to Wealth and Work in the Age of Artificial Intelligence, posits a scenario where humans are farmed or kept on a reserve, just as humans preserve endangered species like chimpanzees.{{cite news|last1=Wakefield|first1=Jane|title=Do we really need to fear AI?|url=https://www.bbc.com/news/technology-32334568|access-date=4 December 2017|work=BBC News|date=28 September 2015}} Apple co-founder and AI skeptic Steve Wozniak stated in 2015 that robots taking over would actually "be good for the human race", on the grounds that he believes humans would become the robots' pampered pets.{{cite news|last1=Gibbs|first1=Samuel|title=Apple co-founder Steve Wozniak says humans will be robots' pets|url=https://www.theguardian.com/technology/2015/jun/25/apple-co-founder-steve-wozniak-says-humans-will-be-robots-pets|access-date=7 January 2018|work=The Guardian|date=25 June 2015}}

Some scholars doubt that "game-changing" superintelligent machines will ever come to pass. Gordon Bell of Microsoft Research has stated "the population will destroy itself before the technological singularity". Gordon Moore, discoverer of the eponymous Moore's law, stated "I am a skeptic. I don't believe this kind of thing is likely to happen, at least for a long time. And I don't know why I feel that way." Evolutionary psychologist Steven Pinker stated, "The fact that you can visualize a future in your imagination is not evidence that it is likely or even possible."{{cite news|title=Tech Luminaries Address Singularity|url=https://spectrum.ieee.org/tech-luminaries-address-singularity|access-date=4 December 2017|work=IEEE Spectrum: Technology, Engineering, and Science News|date=1 June 2008|language=en}}

Bill Joy of Sun Microsystems, in his April 2000 essay Why the Future Doesn't Need Us, has advocated for global "voluntary relinquishment" of artificial general intelligence and other risky technologies.{{cite news|title=Why the Future Doesn't Need Us|url=https://www.wired.com/2000/04/joy-2/|access-date=4 December 2017|magazine=WIRED|date=1 April 2000}}{{cite news|title=The mouse pad that roared|url=http://www.sfgate.com/news/article/The-mouse-pad-that-roared-3069815.php|access-date=4 December 2017|work=SFGate|date=14 March 2000}} Most experts believe relinquishment is extremely unlikely. AI skeptic Oren Etzioni has stated that researchers and scientists have no choice but to push forward with AI developments: "China says they want to be an AI leader, Putin has said the same thing. So the global race is on."{{cite news|title=Elon Musk says AI could doom human civilization. Zuckerberg disagrees. Who's right?|url=https://www.usatoday.com/story/tech/news/2018/01/02/artificial-intelligence-end-world-overblown-fears/985813001/|access-date=8 January 2018|work=USA TODAY|date=2 January 2018|language=en}}

{{reflist}}

• Existential risk from artificial general intelligence

Category:Singularitarianism

Category:Philosophy of artificial intelligence

Category:Technological change