Human Compatible#Russell's three principles
{{short description|2019 book by Stuart J. Russell}}
{{italic title}}
{{Infobox book
| italic title = Human Compatible: Artificial Intelligence and the Problem of Control
| name = Human Compatible: Artificial Intelligence and the Problem of Control
| image = Human Compatible (Stuart J. Russell, 2019) book cover.jpg
| image_size =
| border =
| alt =
| caption = Hardcover edition
| author = Stuart J. Russell
| audio_read_by =
| title_orig =
| orig_lang_code =
| title_working =
| translator =
| illustrator =
| cover_artist =
| country = United States
| language = English
| series =
| release_number =
| subject = AI control problem
| genre = Non-fiction
| set_in =
| publisher = Viking
| publisher2 =
| pub_date = October 8, 2019
| english_pub_date =
| published =
| media_type =
| pages = 352
| awards =
| isbn = 978-0-525-55861-3
| isbn_note =
| oclc = 1083694322
| dewey =
| congress =
| preceded_by =
| followed_by =
| native_wikisource =
| wikisource =
| notes =
| exclude_cover =
| website =
}}
Human Compatible: Artificial Intelligence and the Problem of Control is a 2019 non-fiction book by computer scientist Stuart J. Russell. It asserts that the risk to humanity from advanced artificial intelligence (AI) is a serious concern despite the uncertainty surrounding future progress in AI. It also proposes an approach to the AI control problem.
Summary
Russell begins by asserting that the standard model of AI research, in which the primary definition of success is getting better and better at achieving rigid human-specified goals, is dangerously misguided. Such goals may not reflect what human designers intend, such as by failing to take into account any human values not included in the goals. If an AI developed according to the standard model were to become superintelligent, it would likely not fully reflect human values and could be catastrophic to humanity. Russell asserts that precisely because the timeline for developing human-level or superintelligent AI is highly uncertain, safety research should be begun as soon as possible, as it is also highly uncertain how long it would take to complete such research.
Russell argues that continuing progress in AI capability is inevitable because of economic pressures. Such pressures can already be seen in the development of existing AI technologies such as self-driving cars and personal assistant software. Moreover, human-level AI could be worth many trillions of dollars. Russell then examines the current debate surrounding AI risk. He offers refutations to a number of common arguments dismissing AI risk and attributes much of their persistence to tribalism—AI researchers may see AI risk concerns as an "attack" on their field. Russell reiterates that there are legitimate reasons to take AI risk concerns seriously and that economic pressures make continued innovation in AI inevitable.
Russell then proposes an approach to developing provably beneficial machines that focus on deference to humans. Unlike in the standard model of AI, where the objective is rigid and certain, this approach would have the AI's true objective remain uncertain, with the AI only approaching certainty about it as it gains more information about humans and the world. This uncertainty would, ideally, prevent catastrophic misunderstandings of human preferences and encourage cooperation and communication with humans. Russell concludes by calling for tighter governance of AI research and development as well as cultural introspection about the appropriate amount of autonomy to retain in an AI-dominated world.
= Russell's three principles =
Russell lists three principles to guide the development of beneficial machines. He emphasizes that these principles are not meant to be explicitly coded into the machines; rather, they are intended for human developers. The principles are as follows:{{cite book |last=Russell |first=Stuart |date=October 8, 2019 |title=Human Compatible: Artificial Intelligence and the Problem of Control |url=https://archive.org/details/humancompatiblea0000russ |location=United States |publisher=Viking |isbn=978-0-525-55861-3 |author-link=Stuart J. Russell |oclc=1083694322 |url-access=registration}}{{rp|173}}
{{blockquote|1. The machine's only objective is to maximize the realization of human preferences.
2. The machine is initially uncertain about what those preferences are.
3. The ultimate source of information about human preferences is human behavior.}}
The "preferences" Russell refers to "are all-encompassing; they cover everything you might care about, arbitrarily far into the future."{{rp|173}} Similarly, "behavior" includes any choice between options,{{rp|177}} and the uncertainty is such that some probability, which may be quite small, must be assigned to every logically possible human preference.{{rp|201}}
Russell explores inverse reinforcement learning, in which a machine infers a reward function from observed behavior, as a possible basis for a mechanism for learning human preferences.{{rp|191–193}}
Reception
Several reviewers agreed with the book's arguments. Ian Sample in The Guardian called it "convincing" and "the most important book on AI this year".{{cite news|last=Sample|first=Ian|url=https://www.theguardian.com/books/2019/oct/24/human-compatible-ai-problem-control-stuart-russell-review|title=Human Compatible by Stuart Russell review – AI and our future|date=October 24, 2019|work=The Guardian}} Richard Waters of the Financial Times praised the book's "bracing intellectual rigour".{{cite news |last1=Waters |first1=Richard |title=Human Compatible — can we keep control over a superintelligence? |url=https://www.ft.com/content/0e79832c-ef48-11e9-bfa4-b25f11f42901 |access-date=23 February 2020 |work=www.ft.com |date=18 October 2019}} Kirkus Reviews endorsed it as "a strong case for planning for the day when machines can outsmart us".{{cite news |title=HUMAN COMPATIBLE {{!}} Kirkus Reviews |url=https://www.kirkusreviews.com/book-reviews/stuart-russell/human-compatible/ |access-date=23 February 2020 |work=Kirkus Reviews |date=2019 |language=en}}
The same reviewers characterized the book as "wry and witty", or "accessible" due to its "laconic style and dry humour". Matthew Hutson of the Wall Street Journal said "Mr. Russell's exciting book goes deep while sparkling with dry witticisms".{{cite news|first=Matthew|last=Hutson|url=https://www.wsj.com/articles/human-compatible-and-artificial-intelligence-review-learn-like-a-machine-11574207170|title='Human Compatible' and 'Artificial Intelligence' Review: Learn Like a Machine|date=November 19, 2019|work=The Wall Street Journal}} A Library Journal reviewer called it "The right guide at the right time".{{cite news |last1=Hahn |first1=Jim |title=Human Compatible: Artificial Intelligence and the Problem of Control |url=https://www.libraryjournal.com/?reviewDetail=human-compatible-artificial-intelligence-and-the-problem-of-control |access-date=23 February 2020 |work=Library Journal |date=2019}}
James McConnachie of The Times wrote "This is not quite the popular book that AI urgently needs. Its technical parts are too difficult, and its philosophical ones too easy. But it is fascinating and significant."{{cite news|first=James|last=McConnachie|url=https://www.thetimes.com/culture/books/article/human-compatible-by-stuart-russell-review-an-ai-experts-chilling-warning-k2p0j3hw6|title=Human Compatible by Stuart Russell review — an AI expert's chilling warning|date=October 6, 2019|work=The Times}}
By contrast, Human Compatible was criticized in its Nature review by David Leslie, an Ethics Fellow at the Alan Turing Institute; and similarly in a New York Times opinion essay by Melanie Mitchell. One point of contention was whether superintelligence is possible. Leslie states Russell "fails to convince that we will ever see the arrival of a 'second intelligent species{{'"}},{{Cite journal |last=Leslie |first=David |date=2019-10-02 |title=Raging robots, hapless humans: the AI dystopia |url=https://www.nature.com/articles/d41586-019-02939-0 |journal=Nature |language=en |volume=574 |issue=7776 |pages=32–33 |doi=10.1038/d41586-019-02939-0|bibcode=2019Natur.574...32L |url-access=subscription }} and Mitchell doubts a machine could ever "surpass the generality and flexibility of human intelligence" without losing "the speed, precision, and programmability of a computer".{{Cite news |last=Mitchell |first=Melanie |date=2019-10-31 |title=Opinion {{!}} We Shouldn't be Scared by 'Superintelligent A.I.' |language=en-US |work=The New York Times |url=https://www.nytimes.com/2019/10/31/opinion/superintelligent-artificial-intelligence.html |access-date=2023-07-18 |issn=0362-4331}} A second disagreement was whether intelligent machines would naturally tend to adopt so-called "common sense" moral values. In Russell's thought experiment about a geoengineering robot that "asphyxiates humanity to deacidify the oceans", Leslie "struggles to identify any intelligence". Similarly, Mitchell believes an intelligent robot would naturally tend to be "tempered by the common sense, values and social judgment without which general intelligence cannot exist".{{cite journal |last1=Leslie |first1=David |title=Raging robots, hapless humans: the AI dystopia |journal=Nature |date=2 October 2019 |volume=574 |issue=7776 |pages=32–33 |doi=10.1038/d41586-019-02939-0 |bibcode=2019Natur.574...32L |doi-access= }}{{cite news|first=Melanie|last=Mitchell|url=https://www.nytimes.com/2019/10/31/opinion/superintelligent-artificial-intelligence.html|title=We Shouldn't be Scared by 'Superintelligent A.I.'|date=October 31, 2019|work=The New York Times}}
The book was longlisted for the 2019 Financial Times/McKinsey Award.{{cite news |last1=Hill |first1=Andrew |title=Business Book of the Year Award 2019 — the longlist |url=https://www.ft.com/content/01ef790c-b7a0-11e9-96bd-8e884d3ea203 |access-date=23 February 2020 |work=www.ft.com |date=11 August 2019}}
See also
References
{{reflist|30em}}
External links
- [https://techcrunch.com/2019/10/06/an-interview-with-dr-stuart-russell-author-of-human-compatible-artificial-intelligence-and-the-problem-of-control/ Interview with Stuart J. Russell]
{{Existential risk from artificial intelligence|state=expanded}}
Category:American non-fiction books
Category:2019 non-fiction books
Category:English-language non-fiction books
Category:Existential risk from artificial general intelligence