Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
This dissertation explores the governance issues raised by artificial intelligence (AI) and makes four contributions to the study and practice of AI governance. It connects AI to the literature and practices of responsible research and innovation (RRI) and explores their applicability to AI governance. It provides an assessment of existing AI governance efforts from an RRI perspective, synthesizing a wide range of literatures on AI governance and highlighting several limitations of extant efforts. It explores the value of three different RRI-inspired methods for making AI governance more anticipatory and reflexive: expert elicitation, scenario planning, and formal modeling. Finally, it describes several areas for future work that would put RRI in AI on a sounder footing.
Typology: Thesis
1 / 149
This page cannot be seen from the preview
Don't miss anything!
Responsible Governance of Artificial Intelligence: An Assessment, Theoretical Framework, and Exploration by Miles Brundage A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Approved April 2018 by the Graduate Supervisory Committee: David Guston, Chair Erik Fisher Lauren Keeler Joanna Bryson ARIZONA STATE UNIVERSITY December 2019
i
While artificial intelligence (AI) has seen enormous technical progress in recent years, less progress has occurred in understanding the governance issues raised by AI. In this dissertation, I make four contributions to the study and practice of AI governance. First, I connect AI to the literature and practices of responsible research and innovation (RRI) and explore their applicability to AI governance. I focus in particular on AI’s status as a general purpose technology (GPT), and suggest some of the distinctive challenges for RRI in this context such as the critical importance of publication norms in AI and the need for coordination. Second, I provide an assessment of existing AI governance efforts from an RRI perspective, synthesizing for the first time a wide range of literatures on AI governance and highlighting several limitations of extant efforts. This assessment helps identify areas for methodological exploration. Third, I explore, through several short case studies, the value of three different RRI-inspired methods for making AI governance more anticipatory and reflexive: expert elicitation, scenario planning, and formal modeling. In each case, I explain why these particular methods were deployed, what they produced, and what lessons can be learned for improving the governance of AI in the future. I find that RRI-inspired methods have substantial potential in the context of AI, and early utility to the GPT-oriented perspective on what RRI in AI entails. Finally, I describe several areas for future work that would put RRI in AI on a sounder footing.
iii
Page LIST OF FIGURES ………………………………………………………………………vi CHAPTER 1: INTRODUCTION………………………………………………………………….. 1 2: A FRAMEWORK FOR RESPONSIBLE INNOVATION IN AI …………………. 8 Preliminaries ........................................................................................................... 8 Overview of Responsible Research and Innovation ............................................. 11 AI’s Governance-Related Characteristics ............................................................. 17 The Importance of Publication Norms in RRI for AI ........................................... 28 The Importance of Cooperation in RRI for AI ..................................................... 31 Conclusion ............................................................................................................ 33 3: AN ASSESSMENT OF EXISTING EFFORTS ………………………………….. 34 Introduction ........................................................................................................... 34 Overview of AI Governance ................................................................................. 34 Anticipation........................................................................................................... 42 Inclusion ................................................................................................................ 44 Reflexivity............................................................................................................. 47 Responsiveness ..................................................................................................... 48
vi
Figure Page
Artificial intelligence (AI) has long been a subject of interest to researchers, entrepreneurs, science fiction writers, and the general public. There have been ebbs and flows of attention paid to the field since the mid-twentieth century. AI is currently receiving an unprecedented scale of attention, as shown, e.g., by analysis of New York Times coverage (Fast and Horvitz, 2016). Since the field’s formal beginnings in the mid- 20th century, and especially in recent decades, AI has sparked discussion of ethics and governance, and such discussions have also increased in recent years, as discussed further below. Since roughly 2012, two parallel and related trends have transformed contemporary discussions of AI. First, deep learning, the training of neural networks with multiple hidden layers, has enabled a new wave of human-competitive performance on a wide array of diverse tasks, including image recognition, machine translation, and speech recognition (LeCun et al., 2015). This development builds on a longer history of Internet- related technologies (e.g., search and advertisement placing) leveraging AI techniques, but the application of AI to commercial purposes has received greater attention and enthusiasm in recent years. Second, a substantial increase in discussion of the societal implications of AI has taken place, with participants at different times calling for greater research, formal regulation, and/or some form of self-regulation from the AI community (Brundage and Guston, 2019). Partly inspired by the machine learning “revolution” and the associated growth in commercially and societally impactful applications of AI, these governance discussions have taken place in a wide range of countries and have reached
delivering on RRI in this context demands attention to AI’s status as a general purpose technology (GPT), and the implications that that status has for responsible publication and cooperation among AI developers. I draw on recent developments in the field to illustrate the practical relevance of my proposed framework.^1 Next, I motivate a set of methods for analyzing AI futures more rigorously and reflexively that match the nature of the AI governance problem well, and report on the results of initial efforts in this direction. Finally, I suggest areas for future work. Overall, I found that RRI is a productive framework for AI and that the specific methodological directions I pursue have potential to improve AI governance. In the remainder of this introduction, I briefly preview each of these contributions, which correspond to the order of the following chapters. In Chapter 2 , I give definitions of key terms and analyze AI’s governance-related properties, including its status as a GPT. I briefly describe the history and evolution of thinking on the governance of science and technology, with a focus on the contemporary framework of RRI as my key touchstone in this evolution. I argue that RRI provides a rich framework for thinking about responsibility in AI, but that the challenges of publishing general purpose AI systems^2 have been neglected. Yet these issues loom large in the contemporary challenge of responsible research and innovation in AI. Further, the (^1) In several cases, I have been directly involved in the events I describe, especially with regard to the publication of the GPT-2 system. (^2) I discuss the meaning of generality in more detail later, but briefly I consider there to be a spectrum of increasing generality in which a technology can have more of an ability to be steered toward performing a diverse set of tasks with less human intervention required for each marginal additional task, compared to less general technologies (including in some cases earlier versions of the same AI system). For example, the language model GPT-2 is able to more efficiently adapt to new domains than earlier language models, and the larger versions of the system encapsulate more transferrable knowledge than the smaller versions. Radford and Wu et al, 2019; Solaiman et al., 2019; Brundage et al., 2019.
GPT framing puts a high premium on anticipation of the progress in underlying AI capabilities and the malicious uses toward which AI can be put, which motivates some of the anticipatory efforts described later. In Chapter 3 , I analyze recent developments in AI governance from the perspective of RRI. First, I note the long roots of reflection on societal impacts of AI, and then describe recent developments, especially those occurring in the past few years. I critique these developments with reference to the four “dimensions” of RRI developed by Stilgoe et al. (2013), namely anticipation, reflexivity, inclusion, and responsiveness. I find that anticipatory efforts in AI have generally been underdeveloped, reflexivity in the AI community is too low, inclusion has been narrowly construed, and that responsiveness to surfaced normative considerations has been minimal. This critical assessment provides a baseline for the methodological interventions I discuss in later chapters, which are targeted at filling gaps in the current landscape. In Chapter 4 , I delve into the question of methodology in AI governance analysis: given the RRI framework and the aforementioned characteristics of AI, how ought one go about shaping AI’s development and broader social context positively? I explain why my methodological exploration places a particular focus on the RRI dimensions of anticipation and reflexivity. AI futures are currently highly contested and ill-explored, and both experts and non-experts are insufficiently reflective about the risks, opportunities, and options potentially facing them. These challenges are exacerbated by the generality of AI systems. I contrast AI with energy in various respects. Energy is a technological domain in which anticipatory methods are better developed and more pervasively used, although one in which anticipatory challenges remain. I focus attention
frequent and pervasive uses of expert elicitation in the future. I also discuss the limitations of expert elicitation more generally, given the uncertainties related to technical trends in AI as well as uncertainty about what sorts of expertise are most relevant to governing a GPT. Additionally, I present novel findings related to normative disagreement among AI experts. Finally, I discuss recent collaborative work involving explicit anticipation of possible beneficial and harmful uses of a particular AI system (GPT-2) in order to inform responsible publication decisions. In Chapter 7 , I motivate the use of formal models of AI futures and describe two cases of applying such methods to AI. First, I describe an early effort to design an agent- based model of AI futures, the practical and conceptual challenges of which are informative. The model attempted to capture some salient properties of openness in AI. I describe the design decisions that went into the model’s characterization of openness, which involved issues not (to my knowledge) previously analyzed in the literature, such as the distribution of resources and the absorption rate of shared AI results. In the second case, I describe an effort to reframe the emerging narrative of an “AI arms race” in explicit game theoretic terms, posing the question of whether such a race, if one exists, is best thought of as a Prisoner’s Dilemma, a Stag Hunt, or some other canonical “game” (in the sense used in game theory). I report positive feedback related to this framing and highlight aspects of AI governance for further study which were surfaced as a result of taking this modeling approach. I also discuss more recent collaborative research in which I was involved that pushes this line of thinking further toward identifying concrete policy implications. This work outlines a coherent framework for thinking about the solution of collective action problems in AI--problems made more severe by the generality of AI
systems. The apparent returns on a modest investment in modeling AI development suggests that there is insufficient reflexivity today regarding “hot” AI governance topics such as openness, and that formalization can be one means of increasing reflexivity. In Chapter 8 , I distill lessons learned and future directions from the above analysis. I describe synergies between the methods described in earlier chapters, and identify ways to perform these methods better in the future. I make two recommendations for those involved in governing AI: first, a more systematic effort to identify opportunities in the broad area of “AI for good,” a particularly promising possibility afforded by AI’s status as a GPT; and second, increased attention to inclusion in discussions of AI’s future. Absent such democratization of foresight for and shaping of AI, the anticipatory tools described here might be used to entrench power rather than to steer AI in broadly beneficial directions. Finally, I discuss several areas for future work, including scaling up the methods described, deepening them in various respects, and improving the theoretical foundation of AI governance through comparative analyses of other general purpose technologies.
learning.^4 The “environment” in question might be fully digital, such as a dataset that needs to be labeled, or external, such as the immediately surrounding physical world in the case of an AI-enabled robot. Representative examples of AI systems include search engines, speech recognition systems, semi-autonomous drones, and machine translation systems. The term “AI” has been used to refer to various things, including a research community with the long-term ambition of building more broadly competent digital systems, or the specific technical artifacts already produced by that field, or the systems which researchers in the aforementioned field might aspire to build in the future. Each of these definitions is relevant to questions of governance in different ways: the field of AI is having unprecedented economic and social influence today, many AI systems are already having an impact on society (Brundage and Bryson, 2014; Brundage and Bryson, 2016), and future technical and social developments should inform the nature and degree of our concern about and societal preparation for them (Brundage, 2015; Brundage, 2016a). While each of these is relevant to governance, I primarily focus on issues related to the design and dissemination of AI systems, in which the AI community is a key actor. When I refer to AI being governed, I am referring to the set of institutions, norms, and laws surrounding digital systems that are intended to have some degree of “intelligence,” regardless of whether that intelligence is explicitly modeled on biological organisms such as humans or not. The use of the term “uncertain” in my definition distinguishes AI from other, more static information and communication technologies (ICTs) which are technologies (^4) This definition is adapted from Brundage and Bryson, 2016.
that have full information about their tasks, act deterministically on fully structured data, and cannot be well described as acting in the pursuit of goals. AI systems are agents in the sense that they pursue goals (Russell and Norvig, 2009), but this does not imply human-likeness. AI systems are designed as artifacts to flexibly achieve goals using limited information and computational capacity. “AI system,” as I use the term in this dissertation, can be thought of as shorthand: “AI socio-technical system” would in many cases be a better reflection of the designed^5 and socially embedded nature of the artifacts built by the AI community.^6 Also note that, while much is often made of the definition of AI in popular culture and some policy discussions, the subsequent discussions do not hinge greatly on my definition being used versus another. An alternative definitional cluster focuses on digital systems that perform some behavior which, if done by humans or non-human animals, would be seen as requiring intelligence. Both the generic definition I use and one based on reference to human and non-human animal behavior would yield similar (though not identical) conclusions about the pervasive social and economic applications and implications of AI. A technology capable of substituting for either a subset of “intelligence” generally or a subset of “human-like intelligence” specifically would both have an enormous range of applications, even though these are technically distinct. The definition I use avoids (^5) Many AI systems learn from their experience, but this does not negate the fact that people make a range of design choices when creating and operating them. (^6) Importantly, referring to “AI systems” as agents of societal change does not imply moral agency on the part of the technology. Bryson (2019) notes that “no fact of either biology (the study of life) nor computer science (the study of what is computable) names a necessary point at which human responsibility should end. Responsibility is not a fact of nature. Rather, the problem of governance is as always to design our artefacts—including the law itself—in a way that helps us maintain enough social order so that we can sustain human flourishing.”
disciplines that have contributed to this understanding include history, public policy, philosophy, economics, and especially science and technology studies (STS) (Felt et al., eds, 2017). Various events and trends in the 20th century contributed to greater attention to such issues. Revelations about Nazi medical experiments, for example, spurred calls for ethical treatment of human subjects; allegations of scientific misconduct in the United States created controversy in Congress and heralded greater oversight of federally funded research (Guston, 2000); and rising sensitivity to the military implications of science and innovation in the wake of Hiroshima and Nagasaki as well as the Vietnam War and other events (Moore, 2013) sparked greater debate in the scientific community about issues of social responsibility. The United States government and others have sought to shape science and technology in various ways for centuries, but the US government has been especially explicit about this influence since World War II (Guston, 2000). More recently, there has been a substantial effort aimed at better anticipating and shaping innovation processes as well as outcomes, especially in Europe. While the United States government was for some time a pioneer of assessing the potential societal implications of emerging technologies, having an Office of Technology Assessment (OTA) for this purpose, OTA was terminated in the 1990s as part of broader government budget cuts (Bimber, 1996). In recent decades, European countries have taken the lead in technology assessment (TA), a precursor of RRI, and in public engagement with science and technology more generally. Countries that originally imitated the US’s OTA, such as the Dutch and Danish governments, are now pioneers in methods for fostering democratic deliberation on the
social impacts of technology. These countries have used various terms to refer to this work such as constructive TA (Schot and Rip, 1997). Controversies such as the public debate over genetically modified organisms (GMOs), and many cases in which early signs of technology-related dangers were not heeded until substantial harm had occurred (Harremoës et al., eds., 2001), gave a strong impetus to calls for engagement with science and technology “upstream” in those technologies’ development (Wilsdon and Willis, 2004). Other events such as the Asilomar Conference on recombinant DNA also contributed to the heightening of responsibility discourse in the 20th century (cf. criticisms of the Asilomar model - Jasanoff, Hurlbut, and Saha, 2015). More generally, some have linked RRI in particular and the “responsibilization” of science and technology more generally (Dorbeck-Jung and Shelley-Egan, 2013) to the growing scale and temporal duration of technology’s potential impact, which calls for more sensitivity toward future generations and distant others than was required in earlier phases of human history (Jonas, 1979). The theoretical and empirical literature on responsible research and innovation (RRI), and various associated practices, according to one account, stem from the synthesis of a number of other areas, especially STS, TA, and applied ethics (Grunwald, 2011). The term RRI and its recent antecedents or siblings such as “anticipatory governance” (Guston, 2014) stemmed in large part from rich discussion of the societal implications of nanotechnology and other emerging technologies in the 2000s. Other roots include the ELSI (ethical, legal, and social implications) and ELSA (ethical, legal, and social aspects) discourse stemming in the 1990s in the context of the Human Genome Project, which funded ELSI work in parallel with scientific work. When substantial U.S.
