-
Notifications
You must be signed in to change notification settings - Fork 1
/
aises_1_6
30 lines (30 loc) · 9.01 KB
/
aises_1_6
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html>
<html lang="en" xmlns:epub="http://www.idpf.org/2007/ops" xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Introduction to AI Safety Ethics, and Society</title>
<meta http-equiv="default-style" content="text/html; charset=UTF-8"/>
<link rel="stylesheet" type="text/css" href="../style.css"/>
</head>
<body>
<div class="chapter">
<h1 class="section" id="sec1-6">1.6 Discussion of Connections Between Risks</h1>
<p class="nonindent">So far, we have considered four sources of AI risk separately, but they also interact with each other in complex ways. We give some examples to illustrate how risks are connected.</p>
<p class="nonindent1">Imagine, for instance, that a corporate AI race compels companies to prioritize the rapid development of AIs. This could increase organizational risks in various ways. Perhaps a company could cut costs by putting less money toward information security, leading to one of its AI systems getting leaked. This would increase the probability of someone with malicious intent having the AI system and using it to pursue their harmful objectives. Here, an AI race can increase organizational risks, which in turn can make malicious use more likely.</p>
<p class="nonindent1">In another potential scenario, we could envision the combination of an intense AI race and low organizational safety leading a research team to mistakenly view general capabilities advances as “safety.” This could hasten the development of increasingly capable models, reducing the available time to learn how to make them controllable. The accelerated development would also likely feed back into competitive pressures, meaning that less effort would be spent on ensuring models were controllable. This could give rise to the release of a highly powerful AI system that we lose control over, leading to a catastrophe. Here, competitive pressures and low organizational safety can reinforce AI race dynamics, which can undercut technical safety research and increase the chance of a loss of control.</p>
<p class="nonindent1">Competitive pressures in a military environment could lead to an AI arms race, and increase the potency and autonomy of AI weapons. The deployment of AI-powered weapons, paired with insufficient control of them, would make a loss of control more deadly, potentially existential. These are just a few examples of how these sources of risk might combine, trigger, and reinforce one another.</p>
<p class="nonindent1">It is also worth noting that many existential risks could arise from AIs amplifying existing concerns. Power inequality already exists, but AIs could lock it in and widen the chasm between the powerful and the powerless, perhaps even enabling an unshakable global totalitarian regime. Similarly, AI manipulation could undermine democracy, which would also increase the risk of an irreversible totalitarian regime. Disinformation is already a pervasive problem, but AIs could exacerbate it to a point where we fundamentally undermine our ability to reach consensus or sense a shared reality. AIs could develop more deadly bioweapons and reduce the required technical expertise for obtaining them, greatly increasing existing risks of bioterrorism. AI-enabled cyberattacks could make war more likely, which would increase existential risk. Dramatically accelerated economic automation could lead to long-term erosion of human control and enfeeblement. Each of those issues—power concentration, disinformation, cyberattacks, automation—is causing ongoing harm, and their exacerbation by AIs could eventually lead to a catastrophe from which we might not recover.</p>
<p class="nonindent1">As we can see, ongoing harms, catastrophic risks, and existential risks are deeply intertwined. Historically, existential risk reduction has focused on <em>targeted</em> interventions such as technical AI control research, but the time has come for <em>broad</em> interventions [1] like the many sociotechnical interventions outlined in this chapter.</p>
<p class="nonindent1">In mitigating existential risk, it does not make practical sense to ignore other risks. Ignoring ongoing harms and catastrophic risks normalizes them and could lead us to “drift into danger” [2], as further discussed in chapter Safety Engineering. Overall, since existential risks are connected to less extreme catastrophic risks and other standard risk sources, and because society is increasingly willing to address various risks from AIs, we believe that we should not solely focus on <em>directly</em> targeting existential risks. Instead, we should consider the diffuse, <em>indirect</em> effects of other risks and take a more comprehensive approach to risk management.</p>
<h1 class="section" id="sec1-6">1.7 Conclusion</h1>
<p class="nonindent">In this chapter, we have explored how the development of advanced AIs could lead to catastrophe, stemming from four primary sources of risk: malicious use, AI races, organizational risks, and rogue AIs. This lets us decompose AI risks into four proximate causes: an intentional cause, environmental/structural cause, accidental cause, or an internal cause, respectively. We have considered ways in which AIs might be used maliciously, such as terrorists using AIs to create deadly pathogens. We have looked at how a military or corporate AI race could rush us into giving AIs decision-making powers, leading us down a slippery slope to human disempowerment. We have discussed how inadequate organizational safety could lead to catastrophic accidents. Finally, we have addressed the challenges in reliably controlling advanced AIs, including mechanisms such as proxy gaming and goal drift that might give rise to rogue AIs pursuing undesirable actions without regard for human wellbeing.</p>
<p class="nonindent1">These dangers warrant serious concern. Currently, very few people are working on AI risk reduction. We do not yet know how to control highly advanced AI systems, and existing control methods are already proving inadequate. The inner workings of AIs are not well understood, even by those who create them, and current AIs are by no means highly reliable. As AI capabilities continue to grow at an unprecedented rate, it is plausible that they could surpass human intelligence in nearly all respects relatively soon, creating a pressing need to manage the potential risks.</p>
<p class="nonindent1">The good news is that there are many courses of action we can take to substantially reduce these risks. The potential for malicious use can be mitigated by various measures, such as carefully targeted surveillance and limiting access to the most dangerous AIs. Safety regulations and cooperation between nations and corporations could help us to resist competitive pressures that would drive us down a dangerous path. The probability of accidents can be reduced by a rigorous safety culture, among other factors, and by ensuring that safety advances outpace advances in general AI capabilities. Finally, the risks inherent in building technology that surpasses our own intelligence can be addressed by increased investment in several branches of research on control of AI systems, as well as coordination to ensure that progress does not accelerate to a point where societies are unable to respond or manage risks appropriately.</p>
<p class="nonindent1">The remainder of this book aims to outline the underlying factors that drive these risks in more detail and to provide a foundation for understanding and effectively responding to these risks. Later chapters delve into each type of risk in greater depth. For example, risks from malicious use can be reduced via effective policies and coordination, which are discussed in the Governance chapter. The challenge of AI races arises due to collective action problems, discussed in the corresponding chapter. Organisational risks can only be addressed based on a strong understanding of principles of risk management and system safety outlined in the Safety Engineering and Complex Systems chapters. Risks from rogue AI are mediated by mechanisms such as proxy gaming, deception and power-seeking which are discussed in detail in the Single-Agent Safety chapter. While some chapters are more closely aligned to certain risks, many of the concepts they introduce are cross-cutting. The choice of values and goals embedded into AI systems, as discussed in the Beneficial AI and Machine Ethics, is a general factor that can exacerbate or reduce many of the risks discussed in this chapter.</p>
<p class="nonindent1">Before tackling these issues, we provide a general introduction to core concepts that drive the modern field of AI, to ensure that all readers have a high-level understanding of how today’s AI systems work and how they are produced.</p>
<h2 class="section">References</h2>
<p class="ref">[1] Nick Beckstead. <i>On the overwhelming importance of shaping the far future</i>. 2013.</p>
<p class="ref">[2] Jens Rasmussen. “Risk management in a Dynamic Society: A Modeling Problem”. English. In: <i>Proceedings of the Conference on Human Interaction with Complex Systems,</i> 1996.</p>
</div>
</body>
</html>