Sam Altman思想档案馆
Sam Altman 肖像

文献 公开信 alt-015 机器译文 · 未审校

超级智能的治理

Governance of Superintelligence

来源 OpenAI · 英文原文出处

STATE2 中文可读 · 机器译文,未经人工审校 证据等级 A
中文为默认阅读视图

机器译文 本页中文为机器逐句翻译(qwen3.8-flash),未经人工审校,仅供阅读参考;引用与考据请以英文原文为准。 逐句机译与英文原文 1:1 存档(18 段);未经人工审校。

就我们目前所见的图景而言,可以设想:在未来十年内,AI 系统将在大多数领域超越专家级技能水平,并承担相当于当今最大企业之一那样规模的生产性活动。 Given the picture as we see it now, it’s conceivable that within the next ten years, AI systems will exceed expert skill level in most domains, and carry out as much productive activity as one of today’s largest corporations.

无论从潜在收益还是潜在风险来看,超级智能都将比人类过去不得不应对的其他技术更为强大。我们可以拥有一个显著更加繁荣的未来;但要抵达那里,我们必须管理好风险。鉴于存在性风险(existential risk)的可能性,我们不能只做事后反应。核能是常被引用的、具备这种属性的技术的历史例子;合成生物学是另一个例子。 In terms of both potential upsides and downsides, superintelligence will be more powerful than other technologies humanity has had to contend with in the past. We can have a dramatically more prosperous future; but we have to manage risk to get there. Given the possibility of existential risk, we can’t just be reactive. Nuclear energy is a commonly used historical example of a technology with this property; synthetic biology is another example.

我们也必须缓解当今 AI 技术的风险,但超级智能需要特殊对待与协调。 We must mitigate the risks of today’s AI technology too, but superintelligence will require special treatment and coordination.

一个起点 A starting point

要想有较大概率成功驾驭这一发展进程,有许多重要的想法;我们在此阐述其中三个的初步思考。 There are many ideas that matter for us to have a good chance at successfully navigating this development; here we lay out our initial thinking on three of them.

第一,我们需要在领先的开发力量之间实现某种程度的协调,以确保超级智能的发展方式既能让我们维持安全,又能帮助这些系统平稳融入社会。实现方式有很多种:世界各主要政府可以设立一个项目,让现有的许多努力并入其中;或者我们可以共同商定(由一个下文建议的新组织提供背书力量),把前沿 AI 能力的增长速度限制在每年某个特定速率以内。 First, we need some degree of coordination among the leading development efforts to ensure that the development of superintelligence occurs in a manner that allows us to both maintain safety and help smooth integration of these systems with society. There are many ways this could be implemented; major governments around the world could set up a project that many current efforts become part of, or we could collectively agree (with the backing power of a new organization like the one suggested below) that the rate of growth in AI capability at the frontier is limited to a certain rate per year.

当然,单个公司应当被要求以极高的标准负责任地行事。 And of course, individual companies should be held to an extremely high standard of acting responsibly.

第二,我们最终可能需要一个类似 IAEA(https://www.iaea.org/)的机构来监管超级智能方面的努力;任何超过某一能力阈值(或算力等资源阈值)的开发努力,都需要接受一个国际权威机构的管辖——该机构可以检查系统、要求审计、测试其是否符合安全标准、对部署程度与安全等级施加限制,等等。追踪算力与能源使用能起很大作用,并让我们有理由希望这个想法确实可以落地。作为第一步,各公司可以自愿同意开始实施此类机构未来某天可能要求的内容;作为第二步,单个国家可以着手实施。重要的是,这样的机构应专注于降低存在性风险,而不去处理应当留给单个国家的问题,例如界定 AI 被允许说什么。 Second, we are likely to eventually need something like an IAEA (https://www.iaea.org/) for superintelligence efforts; any effort above a certain capability (or resources like compute) threshold will need to be subject to an international authority that can inspect systems, require audits, test for compliance with safety standards, place restrictions on degrees of deployment and levels of security, etc. Tracking compute and energy usage could go a long way, and give us some hope this idea could actually be implementable. As a first step, companies could voluntarily agree to begin implementing elements of what such an agency might one day require, and as a second, individual countries could implement it. It would be important that such an agency focus on reducing existential risk and not issues that should be left to individual countries, such as defining what an AI should be allowed to say.

第三,我们需要具备让超级智能安全的技术能力。这是一个开放的研究问题(https://openai.com/blog/our-approach-to-alignment-research),我们和其他许多机构正在为此投入大量精力。 Third, we need the technical capability to make a superintelligence safe. This is an open research question (https://openai.com/blog/our-approach-to-alignment-research) that we and others are putting a lot of effort into.

不在讨论范围之内的事项 What’s not in scope

我们认为,重要的是允许公司和开源项目开发低于某一重大能力阈值的模型,而不施加我们在此描述的那类监管(包括许可证或审计这类繁重的机制)。 We think it’s important to allow companies and open-source projects to develop models below a significant capability threshold, without the kind of regulation we describe here  (including burdensome mechanisms like licenses or audits).

当今的系统会在世界上创造巨大价值;尽管它们确实有风险,但这些风险的水平感觉与我们感受到的其他互联网技术相当,而社会可能采取的应对方式看起来也是适当的。 Today’s systems will create tremendous value in the world and, while they do have risks, the level of those risks feel commensurate with other Internet technologies and society’s likely approaches seem appropriate.

相比之下,我们所担忧的那些系统将拥有超越迄今所创造的任何技术的力量;我们应当小心,不要把类似的标准应用到远低于这一门槛的技术上,从而稀释了对它们的关注。 By contrast, the systems we are concerned about will have power beyond any technology yet created, and we should be careful not to water down the focus on them by applying similar standards to technology far below this bar.

公众参与与潜力 Public input and potential

但是,对最强大系统的治理,以及关于其部署的决策,必须接受强有力的公众监督。我们相信,世界各地的人们应当以民主方式决定 AI 系统的边界与默认行为(https://openai.com/blog/how-should-ai-systems-behave)。我们还不知道如何设计这样的机制,但我们计划对其开发进行实验。我们继续认为,在这些宽泛的边界之内,单个用户应当对其所用 AI 的行为拥有大量控制权。 But the governance of the most powerful systems, as well as decisions regarding their deployment, must have strong public oversight. We believe people around the world should democratically decide on the bounds and defaults (https://openai.com/blog/how-should-ai-systems-behave) for AI systems. We don't yet know how to design such a mechanism, but we plan to experiment with its development. We continue to think that, within these wide bounds, individual users should have a lot of control over how the AI they use behaves.

鉴于其中的风险与困难,值得思考一下我们究竟为什么要构建这项技术。 Given the risks and difficulties, it’s worth considering why we are building this technology at all.

在 OpenAI,我们有两条根本理由。第一,我们相信它将带来一个比我们今天所能想象的美好得多的世界(在教育、创造性工作与个人生产力等领域,我们已经看到早期例子)。世界面临许多问题,解决它们需要我们获得更多的帮助;这项技术可以改善我们的社会,而每个人使用这些新工具的创造力一定会让我们惊叹。经济增长与生活质量的提升将是惊人的。 At OpenAI, we have two fundamental reasons. First, we believe it’s going to lead to a much better world than what we can imagine today (we are already seeing early examples of this in areas like education, creative work, and personal productivity). The world faces a lot of problems that we will need much more help to solve; this technology can improve our societies, and the creative ability of everyone to use these new tools is certain to astonish us. The economic growth and increase in quality of life will be astonishing.

第二,我们相信,阻止超级智能的创造,将会带来违反直觉的风险与困难。因为收益如此巨大、构建成本逐年下降、构建者的数量迅速增加、而且它本质上就是我们所在技术路径的一部分,要阻止它将需要某种全球监视体制,即便如此也不保证有效。所以我们必须把它做对。 Second, we believe it would be unintuitively risky and difficult to stop the creation of superintelligence. Because the upsides are so tremendous, the cost to build it decreases each year, the number of actors building it is rapidly increasing, and it’s inherently part of the technological path we are on, stopping it would require something like a global surveillance regime, and even that isn’t guaranteed to work. So we have to get it right.

出处与译稿

来源平台
OpenAI
发表时间
2023 年 5 月 22 日
译稿来源
机器逐句翻译 · qwen3.8-flash
审校状态
未经人工审校(reviewed = 0)
译文存档
正式档案库 · translation 表
原文存档
正式档案库 · segment 表
收录状态
STATE2 · 中文暂不可用

中文译文为阅读默认视图;引用与考据请以英文原文为准。外部链接仅作为来源引用保留。

回到正文开头