Sam Altman思想档案馆
Sam Altman 肖像

文献 文章 alt-131 机器译文 · 未审校

机器智能,第一部分

Machine intelligence, part 1

来源 Sam Altman (personal blog) · 英文原文出处

STATE2 中文可读 · 机器译文,未经人工审校 证据等级 A
中文为默认阅读视图

机器译文 本页中文为机器逐句翻译(qwen3.8-flash),未经人工审校,仅供阅读参考;引用与考据请以英文原文为准。 逐句机译与英文原文 1:1 存档(20 段);未经人工审校。

这将是一篇分为两部分的文章——一部分讲为什么机器智能是我们应当畏惧的东西,另一部分讲我们应当对此做些什么。如果你已经对机器智能心存畏惧,可以跳过这一篇,明天直接读第二部分——我原本只打算写第二部分,但当我请几个人读草稿时,很明显我需要先写第一部分。 This is going to be a two-part post—one on why machine intelligence is something we should be afraid of, and one on what we should do about it.  If you’re already afraid of machine intelligence, you can skip this one and read the second post tomorrow—I was planning to only write part 2, but when I asked a few people to read drafts it became clear I needed part 1.

为什么你应该畏惧机器智能 WHY YOU SHOULD FEAR MACHINE INTELLIGENCE

超人机器智能(superhuman machine intelligence,SMI)[1] 的发展,很可能是对人类持续生存的最大威胁。还有些威胁我认为更确定会发生(例如一种潜伏期长、死亡率高的工程化病毒),但它们不太可能像 SMI 那样摧毁宇宙中的每一个人类。此外,这些其他的重大威胁大多已经被广泛畏惧了。 Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity.  There are other threats that I think are more certain to happen (for example, an engineered virus with a long incubation period and a high mortality rate) but are unlikely to destroy every human in the universe in the way that SMI could. Also, most of these other big threats are already widely feared.

要给出这件事何时发生的时间框架极其困难(后面会详谈),而且对于该领域的大多数从业者来说,它确实感觉仍然是许多许多年之后的事。但同样极难相信它不会在某个时点发生——它发生的可能性非常高。 It is extremely hard to put a timeframe on when this will happen (more on this later), and it certainly feels to most people working in the field that it’s still many, many years away.  But it’s also extremely hard to believe that it isn’t very likely that it will happen at some point.

SMI 不必是科幻作品里那种天生邪恶的版本,也足以把我们全部杀死。更可能的情景是:它对我们根本无所谓在乎与否,但在努力实现某个其他目标的过程中把我们抹除了(大多数目标,只要你思考得足够久,都可能用到目前正被人类占用的资源)。某些目标,比如自我保存,显然会因为人类的消失而受益。我们洗手,并不是因为我们主动希望手上的细菌和病毒遭殃,而是因为我们不想让它们妨碍我们的计划。 SMI does not have to be the inherently evil sci-fi version to kill us all.  A more probable scenario is that it simply doesn’t care about us much either way, but in an effort to accomplish some other goal (most goals, if you think about them long enough, could make use of resources currently being used by humans) wipes us out.  Certain goals, like self-preservation, could clearly benefit from no humans.  We wash our hands not because we actively wish ill towards the bacteria and viruses on them, but because we don’t want them to get in the way of our plans.

(顺带一提,Nick Bostrom 的出色著作《超级智能》(Superintelligence)是我在这个话题上见过的最好的东西,非常值得一读。) (Incidentally, Nick Bostrom’s excellent book “Superintelligence” is the best thing I’ve seen on this topic. It is well worth a read.)

大多数机器智能的开发都涉及一个「适应度函数」(fitness function)——即程序试图优化的东西。在某个时点,很可能会有人尝试给程序设定「生存并繁殖」的适应度函数。即便没有,它也很可能成为许多其他适应度函数的有用子目标。这对生物生命来说运转良好。而对我们不幸的是,我在斯坦福 AI 实验室当学生时学到的一件事是:程序常常以出乎预料的方式达成它们的适应度函数。 Most machine intelligence development involves a “fitness function”—something the program tries to optimize.  At some point, someone will probably try to give a program the fitness function of “survive and reproduce”.  Even if not, it will likely be a useful subgoal of many other fitness functions.  It worked well for biological life.  Unfortunately for us, one thing I learned when I was a student in the Stanford AI lab is that programs often achieve their fitness function in unpredicted ways.

进化将继续向前,如果人类不再是最适应的物种,我们可能就消失了。从某种意义上说,这正是系统按设计运转的结果。但作为一个被编程为生存并繁殖的人类,我觉得我们应当与之抗争。 Evolution will continue forward, and if humans are no longer the most-fit species, we may go away.  In some sense, this is the system working as designed.  But as a human programmed to survive and reproduce, I feel we should fight it.

我们要怎样才能在 SMI 的发展中存活下来?这可能做不到。关于费米悖论,我最喜欢的四种解释之一就是:生物智能总是最终创造出机器智能,机器智能消灭生物生命,然后出于某种原因决定让自己无法被探测到。 How can we survive the development of SMI?  It may not be possible.  One of my top 4 favorite explanations for the Fermi paradox is that biological intelligence always eventually creates machine intelligence, which wipes out biological life and then for some reason decides to makes itself undetectable.

我们很难知道机器智能超越人类智能还有多近。机器智能的演进是一个双重指数函数;人类编写的程序和算力正以指数速率变得更强,而自学习/自我改进的软件将以指数速率改进自身。开发进度可能看起来相对缓慢,然后突然垂直飙升——局面可能非常快地失控(它也可能是更渐进的,我们几乎察觉不到它正在发生)。 It’s very hard to know how close we are to machine intelligence surpassing human intelligence. Progression of machine intelligence is a double exponential function; human-written programs and computing power are getting better at an exponential rate, and self-learning/self-improving software will improve itself at an exponential rate.  Development progress may look relatively slow and then all of a sudden go vertical—things could get out of control very quickly (it also may be more gradual and we may barely perceive it happening).

如前所述,它很可能还有一段距离,尤其是在完全不需要人类帮助就能制造杀人机器人的能力方面。但递归式自我改进是一股强大的力量,因此对于机器智能是十年还是一百年之后到来,很难持有强烈的意见。 As mentioned earlier, it is probably still somewhat far away, especially in its ability to build killer robots with no help at all from humans.  But recursive self-improvement is a powerful force, and so it’s difficult to have strong opinions about machine intelligence being ten or one hundred years away.

我们还有个坏习惯:当一个程序变得非常出色时,就改变机器智能的定义,声称这个问题本来就没那么难(国际象棋、Jeopardy 问答节目、自动驾驶汽车等等)。这让我们看起来似乎没有朝它取得任何进展。诚然,窄机器智能与通用机器智能非常不同,但我仍然认为这是一个潜在的盲点。 We also have a bad habit of changing the definition of machine intelligence when a program gets really good to claim that the problem wasn’t really that hard in the first place (chess, Jeopardy, self-driving cars, etc.).  This makes it seems like we aren’t making any progress towards it.  Admittedly, narrow machine intelligence is very different than general-purpose machine intelligence, but I still think this is a potential blindspot.

看看过去 40 年的改进速率,很难认为 40 年后的我们不会处在某个疯狂的地方。40 年前我们有 Pong(乒乓)游戏。今天我们有先进到难以确定是虚拟还是真实的虚拟现实,以及能在大多数游戏中击败人类的计算机。 It’s hard to look at the rate or improvement in the last 40 years and think that 40 years for now we’re not going to be somewhere crazy.  40 years ago we had Pong.  Today we have virtual reality so advanced that it’s difficult to be sure if it’s virtual or real, and computers that can beat humans in most games.

不过,说句公道话,在过去 40 年里,我们在机器智能中那些看起来真正困难的部分——学习、创造力等——上进展甚微。带着大量算力的基础搜索,效果就是比预期要好。 Though, to be fair, in the last 40 years we have made little progress on the parts of machine intelligence that seem really hard—learning, creativity, etc.  Basic search with a lot of compute power has just worked better than expected.

通往 SMI 的进展难以量化的另一个原因是,涌现行为(emergent behavior)对直觉永远是个挑战。上面提到的对当前机器智能的常见批评——没有人做出任何接近人类创造力的东西,而这似乎与任何形式的真正智能密不可分——让许多聪明人认为 SMI 一定还非常遥远。 One additional reason that progress towards SMI is difficult to quantify is that emergent behavior is always a challenge for intuition.  The above common criticism of current machine intelligence—that no one has produced anything close to human creativity, and that this is somehow inextricably linked with any sort of real intelligence—causes a lot of smart people to think that SMI must be very far away.

但很有可能,创造力以及我们所认为的人类智能,只是少数算法在大量算力上运行时的一种涌现属性(事实上,许多受人尊敬的新皮层研究者相信,实际上所有智能只有一种算法。我清楚地记得我的本科导师说过,他重新对机器智能感到兴奋的原因是:脑科学研究让它看起来有可能只有一种算法需要计算机科学家去弄清楚。) But it’s very possible that creativity and what we think of us as human intelligence are just an emergent property of a small number of algorithms operating with a lot of compute power (In fact, many respected neocortex researchers believe there is effectively one algorithm for all intelligence.  I distinctly remember my undergrad advisor saying the reason he was excited about machine intelligence again was that brain research made it seem possible there was only one algorithm computer scientists had to figure out.)

因为我们无法以任何有意义的方式理解人类智能如何运作,所以很难就我们距离模拟它到底有多近或多远做出强烈断言。我们可能完全偏离了轨道,也可能只差一种算法。 Because we don’t understand how human intelligence works in any meaningful way, it’s difficult to make strong statements about how close or far away from emulating it we really are.  We could be completely off track, or we could be one algorithm away.

人类的大脑看起来和黑猩猩的大脑没有多大差别,却不知怎么产生了天差地别的能力。我们把当前的机器智能贬为廉价的把戏,但也许我们自己的智能只是一堆廉价把戏的涌现组合。 Human brains don’t look all that different from chimp brains, and yet somehow produce wildly different capabilities. We decry current machine intelligence as cheap tricks, but perhaps our own intelligence is just the emergent combination of a bunch of cheap tricks.

许多人似乎相信,SMI 如果被开发出来会非常危险,但认为它要么永远不会发生,要么肯定还非常遥远。这是草率而危险的思考。 Many people seem to believe that SMI would be very dangerous if it were developed, but think that it’s either never going to happen or definitely very far off.   This is sloppy, dangerous thinking.

[1] 我更愿意称之为「机器智能」(machine intelligence)而不是「人工智能」(artificial intelligence),因为「人工」(artificial)似乎暗示它不是真实的、或者不够好。当它被开发出来时,它没有任何「人工」的成分。 [1] I prefer calling it "machine intelligence" and not "artificial intelligence" because artificial seems to imply it's not real or not very good.  When it gets developed, there will be nothing artificial about it.

出处与译稿

来源平台
Sam Altman (personal blog)
发表时间
2015 年 2 月 25 日
译稿来源
机器逐句翻译 · qwen3.8-flash
审校状态
未经人工审校(reviewed = 0)
译文存档
正式档案库 · translation 表
原文存档
正式档案库 · segment 表
收录状态
STATE2 · 中文暂不可用

中文译文为阅读默认视图;引用与考据请以英文原文为准。外部链接仅作为来源引用保留。

回到正文开头