今天,OpenAI 发布了一项新成果(https://blog.openai.com/openai-five/)。我们使用 PPO(Proximal Policy Optimization,近端策略优化)——一种由 OpenAI 发明的通用强化学习算法——训练了一支由 5 个 agent 组成的队伍来玩 Dota,并击败了半职业选手。 Today, OpenAI released a new result (https://blog.openai.com/openai-five/). We used PPO (Proximal Policy Optimization), a general reinforcement learning algorithm invented by OpenAI, to train a team of 5 agents to play Dota and beat semi-pros. ¶
在我看来,在迄今为止 AI 取得过真正进展的所有游戏中,这款游戏最接近现实世界与复杂决策(结合了战略、战术、协同与实时行动)。 This is the game that to me feels closest to the real world and complex decision making (combining strategy, tactics, coordinating, and real-time action) of any game AI had made real progress against so far. ¶
我们训练的 agent 始终以 90—95% 的胜率战胜两周前版本的 agent。我们没有用人类对局来训练——当然,奖励函数是我们设计的,但算法是通过自我对弈训练学会怎么玩的。 The agents we train consistently outperform two-week old agents with a win rate of 90-95%. We did this without training on human-played games—we did design the reward functions, of course, but the algorithm figured out how to play by training against itself. ¶
这意义重大,因为它表明:只要你能投入足够的算力规模,并有一个真正出色、能刻画你所解决问题的模拟环境,深度强化学习就能解决极其困难的问题。我们希望很快用同样的方法去解决非常不同的问题。不难想象,它会被应用到越来越接近真实世界的环境中。 This is a big deal because it shows that deep reinforcement learning can solve extremely hard problems whenever you can throw enough computing scale and a really good simulated environment that captures the problem you’re solving. We hope to use this same approach to solve very different problems soon. It's easy to imagine this being applied to environments that look increasingly like the real world. ¶
世界上有许多问题复杂到根本无法手写解决方案。我预计这将成为机器学习的一个重要分支,也是通往通用智能之路上的重要一步。 There are many problems in the world that are far too complex to hand-code solutions for. I expect this to be a large branch of machine learning, and an important step on the road towards general intelligence. ¶