AAAI Publications, Fourteenth Artificial Intelligence and Interactive Digital Entertainment Conference

Font Size: 
Modular Architecture for StarCraft II with Deep Reinforcement Learning
Dennis Lee, Haoran Tang, Jeffrey O. Zhang, Huazhe Xu, Trevor Darrell, Pieter Abbeel

Last modified: 2018-09-25


We present a novel modular architecture for StarCraft II AI. The architecture splits responsibilities between multiple modules that each control one aspect of the game, such as buildorder selection or tactics. A centralized scheduler reviews macros suggested by all modules and decides their order of execution. An updater keeps track of environment changes and instantiates macros into series of executable actions. Modules in this framework can be optimized independently or jointly via human design, planning, or reinforcement learning. We present the first result of applying deep reinforcement learning techniques to training a modular agent with selfplay, achieving 92% or 86% win rates against the ”Harder” (level 5) built-in Blizzard bot in Zerg vs. Zerg matches, with or without fog-of-war.


reinforcement learning; real-time strategy games; starcraft

Full Text: PDF