Automatic Observer in StarCraft. The idea here is to replace the humans who control the in-game "cameras" for esports audiences with an automated system. Rule-based automated systems already exist, but this system uses (Mask R-CNN) to turn video of the gameplay into objects, and this is then fed into a supervised learning system that uses camera control from humans as its training data. So the idea is it makes an automated system that controls the in-game cameras like humans.

In this example video from StarCraft, it did seem to me like the Automatic Observer did a better job of "anticipating" action and showing it coming, for example by showing attackers marching towards a defended position before they get there, while the rule-based system would jerk you over to the action after it had already started.

Automatic Observer using Human Observing Data in StarCraft

#solidstatelife #ai #computervision #esports