Digital Ventures

Back to blog

AI and the Learning Arena

DIGITAL VENTURES X CHAMP TEEPAGORN October 19, 2017 3:40 AM

2,581

Some may recall a 2011 movie which Hugh Jackman starred in Real Steel.

Although Real Steel was not so popular among critics (59% from Rotten Tomatoes) however the box office was quite good (almost 300 hundred million USD). Real Steel is the regular boxing storyline which tells about an underdog who was inferior in every way. However, in the end, with determination and persistence, he managed to win.

The difference was that Real Steel was not a boxing match between humans but of robots. In the movie’s year 2020, human boxing competitions weren’t popular any longer. The combat between robots has become a new trend that audiences prefer.

Today, although we are closing to the year 2020, we don’t see that future arriving just yet.

Well, that may not be completely true.

Recently, an AI research center with co-founder Elon Musk called OpenAI launched a combat platform for AIs. Two AIs were playing sumo in a virtual arena, the game was called, RoboSumo.

If you watch the game, you may find it uninteresting (visit https://blog.openai.com/competitive-self-play/). The sumo is simple 3D graphics which look like human stick figures while the arena is nothing more than a circular block with no fancy decorations. It may not seem innovative at all, however, RoboSumo’s intelligence is not at the interface but at the AI learning skills.

The AI was not taught to do anything special. They have human body structures but researchers didn’t teach them to even walk. Only simple rules like in sumo games such as pushing the opponent out of the circle were specified. When accomplished, they gain points while the one pushed out of the circle is defeated and lose points.

Following billions of trial and error, the AI began to learn human behavior. For instance, they began squatting to gain low center of gravity and make it more difficult to fall (similar to actual sumo poses). More importantly, the AI learned to trick its opponents. They would move to the edge to trick the opponent to head towards them and they would dodge to the side for the opponent to fall off the arena. Keep in mind that these behaviors are not coding but are AI self-learning abilities from the rules of the game as well as trials and errors.

Come to think of it, this is similar to animal’s evolution. Animals are not born with intelligence. Upon birth, birds can’t instantly fly and insects can’t instantly camouflage. It required several thousand or millions of generations of learning to gain the “intelligent” behaviors (or survival techniques).

OpenAI researchers explain that competitions (such as the RoboSumo) encourage AI to learn faster than its normal evolution (without competition). Maruan Al-Shedivat, an intern from the researching team, shared in an interview with wired that “When you (the AI) interact with other agents you have to adapt; if you don’t you’ll lose.”

Aside from RoboSumo, OpenAI also experimented on similar games to develop other skills. Examples are simple games wherein an AI must kick the ball into the goal while the other has to prevent the ball from entering. Another game requires AI to prevent the opposing player from running into their territory (these games teach movements such as charging or dodging to avoid opponents).

Interestingly, researchers found that AI can “transfer the skills” from one game to a different situation. For instance, the AI that learns to dodge from the sumo game may react to a virtual situation such as strong winds that are created by researchers. Once the wind blows towards the arena, the AI learns to dodge and avoid the wind.

A limitation of this research is that all the situations are within the virtual world. Transferring the learning skills of AIs from the virtual to actual worlds may require further studies and development.

Yet these may be the learnings we need to create multi-purpose AIs in the future.

For now, we may still need to wait for that Real Steel world.