Hi all! On May 11, 1997, a unique event occurred when the Deep Blue computer, developed by IBM, won a 6-game match against world chess champion Garry Kasparov. Since then, many companies have begun to invest in the development of artificial intelligence, which can demonstrate its skills in such situations. In this article I will talk about six projects where the computer again prevailed over humans.
Libratus and Texas Hold’em
In 2017, scientists from Carnegie Mellon University in Pittsburgh decided to create artificial intelligence that could beat professional Texas Hold’em players. They immediately abandoned the analysis of emotions and gestures, since there are no technologies yet capable of implementing such complex algorithms. Therefore, we decided to calculate various combinations and bets.
Each move was analyzed separately, after which artificial intelligence considered the possible consequences. If the opponent made an unexpected maneuver, the program added it to its database and in subsequent games could apply it itself in one of the situations.
This artificial intelligence was called Libratus, and to form strategies for all sorts of situations, Libratus developers used the Bridges supercomputer worth almost $10 million. Bridges played against himself day after day, creating certain winning sequences through trial and error.
When the program was completely ready, four leading players were invited to play as an exhibition match: Jason Les, Dong Kim, Daniel Macauley and Jimmy Chow.The tournament lasted from January 11 to January 31, 2017, the prize fund was $200,000, of which 10% was guaranteed to go to each player. During the tournament, Libratus competed with players during the day, and at night he improved his strategy by analyzing past results. In this way, he could constantly correct the deficiencies found.
By the end of the championship, the artificial intelligence beat the humans, earning $1,800,000 worth of chips, but due to the rules, Libratus himself did not receive any prize money. After the tournament, Dong Kim said in an interview that the longer the game went on, the more it felt like Libratus could see his cards.
BotPrize and Unreal Tournament 2004
The next tournament where artificial intelligence defeated a human is called BotPrize, but it is quite unusual. The first-person shooter Unreal Tournament 2004 was used as a test platform . Developers participating in the tournament must provide created bots that will try to impersonate people. The bot that gets mistaken for a human a certain number of times wins.
The competition was sponsored by 2K Games and called BotPrize. It was officially registered in 2008 and was an adaptation of the Turing test. During it, people and bots anonymously enter into deathmatch mode, the goal of which is to kill as many opponents as possible within a given period of time.
Unreal Tournament 2004 was chosen for a reason. The developers had to take into account not only movement in three-dimensional space, but also frequent fights with several opponents at once. The judges were live players who observed and marked anonymous people as people or bots. If more than 30% of the judges call the bot alive, then it wins the competition. All developers were on equal terms. Before the competition, they had to use the programs GameBots 2004 and Pogamut 3, and the latter contained ready-made bot samples.
And in 2012, two teams won the tournament at once. The UT^2 bot was rated as a living person, reaching 51.9% of the votes. People from the University of Texas worked on its creation, namely: Professor Risto Miikkulainen with doctoral students Jacob Shrum and Igor Karpov.
Another bot from Romanian programmer Mihai Polceanu was called MirrorBot. Unlike his opponent, he was a little more lively and received 52.2% of the votes. Interestingly, real players showed a result of 41.4%.
Both teams said that they had to limit their bots so that they did not become fearsome machines to destroy all opponents. UT^2 reduced the accuracy of hits by increasing the speed of movement and shooting from long distances.
But MirrorBot could imitate the actions of another player in real time. He did this with a slight delay and without complete accuracy, so that from the outside they seemed different. He could remember his target for a while, follow it and dodge, depending on the direction of the enemy’s fire. The bot also forgot about its primary target if another opponent attacked much more aggressively.
IBM Watson and Jeopardy!
After Deep Blue’s victory over Kasparov, IBM was looking for a new challenge. Computers cannot quickly provide clear answers and provide thousands of search results matching keywords. And then IBM employees eventually came up with the idea that it was necessary to create a machine that could answer Jeopardy quiz questions!, which in Russia is adapted to the “Own Game” program.
The point is that on Jeopardy! the program must first decipher the natural human language in the question, figure out what is being asked, and then give the correct answer. Moreover, the shorter the question, the more difficult it is to find the correct solution. Difficulties may also arise if, on the contrary, the program must ask a question in response to the received statement.
By 2006, developers had formed a computer called Watson, but it was very slow and gave about 10% correct answers. The team was given 5 years to correct errors. By 2010, he could already compete with live players, and for 2011 they decided to schedule a demonstration show with famous winners of past matches.
And between IBM and Jeopardy! conflict situations often arose. For example, the show’s management believed that Watson would use the answer button much faster than a normal person. They even suggested making an avatar with a mechanical arm to somehow slow down the process.
But no changes were made, although even the Watson project manager said that the answer button would be activated immediately after the last syllable of the question was read. But at the same time, if Watson does not know how to answer, then the panel will remain untouched.
Actually, how does the processing process take place inside the machine?? First, more than 100 algorithms analyze the question and find many plausible answers. Another set of algorithms sorts them and sets up relationships that prove or disprove the result. Ultimately the answer with the best score will be returned.
During the game, Watson was disconnected from the Internet, but had access to 200 million pages loaded on four terabytes of disk space. Although Watson often answered correctly, he was not perfect. Some questions either confused him or he simply couldn’t find the right answer.
At the end of the show, Watson came out on top with $1 million donated to charities. IBM showed that their machine can quickly recognize a question in English and provide the required answer with high accuracy. This success allowed Watson to be tested in medical institutions in 2013 to improve the quality of staff work, but that’s a completely different story.
AlphaGo and Go
Go is one of the oldest board games. It was believed that a computer is not capable of playing on equal terms with a professional player due to the impossibility of trying out all the available development options. DeepMind decided to challenge this statement and began to develop the AlphaGo program.
The developers had to create an algorithm https://nossaapostacasino.co.uk/bonus/ that imitates human intuition in order to avoid the usual recalculation of combinations. The program was shown 100 thousand games downloaded from the Internet. At first she was forced to imitate human actions, and then she played against herself many times.
As a result, it turned out that AlphaGo began to evaluate the positions of stones on the board and calculate the percentage of victory. When the program chose the most optimal option, it considered further possible combinations in response to its move. Thus, AlphaGo was thinking ten moves ahead.
Percentage of possible events
Percentage of possible events
The interesting thing is that the program did not try to earn as many points as possible, but reacted to the probability of its victory. AlphaGo could easily sacrifice a few points if it could still beat its opponent in the future. Therefore, in some matches she could win with a difference of 0.5-2.5 points.
When AlphaGo beat European champion Fan Hui in 2015, the Go community was ambivalent about the news. Many said that Fan Hui is too weak and these victories are not an indicator of the superiority of artificial intelligence in the game against humans. Lee Sedol, one of the strongest players in Go history, commented that AlphaGo reached the highest amateur level in this match, but not yet professional.
DeepMind decided not to say goodbye to Fan Hui and offered to help him find defects in AlphaGo. He agreed and helped them correct a number of mistakes before the match against Lee Sedol, who was confident of winning 5:0 or 4:1.
The competition took place in March 2016, and all five matches were broadcast in Korean, Chinese, Japanese and English. In total, more than 60 million people from different countries watched the game.
Lee Sedol lost his first three matches. After the second game he said that if he was surprised by the first defeat, then after the second he had no words. In the fourth game, he managed to beat AlphaGo by performing a unique move that even other professional players would not have made at that moment.
AlphaGo also did not expect this maneuver and estimated its occurrence with a probability of 0.007%. After that, she began to re-calculate her moves and made a number of mistakes.
Unfortunately, Lee Sedol lost the final fifth match as well. According to him, he discovered new moments for himself, and his victory in the fourth game gave him such positive emotions that he would not exchange them for anything.
In 2017, there were more matches against a live player. This time they invited Chinese professional player Ke Jie, who was ranked first in his country. After Lee Sedol’s defeat, many players wanted to play with AlphaGo, but they chose Ke Jie, who claimed at the 2016 games that he could defeat the program.
The first three training matches were held online in winter, and Ke Jie lost in all. However, he was still confident that he would win at the official meeting in May. But a miracle did not happen, and he also lost the subsequent games, although the first of them ended with a difference of 0.5 points.
Before the final game, AlphaGo played more exhibition matches. First paired with a live player against the previous version and another live player, and then immediately against five. The result was the same everywhere. The new version of AlphaGo won all the matches against its opponents.
From calm to disappointment.
From calm to disappointment.
Throughout the games, the program showed the ability to make creative decisions, which surprised many players. Some moves contradicted the classical theory of Go, but they proved their effectiveness, and some professionals even began to use them in their games.
Alas, new matches with live players were no longer organized. Part of the team moved on to new projects, and they decided to use the developed AlphaGo algorithms in medicine. This led to DeepMind entering into an agreement with the UK’s National Health Service to apply artificial intelligence to medical data analysis.
AlphaStar and StarCraft 2
After developing programs for turn-based games, DeepMind decided that as a new project it was worth creating artificial intelligence that would beat professionals in the computer game StarCraft 2. Its difference is that it is not turn-based, but actions take place in real time, that is, players simultaneously make decisions.
To train such artificial intelligence, the DeepMind team asked Blizzard for anonymous game data of people participating in StarCraft 2 competitions. After this, the developers loaded the resulting materials into their new program called AlphaStar.
Based on them, artificial intelligence was able to imitate the basic micro- and macrostrategies that are used in StarCraft 2. Then additional development began through simulations of many battles, on the basis of which AlphaStar developed winning strategies. Due to the large amount of data processing, the developers decided to limit themselves to only one of the three races available in the game, namely the Protoss.
A player with the nickname TLO was invited to the first match. He was one of the best professional Protoss players. To his surprise, he lost 5:0. TLO noticed that AlphaStar demonstrated strategies that he had never thought of before. A week later, they invited one of the 10 strongest Protoss players with the nickname MaNa, who also lost with a score of 5:0.
The matches were held according to all professional rules on a special tournament map and without any restrictions. But at the same time, it was noticed that AlphaStar could observe its own and enemy units without moving the camera, that is, the program fully used the reduced game map. And as soon as the developers limited the scope of visibility, like a human, the artificial intelligence immediately began to process the data incorrectly.
Unfortunately, for now, even minimal changes in the rules can affect the work of AlphaStar. If new objects or units are added, the program will have to start training all over again, since it will not be able to immediately adapt to new conditions. Therefore, the DeepMind team still has a long time to work on such a difficult project.
Eurisko and Traveler
In the history of the BattleTech series, I already talked about the Traveler board game. It is a science fiction role-playing game first published in 1977. In it you can travel between different star systems and participate in activities such as exploration, land and space battles, interstellar trade. Tournaments were often held on it, where players figured out how to build a space fleet that would defeat all enemies without exceeding an imaginary defense budget of one trillion credits.
It would seem that since some of the mechanics are not used in competitions, then everything would be quite simple. But that’s not true at all. Players had to consider various factors: how thick the armor was, how much fuel to carry, what types of weapons, engines and computer guidance systems to use.
A powerful engine will make the ship faster, but this may require more fuel, increased armor provides protection, but increases weight and reduces maneuverability. Since a fleet can have up to 100 ships, the number of variables is enormous even for a digital computer.
However, this did not stop Douglas Lenat, an assistant professor in the artificial intelligence program at Stanford University. He wasn’t a fan of Traveler and had never even played it himself. But he always wanted to create a special artificial intelligence that could perform various difficult tasks.
At his disposal was the Xerox PARC research center, where for about a month, for ten hours every night on a hundred computers, a program called Eurisko processed various data to create a victorious space fleet.
As initial data, Lenat loaded an extensive set of rules into the program. Eurisko formed flotillas and staged virtual battles from two to thirty minutes. Every morning Lenat came and studied the data received.
At first everything was bad. Most of the strategies looked impractical or ridiculous. Several times Eurisko suggested changing the rules in order to defeat an opponent. But Lenat was persistent, he corrected the data and then started the emulation again
As a result of long analyses, Eurisko produced the most suitable result, which surprised absolutely everyone at the tournament. The program believed that a swarm of cheap and protected ships would survive and defeat an expensive fleet consisting of maneuverable, heavily armed vessels.
There were ninety-six ships in the formation provided, most of them slow due to their heavy armor. Firepower was based not on a pair of large expensive guns, but on a dozen cheap guns.
Besides this, there were two more trump cards. If Lenat’s fleet was still on the verge of destruction, then a maneuverable rescue capsule would come to the rescue, which could evade all shots over subsequent turns until the enemy flotilla was deprived of fuel and ammunition.
If this tactic were used against Lenat, then in his fleet there was a ship equipped with a complex guidance system and a giant accelerator. Its sole purpose was to destroy enemy lifeboats.
An example of a ship from the tabletop version
An example of a ship from the tabletop version
Lenat approved the choice of Eurisko and went to the tournament. At first, everyone laughed at his fleet and believed that he would lose in the first round. However, after a series of victories, players began to complain that Lenat was cheating and should be disqualified. The judges were unhappy, but the basic rules were not violated, so Lenat was allowed to continue playing, after which he became the winner of the tournament.
The next year, Lenat decided to consolidate his success and show that his victory was not an accident. The situation has become much more complicated. The judges decided to adjust the rules a few days before the tournament. From now on, the fleet will not be able to win unless it is extremely mobile.
Lenat managed to download the new rules, and Eurisko amazed the tournament participants even more. Artificial intelligence has issued a new strategy, according to which critically damaged ships should self-destruct or be fired upon by neighboring friendly ships. Thus, the overall mobility parameter increased significantly and the fleet could easily dominate the enemy.
Without any problems, Lenat won the new championship again. Of course, the tournament judges were dissatisfied with Lenat’s methods. In their opinion, his strategy is too cruel and immoral. If this had happened in real conditions, then none of the military leaders would have sacrificed thousands of their soldiers. The judges also threatened to close the next tournament if Lenat took part in it again.
Douglas decided not to become an even bigger enemy of the Traveler community and agreed to the condition that he would no longer compete. The initial goal was successful, and society saw that artificial intelligence is capable of analyzing numerous variations in such a complex game, which means these developments can be used to solve real problems.
Artificial intelligence still has a very long way to go until it can independently solve various problems. The considered examples showed that even despite the victory over humans, programs have significant drawbacks.
However, these projects gave a good impetus to development in this area. Perhaps in the next decade we will still see small tournaments in some games with the participation of several computers with artificial intelligence.