While observing the fourth out of five games between Lee Sedol, who is a world-renown Go player, and an AI created by DeepMind, Fan Hui , another high level Go player, makes this observation. Fan Hui was handily beaten by the AI prior to this tournament. English is not his native tongue, but I do not think that the quote requires translation. This particular quote stuck with me because of its translation to trading; the level of patience required is high. You must sit, watch, and wait, even if it is cold, like Fan Hui's wolf.
AlphaGo is a computer program designed to compete in the highly competitive game of Go. There is a good chance that you may not have heard of this game. It is mostly popular in Korea and China. It originated in China 3,000 years ago. While the rules are simple, with the goal being to capture an opponent's pieces or surround empty space to gain territory, "Go is a game of profound complexity." Sound familiar? It should to any trader. The rules to the trading game are simple: buy low, sell high, or sell high and buy low. Trading is also a game of profound complexity however, because following those simple rules is a tantalizing challenge.
Another quote regarding the game of Go that can be applied to trading comes from DeepMind's website: "Go is played primarily through intuition and feel, and because of its beauty, subtlety and intellectual depth it has captured the human imagination for centuries." So, it appears we have some similarities between the game of Go and trading. Both are deceptively complex; in the game of Go, there are 10 to the power of 170 possible board configurations, which is more than the number of atoms in the known universe. In trading, the markets can display an almost infinite amount of configurations across time frames. Both games require intuition, which is generally derived from experience. The human brain is powerful and much information resides in the supercomputer that is our subconscious mind. In both Go and trading, when we see a situation that we have seen in the past but we can't consciously recall, we often get a gut feeling or a hunch. Or intuition. In simpler systems, we don't need to rely on this intuition as much. But with such a vast quantity of data to process from a more complex system, we need it.
One of the biggest factors that impedes traders across all markets, asset classes, and time frames, is the constant swim against the current that is human emotion. Human beings have evolved in certain ways since the birth of our species. We can run long distances as no other animal can. We are able to communicate in a much more complex manner than any other creature, leading to what has become a more and more interconnected world. One skill that we did not develop through evolution, however, was the skill of trading in the financial markets. As a matter of fact, for the vast majority of us, the way we are built leads to a wildly error-prone decision-making method. This is not news. The fact that we naturally lack skill at trading has been the subject of many books, blog posts, and papers.
As we continue to harness the power of technology, we are seeing computers become more and more prevalent in the trading world. The initial purpose of these trading systems was to eliminate the human decision making factor. A trader may have had a system that he was executing manually. Let's say that it was a trend-following system that got long when 3 exponential moving averages were below the market and moving higher and a key resistance level was broken. Nothing in the trading world works 100% of the time. And the nature of trend-following, in many cases, is that of many small losses and a couple of large wins that more than offset the losses. But after a trend-follower takes 10 losses in a row, is he going to have the gumption to stick to the system? What if he skips the next trade, which is the big win that he needed because he is scared of another loss? This is an all-too common problem in our world.
So what did the computer-savvy traders do? They simply took their rules, tossed them in a black box, and let the computer execute the trades. Generally, back-testing was done to reinforce confidence in the system. The trader might recognize that, indeed, 10 losses in a row are possible. But according to back-testing on a massive amount of data, that's fine and the system will still be profitable overall.
Coding clear-cut rules to trade the markets is not the daunting task that it once was. There are languages specifically designed for this purpose. But what if the rules aren't so clear cut? What if a trader is relying on his intuition in addition to some rules? What if a Go player is relying on his gut to help guide his decision-making? How can that be coded? How can you possibly code something that you can't explicitly quantify?
"...[AlphaGo] combines Monte-Carlo tree search with a deep neural network that have been trained by supervised learning, from human expert games, and by reinforcement learning, from games of self-play."
So what is a Monte-Carlo tree search and what is a deep neural network?
A Monte-Carlo tree search is simply the analysis of the most promising moves. The search tree expands based on a random sampling of the search space. Many games must be played in order for the tree search to be effective. The game, in this case, Go, is played out until the end using random moves. The final result is then used so we know which moves are going to be better in future games. We are trying to find the moves that have most frequently led to victories. Sound familiar to any traders reading this?
Above is a diagram of the 4 steps in the tree search for one decision. This shows the number of plays that were won by each color (white or black). In this scenario, black is about to move. The 11/21 in the root indicates the number of white wins our of the total plays from this position. Here, white lost and the black nodes get the wins. "This [process] ensures that during selection, each player's choices expand towards the most promising moves for that player, which mirrors the goal of each player to maximize the value of their move."
One major issue with using this method is that it can be applied to a game that has a finite number of moves and a finite length. So the tree search method may be useful for Checkers, Chess, or Go. But the markets are not finite in anyway, aside from trading limits. Anything can happen. One trader could trigger a cascading effect of stops being taken out, resulting in the market breaking down and making a laughing stock of any theories about normal distribution.
Another disadvantage is that when the game faces an expert player, like Lee Sedol, there might be a single branch that was missed during the tree search that leads to a loss for the AI. There is speculation that this is how Lee Sedol won game four. On a quick aside, what an absolute master of pattern recognition to be able to find that "branch."
An artificial neural network, or an ANN, is based on a collection of nodes which are called Artificial Neurons. This name comes from the fact that humans and animals have neural networks, albeit not "artificial." This is an image of what our actual neural networks look like:
And this is a diagram of an artificial neural network:
What the ANN does is improve its performance progressively. For example, an ANN might scan a chart that has been labeled "bull market" or "bear market." With no inherent knowledge about what either one of those things are, the ANN will come up with its own delineating features as to what constitutes each one.
AlphaGo used both of these methods to become a master Go player. Before AlphaGo, researchers believed that an AI would never be able to beat a top professional. As we know, that turned out to not be accurate. I have similarly heard it said that an AI will never be able to beat a top human trader. The difference between Go and the markets is that one is a finite environment and one is boundless. Will that make a difference in how good AI can get? Once an AI has adapted to market conditions, will it be able to adapt again as well as a human being? Can an AI have observed so many market conditions that it will know what move to make next based on a form of intuition?
Many traders know that their particular style only works under certain market conditions. Knowing this, the trader can choose to shut things down during market chop. An AI can certainly be programmed to recognize these conditions as well. The trader can also adapt to trade a new market condition, without necessarily being able to quantify the change. An AI can also do this, but needs a way to quantify whatever the change is.
I'm sure many traders reading this have gotten a feeling that "something doesn't feel right" before. You might be long S&P futures. You can't put your finger on it, but something is telling you to bail. You can tell that it isn't an impulse, but rather intuition (differentiating between separates the good traders from the rest, in my opinion), so you exit. Sure enough, the market plummets. What did you see that triggered this signal from deep within? Was it a certain 5-minute candle and how the DOM was trading? It could be any combination of things that you have seen in the past. The key point is that you can't say exactly what it was, but exiting the trade felt like the right move.
For an AI to be able to make a similar decision, it would have to have a highly developed ANN that has observed many, many market scenarios. It seems like it would be an overwhelming and nearly prohibitively expensive task to develop an AI to be able to do this. Nevertheless, with the amount of money that is poured into trading technology, we can be certain that this is being attempted.
I still believe a good human trader can beat any trading AI or algorithm. For now. We really don't know how far we can push machines, and it is possible that eventually they become better traders than us, using some form of "intuition."
Sources: AlphaGo documentary, deepmind.com, Wikipedia
Informative and helpful information.Thanks for sharing such a nice article. Very briefly explained I like all the content of this article.Algo Trading
ReplyDelete