Symbolic AI, Neural Networks and Deep Learning
Artificial Intelligence (AI) is as old as computer science itself. Calculations, logical deductions, complex assignments… all this was once restricted to humans, until computers came forth.
Even the performance of the early and (from today’s perspective) primitive computing systems seemed to imitate the human mind and it’s accomplishments. No wonder Alan Turing, pioneer of computer science, deeply pondered the concept of AI. With the so-called Turing Test he finally tried to establish a definition. Our own attempt at a viable definition brought us to a matrix we adopted from a standard reference:
| human | rational | |
|---|---|---|
| thought | I. It thinks like a human | II. It thinks rationally |
| action | III. It behaves like a human | IV. It behaves rationally |
Artificial general intelligence (AGI) vs. weak AI
In our last article we not only established a definition for AI systems, but also noted the constantly changing perception of AI: When Kasparov was defeated by Deep Blue in 1997 it was considered a triumph for AI. And indeed Deep Blue matched the requirements II and IV of our matrix beautifully: The “reasoning” of a chess program is completely rational, plus it “behaves” completely rationally, i.e. it chooses the moves that - based on the system’s knowledge - hold the greatest probability of winning. However, we are by now accustomed to unbeatable chess programs and consequently don’t perceive them as AI any longer. We know and understand that the program only calculates all “scenarios” and acts accordingly. The magical aspect is thus lost.
Today, the nature of the public discourse has changed and has nothing to do with these “ancient” chess programs. Using the term AI, the general public and media nowadays usually mean “artificial general intelligence” (AGI). The misleading coverage thus fuels unfounded ideas, expectations and fears, because all current AI systems are “weak” or “specialized” AI. But what is the difference between “strong” and “weak” AI?
The differences between strong and weak AI
To put it bluntly: Strong AI still belongs to the world of science fiction. It usually is at least as intelligent as a human being (or even superior to to it), is aware of itself and able to communicate using natural language. Furthermore it possesses “world knowledge”, i.e. it understands how the real world “works” on a physical, biological, but also social level.
A specialized or weak AI, e.g. a chess computer, a modern image recognition system or even Siri, is always an “one-track specialist”: Although it does it with super-human capacity, it is able to perform exactly one task. Apart from that it is incredibly limited. A chess computer can’t even play Tic-Tac-Toe. It doesn’t know what chess is, what the pieces look like or why you play chess in the first place. This limitation applies even to the most modern image recognition system: The system doesn’t know about cats and dogs, even if it can “recognize” them in images. It has no understanding of “above” or “below”, it doesn’t know about legs or fur. It merely filters complex patterns from images and classifies them, based on certain statistics. Chatbots, Alexa, Siri & Co. are nothing more than Potemkin villages: Human behaviour and understanding are simulated with much diligence and tricks. But as soon as you stray in the least from the predefined setting, these systems are utterly helpless.
| Strong AI (artificial general intelligence) | Weak AI |
|---|---|
| complex physical, biological and social world knowledge | no knowledge at all or enormously specific, database knowledge |
| self-awareness, reflectivity | no self-awareness whatsoever |
| Transfer of learning: found solutions can be applied to similar problems | able to only solve only one (or very few) tasks |
| permanently learning and adapting | learning (or programming) and application strictly separated |
| Science Fiction | State of the art technology |
Are Neural Networks and Deep Learning “strong AI”?
When we delve deeper into the functionality of various AI systems in the upcoming articles, we will always keep our focus on the actual state of research and thus on “weak” AI. Neural Networks can “fake” human behaviour, but we are not dealing with a strong AI. Despite the sensational media coverage, Neural Networks and Deep Learning do not create artificial brains. They are not “better” than the methods used for the programming of those chess computers. They simply work differently and open up new possibilities for digitisation. So what is new in the current AI hype?
Distinction between symbolic AI, Machine Learning, Deep Learning and Neural Networks (NN)
The mentioned chess programs and similar AI systems are nowadays termed “Symbolic” AI. Starting from a top-down approach they try to describe a problem and its parts deterministically with symbols, rules and representations.
The different approach: Deep Learning
“Deep Learning”- like all Machine Learning techniques - on the other hand takes a different approach and uses the bottom-up principle: The algorithm tries to describe a problem stochastically at a lower level, hoping for this to result in useful representations. Most Machine Learning systems proceed like this, Deep Learning being one of them. Deep Learning uses Neural Networks (NN), complex networks that consist of several layers and are therefore “deep”. In the current scientific discourse the terms Deep Learning and artificial Neural Networks are interchangable, so we will use them accordingly.
As the functionality of Symbolic AI is much more obvious than Neural Networks, the general public has recently become more and more accustomed to it. Neural Networks and their results still seem almost “magical” in comparison. While Symbolic AI seems to be almost common nowadays, Deep Learning evokes the idea of a “real” AI. Still we need to clarify: Symbolic AI is not “dumber” or less “real” than Neural Networks. The systems work completely different, have their specific advantages and disadvantages. They even both originated at the same time, the late 50ies.
How does Symbolic AI work?
To understand exactly how Symbolic AI works, we will briefly return to the chess computer. For the first moves of the game, the program accesses a database with (countless) prepared positions and chooses the “right” move. If the database offers no clear solution, the chess AI (in the background) runs millions of combinations of moves and chooses the scenario with the most promising outcome.
How does Deep Learning work?
A good example for a Deep Learning system would be an image recognition program: First it is fed with thousands of images and the information which object class (e.g. dog or cat) the respective image belongs to. With the help of a learning algorithm it adjusts the countless parameters of this layered system step by step - until the wanted results are reliably delivered. However, it is difficult to say exactly how the decision is made. With complex visualizations we can get a certain grasp about how individual layers react to increasingly complex image characteristics. But there is no clearly defined, comprehensible decision-making evident - a Neural Network often seems more like an artificial reflex than an artificial intelligence.
Distinction and Combination of different AI approaches
The field of AI is a wide one - the choice and application of specific algorithms or programming techniques usually determines if it is an AI. Based on our observations, the field of AI sector can be categorized thus:
Neither method is “more real” or “more correct”. The different approaches have specific strengths and weaknesses, qualifying them for different tasks. For modern and successful applications, techniques from Symbolic AI, Neural Networks and Machine Learning are usually combined with each other. Also, not every algorithm belongs exclusively to one camp: More often than not they are successfully intertwined.
Is Symbolic AI „wrong“ and Neural Networks „right“?
The question seems redundant as both approaches try to solve the same problem, albeit differently. Sometimes one approach works, sometimes the other. Admittedly, Symbolic AI has received more attention for a long time because it - to put it simply - yielded better results. Until recently Neural Networks were not as successful because they lacked the computing power, data and algorithmic knowledge to train the efficiently. AI research is also subject to currents and trends. For a long time, Neural Networks were simply frowned upon as an interesting but useless curiosity.
With the rise of the Internet and the ever-increasing amount of available data however, Machine Learning methods, including Neural Networks, became increasingly important since the late 1990s, resulting in the first “big data” boom. Predominant Machine Learning techniques such as Support Vector Machines (SVM) were from then on partially replaced by Deep Learning. In the public perception these seem to be omnipresent right now, solving problems computers were previously unable to solve.
However, since most successful AI applications combine different components, it is important to know the basic methods before exploring current developments. Therefore, the next article will concentrate on several Symbolic AI methods and explain to what extent they can already be useful in inducing “intelligent” behavior.