The sub-symbolic paradigm

Contrasting to Symbolic AI, sub-symbolic systems do not require rules or symbolic representations as inputs. Instead, sub-symbolic programs can learn implicit data representations on their own. Machine learning and deep learning techniques are all examples of sub-symbolic AI models.

Sub-symbolic models can predict some target objectives after extracting patterns from their input. Their training process is more significant than that of the manual symbolic process. With specific techniques, such as NNs, the developer does not even have to process the input data!

Sub-symbolic AI models can be scaled to more significant tasks and datasets effortlessly. Furthermore, sub-symbolic systems learn polytonic relationships, allowing for retraining and updating their previous knowledge. As such, sub-symbolic systems work well with non-stationary datasets. We tabulate the main differences between symbolic and sub-symbolic models as follows:

	Symbolic	Sub-Symbolic
Knowledge base	Manually defined symbolic rules and relations.	Automatic extraction using mathematical models.
Knowledge updates	It depends on the model complexity but is typically manually exhaustive.	Re-training of the model. Typically, an easy process but depending on use cases might be resource exhaustive.
Model development	A manually exhaustive process that tends to be rather complex to capture and define all symbolic rules.	Building the model and training is straightforward.
Missing data	Directly affects the performance of the model.	Can generally deal with missing or incomplete datasets.
Model upkeeping	A challenging and manual process.	Easy.
Model processing efficiency	Sequential evaluation of symbolic rules (slow).	Can be parallelized and scaled up (fast).
Result interpretability	Full traceability.	Ambiguous and complex to interpret.

Table 2.3: A comparison between the symbolic and sub-symbolic paradigms

Comparing both paradigms head to head, one can appreciate sub-symbolic systems’ power and flexibility. Inevitably, the birth of sub-symbolic systems was the primary motivation behind the dethroning of Symbolic AI. Symbolic AI quickly faded away from the spotlight. Funnily enough, its limitations resulted in its inevitable death but are also primarily responsible for its resurrection.

As we got deeper into researching and innovating the sub-symbolic computing area, we were simultaneously digging another hole for ourselves. Yes, sub-symbolic systems gave us ultra-powerful models that dominated and revolutionized every discipline. But as our models continued to grow in complexity, their transparency continued to diminish severely. Today, we are at a point where humans cannot understand the predictions and rationale behind AI. Take self-driving cars, for example. Do we even know what’s going on in the background? Do we understand the decisions behind the countless AI systems throughout the vehicle? Like self-driving cars, many other use cases exist where humans blindly trust the results of some AI algorithm, even though it’s a black box.

Symbolic AI provides numerous benefits, including a highly transparent, traceable, and interpretable reasoning process. So, maybe we are not in a position yet to completely disregard Symbolic AI. Maybe Symbolic AI still has something to offer us. Throughout the rest of this book, we will explore how we can leverage symbolic and sub-symbolic techniques in a hybrid approach to build a robust yet explainable model.