The future of AI between neural and symbolic approaches

The spectacular advances in massive language models have marked a decisive turning point in the recent history of artificial intelligence. Since the emergence of the "transformer" architecture in 2017, the ability of neural networks to process, understand, and generate text has reached unprecedented proportions. This technological revolution burst into the open with the arrival of ChatGPT, propelling generative AI to the forefront of the public and media scene. The operation of these models is based on a seemingly simple principle: they extend fragments of text (the "prompts") in the way deemed most likely from a huge corpus of textual data. This statistical continuation capacity makes it possible not only to produce credible answers, but also to invent stories, formulate hypotheses, and even write content in a wide variety of styles and registers.

The core of this intelligence lies in the deep assimilation of countless texts, thanks to artificial neural networks composed of billions of parameters. At the end of this massive learning phase, the AI ​​acquires a sort of "general knowledge" on a very large number of subjects. The answers it generates thus seem to be based on a real understanding of the themes addressed, while in reality, it functions as a powerful predictive model, optimized to determine the most plausible continuation of a sequence of words. This approach, called connectionist, historically opposes another tradition of AI, called symbolic, where the emphasis is placed on the explicit manipulation of logical rules and symbols.

To ensure the security and relevance of the generated content, deep learning is complemented by adjustment and filtering phases. This involves eliminating, as much as possible, the risk of deviations—inappropriate content, biased or erroneous responses, etc. This step, sometimes called "fine tuning," also aims to specialize the AI ​​in certain areas, to make it more efficient in specific contexts. Nevertheless, the bulk of an LLM's general skills come from the overall statistics of the texts absorbed during the initial training.

The performance of these systems is surprising, to the point that the famous Turing test—which evaluates a machine's ability to pass itself off as human during a written conversation—is now regularly passed. During brief exchanges, it becomes very difficult, if not impossible, to distinguish the responses of a conversational agent from those of a human interlocutor. This feat demonstrates both the scale of the data integrated by these AIs and the effectiveness of the transformer architecture for modeling language structures.

However, these models still encounter notable difficulties in certain areas. Their arithmetic proficiency, for example, remains inferior to that of traditional digital tools. It is not uncommon for a chatbot, even a sophisticated one, to provide an incorrect answer to a multiplication or a basic logical operation. This weakness is intriguing, because computers have long been able to perform exact calculations, even complex ones, as long as they are programmed for this purpose. The source of the problem is structural: generative language models are not designed, at their core, to execute precise mathematical algorithms. Their operation, focused on predicting text sequences, does not allow them to apply strict calculation rules, unless they have memorized enough examples during their training.

Various strategies have been explored to improve this ability. One of the first involves modifying the way numbers are divided into "tokens," i.e., into minimal processing units for the neural network. It turns out that number segmentation, while following the usual reading conventions, is not optimal for calculations. Reversing the order of the digits or dividing the numbers to the right can improve, to a certain extent, the AI's performance on simple operations. Other methods involve associating each digit with its position in the number, or limiting each token to a single digit, in order to simplify the model's internal manipulations. Intensive training phases, fueled by examples of exact calculations, can also occasionally improve performance on specific tasks.

However, even with these tricks, neural networks struggle to acquire true algorithmic mastery of arithmetic operations. They remain dependent on the statistical regularities present in their training data, and quickly show their limits as soon as the complexity or size of the numbers increases. We sometimes observe qualitative leaps during training, with a sudden improvement in performance on certain classes of operations. However, this progress remains circumscribed and fragile, far from the robustness expected of a traditional calculator.

To address this structural weakness, an increasingly widespread solution is to couple the generative model with external tools specialized in calculations. Some conversational agents, such as Gemini or Claude, integrate an external arithmetic module, capable of performing exact operations on large numbers or manipulating truth tables in formal logic. This hybridization makes it possible to obtain precise answers, without the AI ​​itself having to "understand" or learn the calculation algorithms. The use of such modules highlights a fundamental question, debated since the origins of AI: will purely neural approaches one day be enough to endow machines with general intelligence, or will we always have to rely on symbolic techniques for certain tasks?

This debate is rooted in the very history of the discipline. On the one hand, proponents of connectionism seek to mimic the functioning of the human brain, reproducing, through learning, skills derived from experience and exposure to large masses of data. On the other, supporters of symbolic AI rely on the logical and formal manipulation of statements, drawing inspiration from the way humans reason in a structured and explicit manner. Expert systems, which emerged in the 1970s and 1980s, illustrate this second path, with programs capable of diagnosing breakdowns, assisting doctors, or competing with the greatest chess players, using only coded rules and logical inference.

For several decades, symbolic AI dominated the scene, achieving resounding successes in well-defined fields. The Deep Blue program, which beat Garry Kasparov in 1997, is a prime example. But over time, the rise of neural approaches, driven by the rise of computing power and the availability of large databases, has made it possible to overcome the limits of pure symbolism. Today, the best chess or Go players must bow to AIs that learn primarily through experience and massive training, without resorting to explicit rules.

This shift from symbolic to neural has not been without questions. While neural models excel at handling ambiguity, recognizing complex patterns, and adapting to new situations, they suffer from notable weaknesses in three key areas. First, explainability: it is much more difficult to understand what a neural network has "learned," or to identify the causes of an error, than to audit a system based on explicit rules. The billions of parameters in an LLM form a veritable black box, complicating the task of researchers and engineers when it comes to explaining, correcting, or certifying the system's behavior.

Then there's the problem of answer reliability. Neural AI, especially in its generative form, can produce "hallucinations," that is, false or unfounded statements presented with disconcerting aplomb. Correcting these errors systematically remains an unsolved challenge. Unlike symbolic systems, where a rule can be adjusted or a knowledge base enriched, there is no simple, guaranteed method for correcting a neural network that has learned errors or developed biases.

The third point concerns energy efficiency. Training and operating neural models, especially LLMs, requires considerable resources. It is estimated that today, the share of global electricity consumption attributable to AI exceeds 1,5%, and this proportion is expected to double by the end of the decade. Dedicated AI computing centers consume as much electricity as hundreds of thousands of homes, raising major sustainability and environmental impact issues. Conversely, symbolic systems, based on specialized algorithms, are significantly less energy-intensive.

In response to these findings, a third approach has gradually emerged: neurosymbolic AI. This involves combining, within a single system, the learning power of neural networks and the formal rigor of symbolic methods. This rapprochement promises to combine the best of both worlds: the flexibility and adaptability of connectionism, with the explainability, reliability, and energy efficiency of symbolism. Renowned researchers, such as Gary Marcus, have been advocating for years for this hybrid approach, inspired by work on human cognition. Like the brain, which combines automatic processes (intuition, rapid recognition) and deliberate reasoning (logic, formal analysis), AI would benefit from integrating these two dimensions.

The initial results of this approach are promising. Introducing symbolic elements into generative models improves their ability to interpret context, respond accurately, and dynamically adjust to new tasks. Some prototypes of neurosymbolic systems are already demonstrating improved performance in mathematical understanding, logical manipulation, and the interpretation of structured data. The ultimate goal is to take the next step toward AI that is powerful, explainable, and reliable.

However, many questions remain about the scope of this approach. While generative AI has already contributed to solving mathematical theorems or discovering new scientific hypotheses, it is still far from replacing traditional methods of automatic demonstration or formal validation of proofs. AI research is actively exploring these frontiers, hoping to push the limits of what machines can understand and accomplish.

The future of artificial intelligence therefore promises to be both exciting and complex. The complementarity between neural and symbolic methods increasingly appears to be a necessity to overcome the weaknesses of each approach taken in isolation. This synergy paves the way for innovative applications, where AI no longer simply generates text or images, but becomes capable of interpreting the world, reasoning rigorously, and participating in demanding decision-making processes. The path to a general, reliable, and explainable artificial intelligence requires this convergence of paradigms and the continued integration of advances from both traditions.

Ultimately, the dazzling success of generative AI, based on neural networks, has profoundly transformed the technological, economic, and societal landscapes. They open up unprecedented perspectives in automation, creation, and access to information. But their limitations also remind us of the need for an ongoing dialogue between technical innovation, ethical reflection, and environmental responsibility. Only an integrated approach, combining the strengths of connectionism and symbolism, will make it possible to build artificial intelligence systems that meet contemporary and future challenges.

To learn more: consult the Model Context Protocol (MCP) and our article on how to get the most out of language models.