How does RAG work in generative AI?

Large Language Models (LLMs) have revolutionized the field of artificial intelligence with their prowess in understanding and generating text.

Despite their power, these models face challenges such as the production of unfounded content (hallucinations), a lack of up-to-date knowledge, and opaque and difficult-to-trace reasoning processes. It is in this context that the concept of " Generation Augmented by Recovery » (Retrieval-Augmented Generation, RAG) has emerged as a promising solution.

The RAG represents a key development in the approach to LLMs, integrating knowledge from external databases.

This method not only improves the accuracy and credibility of models, but is particularly beneficial for tasks requiring a high level of knowledge. RAG allows models to remain constantly up-to-date and incorporate domain-specific information, thereby merging the intrinsic knowledge of LLMs with the vast and dynamic repositories of external databases.

Summary

• explore the evolution of RAG paradigms, including Naive RAG, Advanced RAG, and Modular RAG,
• while inspecting the foundations of these frameworks: recovery, generation and augmentation techniques.
• We will discuss the cutting-edge technologies underlying these essential components, providing an in-depth understanding of advances in RAG systems.
• We will also look at metrics and benchmarks for evaluating RAG models, as well as the most recent evaluation framework.
• Finally, we will discuss avenues for future research, including the identification of constraints, the expansion of multimodalities, and the progression of the RAG infrastructure and ecosystem.

1/ Definition

The Augmented Generation by Recovery » (Retrieval Augmented Generation (RAG) is an innovative artificial intelligence methodology that enhances the text generation capabilities of language models by combining them with an external search mechanism. At the heart of RAG lies the idea of ​​combining the intrinsic skills of a language model (NLP: Natural Language Processing) with the ability to draw upon a vast and up-to-date body of factual knowledge. This results in a synergy where the model not only relies on what it has "learned" during its training phase, but can also retrieve information from an external database to enrich its output.

Unlike traditional language models which generate responses by relying exclusively on their prior training on large corpora of text (GPT3,5, his training, is the equivalent of reams of paper stacked 100km high... GPT4, we don't know, but probably even more), RAGs integrate an additional information retrieval step. They first search for relevant information in response to a query, then use that data to guide text generation. This approach allows RAG models to produce responses that are not only contextually appropriate but also factually correct and up-to-date.

This distinction is crucial because it solves some of the thorniest problems associated with LLMs, such as generating inaccurate content or difficulty staying up to date with the latest information: this is called hallucination. RAGs then offer a path to more reliable and verifiable text generation, opening the door to more demanding and context-sensitive applications.

2/ Limitations of LLMs

We could write an entire book on the limitations of an LLM (or applications that use them) but let's get straight to the point in the context of the RAG: these LLMs have made possible complex conversations and AI-generated responses that seem natural and human but by testing them, apart from the bluffing side of the form and content, this This last point suffers from limitations which make many practical applications inconvenient.

Generalization and Contextualization

One of the major limitations of LLMs is their tendency to overgeneralize. These models, although trained on large bodies of data, can fail to capture the specific contextual nuances of a given situation. They may provide generic answers that, while grammatically correct, lack relevance to the user's specific needs.

An LLM is a model (of language), not a knowledge base (of rules, facts, etc.). Furthermore, there is no intelligence in an LLM. It is therefore incapable of " take a step back", or to guess the context a priori. GPT3, 3.5, 4 and then 4-turbo nevertheless remain incredibly impressive on our scale. This is due in particular to the arrival of a technique, Transformers, with a paper from the Google Brain teams in June 2017: Attention Is All You Need"It is not the purpose of this post to explain how it works. The fact remains that it is a jump into a new dimension"Natural Language Processing (NLP) has tried various methods of understanding and working with language for 70 years, without ever coming close to what a GPT4-turbo can do today."I take GPT4, but obviously, Llama, Falcon, Claude, Mistral etc are of equivalent nature, even if outside specialization, GPT4 is 'a step ahead')

See in the diagram below a quick timeline of NLP over many years, where grammar theory, symbolism, and statistics have been explored. Neural networks, however, have been around for 20 years in NLP!

Management of Factual Information

LLMs also struggle with factual accuracy. Their "one-size-fits-all" approach to generating responses can lead to what are called "hallucinations" (anthropomorphism, when you hold us…), where the model generates information that seems plausible but is actually incorrect or fabricated. This trend can be particularly problematic in fields where accuracy is crucial, such as medicine or law.

In this context, we find, for example, the recent famous case of an American lawyer having a situation analyzed by GPT-4, which responded very credibly that the proposed strategy was not only viable but had already been implemented in various cases: unfortunately for him (and his client), everything had been invented by the LLM (source)

3/ – Examples of Errors in all directions

LLMs suffer from a lot of bias, we are not going to display them or make a complete inventory (Personally, I've already counted more than 50…) and we will focus on those for which a RAG could be useful.

This paragraph is interesting for two reasons: it allows both to identify biases but also understand why These biases exist. Not everything is due to the training knowledge base (TKB): if a TKB has a bias, the application will see that bias reproduced. If an application has a bias, it's not necessarily the TKB that's at fault.

The examples of bias are offered above in the following format:

• the type/concept/principle of bias
• an example
• the explanation of the reason for this bias.
• how RAG is a solution

3.1/ Factual hallucinations

Example: Ask GPT-4 when the "London Eiffel Tower" was built and get a precise date in response.

ExplanationThe model can generate plausible details based on linguistic patterns even if the information is factually incorrect. It combines known elements (Eiffel Tower, London) to create a new "reality".

Solution: A RAG can verify factual accuracy by consulting external databases to correct or reject incorrect statements before the text is generated.

3.2/ Response out of context

Example: Request advice for an outdated operating system and receive instructions for a modern version.

Explanation: The model often favors more frequently discussed information in the training corpus, ignoring rarer contextual cues.

Solution: By integrating a data retrieval process, a RAG can search and use information specific to the context of the obsolete operating system mentioned, thus providing a relevant answer.

3.3/ Confirmation bias

Example: Falsely confirming that a popular false claim is true because it is often repeated online.

Explanation: The model tracks the frequency of training data, reinforcing frequent statements without verifying their veracity.

Solution: With access to external and up-to-date sources of information, a RAG can confront popular claims with verified facts to avoid reinforcing confirmation bias.

3.4/ Presentation of opinion as fact

Example: Describe a controversial opinion as a widely accepted consensus.

Explanation: Without discerning the nuances between opinion and fact, the model can present polarized opinions as established truths.

solution: A RAG differentiates opinions from facts by retrieving data from credible sources, allowing opinions to be tempered and presented as such rather than as proven facts.

3.5/ Reproduction of stereotypes

Example: Associate stereotypes with specific groups when describing characters or situations.

Explanation: Models learn patterns in existing data, including stereotypes and social biases.

solution: By accessing a variety of sources and perspectives through data retrieval, a RAG can avoid the reproduction of stereotypes by offering more nuanced and diverse representations.

3.6/ Unintentional plagiarism

Example: Generate text that is very similar to an uncited source.

Explanation: By leveraging often-used phrases and structures, the model can reproduce existing content.

solution: A RAG can limit the risk of plagiarism by retrieving information from multiple sources, thereby reducing reliance on specific phrases or structures from a single source.

3.7/ Lack of update

Example: Providing outdated information on topics that have seen recent developments.

Explanation: Models are trained on datasets that may be dated and do not reflect the most current information.

solution: RAGs can draw on information extracted from databases updated in real time, ensuring that the knowledge embedded in text generation is current.

3.8/ Overestimation of confidence

Example: Give answers with certainty on speculative or unknown subjects.

Explanation: Models may generate responses with false confidence due to the tendency to imitate the confident tone of authoritative sources.

solution: RAGs can moderate the confidence of their responses by retrieving and integrating data that reflects the current state of knowledge on a given topic.

3.9/ Literal interpretation of metaphors

Example: To take an expression like "to have a broken heart" literally and suggest medical treatment.

Explanation: Models may lack understanding of figurative language or idiomatic expressions.

solution: A RAG is capable of querying databases to find the common meaning of metaphorical expressions, allowing for a more nuanced and less literal interpretation.

3.10/ Weak narrative coherence

Example: Tell a story with characters and events that change inconsistently.

Explanation: LLMs may struggle to maintain coherence over long passages, leading to fragmented or contradictory narratives.

solution: With data recovery, a RAG can ensure narrative coherence by relying on external information and contexts to maintain story logic and continuity.

And there are many other biases as well.

4/ Techno and ethical constraints of RAG

In addition to content errors, LLMs must overcome technical challenges, such as the need for vast amounts of computing power for training and inference. They are also faced with ethical issues, such as how they handle sensitive data and how they can be used to generate misleading content or disinformation.

These limitations require more sophisticated models capable of understanding and processing information in a more precise and contextual manner. In these cases too, the RAG can offer a way to overcome some of these limitations inherent to LLMs.

Indeed, from a technical point of view, although the use of RAG always requires a certain computing power for data recovery and integration, it can potentially reduce the overall computational burden by limiting the need to generate answers entirely from scratch. By leveraging external information, RAGs can offer more accurate answers with less processing effort compared to an LLM operating in isolation.

Ethically, integrating RAG presents an opportunity to more effectively manage sensitive data and counter misinformation. By accessing verified and trusted databases, RAG models can mitigate the risk of spreading misleading content or false information. Additionally, the ability of RAGs to draw on up-to-date and varied sources of information can contribute to a more balanced and less biased representation, thereby addressing ethical concerns related to bias and fairness.

A work proposal for researchers coupled with journalists : a factual, neutral knowledge base, with explanations (or even a lawyer for the "for" side and a lawyer for the "against" side.) would allow for example, associated with a reading of facts (journalistic type, political speeches etc.) to be able to fact check these facts almost instantly in the light of this knowledge base. (Probably the resulting consequence would be the total discrediting of political discourse, but hey, that's another subject isn't it... )

5/ RAG, how does it work?

The RAG architecture mainly consists of two components:

the recovery component (retrieval) and

the generation component (generation).

This two-dimensional structure allows RAG models to take advantage of both the large data sets available and the advanced text generation capabilities of LLMs.

The "Recovery Component"

The recovery component is the heart of the RAG approach. It is designed to query external databases, information repositories or document indexes to find relevant information in response to a given query. This process leverages advanced information retrieval techniques, such as keyword matching, semantic analysis, and contextual filtering, to ensure that the data retrieved is as accurate and relevant as possible.

1. Keyword Matching:This method involves identifying and selecting information based on specific words or keywords. For example, if you are looking for information on "solar energy," the system will search for and retrieve documents containing those exact terms. It's a simple and straightforward technique, but it can sometimes lack nuance in understanding context or synonyms.
2. Semantic Analysis:Semantic analysis goes beyond simple word matching to understand the meaning and significance behind the words used. It allows the system to recognize that terms like "solar energy" and "photovoltaic" are related, even if they are not identical. This method helps capture more relevant information by understanding the context and linguistic nuances.
3. Contextual Filtering:Contextual filtering assesses the environment in which keywords are used to determine the relevance of the information. For example, it can distinguish whether "Apple" refers to the electronics brand or the fruit, depending on the document's context. This approach refines search results by focusing on documents that best match the desired context.
4. Recognition of Named Entities:This technique identifies and categorizes important elements in a text, such as people's names, organizations, places, dates, and so on. For example, it can recognize "New York" as a place or "Apple" as an organization. This helps to sort and organize information more efficiently, enabling more targeted and relevant searches.

Obviously, there are still other techniques and all of them can be combined.

The "Generation Component"

Once the fetch component has extracted the relevant information, it is passed to the build component. The latter is often a pre-trained language model, similar to those used in traditional LLMs. Its task is to synthesize the retrieved information into a coherent and fluid response. The generation component uses the retrieved information as a factual basis to construct its response, which improves the accuracy and relevance of the generated text.

Interaction between Recovery and Generation

The interaction between the recovery and generation components is crucial. The process is not linear but rather iterative. The generation model can request additional information or clarification from the recovery model, creating a two-way information flow. This approach allows for dynamic adaptation to the user's query, ensuring that the response generated is not only relevant but also informed by the most current and accurate data.

Optimizing the RAG architecture sometimes involves joint training of the recovery and generation components. This allows the system to learn how to better use the retrieved information to improve the quality of text generation. Over time, the system becomes more adept at determining what information to retrieve and how to effectively integrate it into the generation process. In the latter case, we are already in a framework of advanced techniques.

Mechanics of Retrieval and Information Integration in RAG

Triggering Recovery

The process begins with a user request, which is first analyzed by the build component to understand the intent and key details. This initial understanding is then passed to the recovery component. Here, the query is transformed into search terms or semantic queries, designed to efficiently query the database or document index.

Search and Selection of Information

The retrieval component uses various methods seen before to find the most relevant information. It can also leverage machine learning techniques to improve the relevance of search results based on previous returns and the context of the query.

Processing of Retrieved Information

Once the relevant information is retrieved, it is preprocessed to be integrated into the generation process. This may involve condensing data, eliminating redundancies, and structuring the information in a way that makes it easier for the generation component to use.

Integration into Text Generation

The generation component then receives this structured information and uses it as a framework to formulate its response. It embeds them in the context of the original query, using its natural language processing capabilities to generate a coherent and contextually appropriate response.

Iterative Review and Improvement

The process does not end with the first generation of text. The RAG system can evaluate the relevance of its response and, if necessary, initiate the retrieval process again to refine or complete the information. This iterative loop ensures that the final answer is not only relevant but also as informative as possible, reflecting a deep and current understanding of the query asked.

Ultimately, the RAG can function as a dynamic, interactive system, where data retrieval and generation work in concert to produce accurate, contextualized responses. This synergy between retrieval and generation is the key to overcoming the limitations of traditional language models, providing a richer, informed response tailored to the user's specific needs. (reliability, real time, etc.)

Note that this presentation that I am giving you here is generic, as there are so many methods, algorithms, concepts, objectives or architecture for setting up a RAG. Subsequently, simple to slightly more advanced examples will bring this to life in a pragmatic way.

6/ Use Case

Customer Service

In customer service, RAGs can provide quick and accurate responses to customer queries, drawing from an extensive database of FAQs and product documentation.

Academic Research

For academic research, RAGs can help synthesize information from multiple scientific sources, facilitating literature review and analysis of complex data.

Content Writing

RAGs can be used to generate informative and relevant content, such as blog posts, by incorporating up-to-date data and verified references.

Medical Assistance

In the medical field, RAGs can help provide preliminary advice based on described symptoms, referring to medical databases for precise information.

Education and formation

RAGs can be used to create personalized educational materials, tailoring explanations and examples to the specific needs of learners based on educational data.

Social Media Management

RAGs can automate and personalize content creation for social media, generating relevant and engaging posts based on current trends and user interactions.

Juridic assistance

In the legal field, RAGs can assist in the analysis of legal documents and the provision of preliminary advice drawing on databases of laws and case law.

Translation and Localization

RAGs can improve the quality of translations by contextualizing content according to cultural and linguistic nuances, drawing on multilingual databases.

Planning and Logistics

RAGs can optimize logistics planning and management by integrating real-time data on traffic, weather, and supply chains for more informed decisions.

Video Game and Entertainment Development

RAGs can be used in video game development to create dynamic storylines and interactive dialogue, drawing on a wide variety of narrative sources and gameplay data.

In all these cases, as you understand, it is actually a question of using the most reliable knowledge (true, real, recent, certified, etc.) in the form of natural language through an application while using the power of a new generation LLM (GPT-like)

7/ Advantages and Constraints of RAG

Advantages of RAG

1. Increased relevance of responses : Thanks to access to external data, RAGs offer more precise and contextually adapted answers, surpassing the capabilities of traditional LLMs.
2. Ability to stay up to date : RAGs may refer to current information, ensuring that responses reflect the latest available data.
3. Flexibility and Adaptability : RAG's ability to integrate diverse sources allows greater adaptability to different contexts and queries.
4. Improved reliability : By verifying information from external sources, RAGs reduce the risk of spreading misinformation or bias.

Remaining constraints

1. Integration of reliable information sources : Identifying and integrating verified and up-to-date databases represents a major challenge to guarantee the quality of the responses generated.
2. Knowledge database management : Maintaining and updating the databases used by RAGs requires ongoing technical resources and management.
3. Balance between precision and creativity : Finding the balance between using external data for accuracy and preserving creativity and originality in content generation.
4. Privacy and ethics issues : Manage data confidentiality issues and ethical questions related to the use of data from various sources.

In the future, we can expect further integration of RAGs with cutting-edge machine learning techniques, such as deep learning and natural language processing. This synergy will enable the creation of even more intelligent and adaptive systems, capable of understanding and responding to queries with unprecedented precision.

Note that in the context of Gpt4-Turbo, the integration with Bing formulates a sort of real-time RAG, which is yet another case: you have to go and look for the information for each request. Probably OpenAI manages layers of memorization.

8/ RAGs only for the text?

No obviously, we can perfectly imagine situations where searching for information in an external knowledge base does not come back in text form only.

For example, we can imagine RAGs integrating with immersive technologies like augmented reality (AR) and virtual reality (VR), providing enriched user experiences. In areas like education or vocational training, this could revolutionize the way knowledge is acquired and applied.

(Queries => RAG => Text => Image or sequence)

The integration of RAGs into IoT could also lead to smarter home and industrial environments, where connected devices use information retrieved in real time to offer personalized and proactive services.

(Situation => triggers => requests => information => triggers)

The examples are obviously multiple. In medicine, for example, the feedback from a RAG is not necessarily in text form.

9/ Next Steps?

• One of the priorities in the future development of RAGs will be theimprovement of recovery algorithms to ensure even greater accuracy and relevance of the extracted information. This involves perfecting semantic search and contextualization methods.

On this point, the subject is very sensitive and there is no doubt that there is still enormous room for improvement.

• Diversification and expansion of databases and sources of information used by the RAGs will be crucial. This will not only increase the accuracy of responses but also ensure broader coverage of topics and areas.

See for example my proposition above on the basis of facts.

• RAGs will probably find applications in an even wider range of sectors, ranging from healthcare to finance to entertainment and beyond. Their ability to provide informed and contextual answers will be a major asset in many professional fields.

This is a subject that manufacturers must take up: there is a lot of added value to be produced in this area.

• As RAGs gain popularity, establishing ethical standards and protocols for their use will become imperative. This will ensure that these technologies are used responsibly and beneficial to society.

Relatedly: see the subject of the lawsuit between OpenAI and the New York Times: "Can we legally learn everything from everyone?"

10/ Diagrams

In everything that has just been said, it's important to remember that most of the concepts have only been touched upon, and in reality, a complete survey of the various techniques would easily fill a good hundred pages (without the diagrams).

For example, in what has been presented, there is an indistinction between the "naive RAG" form and more advanced uses. As mentioned earlier, in its implementation, retrieving information from a RAG to produce a reliable text-based response can be a simple operation of raw similarity searching based on the evaluation of angles (cosines) between two vectors of dimensions 700 to 1500, or it is possible to implement a complete, dynamic architecture with machine learning, etc.

If we implement the RAG in its simplest meaning and technique: LLM + Knowledge base + similarity, initially we take knowledge, we process it (reading / Chunk, embedding, insertion) then we use a classic Retrieval mechanism with an LLM: similarity and cosine.

This is the RAG below on the left.

The central part like the one on the right shows that we can go further, by not only taking the data as simple knowledge but by carrying out a set of treatments according to the need. These treatments can obviously be extremely complex.

For example, in the image above, when we talk about chunks (i.e. the text components separated from each other of a text to be vectorized), you can have chunks that are quite long: you will then have a Fairly quick but imprecise response. If you opt for short chunks: the response will take longer to obtain but potentially more precise.

In concrete terms, if you put "Around the World in 80 Days" by Jules Verne into chunk format, you can do it by sentence, by paragraphs or by pages: it all depends on your objectives and in a professional or industrial context, and with all the other techniques, and as is often the case, there are no real miracle recipes and "good parameters to know": there is a phase of study / tests / benchmarking to be carried out.

11/ A simple implementation?

Let's do a small example of implementation in 3 stages: simple, improved and with the API (which has the advantage of managing the language processing overlays transparently)

the simple version

First, we prepare the new knowledge that will be needed for the quizzes:

• take a pdf (in the following example, we will consider that it is the book by Jules Verne: "Around the World in 80 Days".
• read it and put it in chunk (= piece of text)
• vectorize it and insert it into a vector database

Then secondly, we do the questioning.

Here is a relatively simple and very commented Python code which describes the operations step by step. # we install the libraries pip install fitz openai pinecone-client

Step 1: we prepare our knowledge base with PDFs for example.

As we see, there are 3 main functions:

• take the text and think of it as a set of sentences or paragraphs
• these sets of sentences / paragraphs (= chunks) are to be vectorized according to a model
• insert it into a reference vector base of a language model.

And how do we do the questioning?

With the following piece of code, which follows the same logic. (but in reverse: instead of inserting after vectorization, we will remove then de-vectorize)

At this stage, the response will only offer extracts of chunks identified

For example, to the question: "how many days will it take to go around the world", the program will answer:

a bit better ?

If you want an answer like "around the world will be done in 80 days" (possibly with sources), you need to add these lines of code to handle "80 days" as a string of characters needed to be found in the answer.

and the display:

With a very high level API like GPT4-turbo-api?

With a natural language overlay directly accessible from the Open Ai APIs with the RAG (= completion) type query function, it is also possible, not to search for "exact information" but simply the information.

The question then becomes: "How long is the trip around the world expected to take?"

the search code is done not like a naive RAG but like an evolved RAG emulated via OPEN AI.

The search code and response become:

And this time, we have a real query in natural language.

Now, you go through all the use cases again, "And there you have it."

(imagine the website of your company, your town hall, your metropolis, etc. etc. which becomes effectively searchable in natural language via a simple chatbot.

12/ A (very) recent example.

During one of my conferences for a large metropolitan area, whether on the town hall website or that of the metropolitan area, I am looking for the concept of "swimming pool, schedules etc…".

This is, "unfortunately," the answer found on both the city's website and the metropolitan area's website. (hidden, this is not the subject here):

And yet, the data is contained in websites every time.

A search by keywords that doesn't even work, in other words, by natural language, is even more of a disaster.

With a RAG, we would have all the timetables and fare classes instantly. It's even so obvious that after having sucked up the entirety of the two sites, I RAGuized them and I got this information. And it's the same for everything else: making an appointment online, and what constraints to have a place in a crèche, and what was decided in the last municipal or urban community council: I even did these live questions during the conference…

13/ And what about fine-tuning?

Fine-tuning is another technology which, while not allowing you to do the same thing (That will be the subject of another post.), is close. In short, imagine you have a neural network (Vision) that can distinguish a tank from a car. Fine-tuning involves retraining it, which takes into account the neural network's ability to recognize cars, and which we will " on train "in order to specialize it (For example, in cars, we will make it recognize a subset "Mercedes / Audi / BMW".

Depending on the need, it is therefore sometimes advantageous to use a RAG, sometimes fine-tuning (sometimes both).

However, here is a comparative table of the two.

14 Conclusion

Summary of Key Points Covered

During our exploration of the concept of Retrieval Augmented Generation (RAG), we discussed several key aspects of this innovative artificial intelligence technology. A RAG represents a significant evolution in the field of language models, by integrating the ability to retrieve information from external databases (or almost real-time source, up to vectorization times) to improve text generation.

We examined the architecture of RAG, which combines an information retrieval component with a text generation component, and how this synergy helps produce more accurate and contextual responses. Various use cases of RAG were explored, demonstrating its applicability in varied sectors.

Additionally, we discussed the benefits of RAG, such as increased relevance of answers and the ability to stay up to date with the most recent information. Remaining constraints, including the integration of reliable information sources and the management of knowledge databases, were also highlighted.

As a practical illustration, we have shared a Python code example to show how RAG can be implemented in a simple way. This code demonstrates how to extract information from a PDF file and insert it into a vector database like Pinecone, then how to query this database to get relevant answers to specific questions. We also talked about the difference between RAG and Finetunning.

And so ?

RAG (Retrieval Augmented Generation) technology clearly represents a significant advance in the field of artificial intelligence and natural language processing by integrating knowledge from external databases with the intrinsic capabilities of LLMs.

RAGs offer an effective solution to the limitations of traditional models, particularly in terms of accuracy, updating information and reducing bias.

Their application in diverse fields, from customer support to academic research, demonstrates their versatility and potential to transform the way we interact with information.

The implementation, without being very complex, may nevertheless require prior engineering work: it is different to consider putting the website of your company or city in a RAG to make an appointment or request documents for a place in a nursery and to put the 11 million documents on its intranet, most of which are defense secrets, to be queried for both training and analysis purposes.

However, in many cases, reasonable Poc can be assembled "in a single day". And their contribution is immediately substantial.

Source: https://www.linkedin.com/pulse/intelligence-artificielle-rag-vincent-pinte-deregnaucourt-jlkge/

To learn more: check out our episode on GPTs and next word prediction et emerging LLM capabilities such as GPT-4.