| Titre : | Boosting LLMs With external knowledge integration |
| Auteurs : | KADDOURI Nour El Houda, Auteur ; Zahaf Ahmed, Directeur de thèse |
| Type de document : | texte manuscrit |
| Editeur : | Université de Saïda – Dr. Moulay Tahar – Faculté des Mathématiques, de l’Informatique et des Télécommunications, 2025/2026 |
| Format : | 92 ص |
| Accompagnement : | CD |
| Note générale : |
General Conclusion
This work presented the design, evaluation, and deployment of a Multi-RAG system combining large language models with structured knowledge retrieval and graph-based reasoning for question answering. The project addressed the fundamental limitations of standard LLMs—namely hallucinations, static knowledge, and limited relational reasoning—by integrating external knowledge sources and graph neural network architectures into the generation pipeline. The theoretical foundations established in Chapter 1 provided the necessary background on language model architectures, pretraining paradigms, and adap- tation strategies, laying the groundwork for understanding both the capabilities and the limitations of modern LLMs. Chapter 2 extended this analysis to the state of the art in Retrieval-Augmented Generation, covering the progression from Naive RAG to Modular and Graph-based RAG systems, and highlighting the role of Graph Neural Networks in enhancing structured knowledge reasoning. The experimental evaluation presented in Chapter 3 compared four distinct approaches on the OpenBookQA dataset. Overall, this project demonstrates that the combination of large language models, structured knowledge retrieval, and graph neural networks represents a promising research direction for building more accurate, interpretable, and knowledge-grounded question answering systems. The deployed Multi RAG application constitutes a concrete and extensible foundation for future research in this area. |
| Langues: | Anglais |
| Index. décimale : | BUC-M 008536 |
| Catégories : |
Master en informatique Spécialité : Intelligence Artificielle Principe et application |
| Note de contenu : |
Table of Contents
General Introduction 11 1 Foundations of Language Models 13 1.1 Introduction ................................................................................................ 13 1.2 Artificial Intelligence and Natural Language Processing ...................... 15 1.2.1 Artificial Intelligence .................................................................... 15 1.2.2 Natural Language Processing (NLP) ........................................... 16 1.2.3 Challenges in NLP ......................................................................... 16 1.3 Deep Learning for NLP ............................................................................. 18 1.3.1 Deep Learning................................................................................ 18 1.3.2 Why Deep Learning in NLP ......................................................... 18 1.3.3 Bag-of-Words Representation...................................................... 18 1.3.4 Deep Learning in NLP .................................................................. 21 1.4 Word Embeddings ...................................................................................... 23 1.4.1 How are embeddings created? ..................................................... 23 1.4.2 Why are embeddings important? ................................................ 23 1.4.3 Types of Embeddings ................................................................... 23 1.4.4 Embeddings for Recommendation Systems ............................... 27 1.4.5 How does this work for songs? .................................................... 27 1.4.6 What does the training actually learn? ..................................... 27 1.4.7 How are recommendations generated? ....................................... 27 1.5 Tokenization Techniques........................................................................... 28 1.5.1 Tokenization ................................................................................... 28 1.5.2 Tokenizer Properties: How Does a Tokenizer Break Down Text? ............................................................................................... 28 1.5.3 Tokenization Method ................................................................... 28 1.5.4 Tokenizer Design Choices ............................................................ 28 1.5.5 Training Data ................................................................................ 29 1.5.6 1.5.5 Tokenization Types .............................................................. 29 1.5.7 Token Embeddings ........................................................................ 33 1.5.8 Contextualized Word Embeddings with Language Models 33 1.6 Word Embeddings Beyond LLMs ............................................................. 34 1.6.1 Word2Vec Algorithm and Training ............................................. 34 1.7 Transformers ............................................................................................... 34 1.7.1 Attention Mechanisms and Previous Limitations ...................... 34 1.7.2 The Transformer Model ............................................................... 35 1.7.3 Overall Architecture ..................................................................... 35 1.7.4 Encoder and Decoder Stacks ....................................................... 35 1.7.5 Attention Mechanism ................................................................... 36 1.7.6 Scaled Dot-Product Attention ..................................................... 36 1.7.7 Multi-Head Attention ................................................................... 36 1.7.8 Types of Transformers.................................................................. 36 1.7.9 Applications.................................................................................... 36 1.7.10 Advantages and Limitations ......................................................... 37 5 1.7.11 Conclusion....................................................................................... 37 1.8 Masked Language Models (MLMs) – BERT and Beyond ...................37 1.8.1 Introduction to Masked Language Models ................................ 37 1.8.2 Architectural Foundations ............................................................ 37 1.8.3 Traditional Downstream Adaptation: Classification Heads 40 1.8.4 Emerging Paradigm: Generative Use of the MLM Head . 40 1.8.5 Advantages of MLM-Based Generative Classification ............... 41 1.8.6 Comparison with Decoder-Only LLMs ........................................ 41 1.8.7 Conclusion and Future Directions ............................................... 42 1.9 Causal Language Models (e.g., GPT) ...................................................... 42 1.9.1 Core Principle ................................................................................ 42 1.9.2 Architecture .................................................................................... 42 1.9.3 Pretraining Objective .................................................................... 43 1.9.4 Applications..................................................................................... 43 1.9.5 Limitations ...................................................................................... 43 1.10 LLMs Fine-Tuning ....................................................................................43 1.10.1 From Pretraining to Adaptation ................................................ 44 1.10.2 Supervised Fine-Tuning (SFT) ..................................................... 44 1.10.3 Effects of Fine-Tuning .................................................................. 44 1.10.4 Advanced LLM Fine-Tuning (Modern Pipeline) ...................... 45 1.11 Conclusion ................................................................................................... 48 2 Retieval Augmented Generation : State of the Art 50 2.1 Introduction and Overview of RAG ......................................................... 50 2.2 Definition of RAG ....................................................................................... 51 2.3 Types of RAG ............................................................................................. 52 2.3.1 Naive RAG .....................................................................................52 2.3.2 Advanced RAG ............................................................................... 52 2.3.3 Modular RAG................................................................................. 53 2.4 Data Types, Retrieval Techniques, and Generation Optimization in RAG.........................................................................................................54 2.4.1 Data Types in RAG ...................................................................... 54 2.4.2 Advanced Retrieval Techniques ................................................... 54 2.4.3 Generation and Context Optimization ........................................ 55 2.4.4 Advanced Retrieval Strategies ..................................................... 55 2.5 Comparison Between RAG and Fine-tuning .......................................... 56 2.6 Evaluation of RAG Systems .................................................................... 57 2.6.1 Retrieval Quality Evaluation ....................................................... 57 2.6.2 Generation Quality Evaluation ................................................... 57 2.7 Applications of RAG .................................................................................. 58 2.7.1 Question Answering Systems ....................................................... 58 2.7.2 Summarization ................................................................................ 58 2.7.3 Personalized Assistants ................................................................. 58 2.7.4 Code Generation and Debugging ................................................. 59 2.7.5 Scientific Research Assistance ...................................................... 59 2.7.6 Legal and Compliance ................................................................... 59 6 2.7.7 Content Creation and Summarization for Marketing . . . 59 2.7.8 Knowledge Management in Enterprises ..................................... 59 2.7.9 Key Advantages in Applications................................................. 59 2.8 Multimodal Retrieval-Augmented Generation (MM-RAG) .................. 59 2.8.1 Introduction et Contexte ............................................................. 59 2.8.2 General Architecture of MM-RAG .............................................. 61 2.8.3 Modalities Used in MM-RAG ...................................................... 63 2.8.4 Advantages of MM-RAG .............................................................. 64 2.8.5 Limitations of MM-RAG .............................................................. 64 2.8.6 Transition Toward Graph-based RAG ........................................ 64 2.9 GraphRAG .................................................................................................. 65 2.9.1 Definition and Core Concepts ..................................................... 65 2.10 KG-RAG ..................................................................................................... 65 2.10.1 Definition and Components ......................................................... 65 2.11 GNN for Knowledge Graphs .................................................................... 66 2.11.1 Definition of Graph Neural Networks ........................................ 66 2.12 Datasets and Benchmarks ........................................................................ 67 2.13 Research Gaps ............................................................................................ 67 2.14 Positioning of Our Work .......................................................................... 68 2.15 Conclusion .................................................................................................. 69 3 Approach and Evaluation 70 3.1 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 3.2 RAG pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 3.2.1 Approach 1: Zero-shot LLM Baseline . . . . . . . . . . . . 70 3.2.2 Approach 2: Boosting LLM Reasoning with KG-RAG via LLM-based Entity Extraction . . . . . . . . . . . . . . . . 71 3.2.3 Approach 3: Boosting LLM Reasoning with KG-RAG via NLP-based Entity Extraction . . . . . . . . . . . . . . . . 72 3.2.4 Approach 4: Boosting LLM Reasoning with KG-RAG via GNN (QAGNN + RoBERTa) . . . . . . . . . . . . . . . . 74 3.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.3.1 Accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.3.2 Confusion Matrix . . . . . . . . . . . . . . . . . . . . . . . 77 3.3.3 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 3.3.4 Results and discussion . . . . . . . . . . . . . . . . . . . . 78 3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 4 System Implementation and Deployment 81 4.1 Introduction ................................................................................................ 81 4.2 General System Architecture ................................................................... 82 4.3 Backend Architecture (FastAPI) ............................................................... 82 4.3.1 Initialization and Resource Loading ........................................... 82 4.3.2 Exposed API Endpoints ............................................................. 83 4.3.3 The /upload Endpoint — Document Graph Construction 83 7 4.3.4 The /chat Endpoint — Approach-Based Routing ................... 83 4.4 Frontend Architecture (React.js) .............................................................. 83 4.4.1 General Interface Overview ......................................................... 84 4.4.2 Left Sidebar — Document Management ................................... 84 4.4.3 Central Chat Area — Question-Answer Interaction ................ 84 4.5 User Interaction Scenarios........................................................................ 85 4.5.1 Scenario 1 — MCQ with Baseline Approach (LLM Only) 85 4.5.2 Scenario 2 — MCQ with Graph Approach(ConceptNet) . 86 4.5.3 Scenario 3 — MCQ with GNN Reasoning Approach............... 86 4.5.4 Scenario 4 — Free Question with GNN-RAG (Without Document) .................................................................................... 87 4.5.5 Scenario 5 — Free Question with GNN-RAG (With Up- loaded Document).......................................................................... 87 4.6 Justification of Deployment Technology Choices................................... 88 4.7 Conclusion .................................................................................................. 88 General Conclusion 90 |
Exemplaires
| Code-barres | Cote | Support | Localisation | Section | Disponibilité |
|---|---|---|---|---|---|
| aucun exemplaire |
Documents numériques (1)
BUC-M 008536 Adobe Acrobat PDF |
