| Titre : | Neural Regression Models Theory and Applications |
| Auteurs : | ALLALI Hafsa, Auteur ; Rouane Rachida, Directeur de thèse |
| Type de document : | texte manuscrit |
| Editeur : | University of Saida - Dr Moulay Tahar. Faculty: Mathematics, Computer Science and Telecommunications Department: Mathematics, 2025/2026 |
| Format : | 73ص |
| Accompagnement : | CD |
| Langues: | Anglais |
| Index. décimale : | BUC-M 008494 |
| Catégories : |
Master Mathématiques Spécialité: Analyse stochastique, statistique des processus et applications |
| Résumé : |
This thesis examines the theoretical foundations and practical applications of two regression
approaches: Multiple Linear Regression (MLR) and Neural Network Regression (NNR). The study begins by establishing a robust mathematical framework for MLR, emphasizing the Ordinary Least Squares (OLS) estimation. However, recognizing the limitations of linear models in capturing complex, high-dimensional patterns, the research shifts toward Neural Networks (NNs) as a flexible alternative. The core of this work investigates the architecture of Neural Networks (NNs) and the mathemat- ical rigor of the Backpropagation algorithm. By situating neural networks within a regression context, we demonstrate their capacity to function as universal approximators for non-linear phenomena. A comparative analysis is conducted to evaluate the predictive performance and stability of both frameworks. The results suggest that while linear regression remains indis- pensable for interpretability and structural inference, neural regression models provide superior accuracy in handling non-linearities and large datasets, offering a powerful tool for predictive modeling in contemporary data science. |
| Note de contenu : |
Contents
Abstract 7 Dedication 8 Acknowledgements 9 General Introduction 10 1 Multiple linear regression 13 1.1 Multiple Regression Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.1.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.1.2 Matrix notation of MLR . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.2 Assumptions of the multiple linear regression model . . . . . . . . . . . . . . . . 15 1.2.1 Stochastic assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.2 Structural hypotheses: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.3 Estimation by Ordinary Least Squares . . . . . . . . . . . . . . . . . . . . . . . 16 1.3.1 Geometric Interpretation of OLS . . . . . . . . . . . . . . . . . . . . . . 18 1.4 Estimation of the variance σ2 ε . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.5 Sampling Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 1.6 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.7 Variance Decomposition and ANOVA Table . . . . . . . . . . . . . . . . . . . . 23 1.8 Statistical Inference and Hypothesis Testing . . . . . . . . . . . . . . . . . . . . 24 1.8.1 Test of Individual Regression Coefficients (t-test) . . . . . . . . . . . . . 24 1.8.2 Overall Significance of the Model (F-test) . . . . . . . . . . . . . . . . . 25 1.8.3 Partial F-test (General Linear Hypothesis) . . . . . . . . . . . . . . . . . 26 1.9 Example: Agricultural Production Analysis . . . . . . . . . . . . . . . . . . . . 26 2 Neural Network Regression 30 2.1 The Perceptron as a Mathematical Model . . . . . . . . . . . . . . . . . . . . . 30 2.1.1 Historical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.1.2 Mathematical Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.1.3 Geometric interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.1.4 Link with Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . 32 2 CONTENTS 2.2 Feedforward neural network model . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.1 Single Hidden Layer Network . . . . . . . . . . . . . . . . . . . . . . . . 33 2.2.2 Matrix formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.2.3 Extension to Multi-layer case . . . . . . . . . . . . . . . . . . . . . . . . 35 2.3 Activation function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.3.1 The mechanism of activation functions . . . . . . . . . . . . . . . . . . . 36 2.3.2 Differentiability of activation function . . . . . . . . . . . . . . . . . . . . 36 2.3.3 Sigmoid/Logistic activation function . . . . . . . . . . . . . . . . . . . . 37 2.3.4 Hyperbolic tangent activation function . . . . . . . . . . . . . . . . . . . 38 2.3.5 Rectified linear unit (ReLU) activation function . . . . . . . . . . . . . . 39 2.3.6 Leaky ReLU activation function . . . . . . . . . . . . . . . . . . . . . . . 40 2.3.7 Activation Function Selection in Neural Networks . . . . . . . . . . . . . 41 2.4 Neural Network Regression Framework . . . . . . . . . . . . . . . . . . . . . . . 41 2.4.1 Regression Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 2.4.2 Empirical Risk Minimization (ERM) . . . . . . . . . . . . . . . . . . . . 42 2.5 Optimization and Backpropagation . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.5.1 Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 2.5.2 The Chain Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.5.3 Backpropagation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.6 Theoretical Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.6.1 Universal Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 2.6.2 Model Complexity and the Bias-Variance Trade-off . . . . . . . . . . . . 50 2.6.3 Complexity Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 2.7 Example: Nonlinear Agricultural Production . . . . . . . . . . . . . . . . . . . . 51 2.8 Theoretical Comparison: Neural Regression vs. MLR . . . . . . . . . . . . . . . 54 3 Numerical Applications 56 3.1 Dataset Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.2 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.2.1 Train-Test Split . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.2.2 Data Normalization for NNR . . . . . . . . . . . . . . . . . . . . . . . . 58 3.3 Multiple Linear Regression Model . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.3.1 Python Code of MLR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 3.3.2 Performance Evaluation of the MLR Model . . . . . . . . . . . . . . . . 60 3.4 Neural Network Regression Model . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.4.1 Architecture of NN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.4.2 Training Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 3.4.3 Python Code of NNR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.4.4 Results of the NNR Model . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3.5 Comparison Between MLR and NNR . . . . . . . . . . . . . . . . . . . . . . . . 65 3 CONTENTS General Conclusion 68 Nomenclature 69 4 |
Exemplaires
| Code-barres | Cote | Support | Localisation | Section | Disponibilité |
|---|---|---|---|---|---|
| aucun exemplaire |
Documents numériques (1)
memoire_ALLALI_Hafsa_(ASSPA).pdf Adobe Acrobat PDF |
