Bergische Universität Wuppertal Fachbereich Mathematik und Naturwissenschaften Applied and Computational Mathematics (ACM)

Team Research Publications Teaching

---

The lecture is suitable for students of mathematics as well as for economics.
The students of economathematics can use it as component AKap.NAaA-a "Selected Topics in Numerical Analysis and Algorithms" in the module of the same name.

Topics of the Lecture:
The aim of the lecture is to introduce the mathematical concepts underlying the theory of approximating a function using machine learning, resp. to be more precise deep learning methods. To this end we give an introduction to the theory and show how this can be used for exploring Neural Networks. To illustrate the applicability of Neural Networks in Finance we consider the pricing of derivatives and the calibration of parametric models. For pricing we consider the Black-Scholes-Merton, the Hull-White and the Heston model. Calibration is considered for the Black-Scholes-Merton and the Heston model.

Outline of the lecture:

1. Foundations of Machine Learning
   • Empirical Risk Minimization (ERM) with Inductive Bias
   • Agnostic Probably Approximately Correct (PAC) Learning
   • Multiclass Classification
   • Generalized Loss Functions, risk function, square loss
   • The Bias-Complexity Tradeoff
   • No-Free-Lunch and Prior Knowledge
   • Decomposition of the error of an ERM predictor
     • Approximation error
     • Estimation error
   • The Vapnik-Chervonenkis (VC) dimension
   • The Fundamental Theorem of PAC learning
   • Lemma of Sauer-Shelah-Perles
   • The uniform convergence property
   • Nonuniform Learnability
     • Structural Risk Minimization (SRM) Learning Paradigm
     • Minimum Description Length (MDL) Paradigm
   • The Computational Complexity of Learning
   • Efficient Algorithms for solving the ERM Problem
2. From Theory to Algorithms
   • Linear Predictors
   • Linear Programming (LP) for the Class of Halfspaces
   • The Perceptron algorithm of Rosenblatt to implement the ERM rule
   • The VC Dimension of the class of (non-)homogenous Halfspaces
   • Linear Regression, Least squares
   • Linear Regression for Polynomial Regression Tasks
   • Logistic Regression, sigmoid function, logistic function
   • Maximum Likelihood approach
   • Generalization of linear predictors: Boosting approach
     • control of bias-complexity tradeoff
     • amplify accuracy of weak learners
   • Adaptive Boosting (AdaBoost)
   • Boosting algorithm: XGBoost
   • Weak Learnability
   • Bagging Ensemble Technique: Random Forest Models
   • Model Selection using SRM and Validation
   • Training Error and Validation Error
   • Error Decomposition Using Validation
   • Convex Learning Problems
   • Surrogate Loss Functions
   • Regularized Loss Minimization (RLM)
   • Ridge Regression
   • Stable learning algorithms
   • Tikhonov Regularization as a Stabilizer
   • Control of the Fitting-Stability Tradeoff
   • Stochastic Gradient Descent (SGD), Adam optimizer
     • projection step
     • variable step size
     • risk minimization, RLM
   • Support Vector Machines (SVM)
   • Optimality Conditions and "Support Vectors"
   • The Kernel Trick
3. Pricing Financial Derivatives
   • Short Introduction to Option Pricing
   • Black-Scholes-Merton, Hull-White and Heston models
   • Applications of Neural Networks to Finance (Pricing, Calibration)

Literature:

• D.P. Berlyand and P.-E. Jabin, Mathematics of Deep Learning - An Introduction, De Gruyter, 2023.
• D.P. Bertsekas, A Course in Reinforcement Learning, Athena Scientific, 2023.
• C.M. Bishop and H. Bishop, Deep Learning - Foundations and Concepts, Springer, 2023. ( Webpage )
• B. Bohn, J. Garcke, and M. Griebel, Algorithmic Mathematics in Machine Learning, SIAM, 2024. ( GitHub )
• S. Bubeck and Nicolo Cesa-Bianchi, Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Now Publishers Incorporate, 2012.
• N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games, Cambridge University Press, 2006.
• M.P. Deisenroth, A.A. Faisal, and C.S. Ong, Mathematics for Machine Learning, Cambridge University Press, 2020.
• F. Fleuret, The Little Book of Deep Learning.
• C. Giraud, Introduction to High-Dimensional Statistics Chapman and Hall / CRC, 2014.
• F. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016.
• T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, 2nd edition, 2009.
• T. Hrycej, B. Bermeitinger, M. Cetto and S. Handschuh, Mathematical Foundations of Data Science, Texts in Computer Science, Springer 2023.
• G. James, D. Witten, T. Hastie, R. Tibshirani, and J. Taylor An Introduction to Statistical Learning, Springer, 2023.
• D.P. Kroese, Z.I. Botev, T. Taimre, and R. Vaisman, Data Science and Machine Learning: Mathematical and Statistical Methods, Chapman and Hall/CRC, Boca Raton, 2019.
• V. Koltchinskii, Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems, Springer, 2011.
• M. Mendez, A. Ianiro, B. Noack and S. Brunton, (eds.), Data-Driven Fluid Mechanics: Combining First Principles and Machine Learning, Cambridge University Press, 2023.
• T. Mitchell, Machine Learning, McGraw-Hill, 1997.
• M. Mohri, A. Rostamizadeh, A. Talwalkar, Foundations of Machine Learning (2nd edition), MIT Press, 2018.
• S.J.D. Prince, Understanding Deep Learning, MIT Press, 2023.
• S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th US edition.
• S. Shalev-Shwartz and S. Ben-David, Understanding Machine Learning: From Theory to Algorithms, Cambridge University Press, 2014.
• towards data science,
• S.-A. Wegner, Mathematische Einführung in Data Science, Springer, 2023.
• P. Wilmott, Machine Learning: An Applied Mathematics Introduction, Panda Ohana Publishing, 2019.

Previous knowledge: Analysis I - III, basic knowledge of ordinary differential equations, stochastics.

Exercises:
For the exercises we recommend to install the Python distribution from Anaconda and install the packages tensorflow, keras, pytorch, matplotlib. If other packages are necessary these can easily installed on demand.
Sheets.

Criteria: Regular participation and participation in the exercise groups, as well as reaching 50% of the possible points on the first seven or the remaining exercise sheets and at least 2/3 of the possible points for the practical tasks.