DALMINE
Overview
Date/time interval
Syllabus
Course Objectives
The Model Identification and Data Analysis (IMAD) course aims to provide the student with the theoretical and practical concepts for estimating mathematical models starting from observations of the phenomenon to be modeled. The course is divided into two parts. The first part deals with the estimation of static models, regression and classification algorithms, the concepts of which are preparatory to the second part of the course and to subsequent courses such as:
- Adaptive Learning, Estimation and Supervision of dynamical systems
- Machine learning and Statistical learning.
The second part of the course concerns the estimation of dynamic models, the use of which is fundamental to put into practice for control and filtering of dynamical systems.
At the end of the course the student will be able to:
- Formulate an estimation problem, identifying the variables of the problem (e.g. input and output data)
- Estimate a static or dynamic model from data, by solving an optimization problem
- Choose the most appropriate model for the type of data available
- Evaluate the goodness of the model estimated from the data
- Apply different estimation methods, depending on the question to be answered by analyzing the data
The course is complementary to the courses of:
- Control Systems Technology
- Controlli Automatici
- Controllo Avanzato e Multivariabile
The course provides the basis for the course of:
- Adaptive learning, estimation and supervision of dynamical systems
Course Prerequisites
The prerequisites are as follows:
- Continuous and discrete time dynamic systems analysis. Block diagrams and feedback. Frequency response. Sampling theorem. Fourier transform.
- Matrices, eigenvalues and eigenvectors, vector spaces.
- Derivatives, integrals, gradients, limits, stationary points of a mathematical function.
- Random variables, probability distributions, moments of a probability distribution.
Teaching Methods
The educational path proposed to the student is as follows:
1) Follow the lecture presented using the slides. They are all available before the start of the course and it is useful for the student to view the slides of a lesson before following the lesson.
2) Study the topics of the lesson with the help of the textbook, slides and personal notes.
3) Respond to the MATLAB exercises prepared. The purpose of these exercises is to verify both knowledge of the concepts exposed, their understanding and the student's ability to use them
4) Follow the frontal exercises carried out on the blackboard (electronic or traditional). The text and the development of the exercises are all available before the start of the course and it is useful for the student to view the development of the exercises before following the lecture.
6) Towards the end of the lessons the student is able to carry out an exam example, proposed with its solution.
Much importance is given to students' active participation in lessons, which is stimulated through continuous dialogue. Students can find the teacher at any time (preferably by appointment) by going to the teacher's office (Automatic Control Laboratory).
Assessment Methods
The exam is done through a written test of 2 hours. It usually consists of 6 questions: 3 exercises and 3 theoretical questions. All exercises and questions are worth 5 to 8 points.
There are no intermediate tests.Performing homework contributes up to +3 points on the final grade.
In order to ensure the effective organization and administration of exams, the instructor may move the next scheduled exam session forward immediately for students who have received a severely failing grade or who, despite being registered, are absent without a valid excuse.
N.B. For computer science students, this course can be integrated with the Controlli Automatici course. The overall grade of the integrated course will be calculated as the average (approximated to the higher integer) of the grades achieved in the two modules.
The instructor reserves the right to request a supplementary oral exam, even after the written exam has been administered, if deemed necessary to verify that students have effectively acquired the knowledge and skills outlined in the course’s learning objectives and to ensure that grades are assigned correctly.
Contents
INTRODUCTION
Review of statistics. Scalar and multivariable random variables. Continuous and discrete probability distributions. Estimation theory and estimators.
PART 1: STATIC SYSTEMS
Linear regression. Formulation of the problem and cost function. Least squares estimation. Gradient descent method. Properties of the least squares estimate. Exercises in MATLAB
Maximum likelihood estimate. Estimation of population parameters. Estimation of linear models.
Logistic regression for classification. Formulation of the problem and cost function. Exercises in MATLAB
Machine learning fundamentals. Introduction to machine learning and data science. Supervised and unsupervised problems. Feasibility of learning. Bias-variance tradeoff. Learning curves. Overfitting. Regularization. Validation, cross-validation and optimal complexity formulas. Exercises in MATLAB
PART 2: DYNAMIC SYSTEMS
Stochastic processes. Stationary stochastic processes. Temporal moments and ergodicity. Z transform and Fourier transform. Power spectral density. Spectral estimate. Deterministic discrete linear dynamical systems. Stochastic discrete linear dynamical systems. Exercises in MATLAB
Stochastic model families. Families of rational spectrum models. Time series: MA, AR, ARMA. Input / output systems: ARX, ARMAX, FIR, OE, BJ.
Prediction. Wold decomposition. All-pass filter and canonical form. Optimal predictor. Optimal predictor for MA, ARMA, ARMX processes. Comparison with Kalman predictor
Identification: fundamental concepts. Prediction Error Minimization methods (PEM). PEM identification of ARX, ARMAX models. Exercises in MATLAB
Identification: analysis and complements. Asymptotic analysis of PEM methods. Persistent excitation. Evaluation of the uncertainty of the PEM estimate. Exercises in MATLAB
Identification: evaluation of the model. Choice of the structure and complexity of the model. Validation and complexity formulas. Residue analysis. Analysis of the uncertainty of the estimate. Simulation vs. prediction of the identified model. Exercises in MATLAB
Online Resources
More information
The materials for following the lessons and for personal preparation are made available prior to the lessons.
The emphasis of the course is mainly theoretical but code examples in Matlab are provided. The practical application of the topics will be mainly in the course Adaptive Learning, Estimation and Supervision of dynamical systems (ALES). The course will allow you to learn skills that are highly sought after by companies, such as the development of analytical models from data, which is essential for professions such as Data Scientist.
For computer science students, this course can be integrated with Controlli Automatici. The Controlli Automatici course provides a complement to the IMAD course, in particular with regard to insights into the theory of prediction through the Kalman predictor and the use of dynamic models identified for control, such as minimum variance control.
The course also provides the basis for the subsequent course Adaptive Learning, Estimation and Supervision of Dynamical Systems (ALES), where we will see, among other things, the concepts of IMAD but extended to MIMO systems.
It is also suggested to take the Control Systems Technology course for a practical understanding of the engineering technologies that make up an industrial control solution.