System Identification and Data Analysis

SYSTEM IDENTIFICATION AND DATA ANALYSIS

Academic Year 2022-2023

Courses

System Identification and Data Analysis (9 CFU):
Master of Science in Artificial Intelligence and Automation Engineering (Curriculum Robotics and Automation)
System Identification (6 CFU):
Master of Science in Engineering Management
Data Analysis (6 CFU):
Master of Science in Artificial Intelligence and Automation Engineering (Curriculum Intelligent Systems)
Master of Science in Applied Mathematics (part of the Data and Financial Analysis course)

Instructors

Office hours (A. Garulli): upon request via e-mail (Torre Rossa, room r202)
Office hours (M. Casini): upon request via e-mail

News

June 14: The registration site for the system identification homework of July 2023 is open.

List of oral exam dates, summer session 2022-23 (please sign up in segreteriaonline):

Oral exam date and hour	Homeworks must be delivered within
June 22, 14.30	June 15
July 13, 14.30	July 6

Another date in late July can be added upon request.

Thesis and research opportunities in the System and Control Group

State Estimation Homework - part (A) of the exam

This homework can be done individually or by teams composed of two students (don't ask for larger teams). The report must be written in LaTeX (using this template) and delivered to the following website at least 7 days before the date of the oral exam: Delivery site for State Estimation Homework.
The homework text can be asked by sending an email to prof. Garulli.

System Identification Homework - part (B) of the exam

Students interested in doing the System Identification homework must send an email to Prof. Casini at least 15 days before the date of the oral exam.

How the exam works: things you must know about the exam organization

Prerequisites

What do I need to know before starting?
- Linear algebra and calculus.
- Basics of probability theory and random variables (though there will be a very short review).
- Basics of dynamic systems theory: input-output and state space representations; transfer function; Z-transform (discrete-time systems).
- Basics of Matlab language.
Some useful links are given below.

How to pass the exam

In order to pass the exam, you have to perform three steps:
- (A) the state estimation homework;
- (B) the system identification homework;
- (C) the oral exam.
Steps (A) and (B) must be done before step (C). Details and schedule of steps (A) and (B) will be comunicated by the instructors during the course. In any case, the reports of homeworks (A) and (B) must be delivered at least 7 days before the oral exam.The dates of the oral exams are usually published in the News section above (they may not coincide with those in segreteriaonline).
The oral exam involves the discussion of the homeworks and questions about all the topics treated during the course (yes, all!).

Students taking the 6 CFU version of the course (either System Identification, or Data Analysis) have to do only steps (B) and (C).

How to do the homeworks

Homeworks will require to write a report explaining the adopted methodologies and the obtained results in a well-documented and concise fashion. Such report must be written in LaTeX (a LaTeX tutorial is available here).
A LaTeX template for the report of the State Estimation Homework is provided here.
A LaTeX template for the report of the System Identification Homework is provided here.
Additional information will be provided during the course.

Golden rules

Rule 1: the right time to do the exam is at the end of the course, during the summer exam session. You are strongly encouraged to do that! In case you need to do the exam in subsequent sessions, you have to ask for the homeworks well in advance, in order to deliver the reports at least 7 days before the oral exam.
Rule 2: do the homeworks by yourself and report them carefully. During the oral exam you will have to explain and motivate the homework solutions.

Bibliography

T. Soderstrom, Discrete-time Stochastic Systems, Springer London Ltd, 2nd ed., 2002. Library code: 269 - 269a.
L. Ljung, System Identification: Theory for the user, 2nd ed., Prentice-Hall, 1999. Library code: 71 - 71a - 71b.

Part 1: Estimation Theory

1.0 Course introduction.
Material: Slides_introduction

1.1 Random variables. Probability distributions. Mean and covariance. Conditional probability. Gaussian variables.
Ref.: [Soderstrom, 2.1-2.4].

1.2 Estimation theory. Parametric estimation. Properties of estimators. Maximum likelihood estimators. Least squares and Gauss-Markov estimators. Bayesian estimation. Minimum mean square error estimators.
Ref.: [Soderstrom, 5.1-5.3].
Material: Notes_estimation_theory; Slides_estimation_theory

1.3 Stochastic processes and time-series prediction. Distributions, mean and covariance function. Stationary processes. Frequency domain representation. Stochastic dynamic systems. Time-series models: AR, MA, ARMA. Time-series prediction.
Ref.: [Soderstrom, 3.1-3.4; 4.1-4.3; 7.1-7.2].
Material: Notes_timeseries
Lab session: Lab_session_time_series; Solution (m-file).
Homework: Homework_time_series; Solution (pdf-file); Solution (m-file).

Part 2: State Estimation (only for students taking the 9 CFU version of the course)

2.1. State estimation for linear systems. Non stationary stochastic systems. The state estimation problem and the Kalman filter. Asymptotic properties of the Kalman filter.
Ref.: [Soderstrom, 3.3; 4.2; 6.1-6.3; 6.7]
Material: Notes_state_estimation (first part)
Lab session: Lab_session_Kalman_Filter; data_labsession_kf.mat; Solution (m-file).

2.2. State estimation for nonlinear systems. State estimation in nonlinear stochastic systems. The Extended Kalman Filter. Advanced nonlinear filtering techniques: unscented filter; sequential Monte Carlo methods.
Ref.: [Soderstrom, 9.1-9.5].
Material: Notes_state_estimation (second part)
Lab session: Lab_session_Nonlinear_State_Estimation; data1_labsession_nse.mat; data2_labsession_nse.mat; Solution (m-file).

Part 3: System Identification

3.1 System identification theory. Identification of linear systems: prediction error methods. Input-output models: ARX, ARMAX, OE, BJ. Least squares estimator for linear regression models. Recursive system identification. Model validation.
Ref.: [Ljung, 1.1-1.4; 3.1-3.2; 4.1-4.2; 7.1-7.3; 10.1-10.2; 11.1-11.2; 16.1-16.6].
Material: Notes_system_identification

3.2 Practical system identification. Use of software tools for system identification.
Ref.: [Ljung, 17.1-17.4].

Useful stuff

Video lectures of System Identification and Data Analysis. Access will be granted upon request from the student Unisi account.
LaTeX tutorial (includes Installation, Quick Start, Commands, etc.)
Matlab download site (create an account using your Unisi email address. Be sure to install the System Identification Toolbox and the Signal Processing Toolbox)
A quick tutorial for dynamic system analysis and simulation with Matlab
Notes of an old course version (in Italian)

Background material

Please report errors or problems to: garulli at ing.unisi.it
Thanks!