Course title
7M9909001
Stochastic Systems for Control and Machine Learning

 CETINKAYA AHMET
Course content
This class introduces stochastic systems and tools for their analysis, emphasizing control engineering and machine learning applications. Stochastic systems are essential building blocks of many technical methods used in different engineering domains. Examples include Kalman filters in control engineering and stochastic gradient descent methods in machine learning. Successful application of such methods require a solid understanding of stochastic systems.
Purpose of class
This class is intended to equip students with essential theoretical knowledge and practical programming skills regarding stochastic systems used in control engineering and machine learning fields.
Goals and objectives
  1. Students will be able to learn properties and applications of discrete-time Markov chains.
  2. Students will be able to understand discrete-time stochastic dynamical systems, their control, and their estimation through Kalman filters.
  3. Students will be able to gain knowledge on stochastic approximation and stochastic gradient descent methods for machine learning applications.
  4. Students will be able to gain practical programming experience on stochastic system simulations.
Language
English
Class schedule

 Class schedule HW assignments (Including preparation and review of the class.) Amount of Time Required
1． Introduction to deterministic and stochastic systems; Review of probability, events, random variables, random vectors Review of lecture notes 190minutes
2． Review of distributions, expectation, covariance matrix, law of large numbers Review of lecture notes 190minutes
3． Introduction to Markov chains; Markov property, Irreducibility, Aperiodicity, Ergodicity, Stationary distributions Review of lecture notes 190minutes
4． Hidden Markov models; Applications of Markov chains; Communication channel modeling in networked control, Google's PageRank algorithm Review of lecture notes 190minutes
5． Introduction to Python and numpy, matplotlib, scipy.stats modules. Programming Markov chains Review of lecture notes 190minutes
6． Programming for learning transition probabilities of Markov chains; Text generation using Markov chains Review of lecture notes, Homework assignment 190minutes
7． Discrete-time stochastic dynamical systems, stochastic difference equations, linear systems, covariance matrix calculation Review of lecture notes 190minutes
8． Optimal control of stochastic systems; Introduction to linear quadratic control and reinforcement learning Review of lecture notes 190minutes
9． Estimation of linear stochastic systems, Kalman filters Review of lecture notes 190minutes
10． Estimation of nonlinear stochastic systems, Extended and unscented Kalman filters Review of lecture notes; Homework assignment 190minutes
11． Programming Kalman filters in Python, Applications of Kalman filters Review of lecture notes 190minutes
12． Introduction to stochastic approximation, Robbins-Monro method, Convergence analysis Review of lecture notes 190minutes
13． Gradient descent; Stochastic gradient descent; Recent stochastic gradient descent algorithms used in training neural networks and deep learning applications Review of lecture notes 190minutes
14． Applications of stochastic gradient descent in training neural networks Review of lecture notes 190minutes
Total. - - 2660minutes
Relationship between 'Goals and Objectives' and 'Course Outcomes'

 Quiz Homework Total.
1. 10% 20% 30%
2. 5% 20% 25%
3. 5% 20% 25%
4. 0% 20% 20%
Total. 20% 80% -
Evaluation method and criteria
There are 2 homework assignments contributing to 80% of the grade (40% + 40%). There will be short quizzes on weeks 3, 5, 7, contributing respectively to 10%, 5%, 5% of the grade. Those who get at least 60% of the full score will pass this course.
Textbooks and reference materials
Lecture notes/slides will be provided for the topics covered in the course. The following books can be used for self-study: R. Serfozo, Basics of Stochastic Processes (Springer); M. H. A. Davis, R.B. Vinter, Stochastic Modeling and Control (Birkhauser); S. Meyn, Control Systems and Reinforcement Learning (Cambridge Uni. Press).
Prerequisites
Linear algebra, Probability and Statistics
Office hours and How to contact professors for questions
  • By appointment. Contact e-mail address: ahmet@shibaura-it.ac.jp
Regionally-oriented
Non-regionally-oriented course
Development of social and professional independence
  • Course that cultivates an ability for utilizing knowledge
Active-learning course
About half of the classes are interactive
Course by professor with work experience
Work experience Work experience and relevance to the course content if applicable
N/A N/A
Education related SDGs:the Sustainable Development Goals
  • 4.QUALITY EDUCATION
Last modified : Tue Sep 06 04:05:20 JST 2022