This class introduces stochastic systems and tools for their analysis, emphasizing control engineering and machine learning
applications. Stochastic systems are essential building blocks of many technical methods used in different engineering domains.
Examples include Kalman filters in control engineering and stochastic gradient descent methods in machine learning. Successful
application of such methods require a solid understanding of stochastic systems.

This class is intended to equip students with essential theoretical knowledge and practical programming skills regarding stochastic
systems used in control engineering and machine learning fields.

- Students will be able to learn properties and applications of discrete-time Markov chains.
- Students will be able to understand discrete-time stochastic dynamical systems, their control, and their estimation through Kalman filters.
- Students will be able to gain knowledge on stochastic approximation and stochastic gradient descent methods for machine learning applications.
- Students will be able to gain practical programming experience on stochastic system simulations.

Class schedule | HW assignments (Including preparation and review of the class.) | Amount of Time Required | |
---|---|---|---|

1． | Introduction to deterministic and stochastic systems; Review of probability, events, random variables, random vectors | Review of lecture notes | 190minutes |

2． | Review of distributions, expectation, covariance matrix, law of large numbers | Review of lecture notes | 190minutes |

3． | Introduction to Markov chains; Markov property, Irreducibility, Aperiodicity, Ergodicity, Stationary distributions | Review of lecture notes | 190minutes |

4． | Hidden Markov models; Applications of Markov chains; Communication channel modeling in networked control, Google's PageRank algorithm | Review of lecture notes | 190minutes |

5． | Introduction to Python and numpy, matplotlib, scipy.stats modules. Programming Markov chains | Review of lecture notes | 190minutes |

6． | Programming for learning transition probabilities of Markov chains; Text generation using Markov chains | Review of lecture notes, Homework assignment | 190minutes |

7． | Discrete-time stochastic dynamical systems, stochastic difference equations, linear systems, covariance matrix calculation | Review of lecture notes | 190minutes |

8． | Optimal control of stochastic systems; Introduction to linear quadratic control and reinforcement learning | Review of lecture notes | 190minutes |

9． | Estimation of linear stochastic systems, Kalman filters | Review of lecture notes | 190minutes |

10． | Estimation of nonlinear stochastic systems, Extended and unscented Kalman filters | Review of lecture notes; Homework assignment | 190minutes |

11． | Programming Kalman filters in Python, Applications of Kalman filters | Review of lecture notes | 190minutes |

12． | Introduction to stochastic approximation, Robbins-Monro method, Convergence analysis | Review of lecture notes | 190minutes |

13． | Gradient descent; Stochastic gradient descent; Recent stochastic gradient descent algorithms used in training neural networks and deep learning applications | Review of lecture notes | 190minutes |

14． | Applications of stochastic gradient descent in training neural networks | Review of lecture notes | 190minutes |

Total. | - | - | 2660minutes |

Quiz | Homework | Total. | |
---|---|---|---|

1. | 10% | 20% | 30% |

2. | 5% | 20% | 25% |

3. | 5% | 20% | 25% |

4. | 0% | 20% | 20% |

Total. | 20% | 80% | - |

There are 2 homework assignments contributing to 80% of the grade (40% + 40%). There will be short quizzes on weeks 3, 5,
7, contributing respectively to 10%, 5%, 5% of the grade. Those who get at least 60% of the full score will pass this course.

Lecture notes/slides will be provided for the topics covered in the course. The following books can be used for self-study:
R. Serfozo, Basics of Stochastic Processes (Springer); M. H. A. Davis, R.B. Vinter, Stochastic Modeling and Control (Birkhauser);
S. Meyn, Control Systems and Reinforcement Learning (Cambridge Uni. Press).

- By appointment. Contact e-mail address: ahmet@shibaura-it.ac.jp

- Course that cultivates an ability for utilizing knowledge

Work experience | Work experience and relevance to the course content if applicable |
---|---|

N/A | N/A |

Last modified : Tue Sep 06 04:05:20 JST 2022