This course is about the measure, representation and communication of information. Since information theory is considered
as subset of communication theory, there are two fundamental questions in communication theory that can be answered by information
theory: What is ultimate data compression and what is the ultimate transmission rate of communication. Accordingly, lectures
will focus on fundamental topics of information theory including information measures, lossless data compression, lossy data
compression, channel coding, relations to and manifestations in other areas (e.g., blockchains, machine learning, etc.)
We are expected to conduct in-person (face-to-face) class in Fall 2022. Those who are oversea and wish to take this class should contact the lecturer via email.
We are expected to conduct in-person (face-to-face) class in Fall 2022. Those who are oversea and wish to take this class should contact the lecturer via email.
Students are expected to learn the fundamental knowledge of information theory to apply to a range of relevant real situations.
| Goals and objectives | Course Outcomes | |
|---|---|---|
| 1. | The students can understand the fundamentals of information theory so that the beauty and utility of information science can be exposed |
A-1
|
| 2. | The students can understand the fundamental limits in communication system. The students can apply the equipped knowledge of information theory to various disciplines in information science |
A-1
|
| 3. | The students themselves can develop good attitude during studying, e.g., proactively discuss with classmates and lecturer, attend the class in time, submit the assignment in time, no cheating during the exam. |
A-1
|
| Class schedule | HW assignments (Including preparation and review of the class.) | Amount of Time Required | |
|---|---|---|---|
| 1. | Introduction to Information quantification and probability & statistic | Self-investigation on information theory | 190分 |
| 2. | Entropy, Joint entropy and conditional entropy | Self-investigation on probability | 190分 |
| 3. | Relative Entropy and Mutual Entropy | Read chapter 2 in the textbook | 190分 |
| 4. | Data processing inequality, sufficient statistic and Fano's inequality | Read chapter 2 in the textbook | 190分 |
| 5. | Asymptotic Equipartition Property | Read chapter 3 and 4 in the textbook | 190分 |
| 6. | Entropy Rate of a stochastic process (II) | Read chapter 3 and 4 in the textbook | 190分 |
| 7. | Mid-term exam and discussions on the solutions | Read chapter 5 in the textbook | 190分 |
| 8. | Data compression (I) | Preparation for mid-term exam | 190分 |
| 9. | Data compression (II) | Read chapter 5 in the textbook | 190分 |
| 10. | Channel Capacity (I) | Read chapter 7 in the textbook | 190分 |
| 11. | Channel Capacity (II) | Read chapter 7 in the textbook | 190分 |
| 12. | Gaussian channel and Differential Entropy | Read chapter 8 and 9 in the textbook | 190分 |
| 13. | - Rate distortion theory - practical applications of information theory |
Read chapter 10 in the textbook | 190分 |
| Investigate concepts in machine learning, image compression and blockchain technology | |||
| 14. | Final exam and discussions on the solutions | Preparation for final exam | 190分 |
| Total. | - | - | 2660分 |
| Mid-term | Final | Attitude during class | Total. | |
|---|---|---|---|---|
| 1. | 15% | 20% | 0% | 35% |
| 2. | 15% | 30% | 0% | 45% |
| 3. | 0% | 0% | 20% | 20% |
| Total. | 30% | 50% | 20% | - |
Midterm exam (30%), Final Exam (50%) and Attitude During Class are the criteria of the grade. More than 60% of the total score
is needed for getting the course credit.
<Note>
Students are marked absent from the class if they are late regardless of the delay time.
If students are absent from more than one third of the total number of classes, the credit of this course cannot be given to them.
Even though students are absent from the class whatever the reason, e.g. sickness, delay of public transportation systems, forgetting to bring the student ID card, it is counted as absence.
<Note>
Students are marked absent from the class if they are late regardless of the delay time.
If students are absent from more than one third of the total number of classes, the credit of this course cannot be given to them.
Even though students are absent from the class whatever the reason, e.g. sickness, delay of public transportation systems, forgetting to bring the student ID card, it is counted as absence.
Elements of information theory - 2nd edition, Thomas M. Cover, Joy A. Thomas, 2006
- Weekdays: From 10:00 - 16:30 by email or face-to-face discussion at 11C05-b New Building, Toyosu Campus. Making an appointment beforehand is highly recommended.
- Course that cultivates an ability for utilizing knowledge
| Work experience | Work experience and relevance to the course content if applicable |
|---|---|
| N/A | N/A |
Last modified : Thu Oct 06 04:03:23 JST 2022