| Program / Major | mDP | Goals |
|---|---|---|
| 先進国際課程 | A-1 | A-1 Students shall obtain basic and advanced knowledge and skills in mathematics, natural and computer sciences as well as presentation skills to communicate on their knowledge with scholars from various fields. |
| (改組前)先進国際課程 | A-1 | A-1 Students shall obtain basic and advanced knowledge and skills in mathematics, natural and computer sciences as well as presentation skills to communicate on their knowledge with scholars from various fields. |
Students are expected to learn the fundamental knowledge of information theory to apply to a range of relevant real situations.
This course delves into the measurement, representation, and communication of information, situating information theory as
a critical subset of communication theory. At the heart of communication theory lie two fundamental questions that information
theory seeks to answer: What is the limit of data compression, and what is the maximum transmission rate of communication?
To address these queries, the course will cover essential topics in information theory, including information measures, lossless
and lossy data compression, channel coding, and the theory’s connections to and implications for other fields such as machine
learning.
- The students can explain the core concepts in information theory (e.g., entropy, mutual information, relative entropy, etc.)
- The students can explain the real-world integration of information theory in data compression, machine learning, etc.
- The student can apply what they learnt for tackling the challenges in their own researches or in practical applications.
| Mid-term Exam | Final Exam | Assignments | Total. | |
|---|---|---|---|---|
| 1. | 10% | 25% | 10% | 45% |
| 2. | 10% | 15% | 5% | 30% |
| 3. | 10% | 10% | 5% | 25% |
| Total. | 30% | 50% | 20% | - |
Grading Criteria:
- Midterm Exam: 30%
- Final Exam: 50%
- Assignment: 20%
- A total score of more than 60% is required to earn course credit.
＜Note＞:
- Students will be marked absent if they arrive more than 10 minutes late to class.
- If students arrive late (by less than 5 minutes) twice, it will be counted as one absence
- If a student is absent for more than one-third of the total number of classes, it means that they do not obtain sufficient knowledge to pass this course.
As the result, they will not be eligible to receive course credit.
- Student cannot use smart phone or PC during the exam.
- Midterm Exam: 30%
- Final Exam: 50%
- Assignment: 20%
- A total score of more than 60% is required to earn course credit.
＜Note＞:
- Students will be marked absent if they arrive more than 10 minutes late to class.
- If students arrive late (by less than 5 minutes) twice, it will be counted as one absence
- If a student is absent for more than one-third of the total number of classes, it means that they do not obtain sufficient knowledge to pass this course.
As the result, they will not be eligible to receive course credit.
- Student cannot use smart phone or PC during the exam.
| Class schedule | HW assignments (Including preparation and review of the class.) | Amount of Time Required | |
|---|---|---|---|
| 1. | Introduction to Information quantification and probability ＆ statistic | Review what you learnt and do the assignments | 190minutes |
| 2. | Entropy, Joint entropy and conditional entropy | Review what you learnt and do the assignments | 190minutes |
| 3. | Relative Entropy and Mutual Entropy | Review what you learnt and do the assignments | 190minutes |
| 4. | Data processing inequality, sufficient statistic and Fano’s inequality | Review what you learnt and do the assignments | 190minutes |
| 5. | Asymptotic Equipartition Property | Review what you learnt and do the assignments | 190minutes |
| 6. | Entropy Rate of a stochastic process | Review what you learnt and do the assignments | 190minutes |
| 7. | Mid-term exam and discussions on the solutions | Review all previous lessons and prepare for mid-term exam | 190minutes |
| 8. | Data compression (I) | Review what you learnt and do the assignments | 190minutes |
| 9. | Data compression (II) | Review what you learnt and do the assignments | 190minutes |
| 10. | Data compression (III) | Review what you learnt and do the assignments | 190minutes |
| 11. | Channel Capacity (I) | Review what you learnt and do the assignments | 190minutes |
| 12. | Channel Capacity (II) | Review what you learnt and do the assignments | 190minutes |
| 13. | Rate distortion theory | Review what you learnt and do the assignments | 190minutes |
| 14. | Final exam and discussions on the solutions | Review all lessons and prepare for final exam | 190minutes |
| Total. | - | - | 2660minutes |
| ways of feedback | specific contents about "Other" |
|---|---|
| Feedback in the class | Feedback on exams, assignment, etc will be done either in class or via scomb, email, etc. |
Elements of information theory - 2nd edition, Thomas M. Cover, Joy A. Thomas, 2006
- Contact professor Phan Xuan Tan based on the appointments via email: tanpx@shibaura-it.ac.jp
- Course that cultivates an ability for utilizing knowledge
| Work experience | Work experience and relevance to the course content if applicable |
|---|---|
| N/A | N/A |
Last modified : Sat Mar 14 13:40:43 JST 2026