Course title
Introduction to Information Processing

Course description
This course is about the measure, representation and communication of information. Since information theory is considered as subset of communication theory, there are two fundamental questions in communication theory that can be answered by information theory: What is ultimate data compression and what is the ultimate transmission rate of communication. Accordingly, lectures will focus on fundamental topics of information theory including information measures, lossless data compression, lossy data compression, channel coding, relations to and manifestations in other areas (e.g., blockchains, machine learning, etc.)

We will conduct in-person (face-to-face) class in Fall 2023. Those who are oversea and wish to take this class should contact the lecturer via email.
Purpose of class
Students are expected to learn the fundamental knowledge of information theory to apply to a range of relevant real situations.
Goals and objectives

Goals and objectives Course Outcomes
1. The students can understand the fundamentals of information theory so that the beauty and utility of information science can be exposed
2. The students can understand the fundamental limits in communication system and can apply the equipped knowledge to various disciplines in information science
3. The students themselves can develop good attitude during studying, e.g., proactively discuss with classmates and lecturer, attend the class in time, submit the assignment in time, no cheating during the exam.
Class schedule

Class schedule HW assignments (Including preparation and review of the class.) Amount of Time Required
1. Introduction to Information quantification and probability & statistic Self-investigation on information theory 190分
2. Entropy, Joint entropy and conditional entropy Self-investigation on probability 190分
3. Relative Entropy and Mutual Entropy Read chapter 2 in the textbook 190分
4. Data processing inequality, sufficient statistic and Fano's inequality Read chapter 2 in the textbook 190分
5. Asymptotic Equipartition Property Read chapter 3 in the textbook 190分
6. Entropy Rate of a stochastic process Read chapter 4 in the textbook 190分
7. Mid-term exam and discussions on the solutions Preparation for mid-term exam 190分
8. Data compression (I) Read chapter 5 in the textbook 190分
9. Data compression (II) Read chapter 5 in the textbook 190分
10. Data compression (III) Read chapter 5 in the textbook 190分
11. Channel Capacity (I) Read chapter 7 in the textbook 190分
12. Channel Capacity (II) Read chapter 7 in the textbook 190分
13. Rate distortion theory Read chapter 10 in the textbook 190分
14. Final exam and discussions on the solutions Preparation for final exam 190分
Total. - - 2660分
Relationship between 'Goals and Objectives' and 'Course Outcomes'

Mid-term Final Attitude during class Total.
1. 15% 20% 0% 35%
2. 15% 30% 0% 45%
3. 0% 0% 20% 20%
Total. 30% 50% 20% -
Evaluation method and criteria
Midterm exam (30%), Final Exam (50%) and Attitude During Class are the criteria of the grade. More than 60% of the total score is needed for getting the course credit.
Students are marked absent from the class if they are late regardless of the delay time.
If students are absent from more than one third of the total number of classes, the credit of this course cannot be given to them.
Even though students are absent from the class whatever the reason, e.g. sickness, delay of public transportation systems, forgetting to bring the student ID card, it is counted as absence.
Feedback on exams, assignments, etc.
ways of feedback specific contents about "Other"
The Others Feedback on exams, assignment, etc will be done either in class or via scomb, email, etc.
Textbooks and reference materials
Elements of information theory - 2nd edition, Thomas M. Cover, Joy A. Thomas, 2006
Probability and statistics
Linear Algebra
Calculus I, II
Office hours and How to contact professors for questions
  • Weekdays: From 10:00 - 16:30 by email or face-to-face discussion at 11C05-b New Building, Toyosu Campus. Making an appointment beforehand is highly recommended.
Non-regionally-oriented course
Development of social and professional independence
  • Course that cultivates an ability for utilizing knowledge
Active-learning course
Most classes are interactive
Course by professor with work experience
Work experience Work experience and relevance to the course content if applicable
Education related SDGs:the Sustainable Development Goals
    Last modified : Sat Sep 09 07:16:50 JST 2023