M2093000
2 Introduction to Information Processing
This course is about the measure, representation and communication of information. Since information theory is considered
as subset of communication theory, there are two fundamental questions in communication theory that can be answered by information
theory: What is ultimate data compression and what is the ultimate transmission rate of communication. Accordingly, lectures
will focus on fundamental topics of information theory including information measures, lossless data compression, lossy data
compression, channel coding, relations to and manifestations in other areas (e.g., blockchains, machine learning, etc.)
We will conduct inperson (facetoface) class in Fall 2023. Those who are oversea and wish to take this class should contact
the lecturer via email.
Students are expected to learn the fundamental knowledge of information theory to apply to a range of relevant real situations.

Goals and objectives 
Course Outcomes 
1. 
The students can understand the fundamentals of information theory so that the beauty and utility of information science can
be exposed

A1

2. 
The students can understand the fundamental limits in communication system and can apply the equipped knowledge to various
disciplines in information science

A1

3. 
The students themselves can develop good attitude during studying, e.g., proactively discuss with classmates and lecturer,
attend the class in time, submit the assignment in time, no cheating during the exam.

A1


Class schedule 
HW assignments (Including preparation and review of the class.) 
Amount of Time Required 
1. 
Introduction to Information quantification and probability & statistic 
Selfinvestigation on information theory 
190分 
2. 
Entropy, Joint entropy and conditional entropy 
Selfinvestigation on probability 
190分 
3. 
Relative Entropy and Mutual Entropy 
Read chapter 2 in the textbook 
190分 
4. 
Data processing inequality, sufficient statistic and Fano's inequality 
Read chapter 2 in the textbook 
190分 
5. 
Asymptotic Equipartition Property 
Read chapter 3 in the textbook 
190分 
6. 
Entropy Rate of a stochastic process 
Read chapter 4 in the textbook 
190分 
7. 
Midterm exam and discussions on the solutions 
Preparation for midterm exam 
190分 
8. 
Data compression (I) 
Read chapter 5 in the textbook 
190分 
9. 
Data compression (II) 
Read chapter 5 in the textbook 
190分 
10. 
Data compression (III) 
Read chapter 5 in the textbook 
190分 
11. 
Channel Capacity (I) 
Read chapter 7 in the textbook 
190分 
12. 
Channel Capacity (II) 
Read chapter 7 in the textbook 
190分 
13. 
Rate distortion theory 
Read chapter 10 in the textbook 
190分 
14. 
Final exam and discussions on the solutions 
Preparation for final exam 
190分 
Total. 
 
 
2660分 
Relationship between 'Goals and Objectives' and 'Course Outcomes'

Midterm 
Final 
Attitude during class 
Total. 
1. 
15% 
20% 
0% 
35% 
2. 
15% 
30% 
0% 
45% 
3. 
0% 
0% 
20% 
20% 
Total. 
30% 
50% 
20% 
 
Evaluation method and criteria
Midterm exam (30%), Final Exam (50%) and Attitude During Class are the criteria of the grade. More than 60% of the total score
is needed for getting the course credit.
<Note>
Students are marked absent from the class if they are late regardless of the delay time.
If students are absent from more than one third of the total number of classes, the credit of this course cannot be given
to them.
Even though students are absent from the class whatever the reason, e.g. sickness, delay of public transportation systems,
forgetting to bring the student ID card, it is counted as absence.
Feedback on exams, assignments, etc.
ways of feedback 
specific contents about "Other" 
The Others 
Feedback on exams, assignment, etc will be done either in class or via scomb, email, etc. 
Textbooks and reference materials
Elements of information theory  2nd edition, Thomas M. Cover, Joy A. Thomas, 2006
Probability and statistics
Linear Algebra
Calculus I, II
Office hours and How to contact professors for questions
 Weekdays: From 10:00  16:30 by email or facetoface discussion at 11C05b New Building, Toyosu Campus. Making an appointment
beforehand is highly recommended.
Nonregionallyoriented course
Development of social and professional independence
 Course that cultivates an ability for utilizing knowledge
Most classes are interactive
Course by professor with work experience
Work experience 
Work experience and relevance to the course content if applicable 
N/A 
N/A 
Education related SDGs:the Sustainable Development Goals
Last modified : Sat Sep 09 07:16:50 JST 2023