M2093000
2 Introduction to Information Processing
This course delves into the measurement, representation, and communication of information, situating information theory as
a critical subset of communication theory. At the heart of communication theory lie two fundamental questions that information
theory seeks to answer: What is the limit of data compression, and what is the maximum transmission rate of communication?
To address these queries, the course will cover essential topics in information theory, including information measures, lossless
and lossy data compression, channel coding, and the theory's connections to and implications for other fields such as machine
learning.
Classes will be conducted inperson (facetoface) in Fall 2024.
Students are expected to learn the fundamental knowledge of information theory to apply to a range of relevant real situations.

Goals and objectives 
Course Outcomes 
1. 
The students can understand the fundamentals of information theory so that the beauty and utility of information science can
be exposed

A1

2. 
The students can understand the fundamental limits in communication system and can apply the equipped knowledge to various
disciplines in information science

A1

3. 
The students themselves can develop good attitude during studying, e.g., proactively discuss with classmates and lecturer,
attend the class in time, submit the assignment in time, no cheating during the exam.

A1

Relationship between 'Goals and Objectives' and 'Course Outcomes'

Midterm 
Final 
Attitude during class 
Total. 
1. 
15% 
20% 
0% 
35% 
2. 
15% 
30% 
0% 
45% 
3. 
0% 
0% 
20% 
20% 
Total. 
30% 
50% 
20% 
 

Class schedule 
HW assignments (Including preparation and review of the class.) 
Amount of Time Required 
1. 
Introduction to Information quantification and probability & statistic 
Review what you learnt and do the assignments 
190分 
2. 
Entropy, Joint entropy and conditional entropy 
Review what you learnt and do the assignments 
190分 
3. 
Relative Entropy and Mutual Entropy 
Review what you learnt and do the assignments 
190分 
4. 
Data processing inequality, sufficient statistic and Fano's inequality 
Review what you learnt and do the assignments 
190分 
5. 
Asymptotic Equipartition Property 
Review what you learnt and do the assignments 
190分 
6. 
Entropy Rate of a stochastic process 
Review what you learnt and do the assignments 
190分 
7. 
Midterm exam and discussions on the solutions 
Review all previous lessons and prepare for midterm exam 
190分 
8. 
Data compression (I) 
Review what you learnt and do the assignments 
190分 
9. 
Data compression (II) 
Review what you learnt and do the assignments 
190分 
10. 
Data compression (III) 
Review what you learnt and do the assignments 
190分 
11. 
Channel Capacity (I) 
Review what you learnt and do the assignments 
190分 
12. 
Channel Capacity (II) 
Review what you learnt and do the assignments 
190分 
13. 
Rate distortion theory 
Review what you learnt and do the assignments 
190分 
14. 
Final exam and discussions on the solutions 
Review all lessons and prepare for final exam 
190分 
Total. 
 
 
2660分 
Goals and objectives (Other Courses)
A:Fundamental Mechanical Engineering 
B:Advanced Mechanical Engineering 
C:Environment and Materials Engineering 
D:Chemistry and Biotechnology 
E:Electrical Engineering and Robotics 
G:Advanced Electronic Engineering 
F:Information and Communications Engineering 
L:Computer Science and Engineering 
H:Urban Infrastructure and Environment 
Evaluation method and criteria
Midterm exam (30%), Final Exam (50%) and Attitude During Class (20%) are the criteria of the grade. More than 60% of the total
score is needed for getting the course credit.
<Note>
Students are marked absent from the class if they are late regardless of the delay time.
If students are absent from more than one third of the total number of classes, the credit of this course cannot be given
to them.
Even though students are absent from the class whatever the reason, e.g. sickness, delay of public transportation systems,
forgetting to bring the student ID card, it is counted as absence.
Feedback on exams, assignments, etc.
ways of feedback 
specific contents about "Other" 
Feedback in the class 
Feedback on exams, assignment, etc will be done either in class or via scomb, email, etc. 
Textbooks and reference materials
Elements of information theory  2nd edition, Thomas M. Cover, Joy A. Thomas, 2006
Probability and statistics
Linear Algebra
Calculus I, II
Office hours and How to contact professors for questions
 Contact based on the appointments
Nonregionallyoriented course
Development of social and professional independence
 Course that cultivates an ability for utilizing knowledge
Most classes are interactive
Course by professor with work experience
Work experience 
Work experience and relevance to the course content if applicable 
N/A 
N/A 
Education related SDGs:the Sustainable Development Goals
 9.INDUSTRY, INNOVATION AND INFRASTRUCTURE
Last modified : Tue Mar 12 04:09:00 JST 2024