Course title
P07102002
Information Theory

 MANO Kazunori Click to show questionnaire result at 2016
Course description
To understand the basic theory of efficient and reliable information storage and transmission in telecommunications. Mathematical modeling of information sources and communication channels and practical coding techniques of Shannon's information theory will be explained. Exercises and practical training will be conducted to help students understand the lectures experimentally.
Purpose of class
It is possible to see the world's information and communication as a system based on Shannon's information theory.
Goals and objectives
  1. You can explain definitions and theorems of Shannon's basic model of information transmission, information entropy, source coding theorem, communication channel capacity, communication channel coding theorem, and basic concepts of error detection and correction.
  2. Through exercises and practical training, you can calculate information entropy, and communication channel capacity, etc., and can process real data for source coding such as Huffman coding and universal coding.
  3. You can explain the applications of information theory in various information and communication systems.
Relationship between 'Goals and Objectives' and 'Course Outcomes'

 Exercise (assignment report) Practical training (data processing) Final examination Total.
1. 15% 40% 55%
2. 20% 20%
3. 15% 10% 25%
Total. 30% 20% 50% -
Language
Japanese
Class schedule

 Class schedule HW assignments (Including preparation and review of the class.) Amount of Time Required
1. Class outline, What is information, Basic model of information transfer, coding theory, Preliminary mathematical knowledge. Read the syllabus. Look up the definition of "information". 60minutes
Review of information transmission models and prior knowledge of mathematics 120minutes
2. Information Source Models and Amount of Information (1): Digital Sources, Stationary and Markov Sources. Read the handout on the information source model and
the amount of information (1) 		70minutes
Review of information source model and amount of information (1) 120minutes
3. Information Source Models and Amount of Information (1): Amount of information, entropy, typical series, divergence, mutual information. Read the handout on the information source model and
the amount of information (2) 		70minutes
Review of information source model and amount of information (2) 120minutes
4. Practice (1): Calculation of amount of information and entropy Practical training (1): Programming and execution of C language programs. 190minutes
5. Communication channel models and codes (1): codes and coding, divisible codes Read the handout on communication channel models and codes (1) 70minutes
Review of communication channel models and codes (1) 120minutes
6. Communication channel models and codes (2): communication channel, communication channel capacity . Read the handout on communication channel models and codes (2) 70minutes
Review of communication channel models and codes (2) 120minutes
7. Source coding (1): Source coding theorem, Shannon coding. Read the handout on Information Source Coding (1) 70minutes
Review of source coding (1). 120minutes
8. Information source coding (2): Huffman coding. Coding of variable length information series. Read the handout on Information Source Coding (2) 70minutes
Review of source coding (2). 120minutes
9. Practice (2): Computation of information source coding (Huffman coding program). Practical training (2). Programming and execution of C language programs. 190minutes
10. Source coding (3): arithmetic coding. Read the handout on Information Source Coding (3) 70minutes
Review of source coding (3). 120minutes
11. Source coding (4): universal coding, dictionary method, block sorting method. Read the handout on Information Source Coding (4) 190minutes
12. Communication channel coding (1): decoding and receiver space, communication channel coding theorem. Read the handout on communication channel coding (1) 70minutes
Review of communication channel coding (1) 120minutes
13. Communication channel coding (2): Error correction coding (single error detection code, single error correction code, linear code, minimum distance). Read the handout on communication channel coding (2) 70minutes
Review of communication channel coding (2) 120minutes
14. Final examination, overall summary. Review and preparation for examinations 190minutes
Total. - - 2650minutes
Evaluation method and criteria
The total score of the course will consist of 50% of the total score of the exercises and practical training, and 50% of the final examination score. The total score of 60% or more is required to pass the course.
Feedback on exams, assignments, etc.
ways of feedback specific contents about "Other"
Feedback in the class
Textbooks and reference materials
Textbooks
　None in particular.
　Materials will be distributed as necessary.

Reference books
(1) "Information Theory for Beginners, 2nd Edition," Hiroshi Inai, Morikita Publishing Co., 2020.
(2) "Information Theory, Revised Second Edition," Hideki Imai, Ohmsha, 2019.
(3) "Mathematical Theory of Communication," Claude E. Shannon and Warren Weaver, translated by Tomohiko Uematsu (Chikuma Gakugei Bunko) 2009.
Prerequisites
The C language is used in the practical training, so students should be familiar with its simple usage.
Office hours and How to contact professors for questions
  • Monday 13:20-15:00
Regionally-oriented
Non-regionally-oriented course
Development of social and professional independence
  • Course that cultivates an ability for utilizing knowledge
  • Course that cultivates a basic problem-solving skills
Active-learning course
More than one class is interactive
Course by professor with work experience
Work experience Work experience and relevance to the course content if applicable
Applicable Based on his experience in the research and development of information and telecommunication systems in a telecommunication company, he teaches basic concepts and specific data processing methods to improve the efficiency and reliability of information storage and transmission in information and telecommunication.
Education related SDGs:the Sustainable Development Goals
  • 4.QUALITY EDUCATION
  • 8.DECENT WORK AND ECONOMIC GROWTH
  • 9.INDUSTRY, INNOVATION AND INFRASTRUCTURE
  • 12.RESPONSIBLE CONSUMPTION & PRODUCTION
Last modified : Thu Feb 27 10:39:47 JST 2025