Course title
330137001
Information Theory

MANO Kazunori
Middle-level Diploma Policy (mDP)
Program / Major mDP Goals
IoT Course DP-4a・2 キャリアを見据えた高度な専門知識
情報処理やネットワーキングに関する技術を駆使し、情報社会の基盤となるIoTシステムを開発できる。
Software Course DP-4b・2 キャリアを見据えた高度な専門知識
ソフトウェア開発手法、情報ネットワーク、および機械学習に関する技術を駆使し、社会的ニーズに適切に対応したソフトウェアを開発できる。
Media Course DP-4c・2 キャリアを見据えた高度な専門知識
画像や音響、サイバースペース等のメディア処理・デザイン技術を駆使し、社会的ニーズに適切に対応したシステムを開発できる。
Data Science Course DP-4d・2 キャリアを見据えた高度な専門知識
多様なデータを収集・分析・予測する技術を駆使し、社会に存在する実際の課題に対してエビデンスを基に解決法を考え提案できる。
Mechatronics Course DP-4・3 専門分野と他分野を関連付ける素養
主軸となる分野の専門知識を他分野と関連付ける分野横断型の知識と行動力を修得し、社会で活用できる。
Architecture and Architectural Engineering Course DP-4a・2 キャリアを見据えた高度な専門知識
国内外を問わず多様な人々と協働しながら、最新技術とシステム思考、柔軟な発想力でこれからの社会に必要とされる空間(建築)を創出できる。
Environmental Systems and Urban Planning Course DP-4b・2 キャリアを見据えた高度な専門知識
国内外を問わず多様な人々と協働しながら、持続可能なまちづくりに必要な環境システム、対策、政策、ビジネス等を提案し、社会実装に寄与できる。
Bioscience Course DP-4a・3 専門分野と他分野を関連付ける素養
主軸となる分野の専門知識を他分野と関連付ける分野横断型の知識と行動力を修得し、社会で活用できる。
Biomedical Engineering Course DP-4b・3 専門分野と他分野を関連付ける素養
主軸となる分野の専門知識を他分野と関連付ける分野横断型の知識と行動力を修得し、社会で活用できる。
Sports Engineering Course DP-4c・3 専門分野と他分野を関連付ける素養
主軸となる分野の専門知識を他分野と関連付ける分野横断型の知識と行動力を修得し、社会で活用できる。
Mathematical Sciences Course DP-4・2 キャリアを見据えた高度な専門知識
現象の背後にある数理構造やデータのパターンを理論的に解析し、社会や自然科学、工学における複雑な課題に対して数理的視点から解決戦略を提案できる。
Mathematical Sciences Course DP-4・3 専門分野と他分野を関連付ける素養
主軸となる分野の専門知識を他分野と関連付ける分野横断型の知識と行動力を修得し、社会で活用できる。
Purpose of class
It is possible to see the world’s information and communication as a system based on Shannon’s information theory.
Course description
To understand the basic theory of efficient and reliable information storage and transmission in telecommunications. Mathematical modeling of information sources and communication channels and practical coding techniques of Shannon’s information theory will be explained. Exercises and practical training will be conducted to help students understand the lectures experimentally.
Goals and objectives
  1. You can explain definitions and theorems of Shannon’s basic model of information transmission, information entropy, source coding theorem, communication channel capacity, communication channel coding theorem, and basic concepts of error detection and correction.
  2. Through exercises and practical training, you can calculate information entropy, and communication channel capacity, etc., and can process real data for source coding such as Huffman coding and universal coding.
  3. You can explain the applications of information theory in various information and communication systems.
Relationship between 'Goals and Objectives' and 'Course Outcomes'

Exercise (assignment report) Practical training (data processing) Final examination Total.
1. 15% 40% 55%
2. 20% 20%
3. 15% 10% 25%
Total. 30% 20% 50% -
Evaluation method and criteria
The total score of the course will consist of 50% of the total score of the exercises and practical training, and 50% of the final examination score. The total score of 60% or more is required to pass the course.
Submitting all assignments and demonstrating a good understanding of the course content will be considered equivalent to a score of 80 points.
Language
Japanese
Class schedule

Class schedule HW assignments (Including preparation and review of the class.) Amount of Time Required
1. Class outline, What is information, Basic model of information transfer, coding theory, Preliminary mathematical knowledge. Read the syllabus. Look up the definition of ”information”. 60minutes
Review of information transmission models and prior knowledge of mathematics 120minutes
2. Information Source Models and Amount of Information (1): Digital Sources, Stationary and Markov Sources. Read the handout on the information source model and
the amount of information (1)
70minutes
Review of information source model and amount of information (1) 120minutes
3. Information Source Models and Amount of Information (1): Amount of information, entropy, typical series, divergence, mutual information. Read the handout on the information source model and
the amount of information (2)
70minutes
Review of information source model and amount of information (2) 120minutes
4. Practice (1): Calculation of amount of information and entropy Practical training (1): Programming and execution of C language programs. 190minutes
5. Communication channel models and codes (1): codes and coding, divisible codes Read the handout on communication channel models and codes (1) 70minutes
Review of communication channel models and codes (1) 120minutes
6. Communication channel models and codes (2): communication channel, communication channel capacity . Read the handout on communication channel models and codes (2) 70minutes
Review of communication channel models and codes (2) 120minutes
7. Source coding (1): Source coding theorem, Shannon coding. Read the handout on Information Source Coding (1) 70minutes
Review of source coding (1). 120minutes
8. Information source coding (2): Huffman coding. Coding of variable length information series. Read the handout on Information Source Coding (2) 70minutes
Review of source coding (2). 120minutes
9. Practice (2): Computation of information source coding (Huffman coding program). Practical training (2). Programming and execution of C language programs. 190minutes
10. Source coding (3): arithmetic coding. Read the handout on Information Source Coding (3) 70minutes
Review of source coding (3). 120minutes
11. Source coding (4): universal coding, dictionary method, block sorting method. Read the handout on Information Source Coding (4) 190minutes
12. Communication channel coding (1): decoding and receiver space, communication channel coding theorem. Read the handout on communication channel coding (1) 70minutes
Review of communication channel coding (1) 120minutes
13. Communication channel coding (2): Error correction coding (single error detection code, single error correction code, linear code, minimum distance). Read the handout on communication channel coding (2) 70minutes
Review of communication channel coding (2) 120minutes
14. Final examination, overall summary. Review and preparation for examinations 190minutes
Total. - - 2650minutes
Feedback on exams, assignments, etc.
ways of feedback specific contents about "Other"
Feedback in the class
Textbooks and reference materials
Textbooks
 None in particular.
 Materials will be distributed as necessary.

Reference books
(1) ”Information Theory for Beginners, 2nd Edition,” Hiroshi Inai, Morikita Publishing Co., 2020.
(2) ”Information Theory, Revised Second Edition,” Hideki Imai, Ohmsha, 2019.
(3) ”Mathematical Theory of Communication,” Claude E. Shannon and Warren Weaver, translated by Tomohiko Uematsu (Chikuma Gakugei Bunko) 2009.
Prerequisites
Basic data processing programming skills (C, Python, etc.).
Office hours and How to contact professors for questions
  • Monday 13:20-15:00
Regionally-oriented
Non-regionally-oriented course
Development of social and professional independence
  • Course that cultivates an ability for utilizing knowledge
  • Course that cultivates a basic problem-solving skills
Active-learning course
More than one class is interactive
Course by professor with work experience
Work experience Work experience and relevance to the course content if applicable
Applicable Based on his experience in the research and development of information and telecommunication systems in a telecommunication company, he teaches basic concepts and specific data processing methods to improve the efficiency and reliability of information storage and transmission in information and telecommunication.
Education related SDGs:the Sustainable Development Goals
  • 4.QUALITY EDUCATION
  • 8.DECENT WORK AND ECONOMIC GROWTH
  • 9.INDUSTRY, INNOVATION AND INFRASTRUCTURE
  • 12.RESPONSIBLE CONSUMPTION & PRODUCTION
Last modified : Sat Mar 14 14:46:26 JST 2026