Information theory is a theory that answers basic questions such as "What is information?", "Is it possible to quantitatively
grasp information?", and "What is the mathematical structure of information?". It also provides the basis for communication
and data compression.
This course covers Shannon's information theory and source coding and channel coding, which are basic technologies for handling digital information.
Digitally coded data such as video, music, voice, and text have a certain degree of redundancy, and it is possible to compress the data size without compromising the quality of the information. However, it is known that the limit of how much data can be compressed is determined by the "amount of information" contained in each piece of data. Shannon theory provides the theoretical framework.
Error correcting codes are also an effective technology for dealing with bit errors that are unavoidable when handling data in various digital systems such as computers and networks, and for correctly transmitting and receiving information.
In this course, we will encourage understanding by carrying out problem exercises both in and outside of class.
This course covers Shannon's information theory and source coding and channel coding, which are basic technologies for handling digital information.
Digitally coded data such as video, music, voice, and text have a certain degree of redundancy, and it is possible to compress the data size without compromising the quality of the information. However, it is known that the limit of how much data can be compressed is determined by the "amount of information" contained in each piece of data. Shannon theory provides the theoretical framework.
Error correcting codes are also an effective technology for dealing with bit errors that are unavoidable when handling data in various digital systems such as computers and networks, and for correctly transmitting and receiving information.
In this course, we will encourage understanding by carrying out problem exercises both in and outside of class.
In the first half, students will learn about the principles of lossless information compression and representative source
coding methods such as Huffman coding, and will also understand the concept of quantifying information based on occurrence
probability (information content and entropy).
In the second half, students will learn about the principles of error correction and master the encoding and decoding operations of basic codes such as parity codes, Hamming codes, and cyclic codes.
Furthermore, students will understand the concept of linear codes and understand encoding and decoding operations using matrices.
In the second half, students will learn about the principles of error correction and master the encoding and decoding operations of basic codes such as parity codes, Hamming codes, and cyclic codes.
Furthermore, students will understand the concept of linear codes and understand encoding and decoding operations using matrices.
- The relationship between information content, entropy, and average code length can be correctly explained.
- For a given model of a communication channel, one can select a coding scheme that allows the communication of information with no more than a specified decoding error rate.
- To understand the concept of linear codes and be able to perform encoding and decoding calculations using matrices.
- To understand the concept of error correcting codes and be able to perform specific encoding and decoding calculations for basic methods such as parity codes, Hamming codes, and cyclic codes.
| 中間試験 | 期末試験 | Total. | |
|---|---|---|---|
| 1. | 20% | 20% | |
| 2. | 20% | 20% | |
| 3. | 30% | 30% | |
| 4. | 30% | 30% | |
| Total. | 40% | 60% | - |
The midterm exam, exercises, reports, and quizzes will be rated at 40%, and the final exam will be rated at 60%.
A total score of at least 60% will be considered a pass.
Scoring guideline: You can earn 60 points or more if you thoroughly review the content covered in each lesson and are able to derive the answers to the examples and exercises covered in class, as well as similar problems.
A total score of at least 60% will be considered a pass.
Scoring guideline: You can earn 60 points or more if you thoroughly review the content covered in each lesson and are able to derive the answers to the examples and exercises covered in class, as well as similar problems.
| Class schedule | HW assignments (Including preparation and review of the class.) | Amount of Time Required | |
|---|---|---|---|
| 1. | What is information theory? - The role of codes in digital systems Source coding and channel coding - Principles of error-correcting codes Using repetition codes as an example |
Review high school course "Probability and Statistics." | 50minutes |
| Review the high school course "Logarithms and Their Properties." | 50minutes | ||
| Read Chapter 1 of the textbook. | 80minutes | ||
| 2. | Information and entropy - Information context - Entropy - Properties of entropy - Joint entropy - Conditional entropy - Mutual information |
Read Chapter 2 of the textbook. | 190minutes |
| 3. | Information source model (1) - Statistical representation of information source - Basic model of information source - Markov information source |
Read Chapter 3, sections 3.1 to 3.3 in the textbook. | 190minutes |
| 4. | Information source model (2) - Stochastic model of Markov source - Entropy of information source |
Read Chapter 3, sections 3.4 to 3.5 in the textbook. | 190minutes |
| 5. | Source coding and its limitations - Basics of source coding - Conditions for efficient codes - Code trees - Craft's inequality - Bounds on the average code length |
Read Chapter 4 of the textbook. | 190minutes |
| 6. | Source coding methods - Huffman coding - Block Huffman coding |
Read Chapter 5, sections 5.1 to 5.2 in the textbook. | 190minutes |
| 7. | Mid-term exam and the answer explanation after the exam | Review in preparation for the mid-term exam. | 190minutes |
| 8. | Data transmission channel models - Statistical representation of channels - Stationary channels without memory - Additive binary channels |
Read Chapter 6 of the textbook. | 190minutes |
| 9. | Limits of channel coding - What is channel coding? - Channel capacity - Channel coding theorem |
Read Chapter 7, sections 7.1 to 7.3 in the textbook. | 190minutes |
| 10. | Basics of linear codes - Single parity check codes - Systematic codes and linear codes - Horizontal and vertical parity check codes |
Read Chapter 8, section 8.1 in the textbook. | 190minutes |
| 11. | Hamming code - (7,4)-Hamming code - Generator matrix and check matrix - General case of Hamming code |
Read Chapter 8, section 8.2 in the textbook. | 190minutes |
| 12. | Cyclic codes (1) - Polynomial representation of binary sequences - How to construct cyclic codes |
Read Chapter 8, sections 8.3.1 to 8.3.2 in the textbook. | 190minutes |
| 13. | Cyclic Codes (2) - Error detection and correction capabilities of cyclic codes |
Read through Chapter 8, sections 8.3.3 to 8.3.4 in the textbook. | 190minutes |
| 14. | Final exam and the answer explanation after the exam | Review in preparation for the final exam. | 190minutes |
| Total. | - | - | 2650minutes |
| ways of feedback | specific contents about "Other" |
|---|---|
| Feedback in the class |
"Information Theory from the Basics" (in Japanese), by Atsuyoshi Nakamura, Takuya Kita, Shinichi Minato, and Yoshihiro Hirose,
Muisuri Publishing.
- Course that cultivates a basic problem-solving skills
| Work experience | Work experience and relevance to the course content if applicable |
|---|---|
| N/A | 該当しない |
Last modified : Sat Mar 08 04:28:18 JST 2025