Opportunities to utilize audio and visual data are increasing across a wide range of services and systems in contemporary
society. In order to effectively make use of such data, it is necessary to understand their characteristics and to apply appropriate
data processing methods according to specific contexts and objectives.
This course aims to provide students with fundamental knowledge of audio and visual data and their processing techniques. Through simple production activities that apply this knowledge, students will develop a basic understanding of audio and visual processing by integrating both theoretical and practical perspectives.
This course aims to provide students with fundamental knowledge of audio and visual data and their processing techniques. Through simple production activities that apply this knowledge, students will develop a basic understanding of audio and visual processing by integrating both theoretical and practical perspectives.
Students will acquire fundamental knowledge of the elements and characteristics of audio and visual data, as well as basic
processing techniques, and will be able to apply this knowledge to produce actual creative works.
- Explain the fundamental characteristics of audio and visual data.
- Explain the major processing methods for audio and visual data according to specific situations and objectives.
- Apply knowledge of audio and visual data processing in practice to create audio-visual works.
| Tasks | Presentation | Total. | |
|---|---|---|---|
| 1. | 20% | 10% | 30% |
| 2. | 20% | 10% | 30% |
| 3. | 40% | 40% | |
| Total. | 80% | 20% | - |
| Class schedule | HW assignments (Including preparation and review of the class.) | Amount of Time Required | |
|---|---|---|---|
| 1. | Orientation & Fundamentals of Sound — What Sounds Cannot Be Transmitted via Zoom or Telephone? As an orientation to the course, students will learn that audio and visual data are utilized in many aspects of everyday life and that such data are processed differently depending on context and purpose. The class will then cover the three fundamental elements of sound, the characteristics of sound waves, and human auditory perception, in order to develop an understanding of the mechanisms of sound. |
Set up the working environment for TouchDesigner. Complete other assigned tasks. |
200minutes |
| 2. | Analog and Digital Data, and the Fourier Transform — The Foundations of Noise Reduction, Data Compression, and Wi-Fi — This class examines the differences between analog and digital data, while developing an understanding of the advantages and limitations of digital data. Students will also learn the fundamentals of the Fourier transform. Using FFT in TouchDesigner, they will conduct frequency analysis of audio signals and attempt to create a simple VJ work that changes according to different frequency bands. |
Create a VJ work based on frequency analysis. Complete other assigned tasks. |
200minutes |
| 3. | How Visual Images Work — Why Do Flipbooks Need to Be Flipped? — This class provides an overview of how humans perceive visual images by introducing key characteristics of human vision. Based on these characteristics, students will learn the fundamental elements of visual data. In addition, students will deepen their understanding of persistence of vision and the mechanisms of moving images by constructing a zoetrope as a hands-on activity. |
Complete the zoetrope. Complete other assigned tasks. |
200minutes |
| 4. | Image and Video Data — Why Do Professionals Use Vimeo Instead of YouTube? — This class covers how images are handled as digital data, including differences between raster and vector graphics and between color models, in order to develop an understanding of their appropriate use. Students will also study the major compression methods for image and video data, and learn how to select suitable formats depending on purpose and context. |
Conduct the assigned task | 200minutes |
| 5. | The Internet and Audio-Visual Data — Let’s Build Zoom from Scratch — This class explores methods for processing audio and visual data in networked and internet-based environments. Using TouchDesigner, students will implement NDI communication and attempt to build a simple video calling system that operates over the internet. |
Create a small-scale work utilizing IP-based communication. Complete other assigned tasks. |
200minutes |
| 6. | Data Analysis and Application — How Can Data Be Used for Specific Purposes? — This class introduces various methods for analyzing visual data. Students will then gain hands-on experience with face tracking and hand tracking using the Python library MediaPipe in TouchDesigner, and will attempt to create content that applies these techniques. |
Create a small-scale work utilizing MediaPipe. Complete other assigned tasks. |
200minutes |
| 7. | Audio and Video Data Acquisition — How Can Data Be Recorded Appropriately? — This class reviews the knowledge acquired throughout the course while introducing the mechanisms and characteristics of microphones and cameras. Students will then use these devices to engage in audio recording, video shooting, and basic editing. Through these activities, they will develop an understanding of how differences in data formats, recording methods, and compression techniques affect audio-visual data. |
Create a small-scale multimodal work. Complete other assigned tasks. |
200minutes |
| Total. | - | - | 1400minutes |
Each assignment is graded on a 10-point scale, and the total score for all assignments is scaled to a maximum of 80 points.
The final grade is calculated on a 100-point scale, consisting of the total assignment score (out of 80 points) and class
participation, including questions and discussions during class sessions as well as the content of presentations given when
presenting one’s own work (out of 20 points). A total score of 60 points or higher is required to pass the course.
Please note that the evaluation methods may be revised as necessary in response to the progress of the course or changes in assignment content. Any such changes will be explained during class.
Please note that the evaluation methods may be revised as necessary in response to the progress of the course or changes in assignment content. Any such changes will be explained during class.
| ways of feedback | specific contents about "Other" |
|---|---|
| Feedback in the class | 授業時間外でも必要な場合は適宜メールやLMS等でフォローアップを行う。 |
It is desirable for students to have a basic understanding of the differences between analog and digital data, as well as
the Fourier transform, prior to the class.
- Support will be provided before the start of class or immediately after class. Follow-up will be conducted as needed via email, the LMS, or other appropriate channels.
- Course that cultivates a basic problem-solving skills
| Work experience | Work experience and relevance to the course content if applicable |
|---|---|
| Applicable | The course is taught by an instructor with professional experience in audio and visual processing. |
- 9.INDUSTRY, INNOVATION AND INFRASTRUCTURE
- 12.RESPONSIBLE CONSUMPTION & PRODUCTION
Last modified : Sat Mar 14 13:24:18 JST 2026