Analyzing Jo-Ha-Kyu Mechanism in Japanese Traditional Performing Art Ningyo Joruri

ABSTRACT

IoT devices that connect people without physical contact become more and more important after the COVID-19 impact. However, strange appearances and movements performed by IoT devices (interactive humanoid robots) cause human discomfort, so-called the uncanny valley, preventing widespread use of humanoid IoT devices. On the contrary, a Japanese traditional performing art named Ningyo Joruri (puppet theater) is recognized as a UNESCO intangible cultural heritage, and the sophisticated puppet motions and its unique music style somehow can avoid causing human discomfort even if the appearance of puppets is close enough to humans. One of the most important factors in empathizing humans with the puppet without uncomfortable is the modulation technique of both music tempo and motion speed known as Jo-Ha-Kyu. In this study, we analyzed Ningyo Joruri based on the Jo-Ha-Kyu mechanism, which is an art concept adopted in the puppet theater to interact with audiences according to modulation of the tempo. First, we obtained puppet movements using motion capture systems with the music. Second, we detected the changing tempo in Ningyo Joruri using the deep learning method to demonstrate the Jo-Ha-Kyu mechanism quantitatively. Finally, we showed the correlation of Jo-Ha-Kyu between Ningyo Joruri music and puppet manipulation techniques in the frequency domain using the Hilbert Huang transform. Our results revealed that low-frequency movements play an important role in synchronizing motion to the tempo of corresponding music, presenting novel knowledge to motion designers for humanoid robots IoT devices. © 2022 IEEE.