学术前沿丨音乐人工智能与音乐信息科技系系列讲座——人机协同表演研究，基于大语言模型的音乐标注与检索
更新时间：2025-12-19 22:17 浏览量：1
主讲嘉宾：Juhan Nam教授
主持人：李小兵教授
讲座一：人机协同表演研究
时间：2025年12月22日 11:00-12:30
讲座二：基于大语言模型的音乐标注与检索
时间：2025年12月22日 16:00-17:30
地点：音乐人工智能实验室
Lecture 1: Human-AI Ensemble Performance
摘要：
人机协同表演重新定义了音乐协作范式，需要交互系统具备细腻且实时的通信能力。本讲座将探讨支撑此类交互的基础技术，重点关注稳健、低延迟、多模态处理技术栈的开发。研究范围涵盖音乐信息检索（MIR）的核心任务，包括实时钢琴转谱、乐谱跟随，以及视觉处理层面的演奏者提示检测与响应式表演可视化。讲座将梳理上述领域的技术前沿进展，并展示由前沿 MIR 技术驱动的成功舞台表演案例研究。
Abstract:
Human-AI ensemble performance redefines musical collaboration, requiring interactive systems capable of nuanced, real-time communication. This talk investigates the foundational technologies enabling this interaction, focusing on the development of robust, low-latency, and multimodal processing pipelines. The scope covers core Music Information Retrieval (MIR) tasks, including real-time piano tranion, score following as well as visual processing tasks such as performer cue detection and reactive performance visualization. Technical advancements across these domains will be discussed, culminating in case studies of stage performances successfully driven by state-of-the-art MIR technology.
2
讲座二：基于大语言模型的音乐标注与检索
Lecture 2: LLM-Powered Music Annotation and Retrieval
摘要：
大语言模型（LLM）的广泛应用已将先进的语言通信集成到众多工作流程中，从根本上改变了知识获取与处理的方式。在音乐信息检索（MIR）领域，LLM 正引发音乐搜索与推荐系统的范式变革。本讲座将介绍音乐-文本理解的演变过程，即从传统的单模态分类模型转向融合性的多模态音乐 LLM。讲座将展示此类模型如何借助富有表现力的自然语言描述与动态多轮对话接口，生成丰富的内容标注并实现精细化的音乐检索。讨论将进一步剖析这一转型的技术基础及其对未来 MIR 应用的深远影响。
Abstract:
The widespread deployment of Large Language Models (LLMs) has integrated advanced, language-based communication into numerous workflows, fundamentally altering how knowledge is accessed and processed. Within the domain of Music Information Retrieval (MIR), LLMs are initiating a paradigm shift for music search and recommendation systems. This talk presents the evolution of music-text understanding, moving beyond conventional, single-modality classification models toward integrated multimodal musical LLMs. We will demonstrate how these advancements enable the generation of rich content annotations and facilitate fine-grained music retrieval by leveraging both expressive natural language deions and dynamic, multi-turn conversational interfaces. The discussion will cover the technical underpinnings and implications of this transition for future MIR applications.
嘉宾简介
Juhan Nam教授
Juhan Nam现任韩国科学技术院（KAIST）教授、文化科技研究生院音乐与音频计算实验室主任。研究聚焦于音乐信息检索、音频信号处理及人工智能音乐应用等。他同时担任 Sumi Jo 表演艺术研究中心主任，致力于与艺术家协作开发面向音乐表演与教育的创新技术。他于斯坦福大学计算机音乐与声学研究中心（CCRMA）获音乐博士学位，在开启学术生涯之前，他曾任职于英昌乐器（Young Chang / Kurzweil），负责合成器和数码钢琴的研发工作。他亦担任第26届国际音乐信息检索会议ISMIR 2025主席。
Juhan Nam is a Professor at the Korea Advanced Institute of Science and Technology (KAIST), South Korea. He leads the Music and Audio Computing Lab at the Graduate School of Culture Technology, where his research focuses on music information retrieval, audio signal processing, and AI-based music applications. He also serves as the Director of the Sumi Jo Performing Arts Research Center, fostering collaborations with artists to develop innovative technologies for music performance and education. He received his Ph.D. in Music from Stanford University, where he studied at the Center for Computer Research in Music and Acoustics (CCRMA). Before his academic career, he worked at Young Chang (Kurzweil), developing synthesizers and digital pianos. He also was the General Chair of the 26th International Society for Music Information Retrieval Conference (ISMIR 2025).
供稿：音乐人工智能与音乐信息科技系
设计：张裕欣
“央音大讲堂”第11讲丨文化的本质与创造新的文化生命体 ——学习习近平文化思想
院长于红梅会见阿德莱德大学代表团一行