Skip to main content
Sound Design

Shangyun Wu

Upcoming Event

《 SHANGAVATAR 》ELECTRIFY , 25th, July 2022 Iklectik

Conference Papers

2020 《 Implementation of Event Classification and Spatio-temporal Information Integration Analysis based on Meteorological Records in Historical Texts: A Digital Humanities Case Study on Climate Change and Social Trends 》 Digital Humanities 2020

2019 《 Climate Event Classification Based on Historical Meteorological Records and Its Presentation on A Spatio-Temporal Research Platform 》 Digital Humanities 2019


Awards

2022 Design Education Trust - 4D Project Award ( UK ) 

2022 Shortlisted Taoyuan Technology Performing Arts Award ( TW ) 


Exhibitions

2021《 In Between 》 Online ( TW ) 

2020《 In Between 》 Science Gallery Venice ( IT ) 


Performances 

2021 《 Rising, Warming, Melting 》AIR ( COP26 ) Soundcamp & IKLECTIK London ( UK ) 

2016 《 Taipei Impropotu 》P Festival with Berlin Quartet ( TW ) 


Degree Details

School of DesignInformation Experience Design (MA)Sound Design

Show Location: Battersea campus: Studio Building, First Floor

ShangYunWu is a sound and media researcher, designer and creative technologist from Taiwan. Prior to joining the RCA I worked for a pop science media company, and did some independent research for Academia Sinica in Taiwan.

I have also composed music for different mediums, including the social advisement soundtrack for Cerruti 1881 SS2019 Backtage, piano composition for Taipei Sinfonietta and Philharmonic Orchestra, press conference music for Fashion Designer Mr. Sun and the ceremony music for Children Film Festival in Taiwan. I am fascinated by the process of bridging these two worlds.

Since starting IED, my work has focused on “ SOUND+MEDIA ” as the research topic aiming to shift perspective from how information is sent to how it is sought for shaping the human's communication patterns, especially focusing on sonic information. Therefore, I chose synthetic media - AI Voice as the topic for my final project.


A Visual Design
Documentary Film

< I, AI CREATE TOGETHER - Shangavatar > is an experimental media and sound project aiming to clone Shangyun herself as an avatar for replacing the social activities and performance activities which need to face a wider audience in her life. 


Medium:

Voice, 3D Modal, Camera, Screen, Speakers, Headphone, Computer, Artificial Intelligence Algorithm

Size:

Dimensions Variable
A Voice Video
Voice, Emotion and Identity Design Reference

As a sound and media researcher, designer and engineer, I have spent years in these two specialties. During my time at the RCA I chose one of the intersections between sound and media which is voice. With synthetic media technology, the AI voice with emotion could offer the capability of suggesting identity, and this could be applied in creating digital avatars and in different mediums. 

In this project, I trained my voice into an AI voice that carries different emotions. The voice becomes the signature of the avatar when it engages with humans in small talk. As a memory of the academic life experience in the UK, the dialog is designed with British daily conversation, that is, it’s English with a slightly Taiwanese accent. Also the AI voice is designed as a part of the interactive music system while Shangavatar plays the role as a musician. 

The showcase of Shangavatar will be on 25th July in iklectik. You are welcome to come and join us! 

API Support : Resemble AI 

copyright © 2022 ShangYunWu all rights reserved 

Medium:

Voice, Camera, Speakers, Headphones, Computer, Artificial Intelligence Algorithm

Size:

Dimensions Variable
My Facial Expression While I Talk
My Facial Expression While I Talk
Shangavatar's Facial Expressions While Ze Talks
Shangavatar's Facial Expressions While Ze Talks
Facial Expression, Emotion and Identity Design Reference

One of the key points of defining the avatar as Shangyun ( Me ) includes the facial expression while I talk with others. It’s a long process for mapping the real data of facial expression with my real 3D scan model. The negotiation between a realistic facial expression and the preset animation took me lots of time to decide. Through the whole research and development process, I decided to take my real facial expression as the physical feature is a scientific choice while designing the identity of the avatar.

copyright © 2022 ShangYunWu all rights reserved 

Medium:

3D modal, Camera, Screens, Computer, Artificial Intelligence Algorithm

Size:

Dimensions Variable
Shangavatar zir way in growing up.

The previous Shangavatars are shown in the above videos. According to  “ Self, Identity, and Social Institutions'' by D. Heise, N. MacKinnon, it is the emotional voice and realistic facial expressions that carry the sense of personal identity. To test this, Shangavatar was designed as different iterations with varying appearances and variations on the AI voice. These include a cartoon version, a realistic photo version and a few 3d model versions..etc.

Medium:

2D Visual, Photos, 3D Modal, Voice, Camera, Screen, Speakers, Computer, Artificial Intelligence Algorithm

Size:

Dimensions Variable
How can you engage with Shangavatar
How can you engage with Shangavatar

The starting point of designing Shangavatar is due to a shangyun’s personal need. That is, the whole interaction between the avatar and other humans is based on Shangyun’s ( my )  social interactions and experiences based on a human’s visual and auditory perception.

I endowed the avatar with a function based on the concept of a “meaningful glance”, to detect human presence and emotion in a specific spatial area . Once we enter the area, the avatar will attempt to engage us in conversion. The avatar can talk with appropriate emotion and give the relevant feedback which is what humans do most of the time in a social venue. 

This project is based upon research in the following books,  “ Self, Identity, and Social Institutions'' by D. Heise, N. MacKinnon, “ Musical Robots and Interactive Multimodal Systems '' by Jorge Solis, Kia Ng (auth.) and “Designing Sociable Robots” by Cynthia L. Breezee. 

copyright © 2022 ShangYunWu all rights reserved 

Medium:

Voice, 3D Modal, Camera, Screen, Speakers, Headphone, Computer, Artificial Intelligence Algorithm

Size:

Dimensions Variable
The Upcoming Events With Shangavatar

The project is designed based on Shangyun's personal experience. Thus, it is designed to have a small talk before and after a performance. In order to convey the whole experience to the audience, the shown mock-up sketch is designed with a scenario from a conversation to a facial detection screen till a performative installation. 

However, Shangavatar, only recently born, will be shown in the RCA’s physical graduate show as a social character role. At the same time, it will participate in a music showcase in Ikletic on 25th, July. You are welcome to save your day to join its showcase. ;) 

copyright © 2022 ShangYunWu all rights reserved 

Medium:

Voice, 3D Modal, Camera, Screen, Speakers, Headphone, Computer, Artificial Intelligence Algorithm

Size:

Dimensions Variable

Creating Shangavatar is a long process, I would like to thank for those who helps me to create Shangavatar with their professional skills and time, especially my friends and family for mental supports with love. This project is for those who love freedom.


Creator - ShangYunWu

Sound and Voice Technician - ShangYunWu

Music Producer - ShangYunWu

Small Talk Consultant - Sophie Maria Vickers

AI Voice Model and API Technical Supporter - Resemble AI 

First Avatar Visual Designer - Lara Pfennigdorff

3D Model Design - ShangYunWu

3D Model Technician - Mike Faulknr

3D Model Consultant - Mike Faulknr, Kam Raoofi , Kyle Ramsey

3D Scan Technician - Justin Calder, Kar Leung Wai

Interaction Designer - ShangYunWu

Interaction Technician - ShangYunWu

Installation Designer - ShangYunWu

Installation Consultant - Rodrigo Canas 

Concept Film Director - ChunTienHsiao

Concept Film Producer - ShangYunWu

Documentary Film Supporters - Chun Tien, Wen Pay, Jun Jun, Shi Ya, Yushen Wu, Ziqin Hou, Wen Hui

Special Thanks - Resemble AI , Professor Robert Rowe , Dr Matt Lewis


Design Education Trust - 4D Project Award