With a camera in hand, I decided to head down to Orchard Road for a little photo walk. Strolling along the crowded streets, The Buskers' lively performances juxtapose the common chatters and strolls of others. With this, I knew that I found a great subject for a mini photo series. The photographs did a good job at telling the narratives of The Buskers by freezing a moment of performance, but what if we can hear the photographs? Would it help us to understand the stories better?
To explore the addition of sensory modalities to photographs, by starting with and focusing on the addition of the sense of hearing through adding soundscapes.
An audio reactive photograph that has boxes with the object/person’s name popping up when the said objects/person are audible in the soundscape.
Within Adobe Illustrator, I drawn boxes that covers the elements which I want to highlight and coupled it with a text label. I used the image as a reference throughout this process to gauge the size of the boxes.
After compiling a soundscape relevant to the relevant photographs, I put it into TouchDesigner to isolate and extract specific audio data regarding to each part of the soundscape (I.E Isolating Sound of Traffic, Er Hu instrument). With these individual audio data, I normalized and mapped them to trigger it's corresponding boxes, for example, letting the sound of Er Hu trigger the appearance of the Er Hu box and text. This was done for the different components in the photograph and soundscape to create the audio-reactive visual.