Ng Si Ying is a Singaporean rattan artisan (@atinymaker) that specialises in the crafting with Rattan. She has crafted bags, rings and even airpods cases just by using rattan. This experience is meant for participants to feel the touch of the material she works with and also to hear about her unique story as a rattan artisan.
To explore the addition of tactile interaction within a audio-visual interface through a interaction with a rattan board.
A touch-visual-audio interface. Participants touches a rattan board triggering a visual change on the image on screen (Image goes from B&W to colored). A audio interview starts playing when the image is fully colored.
To have a visual transformation on screen that seems obvious, I changed the image from black and white to colored, symbolizing the image coming to life upon interaction. To achieve this, I started by downloading a portrait of the rattan artisan and created two images: one black and white and one colored. I added some instructions on the black and white version to help guide the participant on what to do.
To create the experience where touching the rattan triggers the interview audio to play as well as the visual change in the photograph, I used a plugin called MediaPipe that registers hand data. I directed the plugin to just detect the full hand and for it to generate a change in value when a hand is detected. I positioned the camera to frame the rattan tightly, and mapped the hand detection range to be within the area of the rattan. The change of value when hand is detected is used to trigger the saturated image to overlay the B&W image and to also trigger the audio interview, giving the illusion that the image is coming to life.
