Interaction - Bachelor
Flux is an interactive audio-visual experience designed for DJs and small-scale music events. It utilises projection mapping, generative audio-reactive visuals, and live video input to transform simple spaces into unique, immersive environments. Flux makes high-impact visuals accessible while exploring how technology can bring people together through sound, light, and movement.
Flux is a system created with TouchDesigner and BlackHole that takes video and audio input and processes it to create a visual spectacle that can turn an ordinary room into an immersive audio-visual experience.
TouchDesigner is a node based, visual programming software. Within TouchDesigner the Flux system is a network that consists of two video inputs (one for the performer and one for the audience) and an audio input that receives music the DJ is playing. Unique visuals are computer generated and animated to be audio-reactive in addition to composites of videos and text all of which are then mapped and projected to surfaces behind a DJ in real time.
Through a series of nodes the audio is analysed, splitting it into high frequencies (snare), low frequencies (kick) and a combination of the two (rhythm) and these parameters are used as references to drive the visuals. The video inputs are processed to aesthetically match the visualisation, including a cloud of pixels for the audience camera that flows based on the audience’s movement. Kantan Mapper (TouchDesigner’s mapping feature) is used to “map” i.e. draw and layer shapes on the wall behind the DJ. The processed TOPS (texture operators) of these networks are then assigned to the shapes and the entire scene is sent to a projector. To increase the visual interest of the projections white projection surfaces including flowers, abstract shapes and a picture frame were constructed and fixed behind the DJ to be mapped onto.
The result is live visuals that are audio-reactive to the DJ’s music and to the movement of the audience, enhancing atmosphere and immersion.
Having worked in hospitality and live events for over 10 years, and immersing myself in music culture as I grew up, I knew I wanted my capstone project to be something that deeply reflected my interests and passions. Throughout my degree I have been given the opportunity to learn a variety of different skills but something that has always inspired me has been immersive and interactive art. I believe I will have many opportunities in my career to create well designed products, systems or services and therefore I wanted to take this opportunity to design an experience and learn about a technology that has always been fascinating to me: projection mapping. To explore this I came up with the brief:
How can I enhance the atmosphere of small music events with projection mapping?
During my research I identified local DJs and concert-goers as my two primary users, and I analysed their responses for insights into the goals that really mattered for them. At small events DJs and event producers rarely have large budgets for visuals, however almost every respondant to my survey was emphatic about lighting and visuals playing a crucial role in enhancing atmosphere. One of the largest issues was the expense of the equipment, the set up and the technical expertise needed to have “big show” lighting, so I focused on creating an audio-visual system that could have high impact but reduce cost by removing the need for a dedicated AV technitian or multitudes of heavy equipment.
Beneath the obvious objectives of having a great time and immersing themselves in the music, I discovered that both the performers and the audience were looking for “success queues” (smiling, dancing, body movement, attention) to increase their confidence and pleasure within the environment. A goal that surfaced that took me by surprise was that users were also concerned with accessibility, safety and community values being upheld.
In my opinion, most punters primarily listen with their eyes.
performer survey quote
With the DJ and audience goals in mind I experimented with various projection mapping softwares and aesthetics iterating with user testing to create an experience that engages the senses while solving user goals. In order to do this I assigned different mapping shapes to achieving different user goals:
Several audience members had accessibility concerns such as vision sensitivity from strobing lights and being unable to see the DJ due to short stature, to accommodate this I ensured that the projections did not strobe or change in a volatile manner, and added a camera feed of the DJ at the top of the wall to increase visibility.
Both the audience and the performers expressed concern with the crowd being too aggressive, intoxicated or not adhering to community values such as treating others with respect. To address this I included subtle visualisations that served as reminders of the values of the event.
Besides the effect of visually interesting projections themselves causing users to feel more immersed and therefore more likely to smile or dance, the inclusion of a video feed pointed towards the audience was found in my research to cause the audience to come closer to the DJ and move more, retaining engagement and signalling success to the DJ, which increased confidence.
David Lynch once said that “sound is 50% of film”. My theory was that lighting is the other 50% of music, a theory that was proved unanimously by the 60 respondants of my survey. Lighting was specifically named by performers as something they wish they had access to and named by concert-goers are something that can enhance the experience of live music immensely. As such I iterated on the visuals themselves. As a projection system like Flux could go in any direction aesthetically, for this project I achieved the visuals according to a brief by my Subject Matter Expert DJ whose brief was “disco, groovy and fluid”.
As the visuals are unique and computer generated in real time, once the system is set up to run, it can be left to generate visuals for the entire set. Theoretically, with just a few minor adjustments in the shapes that are mapped onto the walls, the foam projection surfaces, colour tweaks or varying video uploads, this system could be a “set and forget” option for DJs playing at house parties, raves or small events who cannot afford to purchase expensive lighting equipment or pay dedicated AV staff. The equipment and materials necessary to achieve this are also much simpler and lighter than traditional lighting rigging.
Originally I was going to design a system that the DJ could control while performing, however in my research I discovered that the performers have very little capacity to focus on anything other than mixing, to accomodate this while still keeping the experience low cost I decided to have the visuals react to things that were already happening during the event (DJ performing, crowd moving, music playing) so that it could be dynamic, whilst hands off.
Di is an Interaction Designer influenced by her background in hospitality and passion for film, art and music. With a focus on thorough UX research and testing she endeavours to blend creativity and empathy to design experiences that are functional, accessible and emotionally resonant.