Lasse Laursen

DJ Interface with Remote Audience Feedback


Summary

Streaming a live performance (music, gaming, theater, etc.) is common place these days. While the technology to deliver the content has massively improved over the past years, the methods with which to connect the performer with the audience has remained surprisingly static. The most advanced integration of audience feedback that you are likely to see is a chat window or a poll placed along side the stream itself.

This project aims at breaking down this barrier by integrating the audience feedback directly into the performers interface. We chose to focus on discjockeys (DJs), as audience feedback is key during their performances. DJs rely heavily on the direct and indirect feedback from an audience to make informed decisions. When the DJ is performing remotely (i.e. not in the same location as the audience), that feedback vanishes. Of course, a remote performance is not the same as a local performance, and the DJs needs may not be the same in these two situations. However, given the option, most DJs still prefer to know what their audience thinks/feels despite being less inclined to take it into consideration.

Associated Publications

  • A Multi-Touch DJ Interface with Remote Audience Feedback
    Lasse Farnung Laursen, Masataka Goto, Takeo Igarashi, in ACM MM '14: The 22nd ACM International Conference on Multimedia, pp. 1225-1228
    Published PDF

Quick Facts

  • Subjected to two formal user studies involving 7 DJs.
  • Four 2-hour live shows conducted (as part of user study) with an average of 120 live listeners on Slayradio.org.
  • Improved interface actively in use semi-weekly on Slayradio.org
  • Built from 'scratch' using C++ over the course of a year.
  • Cinder used as base framework with BASS as the main audio framework.
  • Custom built GUI library built from the ground-up supporting standard tap, drag, pinch gestures.
  • MT3200 touch-device produced by FlatFrog. The touch-device was discontinued in 2014.

Resources

DJ interface

The interface consists of a common audio time-line visualization, with general playback and visualization controls at the bottom and a set of drop-down menus for loading audio marked. The entire time-line supports pinch gesture zooming for an ease of overview and/or detail. All loaded audio elements support intuitive drag-and-drop style interaction. Two modes of visualized playback are supported; static time-line with a moving playback line, or moving time-line with a static playback line. The former is ideal for planning ahead and setting up audio elements in advance. The latter mode of visualization is a more dynamic ‘on-the-fly’ style interaction mode. Switching between the two modes is possible at any time to suit the DJ’s current mood and needs.

The interface uses a MT3200 touch-device produced by FlatFrog. The devices supports up to 41 simultaneous touches, as does the DJ interface. The size of the screen easily allows two simultaneous users.

Development

One of the things I most often think about after having worked on a project for a longer period of time, is if I made the right choices along the way. While regrets are a waste of energy, it's healthy to assess whether or nothing things 'could' have gone better so as not to repeat any mistakes. This project has been one of the hardest ones in terms of convincing myself one way or the other. Most likely due to the fact that it was (and still is) one of the most ambitious projects I ever undertook.

Developing this interface has been - by-far - the most encompassing system I've had to develop as it came with some iron-clad requirements:

  • High Performance - The system must respond quickly. Touch-technology is laggy enough on its own. If the software layer introduces any additional lag, the interface can quickly become overly frustrating to use.
  • Robust - The system must be stable enough to run continuously for several hours with constant end-user interaction.
After surveying many of the available frameworks and libraries I opted to use C++ and building on Cinder as the base application framework. While many competing frameworks offered more functionality out of the box, none of it seemed ideal for my goals (back in 2012). I also investigated a number of existing GUI libraries, but opted to build my own as the existing selection had a huge amount of legacy code (being originally designed for both mouse and keyboard, with touch tacked on). The GUI library powering the DJ interface is touch first, mouse/keyboard second and allowed me to quickly build some intuitive widgets. For example, songs are loaded by simply dragging them 'off' the dropdown and on to the timeline interface.

In retrospect, using C++ has undoubtedly hampered quick iterative development. Given a dedicated team I would probably have opted to develop a prototyping sandbox system in Python to go along with the actual application implementation. However, given that the system was intended to be used live during the final user study, I opted to focus on the version that would end up performing the best, with albeit fewer design iterations.

Observations and results

Here are a few things we learned throughout this project in regards to integrated remote audience feedback:

  • Preference for comments - The DJs generally preferred comments to likes, as they felt it showed more of an effort on behalf of the audience member.
  • Filter use was non-existant - The DJs never filtered any of the feedback at any point during the two-hour shows. However, this is likely due to the interface being very simple overall, thus making the feedback not intrusive.
  • Application in a local setting - Some DJs expressed an interest in having this type of feedback available during a local performance.
  • No strong differentiation between feedback types - Overall, the DJs did not seem to value the feedback differently depending on the type (gold/bronze).

The participating DJs also made a number of interesting observations:

The feedback was helpful. Particularly if I was doing an online set, that would be really useful. [The feedback] is more direct. If we use a dance floor analogy; People will leave the dance floor [due to fatigue or to get a drink,] not cause they don’t like the music [...]. [...] So in that way, dancing is kind of an imprecise metric.DJ Modality
I wouldn’t do a live [remote] mix, and then get some negative feedback, and [...] just switch genres. I might do that at a [local] club.Phillip Huey
It gives you inspiration and energy. [...] It works. I was surprised.Mark Jackson
It’s interesting to see how people react and what they do. Will it heavily influence what I do? I don’t think so.Matthew Gammon
It would help me understand my audience better, know what they want to hear, which could lead to better parties or sets. That’s important. [...] I can see a lot of uses for that.DJ Vivid

Gallery

A very early pre-alpha prototype screen-shot featuring a basic time-line and very sparse GUI functionality .
A very early pre-alpha prototype screen-shot featuring a basic time-line and very sparse GUI functionality .
A very early pre-alpha prototype screen-shot featuring a basic time-line and very sparse GUI functionality .
An early pre-alpha prototype screen-shot showcasing some of the integrated audience feedback. Initially, audience members could only give overall positive/negative feedback and positive/negative feedback with regards to the volume (too loud / too quiet).
An early pre-alpha prototype screen-shot showcasing some of the integrated audience feedback. Initially, audience members could only give overall positive/negative feedback and positive/negative feedback with regards to the volume (too loud / too quiet).
An early pre-alpha prototype screen-shot showcasing some of the integrated audience feedback. Initially, audience members could only give overall positive/negative feedback and positive/negative feedback with regards to the volume (too loud / too quiet).
A screen-shot of the interface used in four two-hour live shows (shown here with mock-feedback). This version support simplified positive feedback, along with user comments and a listener histogram. All of the feedback can be toggled on/off at any time during a performance. The system distinguishes between two types of listeners (gold / bronze) who represent those known and not-known to the DJ, respectively.
A screen-shot of the interface used in four two-hour live shows (shown here with mock-feedback). This version support simplified positive feedback, along with user comments and a listener histogram. All of the feedback can be toggled on/off at any time during a performance. The system distinguishes between two types of listeners (gold / bronze) who represent those known and not-known to the DJ, respectively.
A screen-shot of the interface used in four two-hour live shows (shown here with mock-feedback). This version support simplified positive feedback, along with user comments and a listener histogram. All of the feedback can be toggled on/off at any time during a performance. The system distinguishes between two types of listeners (gold / bronze) who represent those known and not-known to the DJ, respectively.
An early screen-shot of the audience feedback interface. It consisted of only four buttons enabling the simple discreet feeback to be sent directly to the DJ interface.
An early screen-shot of the audience feedback interface. It consisted of only four buttons enabling the simple discreet feeback to be sent directly to the DJ interface.
An early screen-shot of the audience feedback interface. It consisted of only four buttons enabling the simple discreet feeback to be sent directly to the DJ interface.
A screen-shot of the audience (web-based) interface used in the four two-hour live shows (shown here with mock feedback). The users could actively participate by sending both a discrete 'like' or a comment.
A screen-shot of the audience (web-based) interface used in the four two-hour live shows (shown here with mock feedback). The users could actively participate by sending both a discrete 'like' or a comment.
A screen-shot of the audience (web-based) interface used in the four two-hour live shows (shown here with mock feedback). The users could actively participate by sending both a discrete 'like' or a comment.
The broadcasting setup as used in each of the four two-hour live shows at the University of Tokyo, Igarashi Design Lab.
The broadcasting setup as used in each of the four two-hour live shows at the University of Tokyo, Igarashi Design Lab.
The broadcasting setup as used in each of the four two-hour live shows at the University of Tokyo, Igarashi Design Lab.
A commemorative collage of many of the participants who helped test the DJ interface during its early development.
A commemorative collage of many of the participants who helped test the DJ interface during its early development.
A commemorative collage of many of the participants who helped test the DJ interface during its early development.

This project wouldn't have come to fruition if it wasn't for the help of both my supervisors Masataka Goto and Takeo Igarashi, Slaygon and SlayRadio.org, as well as all of the participating DJs.


© Lasse Laursen 2015 - 2021