Mapping Design Spaces for Audience
Participation in Live-Streaming
Live streaming sites such as Twitch offer new ways for remote audiences to engage with and affect gameplay. This work introduced and validates a theme map of audience participation in game live streaming for student designers. This map is a lens that reveals relationships among themes and sub-themes of Agency, Pacing, and Community---to explore, reflect upon, describe, and make sense of emerging, complex design spaces. This was the first attempt articulate such a lens, and to provide a reflective tool to support future research and education.
Identifying Themes and Developing Theme Map
“Design for Crowd and Cloud (Twitch Plays Game Design)'' was an undergraduate course at Carnegie Mellon University that taught game design for audience participation in live-streaming games on Twitch. In the final project, 8 student teams developed either an original game that took advantage of "crowd and cloud'' play dynamics, or a Twitch modification that addressed a problem identified by the class.
Process Documents: Each group produced a set of process documents as part of the final project submission. These process documents embodied " action-notes,'' the goal of which was to support design ideation, and to trace the emerging rationale of a project. Types of documents included a "Concept Document,’’ “Design Iteration,” `”Trial and Error,” “Prototyping," “Playtest Report,” “Notes” “Process Book” “Feature List,” “Interim Milestones,” “Research Interview,” and “Final Project Pitch.”
An author and I conducted a thematic analysis procedure, a qualitative method for identifying, analysing, and reporting themes that is more flexible than traditional grounded theory. We employ the six-phase framework developed by Braun and Clarke (2006) to identify important themes and sub-themes in the data. We borrowed the notion of sensitizing concepts from grounded theory to visually organize the theme map. Sensitizing concepts are interpretive devices that offer ways of seeing, organizing, and understanding an experience, "drawing attention to important features of social interaction.''
Identifying Thematic Relationships and Constructing Map
The first two steps of thematic analysis are becoming familiar with the data, and generating initial codes. First, we read and reread the dataset, and made notes of their first impressions. In the second step, we iteratively open-coded the dataset, and created an initial set of open-codes. After this, we coded the remaining six projects using these set, discussed additional codes they observed, and added them to the set. After open-coding the data, we searched for themes (step 3), reviewed them (step 4), and defined them in a codebook (step 5). Before writing up the analysis (step 6), we further organized the themes by three high-level sensitizing concepts, borrowed from grounded theory.
We identified themes by first examining the open codes, and grouping clearly related codes together in a Google sheet. After grouping the related codes, we discussed the broader themes that each code grouping could represent, and created an initial codebook with a theme name and description. After this, we reviewed the themes. They considered whether the data supported the themes, whether there were clear distinctions between themes, and whether they had missed any important themes. The final product was a set of 19 themes divided into 58 sub-themes. We found that the large number of sub-themes were necessary to fully describe the complex dataset. We also identified three sensitizing concepts, Agency, Pacing, and Community that encapsulated the themes and sub-themes. These sensitizing concepts were reflected in the literature, and were used to organize the theme map.
We constructed the theme map by diagramming semantic relationships between sensitizing concepts, themes, and sub-themes onto a three set Venn diagram. This theme map was based on concept mapping, which allows students to understand relationships between concepts and related domains; manipulating a complex set of relationships in a diagram helps them remember and analyze component parts. The full map can be found here.
Validating Theme Map Usability and Value
We validated the theme map with a second set of student designers, using it to analyze two existing audience participation projects. Our goal was to show that the concept map could be applied beyond the course, in order to understand (1) how clearly and consistently project designers mapped project design spaces using our theme map and (2) how useful and valuable designers found the map.
We investigated two projects, Commit to the Bits and Echoes, asking team members to map out their respective projects using our theme map and step-by-step instructions, and to respond to questions about the exercise and map. We recruited 7 participants that worked on one of the two projects as either a designer or developer. Five participants contributed to the Commit to the Bits project, and two participants contributed to Echoes. Interviews lasted 30 minutes, beginning with a brief introduction of the research and our theme map, followed by a think-aloud analysis exercise, and a set of reflective questions.
The interviews found that participant across both projects agreed that the map was clear, fit their projects well, and was useful. Further, participants agreed that the mapping instructions and theme map fit their projects well, and strongly agreed that the map was a good fit for their respective projects. Users in both projects thought the theme map would be useful, both as a comparison and reflection tool, and as part of a larger design process. One user explained that a strength of the design space map was seeing “detail in categorized interactions,” and that connecting these themes and sub themes, [helped us] understand the strengths of our project.” Another user likewise explained that the map would help clearly present the main themes of a project to an audience, and to compare different projects. Participants also wanted to use the map during the design process, and several users highlighted the value of the map as helping to keep the focus of the project consistent.
For more details on the map and findings, see the CHI '21 paper.