the Co-creation Space: Open Pilots with INO and LICEU
OPEN PILOT: Irish National Opera
We explored the value of the Co-Creation Space for materials co-creation through an open pilot with an Irish opera composition workshop. Participants joined a private group which kept their interactions separate from the other pilot, and interacted with the CCS functionality described in section 3. The pilot ran over the course of 12 weeks from April to June 2021; Thirteen (13) participants were recruited by the project. Participants were primarily older adults from the Dublin suburb Tallaght between the ages of 30 and 70.
OPEN PILOT: Liceu Opera
We explored the value of the Co-Creation Space for performance co-creation through a Spanish choir open pilot, the goal of which was to support 11 choirs to learn to perform a new opera together. The pilot ran for 10 months, from October 2021 to July 2022. Two hundred and six (206) users from 11 community choirs participated in the trial, including 198 non-professionals and 8 professionals from 11 choirs. All users were participating in the creation of La Gata Perduda, an opera being co-created by the LICEU opera house in Barcelona.
Open Pilot Procedure
At the beginning of each pilot, professionals learned about the CCS during a meeting, and registered for the tool. Then, facilitators introduced the tool to individual choirs, and helped participants register for the CCS and fill out consent forms online. Participants were given a chance to ask questions, and were given a quick-start reference guide and tutorial video. At this time, participants were told that they would use the CCS to communicate about different forms of media, and that they could decide how formally or informally they should use the tool. After each pilot ended, focus groups were conducted with professional and non-professional participants. Participants were asked to consider how they envisioned using the tool at the beginning of the study, and how it compared to their actual use. They were also asked what formal and informal purposes they used the CCS for, how useful the tool was for them, and how the pilot experience would had been different on another social media platform.
Mixed Methods analysis
My team and I hypothesized that the CCS would be valuable to participants during moments when they needed to discuss and reflect on co-creation materials. We validated this hypothesis by triangulating four methods: 1) a quantitative report summarizing usage of the tool across the workshop period, 2) a content analysis of CCS text, 3) a network analysis of interactions, and 4) a qualitative group interview of participant feedback collected on the last day of the workshop.
Summary of Use
My team and I compared professional and non-professional use of the tool for the two pilots over the length of each pilot. This included viewers per week, multimedia interactions, posts, multimedia uploads, comments, and emoji reactions. We interpreted results of usage in context of pilot activities.
Text Content Analysis
With the help of a second coder, I performed a content analysis of the open pilot text was performed through a qualitative coding procedure. First, we independently looked for open codes across the text of the two open pilots. After this, we created a codebook of 13 codes grouped into 5 high level categories representing the focus of discussion. After creating an initial codebook, we performed an IRR on a subset of data, and generated an initial score of 85%. We discussed the text together, added two categories, and performed a second IRR, with a score of 94%. After this, one coder coded the rest of the dataset, and made notes on what they were not sure about. Finally, we went through the text together, and came to agreement about uncertain codes. In addition to the text coding, an automatic sentiment analysis was performed on the text using the AWS Comprehend cloud service that generated a positive, neutral, negative, or mixed}response for each text. In parallel to coding the text, we checked agreement with the sentiment analysis.
We visualized the differences in discussion of the open pilots as treemaps, with higher saturation indicating more positive sentiment.
Network Analysis
Our team performed a network analysis of interactions from the 1st through the 12th week of the pilot to understand the structure of the discussions taking place. We built a network with users as nodes, using node size corresponding to total posts and comments a user created. We used size to order the network in a circular layout, and color to distinguish between professionals and non-professionals. Links between nodes represent the comments made by a user to a post by another user, with line width proportionally representing the number of bidirectional replies.
To understand how this structure evolved over time, we graphed all content that had been created up to a point in time; for example, the graph for week 3 counts all posts and comments created in weeks 1, 2 and 3. Thus, the network grows by increasing nodes, links, or strengthening links between users.
Post Pilot Interviews
Participants from the two pilots were interviewed at the end of each pilot over Zoom. Six (6) users from the INO participated in a qualitative group interview on the last day of the workshop, and 13 participants from the LICEU trial on the last week of the trial. Participant responses were analyzed by grouping responses to usage of the tool, and to its value.
These interviews, in combination with the quantitative data, led to a better understanding how to support discussion processes in different co-creation dynamics. We found that the relative differences in use and value of the Co-Creation Space between the two trials suggests that technology for artistic co-creation should support flexible space segmentation, direct and indirect communication channels, and should consider new tool features in the context of existing app ecosystems.
Needs for Divergent Co-creation Dynamics
Having reflected on the differences in dynamics between the two pilots, and we identified three technology needs to support divergent artistic co-creation: 1) flexible space segmentation, 2) explicit and implicit communication methods for technical and emotional needs, and 3) the importance of considering feature affordances in context of existing app ecosystems.
Need 1: Support Flexible Space Segmentation
A primary difference between the Irish and Spanish pilots were the number of participants and the existence of sub-groups; in the Irish pilot, all participants worked together as a group, whereas the Spanish pilot was formed from pre-existing groups. This resulted in differences in use and value of the CCS. While Irish participants liked the simplicity of the text timeline, Spanish participants wanted to create clearly marked sections for different voices, similar to that which one choir leader had done using Telegram. Further, Spanish participants wanted a distinct photo repository of memories, a notable difference between the pilots; since the Irish pilot was conducted over Zoom and did not include a rehearsal or show, participants did not need to separate technical from informal media, whereas Spanish participants did. This suggests that artistic co-creation tools should support flexible space segmentation that allows communities to self-segment into sub-groups, and to segment posts into distinct channels.
Need 2: Support Direct and Indirect Communication Channels
Differences in participant groups led to differences in direct and indirect communication needs. Unlike Irish participants, Spanish participants wanted segmented professional and internal channels that differentiated official notifications from teachers and directors. For official channels, Spanish participants requested ways to gauge participant comprehension. Such as a reading check confirmation, “like in WhatsApp...[but where] participants would click to confirm that they had actually read what was sent to them” (P11). Further, Spanish participants needed indirect ways of communicating emotions; they did not feel comfortable posting personal concerns to a forum of so many people. Instead, one interviewee suggested including an indirect “how you are feeling” rating system about rehearsals, with a chance to provide “clarification and comments”’ (P9) about positive or negative experiences. To support such differences in communication, artistic co-creation tools should support both direct and indirect communication channels that give users a chance to view and respond to content in ways they feel most comfortable.
Need 3: Consider Features in Context of Existing App Ecosystems
Maximizing user experience and ease of development led to practical design choices, such as building a responsive web-based tool that could be used on both computers and mobile devices. Further, we scoped the CCS to focus on discussion and reflection dynamics based on gathered user requirements, and created a clean aesthetic that could fit different co-creation needs. However, our open pilot findings reveal tension between our practical design choices and some affordances users were accustomed to, such as getting pop-up phone notifications, a difficult feature to implement in browser applications. Relatedly, younger users from the Spanish pilot were put off by our simple interface because they expected a tool connected to existing social media. Notably, this is a stark contrast to Irish participants, who valued the private space afforded by our tool. These findings highlight tensions between using existing tools for artistic co-creation, and designing new community tools in an already saturated social app ecosystem.