#Craft: Critical Response

Welcome to the second post in the #craft series. In this series of posts Storyteller and Studio Lead Hannah Nicklin will be touching on elements of the craft of storytelling in games. This might be vocabulary, useful exercises, opinion pieces, or in this case, a reflection on a method for crit and critical response, and how it can help us all think about how we seek and offer good feedback. Title illustration by Lizzy Stewart.

This is a post about the craft of feedback. Being able to offer and receive good feedback across disciplines is one of the crucial aspects of game development, and yet one which we can too-often have little room to exercise.

A bit of context: when I worked in performance, I worked in a part of it particularly called 'devised' performance - in devised performance people set rules and use improv games to make and test material iteratively (like how a band composes music together, compared to how a composer writes music interpreted by an orchestra). And a really common part of the process are 'work in progress' performances, where the team will test the performance in front of an audience, to see how they react to it, and to get a ‘feel’ for how it hangs together in the room. It's basically a playtest. But because you can't always see how the audience is receiving it (because you're often in it!) it's important to talk to the audience afterwards. In a ‘post-show discussion’.

In devising theatre, post-show discussions are how you get feedback from your audiences about how the work is working (or not). They can sometimes be great, but often can end up being run off track, or moderated by an artist who doesn't know what their questions are, or who is asked questions by an audience that make them defensive.

One of the main skills of feedback is knowing how to listen to and give it - but it's not something we're all naturally versed in - especially when it might not be a part of our education or online peer networks (who wants to hear about what can be improved on Twitter!).

Which is why I particularly love a system devised by Liz Lerman called Critical Response. It basically is a mechanic for supportive, good feedback. It's incredibly valuable, I teach it to students, use it in my own practice, and I have also run productive 2-day playtesting workshops based on it. And I'd like to tell you a bit more about it today, as part of developing our craft around feedback.

Economies of Time

Why do I want to share Critical Response with game devs? Well, while one of the extremely stressful things about working in performance/art was the short project timescales, it was also in some ways one of the most positive things about it. When a ‘long’ project is 12 weeks long, you do a lot of them. This involves a huge amount of overhead as a freelancer - applying for funding, organising teams, reflecting and moving on - but it also allows you to develop your work, expose it to the public, learn from it, and move on, many times a year.

One of the joys of working in games is that you can secure a ‘short’ piece of work with a company which is a year long (the security!!). One of the sorrows of working in games is that you can take several years to pull together something playable and polished enough to be legible (especially in the realms of game storytelling), and by that point you have a lot of personal and professional investment in its quality and success. Along with small and Very Online dev communities (again, Twitter, the worst place for supportive feedback), this often results in a lot of empty positivity and kind words, and criticism is left to the reviewers. Then, because we are not practiced at hearing crit, nor prepared or able to act upon it post-launch, it can be hard to process that crit usefully.

In games we do sometimes seek input in structured ways - such as playtesting - but playtesting in casual settings often only works for the kind of games where the joy/core loop is held in 45 second pieces of gameplay. Mechanical joy is rarely the only thing you need to test in a game. What about a story that spans over 10 hours of a game? Or whether or not the moment-to-moment design choices chime with longer-lived parts of the game design?

That’s why I want to offer a structure for thinking through feedback - it’s a shortcut to the kind of practice you’d learn over 10 projects, which could be a year in other art forms, but is 20-30 years in mid-scale Indie Games.

How to listen.

Crit is super common to art-school derived practices - students will regularly get together and present artworks, proposals, readings, scripts together, and then take turns to productively feedback on where and how it can be improved. ‘Crit’ in these terms is almost always done between peers - and constructed around a level of expertise in the form. It’s also something that students will get into the practice of. And will be situated differently to feedback from non-expert audiences, and critics.

When you seek feedback, the first thing to do is to think about who you’re seeking it from. It can often be with a mix of peers with expertise, experts in the critique of games but not in developing them, students with burgeoning expertise, or in ‘focus group’ style sessions, with non-expert audiences, who are less likely to be able to suggest solutions, or even unable to fully articulate what the problem is.

These different kinds of audiences you can seek feedback from require wildly different kinds of listening and interpreting, and the wonderful thing about Liz Lerman’s Critical Response methodology is that it provides a structure to identify where precisely you are on that spectrum.

In identifying where you are on that spectrum the most important thing is to understand that you should be prepared to be flexible; Lerman has a system, but your job in using it is work out how it can work for you, and not use it as a series of survey questions, but as a means of opening up a conversation. It’s about facilitating dialogue between designers/artists, peers and audiences/players, not about making you stare at a piece of paper and ask the same exact questions of everyone.

If you want to read it all first hand, by the way, do purchase her e-book, here.

Critical Response

Lerman first talks about the key roles involved in a good critical response dialogue, how they should be approached, and then about ‘4 stages’ to facilitate the feedback. Here’s all of that, handily summarised:

The Roles: The Artist

The artist (for us, specifically, the game designer/game design team) needs to be ready and open to offering their work-in-progress game. That means that they understand the context of the feedback (where they're at, what they hope it will discuss), that they have questions they want answered, and that they also know what they don’t want feedback on. They need to be prepared to question their work in conversation with other people. This role is not to be underestimated in terms of preparation! It’s not just producing a build: it’s asking yourself things like:

• What is it I’m showing and how can I present it usefully?
• Who do I invite to look at it?
• What questions do I have that will help me right now?
• What isn’t it helpful to discuss right now?
• How can I prepare myself to listen usefully - to understand what is useful criticism, and what might be more taste-related or tackling something I haven’t addressed yet?
• How do I guide the feedback towards experience and not solutions.
• How do I give myself space to reflect on what I disagree with, and what I think is worth considering. (It’s still your work!)

The Roles: Responders

These people will offer feedback, and should be willing to follow the frameworks for the feedback you're setting up. They will have been invited depending on what the artist wants to test - are they peers (more articulate at early stage feedback, but not the majority of your audience), are they your target audience (and how can you support them to feel articulate)? Are they new audiences, or regular gamers? They should know why they’ve been invited, feel comfortable about what they need to do, and how the day will work.

They also need to be resourced appropriately - you may do an exchange of feedback with a peer, students should expect lunch/mentoring perhaps, and you may pay people ‘off the street’ for their time.

The Roles: The Facilitator

This can sometimes be the artist/dev, though it’s often better if it’s a ‘neutral person’ (i.e. not the person who directly made a feature). It’s the facilitator's job to keep the process in step, and use the process to frame useful questions and responses from the responders. They will lead the responders through 4 basic steps.

The 4 Stages of Feedback

The following 'stages' of feedback, are what the facilitator should lead the playtester through after they have sat with the game for a period of your choosing. They are particularly arranged like this because they stop responders offering opinions straight away that could make the artist react defensively, which would stop the discussion, and because they direct the responder’s thoughts in useful directions, and then open it up for feedback and questions that might not have occurred to the artist.

1. Statements of Meaning: I usually summarise this as ‘how did the work make you feel?’ you might want to use a different phrase for different genre of game, something like 'how did it feel to play?' - but generally this question is about the overall effect of it. It should give you a useful impression of how the game works - the word 'feel' is important because it's subjective, and unconnected to judgements. ‘What did you think’ might tend to focus on problems and perceived solutions, which eventually can be useful, but first up it's useful to understand the effect it's having - not the effect people think it should have. It's also an easy way to begin (not technically challenging - everyone has a subjective experience) and gives a neat framework to understand the effect/vibe/experience of the game, how it's working as a 'big picture'.
   
2. Artist as Questioner: The artist/dev asks questions about the work. Which they have prepared in advance.  After each question the responder can answer, they can express opinions in direct response to the question, but it’s best to not ask for change suggestions here, yet. Example questions: did you understand what to do? What did you enjoy? Were you frustrated at any point? How did you feel about X feature? ...
   
3. Neutral Questions: Responders are invited to ask neutral questions of the artist about the work. The artist/dev responds - but try not to apologise, make excuses, or talk at length - it's the questions which are the useful things here. Questions are neutral when they don’t disguise an opinion. For example, if you are discussing the colour scheme, “Why do you use that weird red?” isn’t a neutral question. “What guides your colour choices?” is.
   
4. Opinion Time: Responders state opinions, subject to permission from the artist/dev - the permission is basically so you talk about things that are useful, rather than things you can’t or haven’t yet work on. You begin: “I have an opinion about ______, would you like to hear it?” The artist/dev can say ‘yes’ or ‘not right now’ etc., and doesn’t have to give a reason. It might be someone asks to give an opinion on the sound you've used, but it's all placeholder at the moment, in that case you might say 'not right now'.

It’s important to note this is not a rigid framework but points from which conversations can be had. This is not a survey! It is a structure for a conversation.

The importance of Peer Networks

Knowing about this excellent system, and how to put it into practice as a whole is great, but you can also drawn on smaller aspects of it in application to internal critical discussions. Permissioned opinions are super useful for mock ups, for example, where you might want feedback on the shape of a design, but not its colours right now; or asking for 'neutral questions' to be used in an internal feedback session, and making sure that someone who's not the developer of that specific feature is holding the discussion. Knowing what your questions are helps immensely. Asking people what questions they have about their work helps you frame feedback most usefully.

Likewise crit and good crit-culture goes beyond a single methodology and moment of asking for feedback though - it’s also a practice you need to exercise to be able to offer and receive good feedback in general.

Places like Twitter are often a really good useful place of solidarity for a distributed and global community like gamedev - but they’re also often places of promotion and of exposure (exposure to attack or annoyance by strangers with parasocial ideas about appropriate things to say).

Public online spaces are good places to network but rarely useful for critical discussion, as crit will often need you to open up in ways which can be more vulnerable than is comfortable in a public place, or a place of promotion.

In a non-proximate industry like gamedev this is where Discords and Slacks can be super useful ways to grow or gather peer networks. I’d like to make an argument for peer networks. Perhaps you have a ready-formed network from a game design degree? Perhaps you have a few close friends you want to do monthly book-group style crit sessions of cool stuff from itch.io to develop your general crit skills? Perhaps you want a network of people who do a similar role to you to check your working on ideas before you’re ready to present them to a different ‘department’ in your studio?

It’s worth thinking if you can set these up! If you have one or two people you can think of to bring into a space or crit structure, I’m certain it will grow.

And if NDAs and secrecy in tech-circles also makes these spaces too vulnerable, asking your team to resource you in NDA’ing some key peer-mentors (having a couple of peers you check in with once a month or so in an hour long reciprocal chat) - where you can vent and ask how they would have tackled something, or what they’re working on - can also be super useful.

Processing Crit.

One of the things Lerman talks about is being prepared for crit. This means knowing where you are, what you’ve done, and the questions you have which will help guide you to where you want to go next. You need to know what your intentions are/were, and you also need to be comfortable with the potential experience of listening to how you’re not meeting them yet.

People will often try and be solution-focussed when you might need to encourage a conversation which is problem-focussed, or vice-versa, depending on whether you’re with playtesters or peers!

One of the things I would strongly recommend is that (if you’re not able to have a ‘neutral’ person hold the conversation) is that you record the conversation to return to. Sometimes you have to hear a piece of crit, and allow your instinct to recoil a little, before sitting with it and turning it over in your head and your heart, and taking what’s useful from it.

In the end, people doing this peer or playtesting work are always aiming to be helpful - and when you solicit that help, you need to make sure you’re giving yourself the room to process it.