2021 Writing Internship - What We Did, and What We Learned.

Studio Lead Hannah Nicklin does a deep dive on the design of the Intern Recruitment process, what we did, how it worked, and what we learned. You can also take a look at Char Putney's advice to applicants post, and read Ida Hartmann - our first intern recruited through this process - speak about her individual experience of it.

Opening Credits

I want to start off by saying that while I set the framework for our internship - the pay rate, the aims, and worked closely with others on the copy and its presentation, it was also co-led and administered by the hard work of Creative Producer Ben Wilson, and the bulk assessment was in tandem with Story Lead Char Putney - there's no way this would have existed - or have been so thoroughly evaluated - without their hard work. We're also grateful to the input of the full Die Gute Fabrik team on the video, and other aspects. And the Board and Co-Owners, for supporting my wish to provide entry-level positions as well as expert-level positions in a diverse team.

Also, if at any point any of the language in here feels a little legal or careful it's likely because our excellent Danish lawyers have helped us make sure we're saying things correctly and in a way that doesn't make us liable legally. I would of course always love to be more radical, but if I don't have a company to make games with because someone sued us, I can't provide any internships at all.

Finally I would like to credit the more general industry level advocacy of groups like POC in Play - who help all of us do better in designing work opportunities which are open to as many people as possible.

The Aim

I wanted to produce an entry-level Narrative Design & Writing Internship recruitment process that did the following:

• Open & welcoming to people from other disciplines transitioning to games
• Open & welcoming to people with caring responsibilities, older
• Open & welcoming to marginalised folks
• Easy to apply to in stage one and with no additional labour apart from a simple light-touch first application process
• A paid test.

Part of this was in the design of the actual internship - if we want to be welcoming to people transitioning between industries who might be mid-career, or single parents, etc., we need the internship to be paid, and paid in a way that a single parent can support a family with. This is in part solved by the fact I run a 'flat rate' in the studio (which stops pay disparity across both disciplines and for marginalised folks), which I also extended to the internship, and includes making sure that any writing tests were paid.

The rate for these is based on a Danish tax rate and cost of living (for example, I pay 40-50% tax, do a weekly shop of fairly middle class standards of around $150-200 USD to feed one person, and pay rent and bills of a small 1 bedroom apartment costing around $2000 USD per month, which leaves me around $500-750USD disposable income a month) it is enough for a single person to be comfortable, but most importantly, for a single parent to live their life. That's my measure of the flat rate for the studio.

I also included the following in the design of the internship:

1. A laptop - one of the biggest barriers to working in games for many people is affording the hardware to run the software. I wanted to solve this up front.
2. A mentoring programme as well as the work - designed around the mentee, allowing them to choose who on the team they wanted to learn from. Meaning that if being mentored by a Black or LGBTQ+ or woman was more comfortable or useful to them, we could resource that.
3. Remote, and accepting applications for anyone with a decent timezone match (we said 4 hours of working hour cross over 10am-6pm). Plenty of applicants explicitly ignored this, and I'm sorry to everyone who applied from e.g. West Coast America, or Australia, New Zealand etc. Many roles can work asynchronously, people do run game dev that way. But this role's direct supervisor felt (and I agreed) that this particular role did need both a decent crossover with us in order to be supportive, and also to attend the meetings and mentorships that would make it effective. Many applicants did say that they would be happy working e.g. 2am-8am if it meant they could do the role, and while I might consider that for a peer who could explain why it would fit with their work/life balance, caring responsibilities, and health and wellbeing - in this case I didn't feel like informed consent was possible because of the power differential of an entry level position in a hugely under-supported field. (That is also why I'm writing this post - to perhaps equip more companies around the world in other timezones with the experience I gained in running one!)
4. Constructing the 'what we're looking for' in a way which was explicit that it was about aptitude (which anyone can have) and not experience (which privileged folks are more likely to have access to.
5. Where possible using accessible and simple English (though of course as the applicant will have to write in English, we did need to ask for samples in English). (We did not ask for English-as-first-language people, though, it should be noted. Fluency only). A lot of business acronyms (WFH, 0.5, pro rata, etc. are country-specific, e.g. in Denmark people don't write 'part time' as 0.5 or 0.25, etc.)
6. Including a video which featured us all as people, saying hello, showing the present scale and diversity of the team. Plus a clear contact for asking questions (thanks, Ben, for fielding most of those - I only handled the ones sent to us over Twitter DMs).
7. And a 3-step process, Stage One: which was very simple and light touch to begin with meaning the barrier to entry was low and open, then Stage Two: a paid test using some mocked-up tasks which a very much like tasks the intern would do as a part of the role (a research task, a dialogue task, and a world-building task), followed by Stage Three: an interview. We had the funding only for 15 paid tests, though, which was a tough first round cutoff.

You can re-read the advert text here if you want to track those in the application. And here's our video!

The Mailing List

I also had a section in the Stage 1 application asking people if they wanted (all GDPR-compliant) to opt into a mailing list for future DGF opportunities. AND a second mailing list which we have permission to send other opportunities to.

That means I have a rich, diverse, and wonderfully enthusiastic GDPR-compliant mailing list of people looking for entry-level work in storytelling in games. If you are at all interested in setting up your own internship, please reach out to us and we will send them your job ad!

What it cost:

If you are looking to pitch something similar in your own studio, I can summarise the headline costs of administering the process in person hours, and the cost of the 5 month internship itself:

Costs:

• Laptop: approx $1000 USD
• 16 Paid Tests (we stretched to one more). Around $240 USD (1500 DKK) for a 4 hour test = $4000 USD (rounding up)
• Wage expressed as a day rate (I won't give the Danish employer costs) and rounded to 9 days per month (between 8 and 10 depending on length of month) = $21,000 USD (after Danish base rate tax that would be around $12,600 for 5 months)

Total Monetary Cost (remembering that you also get, y'know, 5 months of an extra person working on the game!) $26,000 USD.

Person hours administering entire process featuring ~3000 applicants.

• Application process drawing up - approx 20 person hours across two people
• Stage One assessment person hours - approx 50 across three people
• Stage Two assessment person hours - approx 20 across three people
• Stage Three assessment person hours - approx 20 across three people
• Mentoring (in addition to onboarding and supervision which isn't covered here) - approx 20 person hours over the period of the 5 months.

Total person hours: 130 person hours, which if a day is 7 hours long, is approx 19 days (rounded) of person time.

The Challenges

If I weren't in charge of the company, and well-supported by both the Board & Co-owners, and by the hard work of Ben Wilson, I might have been more challenged in arguing to resource the opportunity in terms of pay structure, funding, equipment, and the argument for investing in emerging talent at all. As it is the biggest challenge I faced was where the company is based, that is:

• Danish legal frameworks

Unlike in the UK and USA (I don't know enough to point to other examples) positive action (i.e. directly appealing to or shortlisting around 'protected groups' (UK term) of under-represented people in an industry or country) has no general supporting legal framework in Denmark.

There is a law protecting against negative discrimination. The ‘law prohibiting discrimination in the labour market, etc.’ (in Danish: Bekendtgørelse af lov om forbud mod forskelsbehandling på arbejdsmarkedet m.v.). The law prohibiting employment discrimination prohibits both direct and indirect discrimination. This is on the basis of: Race, skin colour or ethnic origin, Religion or faith, LGBTQ+ identity, National or social origin, Political views, Age, Disability.

But - largely speaking - positive action would also breach this legislation - as I wouldn't be treating unmarginalised people equally. There are some exceptions, but most of them involve applying to a government ministry for the relevant industry for permission. I would have to go through the process of appealing to the Ministry for Employment if I wanted to make this opportunity open only to marginalised folks. This is out of my reach right now for a number of reasons - preparing the case isn't something I had time for and would probably reduce 3 internships over 3 years into 1 over 3 years, and with no guarantee that my case would be considered justified by the end (in fact for cultural reasons I'll touch on shortly, I would expect it not to be - examples of ministry exceptions given to me by our legal team were religious groups, political parties etc.).

Also, while it is permissible to encourage someone from an underrepresented group to apply for employment, it is a condition the job advertisement states that all applicants will be considered and that the underrepresented group does not have priority. That’s why our lawyers said it was necessary to add the sentence “We encourage anyone interested in the internship to apply, irrespective of race, colour, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability or age” - note the 'irrespective' set up as opposed to reaching out specifically to marginalised groups. That kind of language feels mealy-mouthed and not honest in the context of an international industry, but in Denmark, it is the best we can legally do.

Why does this matter? And how come it works this way? The politics of Labour in Denmark are very different to the UK and USA in some excellent ways - it's an incredibly unionised workforce (around 70% of the workforce) and instead of a minimum wage, most industry's wages are set by collective bargaining - at the behest of the workers, and not the state. (In fact this detail is also stopping an EU-wide minimum wage being set because it's much lower than the collective bargaining and unions worry that it would drag down average wages.).

However there are also some downsides to a Labour market which is set up around industries and unions and resistant to state intervention - this (in my opinion but I am of course eager to hear from Danish experts in the matter) leads to little state-level proactive equalities legislation - that is it produces legislation that defends against discrimination on an individual level (and that can be proven, and that someone has the resources to prove in court/with the support of their union). Rather than materially working the structures that already exist proactively (how I would describe positive action).  

Then on top of that there is a wider cultural issue of silence around all forms of discrimination - perhaps born out of the same 'Janteloven'-driven solidarity which also makes the unions so strong - of a populace who have made so much progress in workers' rights and the welfare state that there is (from my outsider perspective) a sense that equality is 'solved' and that you shouldn't stand out.  

This matters to a Danish-based company, with a British CEO, recruiting people from all over the world. There is a mis-match in language in appealing outside of Denmark, and in operating in Denmark the structures that do exist are discretionary, and underlain by a culture which often refuses to acknowledge structural oppression.

There is - and I have seen this in the media and press as recent #metoo and BlackLivesMatter discourses have reached Denmark - real resistance to the idea racism or sexism or anti-LGBTQ+ attitudes can be possible in a society which prides itself on being equal - discussions in parliament are defensive. And at a state level it's hard to find that discrimination is even being counted (again, maybe I miss this as I read most of my Danish media in translation, but I rarely hear of polls and statistical studies accounting for discrimination in Denmark, and EU-wide studies and articles tend to corroborate my view here).  UK and American-led discourses around anti-racism, for example, are often articulated as an 'import' to Denmark (an excellent article on hyggeracism and the specific flavour of Danish prejudice which is about 'not talking about it' can be read here).  

As a migrant I'm just going to point out that I feel very lucky to live in Denmark for many reasons - and that my critique here is the best way I know of being a citizen; holding a society to account as well as celebrating it where it's due. I want to be a part of making Denmark even better. I hope in highlighting the specific cultural and legal restrictions in even setting up this posting I can show how it makes the Danish business I run less competitive, and less constructive. In the UK I may have even set up a traineeship aimed specifically at a marginalised group. Whereas in Denmark it's not only illegal for me to do so - I cannot even explicitly encourage marginalised groups to apply over others. It needs to be 'irrespective'.

(There was a lot of back and forth with this section with our lawyers - thanks again for the support, if you're reading!)

This is a problem for us in videogames - an international industry so influenced by relatively progressive UK and USA diversity practices. It means that someone glancing at our advert would assume we're not welcoming because we didn't explicitly state it. It meant I had instead to encode it in non-explicit ways: a video from all members of our existing diverse team, and find ways to make sure in practice the application was as accessible as possible without promoting it as so. That it allows you to articulate your pronouns if you wish (but does not require it). That allows you to share with us your context if you wish (but does not require it).

I suspect if I really want to make a wider difference in the game scene in Denmark - and in its ability to advertise to and recruit diverse international talent - I should also spend some of my time lobbying my union (PROSA), and other tech and creative industry unions and local political representative for protection for principles of Positive Action in Labour law and practices. I have put this in my to do list for later on in the year.

And for absolute clarity: please do not think that I'm suggesting a unionised workforce is anything other than good. It is just part of the context which has in this case produced the circumstance I'm describing - nor is it inevitable a unionised workforce would produce this, unionism can and should be internationalist and anti-racist. It also often hasn't been - I know that as the descendent of union activists going back generations.  Please don't read this as a simplistic conclusion, or that I'm even anywhere near having understood all of the fine grain Danish context for it yet.

The Marginalisation Question & Assessment Criteria.

I draw up this complex context, however, because led to one of the most unsatisfactory compromises in the application process: our marginalisation question. I wanted to give people room to discuss the context of themselves and how that affects their experience - to give them room to be heard and seen. I also wanted to track the success of our application in being accessible to diverse candidates. I also wanted to ask in a way which acknowledges that for example, a Black person applying who lives in Nigeria is marginalised in the wider games industry, but wouldn't experience the day-to-day racism in their career that at Black person living in the United States, France, or Denmark would.

But on the emphatic advice of our lawyers we couldn't ask anything specific and mandatory (and be accused of discrimination on its basis). They also recommended we formally write up our criteria and process in advance in a system which could capture date/time of its being first drafted (GoogleDocs) so that if we were challenged by someone who thought we hadn't treated their application equally because e.g. they were a cis white man we could point to how our process and criteria were 'equal'.

This goes against my actual beliefs and well-research principles around 'equal' - in the context of structural oppressions when not everyone will be starting from the same place ('equal' is not always 'fair'). But it's the law in Denmark. The marginalisation question was not a part of our 'marking' sheets (which were filtered out by some clever spreadsheet work by Ben). And we have a clear scoring criteria for each question which we can point to.

I'm choosing not to share this criteria here (and indeed openly) in detail because I believe that privileged people are often taught to 'write to the test' in their education, and that to do so might disadvantage people in future application processes who aren't taught this skill because of a test-focussed education system, or because of tutoring or private schooling. But here is the criteria for one of the questions with weighing and specifics removed as an example:

• Good basic storytelling instincts
• A good ear for dialogue, tone and format
• A fresh perspective.
• Fluency and flair.
• Potential for development within the context of our project and team.

I cannot also meaningfully (or in good faith - as it wasn't for public consumption) summarise our responses to the Marginalisation question, but I can confirm very roughly that over 75% of our applicants were in some way from a background marginal in the Western games industry (which might be: people of colour, LGBTQ+ folks, women, people living with disabilities, and people from non-Western countries). This is something I believe to at least be in some part a vindication of our hard work in encoding what we couldn't be explicit about.

However, there are some real problems with our solution to measuring the success of this outside of more simple (but not legally compatible) monitoring using tick boxes around race, gender, country of residence, sexuality, physical impairments, mental health, etc. Which are at least simple, dry, and crucially - low on labour. Some of the applicants wrote huge essays in the marginalisation field which I am very grateful for their sharing - but equally I would hate for applicants to feel as though they had to perform their trauma in order to earn the position. It was an optional field, and we were clear it was for monitoring, and some people welcomed the nuance of being able to talk about their context, but I'm certain it also weighed on others.

Another person also told us that 'marginalisation' specifically is a problematic term (it is!) and we're better asking how people relate to hegemonic power structures in their work/life (it was a bit more complicated than this as a proposal but again, I'll avoid directly quoting something not for public consumption). And that's both true, and I believe, begins to be a less accessible way to present the question - some people outside of USA and UK diversity discourse might struggle to assess how to respond to a complicated question about hegemonic power structures. This isn't something I have a good solution to yet, and it's likely I will work the most hard in the next intern recruitment process (if we can resource one - it depends on company funding situations) on workshopping and phrasing and testing how to include room for people to be seen and heard in this way, but not feel pressured or diminished by it.

We also checked this specifically in our survey, and while 2/3 of people were happy with how the question was phrased, there was a significant enough dissatisfaction that I want to continue to work on it.

The Response

The response to the posting was in some ways a vindication of all the above hard work. I will break it down more soon, but we had several thousand people apply, at least ten times more than we expected. We had worried about making sure we advertised the position via advocacy groups and via personalities who have diverse following, but in the end we didn't need to do that much outreach because enough was designed in the fabric of the posting that people shared it for us, without having to do targeted outreach (though Ben still did research and do this in a smaller way).

This presented its own challenges, we had to massively extend our assessment period in a way which made some applicants feel hard done by. We only had me, Char, and Ben, and we also had to do our jobs at the same time. We collected email addresses but had to write to them via a mailing list because of the magnitude of applications, for that reason, typos in the email address and junk mail filters meant a few people never heard back from us. We emailed everyone with a new timeline as soon as we were aware we would need more time, but it was still frustrating to some applicants. And it also absolutely meant there was no meaningful way for us to be able to offer feedback to Stage 1 applicants. I don't know that a studio of our size can solve that problem.

Evaluation Methodology

We have included both quantitative and qualitative methods in evaluating the effectiveness of our posting. Listening to people's feedback throughout, running a survey for applicants which collected both numbers and 'free' text, and also on the basis of  some necessary 'eyeballing' - i.e. that we're not actually (as mentioned) in Denmark allowed to meaningfully collect any information on our applicants beyond an optional question on 'marginalisation' which was long argued over and edited with the help of our excellent Danish lawyers.

I've already touched upon the complicated local context when speaking to international expectations around positive action and monitoring applicant's relative diversity - much of which we aren't able to do from a Danish context, and won't repeat that above.

A screengrab of the feedback survey can be seen here. It was sent to all applicants who agreed to be on our mailing list, and over 300 people responded - more than 10% of total applicants. We also Thanks especially to everyone who took the time to fill out the survey! At the end of this post you can go to the Notion Page where we've summarised everything.

Quantitative feedback

Here's a summary of advert and application-completion accessibility as garnered from our feedback survey, here 1 is the most negative, and 10 the most positive. Extra credits here to Ben Wilson for compiling it all and producing the nice graphs <3

And a summary of the countries of residence of applicants. We had applicants from 91 countries in total.

Overall I'm pleased with these responses. I think an FAQ will improve the accessibility score, and we're better equipped to build one having run the process once, and I think a spread of applications of 91 countries is excellent, and especially showing a top 10 which isn't only European (though obviously our time zone needs will curtail the scope this can ever reach. And there are a lot of chancers in Australia.)

Written Feedback from respondents: Positive

This is supposed to be a what-we-learned post, so I'm not going to dwell too much on these, but it was usefully affirming to see that some of what we tried to do was successful for some people. People spoke positively about the following themes:

• The post was accessible and the company seemed welcoming.
• The process was streamlined and had a good low bar of energy for entry.
• It seemed fair and inclusive.
• It was good that the company responded to everyone at every stage of the process.
• Opening the opportunity internationally.
• Specific and open questions instead of sending a cover letter.
• Open to non-game discipline experience.

Feedback from respondents: Negative.

I'm going to highlight some common themes here. With the full context of this post, you can hopefully understand why some of these things were common themes:

1. No individual feedback
2. Too long to respond
3. Application was too general feeling
4. Position was too well-paid (!)
5. No clue was given as to what you wanted/how it would be assessed.

Point 4 is interesting. Some people felt like the pay meant that it was too competitive and likely to attract people who weren't entry level. But I'm confident in my process here - the flat rate in the company is meaningful to me as a Studio Lead*, and not something I'm willing to compromise on. Also, as someone assessing the applications I can assure people that we are also looking for people who will grow with and benefit most from working with us - this absolutely does disqualify very experienced people.

* Read this thread for a bit more context on this, it's a tad complicated but here isn't really the place to go into it.

On points 3 & 5: Many of the things which made the application feel general (and why we didn't list how things would be 'marked') are part of the intentional design of the process so that we could try and make it as accessible to people from many levels of experience and background. Some people (especially those who have come from middle and upper class tutored backgrounds and schooling) are taught how to 'write to the marking' in a way in which other cultures and class backgrounds are not. I think the solution to this for next time is an FAQ.

I want to be clear that I'm not dismissing these feedbacks. I think it could be easy to read through the 'what would you change' free-feedback and discount it using one or all of the following though processes:

• All of these have a direct opposite opinion in the 'what was good about it' feedback, and so they 'cancel out'.
• Many of these things were beyond our physical capacity.
• Many of the things critiqued were part of the intentional design of the process so that we could try and make it as accessible to people from many levels of experience and background.

But I want to be clear, even if I want to keep some of these intentional design decisions in the process and I can't add more resources in to respond to each Stage 1 applicant individually, I can do better at providing the applicants information on what to expect. It was hard to strike a balance for the position's copy between readable and easy to process, and answering every question. The same too with the form - provide an application form wide enough to welcome all, but specific enough to make you feel like you know what's 'wanted'. But now we've done this evaluation work we can provide an accompanying FAQ which explains (along with other common questions):

• We can't offer feedback at Stage 1 because of our resources (2-3 people who also have to do the rest of their jobs while processing applicants).
• We have left the form short and the questions general for accessibility - because we don't want anyone to count themselves out, and we're interested in people transitioning between disciplines as well as e.g. recent game graduates. Also because we think unpaid tests are unfair, and that greater specification privileges certain backgrounds.
• Why (if we continue to do so) we choose not to share our assessment criteria. I'm not sure about this and will think on it more with the team.
• Positive advice on how to apply, now summarised in an excellent blog post by Char Putney.

Conclusions

I've been writing this post for a long time, and you've been reading for a long time too! So I'm going to be short and sweet here about our takeaways.

Good things:

• I'm proud of how we resourced this application process.
• Providing tech, money, and implicit context in the wording, video and presentation of the company made a difference to those applying.
• The laptop wasn't a priority for most (0.3% of respondents) but its cost is such a minor factor in the whole of the things, I feel like it's valuable to include (and demonstrates our ethos).
• We reached a large pool of countries of residence through the remote nature of the listing.
• Those who made it to Stage 2 and 3 were grateful for the feedback we were able to resource.
• 15.6% of people applied because of the way the posting was worded, 8% listed the video as a motivation for applying, and 7% the People Page (i.e. our existing team).
• When asked about the top 3 priorities for applying, 11.5% counted the company ethos among them, 10.5% the pay, 12.6% remote work, and 8% the mentorship.

I believe this vindicates a lot of our efforts.

What I will do differently next time:

• Prepare a timescale that can accommodate 3000 applicants
• Provide an FAQ on response time, feedback to be expected, how to apply, and many of the other FAQs we received. I will explain the reasoning behind some of the common negative themes which I think enhance the accessibility of the posting.
• Ask people to check their email is correct by submitting it twice, and to whitelist the company email.
• Make sure our server can handle a lot of sudden attention.
• Because of the popularity of the ad, some applicants (I didn't have room to go into this but it sucked) applied via third party websites who - without our permission - copied our ad, but made you pay to sign up to be linked to our page, and others who took their submission and didn't send it anywhere. We need to be clear the only place to apply is via our form.
• Work harder on the means of phrasing the marginalisation question.
• Have a tick box where people specifically have to agree they fit the time zone requirements.
• Work with Ben and Char to think about how much of our assessment criteria it's supportive to be transparent about.
• Workshop the marginalisation question (and run it past our Danish lawyers again).
• Try and represent the needs of the company & games industry within the larger diversity and labour discourse of Denmark (not just through lobbying, but also being on industry panels, education panels, speak at events etc., I'm already doing this a fair amount so really 'how do I change the law' is the question here).

I hope that's useful, interesting, and a valuable insight in some way to anyone who's made it this far into the article. Please do get in contact if we can help you advocate in any way within your own companies to build more opportunities like this. And remember we have a mailing list of people eager to hear from you when you do!

I know I put the credits up front, but I'm especially going to foreground Ben Wilson and Char Putney here, for being the other 2/3 of the 130 person hours which went into this process.

Full transparency

Finally, you can also see a summary and excerpts of the 'free' feedback all laid out in one place in this public Notion page.

Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.

A new tool that blends your everyday work apps into one. It’s the all-in-one workspace for you and your team

Notion