A ghost game prototype for mixed reality

The 2015 film It Follows is one of my favourite horror films. Besides the disorienting feel, an unsettling soundtrack and measured pacing, it has a very simple premise. Everyday people are turned into objects of abject terror.

In a nutshell and without spoilers, a spirit (IT) takes the shape of regular people. IT follows its victim until IT catches and kills them. IT doesn’t run and scream towards you. IT just walks. Silently. At a very steady constant pace. At first, one can take comfort in that walking pace. There’s more than enough time to run away, right? But it’s the single minded focus with which IT comes towards you which becomes truly terrifying.

One of the creepier versions of IT…

Because you soon realise that you can never rest. You must be alert and plan ahead. How much time do you have before IT catches up? How long can you spend at a coffee shop? What if your bus is late? Can you even afford to sleep?

It’s a wonderful riff on the traditional cat-and-mouse game that serves up a tasty dish of impending dread. So of course I thought, how do you turn that feeling into an experience? And specifically, a mixed reality game?

Mixed reality versus virtual reality

The mixed reality universe includes virtual objects that interact with the real world environment (including real objects, people, locations and/or systems). This new technological approach offers a host of opportunities to reinvent a good scary story, above and beyond what could be experienced in virtual reality alone.

  • Enhanced narrative. The object of fear is often a supernatural figure such as a ghost or spirit who can only be seen by the protagonist. This works well in mixed reality as content can be filtered or restricted depending on a player’s level of access. That is, only the player/protagonist can see the ghost. Moreover, seeing virtual objects in the real world catapults that ghost story straight into the player’s own reality.
  • Strange things in familiar settings. A mixed reality experience plays out in the real world, in familiar and natural surroundings to the player. Additionally, the juxtaposition of disparate worlds (natural vs supernatural, safe vs dangerous, familiar vs strange) can disorientate and intensify feelings of anxiety. A few hours ago you saw a ghost in the kitchen. Now you’re making lunch when something flickers in your periphery… but you’re not wearing your headset anymore. Unsettled much? Playing out a ghost story in everyday life can also lull you into a false sense of security. As you muddle through the normal aspects of your daily routine you could be most vulnerable to surprise.
  • Enhanced characterisation of non-playing characters. Supernatural figure(s) are often bestowed with magical qualities that defy what is normally possible. For example, walking through walls, levitating through ceilings, and materialising from everyday household objects. This might be cool in the virtual reality version of a haunted house but experiencing exactly the same event in your own house can only heighten that experience.
  • The internet of things as your supporting cast. It only takes a little fuel to spark the fire of imagination. Combining mixed reality with smart objects can transform the most mundane events into moments of heart stopping terror. The microwave turns on for no reason, hall lights flicker, speakers spontaneously blare creepy music. Horror movies are full of these delicious moments.

So how do we flesh out a mixed reality horror game that can be played in the real world?

Mixed reality game design

Using the basic premise of the film It Follows we can quickly outline some key features of the game design.

  • The opponent. IT is a spirit that haunts a single victim at a time. The beauty of a mixed reality game for “It Follows” is twofold. Not only is the real world environment part of the game space but everyone in it becomes a character by default because the film features regular people as IT. You now have a cast of thousands from which the player must try to distinguish IT.
  • Player Mode. The film denotes IT can only follow one victim at a time. We’ll call this single-player mode.
  • Objective. Evade IT. In the film, a victim can pass IT on to someone else by having sex with them. But for this exercise, we’ll stick to single player format so that we can fully explore how this game can play out in mixed reality.
  • Win/Lose definition: In the film, if IT catches you, you’re dead. Pretty simple loss scenario. But the film doesn’t have a specific win scenario. This is something we can also explore.

But there are other factors to consider with our mixed reality designer hat on:

  • How does the player and IT navigate the built environment?
  • Are there safe zones? What are the rules associated with their use?
  • How practical is this game when the player has to go to work, school etc?

The best way to understand and test different rules and game mechanics is to create prototypes that test various hypothesis. So let’s get started.

Prototype 01: spreadsheet

I modelled basic game mechanics in a spreadsheet to understand the parameters that a single player would face while they were being followed by IT in the real world. I included a lot of assumptions as a starting point.

IT walks at a steady pace. The average walking speed is around 5 meters per second. For this basic prototype, I assumed the player also walks at the same speed. I took a map of my local area, pinpointed several locations and calculated the time to walk between these points to create a player’s typical journey over several hours. I then modelled different walking routes and stopping times to see if the player could evade IT successfully.

Spreadsheet model for It Follows mixed reality game.

What I learned:

  • This experience works well as a game. IT travels at a leisurely walking pace but it’s surprising how quickly IT can catch up with you. Yes, if you walk faster you’ll gain some distance but stopping, even briefly, will allow IT to catch up quickly. Even while playing with such a simple model I felt the urgency to move faster and not stop for too long.
  • A starting point. In the film, IT is an entity that is always following someone. Even when IT has been passed on to another player IT just starts following the new player. That is, IT doesn’t respawn in new locations. But in a mixed reality game for single players, I will need to decide where IT is located when the game commences. In my spreadsheet I randomised this element.
  • IT follows you, not your route. My spreadsheet was based on IT following a single player. But after testing a few routes I realised the spreadsheet model had a serious flaw: IT followed the player’s route instead of the player’s current location. The model didn’t take into account that when the player changes direction, IT would also adjust its course so that IT is always heading towards the player, not just following the player’s previous route. My model was too linear. So the spreadsheet needs to be a lot more complicated and include geo coordinates for the simultaneous locations of both IT and the player.
  • Player access to information. While playing with the model I quickly realised I was privy to very important information that impacted the way I played the game. At any given time, I knew exactly how far IT was from me, both in distance and time. I could calculate exactly how long I could stay in one location before needing to move on. But in the movie, victims had no idea where IT was until they had visual confirmation. Even then, they had to distinguish IT from a crowd. This gives the victim only a few minutes warning (if they’re lucky). This is critical for player experience. If a player has too much information they will feel relaxed and in control i.e Not very scared at all. But too little information will make the player feel hopeless and anxious all the time. They might just give up. So I must decide how much information to reveal to players or risk an unbalanced play experience i.e The game will be too easy or too hard.
  • A lot more questions. If the player catches public transport could IT board the same vehicle and keep chase? Does IT ignore the built environment of the real world and just walk through cars and buildings towards the player?

My spreadsheet model, although basic, surfaced a lot of interesting points and questions. I could continue to tweak the model and attempt to answer all these questions but it would take considerable time and not necessarily capture all the nuances of playing this game in real life. So leaning on my experience as an experience designer, I decided it would be quicker and easier to test different scenarios and hypothesis using very basic rapid prototyping techniques.

Prototype 02: Paper

Using hexagonal graph paper, a few coloured pens and dice, I quickly playtested different versions of the game. A few elements remained unchanged between versions.

  • One player was IT (red)
  • The other player was the victim (blue).
  • Each player rolls the dice to determine how far they can travel in a single move. They can move in any  direction.
  • The game ends when IT lands on the same or an adjacent tile to the player.

In each iteration I changed only one parameter to test that feature alone. If I tested more than one feature at a time it would be difficult to discern the impact between them. In the image below, I’ve mapped out all the various iterations and how each play test explored a different game mechanic.

Prototype/ playtest iteration framework.

During the playtests I played IT while a friend played as the victim. We recorded our moves on paper.

Version A. Both players can only move in a straight line. Victim is caught within 7 moves.

Version A prototype.


  • This initial prototype demonstrated within a few moves exactly what the spreadsheet model could not: IT will always turn in the direction of the victim.
  • Despite the initial distance IT catches up to the victim surprisingly quickly.
  • My friend who played the victim (and had also seen the movie It Follows) described how anxious he felt while rolling low dice values and IT continued to get closer.
  • We need to test parameters that mimic playing this game in the real world.

Version B. Test element: safe zones are introduced (green areas) where a victim can enter but IT cannot.

Version B prototype.


  • The victim headed straight for a safe zone and tried to stay there for as long as possible. To prevent them from just staying there forever the game needs a game mechanic to draw them out. (In real life this would happen naturally as players have to go to work, school, run errands etc.) We explore this idea further in version D.
  • This prototype allowed IT and the victim to cross paths. So we added a new rule: the victim and IT can’t cross paths in the same move, otherwise the victim will be caught and the game ends.
  • Safe zones in real life have ingress/egress limitations, for example you can only enter via existing points e.g. doors and potentially windows. The ingress/egress points in this prototype are not sophisticated enough to represent the real world. A design iteration could be once the victim has entered a safe zone (e.g. via door A) IT loses track of them. IT would stop and wait at that point. Once the victim leaves the building (via door B), IT becomes aware of the victim’s current location and follows in that direction. This idea is illustrated in the image below. Ideally the victim should use an egress point as far as possible from the original ingress point, forcing IT to travel around the entire building and buying the victim some extra time.
The ingress and egress points of a safety zone. Blue path indicates “victim”. Red path indicates “IT”. Note, while the victim is within the safety zone, IT has no knowledge as to their location (except within the building) and is forced to wait at the ingress point (A) until the victim leaves the building (point B).

Version D. Both players can only move in a straight line. There are safe zones. Test element: the victim must complete an objective (land in all three locations, not just pass through). The game ends when the victim has completed the objective (win condition) or is caught by IT (loss condition).

As you can see from the prototype test below, this was by far the longest game before IT caught the victim.

Version D prototype.


  • This had the best game balance. There was a nice ebb and flow between high tension moments when IT almost caught the victim and when the victim was forced to complete objectives that sometimes put them in danger of being caught.

I won’t post all the subsequent versions here but you get the idea. I was learning a lot of from each version of the game. Talking aloud with the other player allowed us to express and document how we felt while playing the game which was really helpful to isolate and understand moments of frustration, anxiety, or confidence.  The next interesting iteration was Version G.

Version G. Test element: IT could always see where the victim was located but not visa versa until they were within 7 tiles of each other (i.e. a minimum distance of 2 moves = rolling the dice and getting 6 + 1). We playtested this feature by placing a card between the two player sheets (like playing Battleship). By this stage of prototyping we had also introduced ingress/egress points for safe zones (noted in yellow).

Version G prototype.

The interesting finding from this playtest was that the victim found themselves in a constant state of anxiety. Each move was stressful because they did not know whether they were moving towards or away from IT. When the location of IT was revealed, it was almost a relief. (The situation was likened to knowing there’s a spider in your bathroom: it’s worse when it suddenly disappears and you have no idea where it went.)

Summary of findings

So what did we learn from all these prototypes that we didn’t know before?

  • Being safe does not equal fun. Safety zones are an important game mechanic. But you can’t just hide in safe zones forever. There’s no fun in that. The sweetest tension is when you’re almost caught and the flood of relief when you escape.
  • Push and pull. Players rush into safety zones but need something to draw them out. We introduced specific objectives that players had to meet in our playtests. This mimicked how a mixed reality game could play out in real life as players already have obligations that will force them outside (going to work, leaving for school etc).
  • Where is IT?  The most difficult element to game balance is the amount of information to reveal to players about IT’s current location. In Version G we limited this to 2 dice rolls. In a mixed reality game you could reveal IT’s location in several ways. For example, a player could have one chance each day to know IT’s current location. The player would have to use this opportunity wisely. Knowing IT’s location first thing in the morning would allow the player to calculate how far IT was (distance and time) and plan their movements from then on.
  • Respawning. Once IT catches the player the game is over. Although our playtests had the player and IT start the next game in the same original positions (this was for version comparison) in mixed reality we would program IT to respawn randomly. This prevents a player from predicting IT’s path and then creating an optimal getaway route. Random respawn locations also introduces variety since the player would most likely have the same start point each day (home).
  • Win condition. Perhaps the game does not have a specific win condition. Rather, it could be an endless running game like Temple Run, where the player’s aim is to simply last longer than she did last time and earn her position on a ranked leaderboard.

User Interfaces (UI)

I’ll be exploring mixed reality user interfaces in a future post but for now, I’d like to focus on what the player would see if they were playing this game.

Keeping the UI to a bare minimum has a number of benefits:

  • The game would be played over a long period, so minimal UI allows the player to get on with their daily tasks without distraction or annoyance.
  • It shouldn’t conflict or detract from other mixed reality UI that may be engaged for work or entertainment purposes. (In the future, people will rely on mixed reality heavily to communicate, work and learn.)
  • It lulls the player into a sense of normalcy.

The image below outlines a simple UI for various game states.

The UI displays five game states:

  • Game in play denoted by countdown timer (days, hours, minutes, seconds).
  • IT contact less than 10 minutes: timer turns red.
  • IT contact less than 1 minute: player’s peripheral vision turns red and IT is identified via a red sphere. Audio includes a heartbeat.
  • IT contact less than 15 seconds: player’s vision turns red. Audio heartbeat gets faster and faster.
  • Game over: game over announcement appears.

The key design element is a countdown timer. The prototyping playtests (specifically version G) revealed that the player required a minimal level of information as to IT’s current location for optimum game balance. (Too little and the player didn’t have time to make decisions, too much and IT became too easy to evade.)

Note, these images are only 2D representations of the player’s field of view. The countdown timer could be displayed anywhere within the player’s field of view. In fact, it could be “attached” to any object in the real world. For example, the player could simply tap the back of their hand to display the timer display on their skin.

Mixed reality considerations

Built Environment

This game would be great to play outdoors. In a forest, park, or city centre. Navigating this environment would require IT to walk around objects (trees,  sign posts, cars). Street navigation can also be aided through tools such as Open Street Maps but we also need real time object recognition and spatial mapping of the environment. Even more so if we allow IT to follow the player into indoor public spaces (libraries, shopping malls, museums). We need to recognise the layout of enclosed spaces and objects located within them. Mapping internal spaces is tricky but possible. This has been demonstrated through mixed reality hardware such as Microsoft HololensMeta and Google Tango.

Did anyone lock the front door? (Still from the film “It Follows” via Film-Grab.com).

Following the film narrative, IT can enter rooms via conventional ingress/egress points such as doors or windows, but only when these are open. Within enclosed spaces IT cannot pass through walls, floors or ceilings, navigating only as a regular person would from room to room via internal doors.

Safety zones (areas where IT cannot enter)

The prototypes outlined earlier revealed the importance of safety zones. Essential for not only gameplay but also for integrating the game into the player’s daily life. If you go to school or work, you’ll be based in the same location for at least several hours. This would give IT ample time to catch up with you (even if you gained time earlier via public transport or a car). Translated into a game mechanic, the player could designate several areas as safe zones. As the player levels up, the number of safe areas could decrease and increasing difficulty (i.e. as you become more experienced ,there are less places to hide).

Players’ physical safety

The break out augmented reality hit of 2016 was, without a doubt, Pokémon Go. But the game’s mainstream popularity also created safety hazards for players. The game’s official website outlined Safety FAQs which included a note on physical safety.

“PokéStops and Gyms are created from historical sites, public artwork, and user-designated locations. They exist in many places, including trails, parks, and urban areas. The safety of any given area depends on the user, the time of day, and many other factors. We encourage users to use their own judgment about which parts of the city or countryside they feel safe going to at various times of day or night.”

Pokemon Go. Image by PaintImpact via Flickr.

There have been a number of stories (some perhaps overstated by the media in the hype following game launch and surge in popularity) of players entering dangerous areas, trespassing, or just not watching where they were going because they were too focused on their screens (including walking out into traffic). This was a danger to all players not just younger ones. Designs should incorporate as much context or location-based data as possible to warn players of potential danger. In addition the colour, shape and placement of UI elements within the player’s field of vision should not obscure vision in a dangerous way. This is another benefit of mixed reality over augmented reality.

Pokémon are displayed over your phone’s view of the world. In mixed reality, Pokémon would be an object in the world. For example, the image below shows a Nidoran in front of a dog. A mixed reality version of Pokemon Go would allow the Nidoran to run behind the dog.

Pokemon Go. Image by Albert Hsieh via Flickr.

Unfortunately not all situations can be foreseen by designers. The system was also “gamed” by criminal opportunists who created Pokéstops to lure unsuspecting players into secluded areas. It is important to learn from these early examples of augmented reality games as well as behaviour from massively multiplayer online games (MMOGs). They can help us understand how players might interact with the game and each other, to design systems and game mechanics that minimise risk while ensuring everyone still has fun.

Players’ psychological safety

We have an obligation to our players to consider their comfort and well being, even when they have opted in and provided consent to playing a game like this. Video games have age restrictions recommended by the Entertainment Software Rating Board (ESRB).  Theme park rides provide warnings to people based on height and medical conditions. I believe mixed reality games will also require ratings or adequate warnings to users.

While researching this topic, I really immersed myself and imagined how it would feel to play IT Follows in real life in various situations and locations. Perhaps I had primed myself too well. One night, I went to bed early. I must have heard something because I woke up. I was still half asleep when I rolled over in bed. In the darkness, I saw a figure standing over me. (It turned out my partner had come into the room and been talking to me for a few seconds but didn’t realise I had already fallen asleep.) Well my friends. That scared the crap out of me. This personal experience leads me to believe that your home should always remain a safe zone. Additionally you should only be allowed to play this game of your own volition. A person should not be forced to play or be tricked into playing this game.

You may think I’m overstating the power a simple game like this has to scare people. But if you want an insight into how scary the horror genre could become while experienced within virtual or mixed reality, I strongly recommend watching the “Playtest” episode from the outstanding TV series, Black Mirror.

I am a techno-idealist at heart and truly believe in the potential for mixed reality to create, facilitate, assist and guide people in the future. But it also has the power to confuse, disorient and harm people. As computing power improves it will become increasingly difficult to distinguish between what is real and what is not.  What if you couldn’t trust what you saw or heard?  How would you differentiate between a mixed reality experience and the symptoms of a psychological disorder? Are they projected images or hallucinations? Are those voices part of a game or in your head? As the classic film The Game so aptly presents, you can quickly become entwined within your own paranoia. Unfortunately there is a significant risk of gaslighting in mixed reality.

Mixed reality designers of the future must recognise these potential issues from the outset and design experiences accordingly. Of course, this should be just part and parcel of what a designer is trying to achieve, which is an ideal user experience.

One thought on “A ghost game prototype for mixed reality

Comments are closed.