Studio Cypher: Category postmortem

Ludium 2 Thoughts

Will Emigh — Wed, 04 Jul 2007 20:26:00 -0700

Why a game?

If we want to argue that games can have serious outcomes, at least some of our conferences should also be games. The first Ludium was a hybrid of sorts, with several mini-games only loosely connected to the main thrust of the conference. This Ludium was designed from the ground up as a play experience that would also have a serious outcome: a vision of how video games and policy should mesh.

Some have argued that the game framework of the Ludium warped the results (in particular, critics find the statements of policy to be relatively weak). I think you’d find the same broadness in any statements acceptable to a reasonably large heterogeneous group. In addition, any function with a similar goal is going to be game-like in some respects. Robert’s Rules of Order are merely a more familiar set of rules.

In particular, we designed the game to encourage the behavior we found most useful at the last Ludium and other conferences. Primarily, we wanted to make sure that everyone felt that their ideas had been heard. We wanted the Ludium to end with a group of people who were invested in the results.

What worked well?

Based on people’s comments after the Ludium, I think we met that basic goal very well. Even though not everyone agreed with all of the statements, they seemed to agree that they deserved to be discussed. And, unlike the first Ludium, there was a much greater sense of cohesion at the end, despite the sometimes vigorous debates during the conference.

Probably the central reason that everything went as well as it did is something that we can’t take the credit for: the attendees. We had a great group of people from diverse backgrounds who played the game with an eye towards the future. It would have been great to have more people and more stakeholders represented, but the people who did come were dedicated and cooperated even more than we expected.

A design choice that really made a difference was starting with small groups and merging them over the course of the Ludium. As a member of a group of 2-4 people, each player was able to have at least some say in the platforms that were considered. It was heartening to us that many of the statements coming from these distinct groups were very closely related. That indicates to me that there is a broad consensus about the most important areas of video game policy.

What could be improved?

In retrospect, the spies of Koithuo were the most game-like element of the Ludium. These three individuals were given a pre-defined list of ten concepts before the Ludium began, with the goal of getting as many of these as possible into the final platform. When designing the game, we had no way of knowing which aspects of the industry would be represented. To make sure that there would be some input from the industry as a whole, we used a Delphi study to find the ten most important policy ideas among a wide range of stakeholders. That ensured that these points would at least be discussed during the Ludium.

This turned out to be an unnecessary precaution, since all of the statements given to the spies were independently introduced on the first day. Given the makeup of the conference, we probably could have done away with the spies entirely. With a more antagonistic set of players, a less universal goal, or a more controversial topic, the spies could still provide a valuable service.

Procedurally, the biggest problem was that it took too long to tally votes. The scoring system required detailed record-keeping of who sponsored which statements. There were many more mergers of statements than expected, which made calculating the results slower than in playtesting. Physically entering the ballots into the computer also took too long. In the future, the IP system could be replaced with a simpler scoring system and ballot-entering could be done in parallel on multiple computers.

Was it a success?

It’s still early to say whether or not the policy will lead to real-world changes in video game policy. Nevertheless, there’s no doubt in my mind that the game itself was successful. The people who attended the Ludium left with a feeling of consensus and produced a document that has already fostered myriad discussions. The statements produced by the people of the Ludium won’t end the debate about video games and policy, but they show promise as a beginning.

PFF Postmortem

Will Emigh — Wed, 13 Sep 2006 11:04:00 -0700

Now that enough time has passed for us to gain some perspective on the game (and ruminate on your comments in our survey), we’ve looked over Perfect Friends Forever and decided what worked, what didn’t, and what we’ll do differently next time. Overall, we’re quite happy with this episode. It went much more smoothly than OITC and hit most of the goals we set for it.

Without further ado, here are our thoughts on the second Cyphers adventure!

Overview

Perfect Friends Forever started July 1st and ran through July 31st. The first week consisted of a slow introduction to the PFF website, where Madame Z was trying to start a dating service using ghosts as advisors. Players were also pointed to the Bay Horse, MO website and the blog for PFF’s webmaster, KH.

Once the game began in earnest, players discovered that Madame Z had been threatened, presumably by one of her ghostly advisors turned poltergeist. At this point, players were given limited access to the ghosts in the form of puzzles. As each ghost’s puzzles were solved, they revealed more about themselves and the rest of the household, including the events surrounding the death threat.

Madame Z hired a private investigator named Anton Jeffries to help her determine which ghost was the culprit, since she wasn’t willing to fully open the site with a poltergeist on the loose. A mysterious vision led players to open Anton’s case notes, where he tracked his progress throughout the story.

During his investigation, Anton discovered that none of the ghosts were exactly as they seemed but didn’t come any closer to figuring out who had created the threats. He sent a copy of Wylie’s will to Wakeful Agents, but its supposed “treasure map” was never solved.

Soon after the investigation started, Madame Z was attacked by a grave robber, seemingly in an attempt to achieve the earlier threat. Madame Z survived with a small concussion after her screams brought Anton running to her rescue.

With no solid leads, Anton conferred with the players and set up a trap to catch the culprit. Although the trap was basically successful, players had to provide further evidence to finish the case and see the perpetrator punished.

The Wylie Treasure remains unfound… if it even exists.

What Went Right

Casual fun (without making it easy)

One of our major goals going into PFF was to make it easier for people to play the game. Out in the Cold had so many updates that it was hard to enjoy the game casually. PFF only had updates twice a week, which made it easier for people to follow without spending significant time on the site. To keep more involved players interested, we made email much more important and put up new puzzles every day. In addition, players were occasionally able to ask questions of advisors as they answered their puzzles, giving them opportunities to get involved whenever they had free time.

Puzzles

One of our design goals for PFF was to have more puzzles, since Out in the Cold focused almost exclusively on storytelling. In this respect, PFF succeeded admirably, with over 75 puzzles over the course of the game! Having new puzzles available every day helped offset the fact that major story updates only happened twice a week. It also encouraged player involvement.

Community involvement

We restricted the number of puzzles any one person could answer (and served them in a semi-random order) to encourage people to work together as a community. No one person could access all of the comments made by the ghosts, so nobody could completely experience the game without working with others. This environment allowed even casual players to give valuable information to everyone else.

More Characters

In addition to the six ghostly advisors, PFF had a varied cast, some directly accessible and some not. Having multiple characters available made it easier for players to build a unique rapport, one of the major strengths of MultiPlayer Novels. Even characters who weren’t accessible added depth to the game by giving new perspectives on events of which players had already seen one aspect.

What Went Wrong

Plot Input

It can be hard to determine exactly, but our impression is that players didn’t feel like they had a lot of say in where the plot went. This might be because there were few obvious branching points in the plot. Despite this impression, there were actually several areas where players had impact. For example, Stanley’s role was originally much smaller, Jasmine wasn’t going to break with Nickolas, and the treasure was going to be found. In the future, we need to make sure that players notice that they’re affecting the plot as they’re doing it.

Unclear intermediate goals

Although the overall goal was clear (figure out who threatened Madame Z), it wasn’t always clear what the players should be working towards. This was mitigated by the obvious default, puzzle solving. For those who aren’t puzzle solvers, this wasn’t great. It also made it easy to feel disconnected from the overall story.

Slow interactions

Another aspect that disconnected players from the story was the slow and intermittent nature of most character interactions. We had some problems with connections, moving their appearance back from noon to almost midnight over the course of the game. In addition, the questions players could ask the advisors were answered intermittently. This was obviously frustrating and should have been resolved before the game started.

Lessons Learned

In general, PFF went a lot more smoothly than OITC. More people got involved and stayed involved. There are still definitely areas for improvement, though. PFF avoided the major pitfalls from OITC but also lacked some of the real high points. In general, I think this was because the game was a little disjointed with the puzzles and gameplay generally separate from the story.

Mix story and puzzles

One major way to fix this is to integrate the story and puzzles more. There were really only two serious puzzles central to the story. It was enjoyable to have consistent puzzles that weren’t required to move the story along, but additional puzzles would also have given players a real feeling of power in the story.

Clearer small goals

Another way to improve players’ control over the story would be to make the minor goals clearer. For example, instead of Anton asking, “Which ghost did it?” he could ask, “What is each ghost’s alibi for the night of the threat?” Although they’re functionally similar, and the latter leads into the former, the second question asks players to perform a very specific function that they can accomplish within a couple of days. This allows for better feedback for the players, since they can receive rewards every couple of days if they’re on the right track.

Better reward structure

A better reward structure overall would also make the game more enjoyable. The connections worked well for what they were, but the rewards were too unpredictable and, often, uninteresting (as when solving a puzzle just increased the connection). A better system would have some immediately apparent goals as well as random rewards. Knowing that you’d get to ask a question at 18% connection would give players something to work towards. If they also got something at 12%, that’s great, but small goals are important as well.

Conclusion

So far, we’ve mostly been working within existing ARG frameworks. Over the next several weeks, we’re going to be working on some new concepts that aren’t directly related but that will be connected and will certainly be informed by the things we’ve learned with our first two MPNs. We’re also thinking of starting a behind-the-scenes Podcast if there’s enough interest.

Thanks for your interest and support! We look forward to entertaining you again soon.

OITC Postmortem

Will Emigh — Sun, 25 Jun 2006 19:23:00 -0700

Now that most people have filled out the survey, we can talk about our impression of Out in the Cold without influencing answers (but keep filling out those surveys! One lucky respondent will win a free subscription to episode 2). Overall, we’re very pleased with the first episode. It was a real experiment in a lot of ways. Nobody has really tried a subscription since Majestic, most MPNs are much longer, and it’s the first part of a serial story. There’s definitely room for improvement, but the response has been good enough that we know we’re on the right track. That said, I’m going to take you through the good, the bad, and the ugly from the inside.

Overview

Out in the Cold lasted the six weeks from May 1st through June 13th. The first week was the pre-game Batch Processor puzzle (more on that later) with the full game lasting five weeks. The story was about the mysterious disappearance of Charles “Chuck” Lehner, an Agent for the Cyphers (a sort of supernatural preservation society.)

The story launched with a postcard leading to Chuck’s dead drop where he mentioned working on a possibly dangerous project. From there, the Wakeful Agents (our players) found his website and then the website of one of his friends, Sue Terrini. Sue had heard of the Cyphers from Chuck and attempted to contact them when Chuck missed a meeting with her.

With Sue doing the legwork, players discovered that another of Chuck’s protégés, John Match, had accidentally broken an ancient artifact and released a doppelganger from confinement. The doppelganger rewarded John with wishes, each coming with a terrible cost. John realized the cost too late, after causing the doppelganger to kidnap Chuck.

Chuck managed to escape the doppelganger and send a package to Sue containing the first information on how to fight the doppelganger. Meanwhile, John made his way to several of Chuck’s caches of information which held further clues about the doppelganger’s past. Finally, Wakeful Agents in Seattle and San Francisco were able to discover a ritual (and the antipode of that ritual) to bind the doppelganger to John’s body. Such a binding would kill John and give the doppelganger his place in the world. Unfortunately there seemed to be no choice but to complete the ritual, as the doppelganger threatened John’s sister.

While Sue performed the rituals on John, weakening him and strengthening the doppelganger, the Wakeful Agents performed the antipode rituals to strengthen John and weaken the doppelganger. Wakeful Agents also searched for an antipode to the final sacrifice alluded to in Chuck’s notes. After several days of this, John ran off to sacrifice himself, not realizing that Sue and the Wakeful Agents had a plan to save him.

The evening of the sacrifice, the Cypher Kevin sent Sue the information from Chuck’s Chicago cache, which included the antipode sacrifice. Following that, Sue was able to create a “hollow man” and imbue it with the energy the Wakeful Agents had created with their antipode rituals. At the last second, Sue managed to stop John and send the hollow man to the doppelganger as the sacrifice in his place.

The doppelganger took the bait and was trapped within the hollow man. Unfortunately, the doppelganger was still not helpless and almost killed Sue before John was able to pry the clay sigil from the doppelganger’s new chest, separating the body from its influence. With the doppelganger safely trapped in the clay piece, Sue took John and his sister to the hospital.

Pleased with her ability, Kevin asked her to join the Cyphers but Sue decided that she had to find out what had happened to Chuck before she could do so.

What Went Right

The Story

Given our commitment to a serial structure for our MultiPlayer Novels (MPNs), story is incredibly important to us. Even though it only lasted 5 weeks, we were able to tell a story with character growth, hints about the Cyphers’ world, and a definite conclusion while still leaving space to extend the story in the future. Sue in particular had a really strong presence, which lead to some great interaction with the players.

Physical Items

People like getting stuff in the mail and many MPNs take advantage of that. To give just two recent examples, Who is Benjamin Stove had “Random Crap” and Catching the Wish has comic books and tickets. We want all of our Wakeful Agents to get a souvenir for each case so that they have a physical reminder of their participation. Out in the Cold’s item was a clay hexagon matching one of those used by Sue to trap the doppelganger.

The only thing cooler than a physical memento is a physical memento used in the game! Wakeful Agents took a series of pictures with their clay tiles as part of the antipode rituals. We got a real kick out of seeing people’s interpretation of things like the opposite of an ‘empty house’ and our impression is that most people had fun taking the pictures and expressing their creativity to us and the other players.

The Wakeful Agents

Of course, that physical items wouldn’t be cool without the players. No MPN could survive without player interaction. It’s been part of the common wisdom in the community that nobody will pay for a good story in this format. Given that, we had low expectations on player signup for our initial story. We were overwhelmed with the response we got. People are more than willing to support an MPN that interests them.

So much for the business side of things. On a personal level, we had some great players who really got into the story and interacted with the characters (and with us!). We hope many of you will come back for our next episode and share your experience with the newcomers as well as keeping it fun for us to run the story.

What Went Wrong

Pacing

Even though the story worked well overall, there were a couple of points where it bogged down. The most obvious point is the (infamous) Batch Processor puzzle. Basically, players were given a fragment of text and the amount of ‘poison’ and number of ‘parasites’ in that passage. They were able to make a guess on the next text and finish the puzzle once they’d gotten three correct guesses in a row.

The Batch Processor was our pre-game activity to give people a chance to sign up before the game started in earnest. Unfortunately, the puzzle dragged on, wasn’t all that interesting by itself, and didn’t engage those players who aren’t interested in puzzles (yes, they do exist). We still think that the basic idea of a collaborative puzzle with immediate feedback is a good one, but the Batch Processor is a shining example of how not to do it.

The story also slowed down in the middle for about a week as players searched for and examined Chuck’s information caches. Having physical caches is fun, but it always takes some time to organize retrieval. Since the story hinged on the contents of those caches, not much could happen while they were being analyzed. In the future, we’ll have more than one storyline (and more than one character!) so that if one part of the story is dragging, people can still engage with the rest of it.

Communication

It’s difficult as a Puppetmaster to figure out exactly how much to communicate with your players. Direct emails are out for obvious reasons, but even if you speak through a character, it breaks immersion. There’s also a balance between total realism (characters only speaking about what they’d actually know) and game balance (sometimes players need to know what to do). It’s also difficult to know how much to communicate with players as a company in this industry. The major players tend to be pretty secretive, for obvious reasons, so we defaulted to that. This lead to some miscommunication, so expect to hear more from us in the future (starting with this postmortem)!

Once again, the most obvious problem with communication was the Batch Processor. We didn’t make it clear enough that it was a pre-game puzzle and that the story wasn’t going to kick into gear for at least several days. This lead to frustration among the players as their first impression of the game was a poor one. In addition, there were two typos in the text (one a leftover from a prior, more difficult version of the puzzle) that made it almost impossible to solve. It’s a good thing that ARGonauts regularly solve the impossible!

Unclear in-game goals were also an issue at times. One of the great things about MPNs is that they’re so open-ended that players can get involved in myriad interesting ways. In the beginning, we erred towards open-endedness and gave the players very little direction beyond the initial “find Chuck Lehner” call to action. This lead to some of the pacing issues mentioned earlier as we waited for players to decide what to do next. By the end, we’d realized our mistake and gave players an opportunity to express their creativity and get involved within the well-defined framework of the rituals.

Outside of the game world, we did a poor job communicating what was subscriber-only and what was public. It was never our intention that non-subscribers be excluded from the game as much as they were. Non-Cyphers characters will talk to anyone, just as they do in other MPNs. And of course, non-subscribers can visit public websites and solve most of the puzzles. The Wakeful Agent-specific content includes communication with the Cyphers themselves and some unique content, including additional puzzles (although subscribers are welcome to work with non-subscribers to solve these puzzles).

Logistic Problems

We’re back to our old friend the Batch Processor. Since the ‘poison’ and ‘parasites’ couldn’t be counted by computer, we counted it manually, leading to human error (a missing comma) after our normal checks. We also originally had a different definition of ‘parasites’ and didn’t change the initial clue when we changed the definition [Update: this happened during prototyping, before the game actually launched, not in response to the players]. Finally, the code for checking guesses over time wasn’t properly tested for things like a long pause between guesses. The problems we had with the Batch Processor were especially egregious because this was a long-term (several day) puzzle and it was players’ first foray into the Cyphers’ world.

Mailing out items was also a lot more problematic than expected. If the code stamped on the postcard was in the wrong place, the post office routing code couldn’t be scanned by machine, causing some postcards to arrive late and occasionally partially destroyed! The clay tablets were too thick to go through a machine sorter so they also had problems. Some of the tablets cracked while others came through with a destroyed envelope.

Lessons Learned

We’ve obviously learned a lot. What are we going to do differently next time? Since we encountered many of these issues in the early stages of the story, many of these lessons have been integrated into our next story. Some of it we’re acting on right now by making this postmortem public. As a small company creating short stories, we’re able to stay agile and really make changes based on user feedback.

More Characters

More characters are more interesting and make the story more complex without increasing the workload too much. It also means that some characters can be waiting on things without the entire story dragging to a halt.

Better mix of activities

Out in the Cold was our first real attempt to woo new players while still interesting experienced players. As a result, it didn’t totally satisfy either. Experiences players were frustrated with the number of puzzles and casual players often had trouble keeping up with the in-game content. In our next episode, we’re going to have more optional activities (mostly puzzles) so that casual players can keep up but experienced players (and players with more time) will still have something to do whenever they feel like playing.

Clearer goals

Many people are turned off when they don’t know what to do. To make our MPNs more fun, we’re going to be clearer about what our characters want and need at any given time. Not everything will be a direct call to action but at any given time there will be a clear goal to work towards. As part of this (and our attempt to create a better mix for casual and experienced players), the next episode will have major story updates only twice a week even though there will always be things to do.

Conclusion

Even though Out in the Cold wasn’t perfect, we feel like we’re on the right track. A lot of people had fun, we had fun, and we learned a lot! Our next story, which launches its pre-game July 1st, was designed with these lessons in mind. This will make it better and your feedback will make it better still. We look forward to playing with you soon!