Game boss interview: Rob Pardo says playtesting is critical to game design

Rob Pardo developed his relentless focus on excellent game design when he was at Blizzard Entertainment, making games, such as StarCraft, Warcraft III, and World of Warcraft. He learned to experiment and playtest and to do that over and over again until the game felt right. He also learned to let go of games that didn’t meet the highest quality standards and move on to the next project.

He took those sentiments with him when he left Blizzard in 2014, after 17 years at the big game publisher. He was an adviser at Unity for a while and then moved on last year to start Bonfire Studios, which raised $25 million from Andreessen Horowitz and Riot Games. Pardo is now working on a secret project, but he still believes that excellence is something that you try to do every day. He has recruited other like-minded professionals, from Min Kim, former CEO of Nexon America, to former game TV broadcaster Morgan Webb.

Pardo is going to give a talk at The View conference, an event that takes place in Turin, Italy, on October 23 to October 27. I did an interview with him, and he gave me a preview of his talk, dubbed “Excellence Is a Journey, not the Destination.” In our conversation, I tried to extract some stories that help define excellence and quality from his perspective.

He brought up an example of working with Blizzard cofounder Allen Adham on testing a unit in StarCraft. They obsessed over the details and then changed it based on what they found. That iterative process of playing and testing is a critical part of game design, Pardo said. He had some painful times when the work didn’t pay off like when Blizzard canceled Titan, a massively multiplayer online game that Pardo was working on.

Here’s an edited transcript of our interview.

GamesBeat: What is your talk about?

Rob Pardo: I called it “Excellence Is a Journey.” You don’t get these opportunities very much in your career. I had a very long career at Blizzard. I thought that before I get too deep into Bonfire, it’s a good opportunity to reflect back — for myself more than anything and extract all the lessons I learned from the time I was there and try to turn those into things I’ve learned, things that I think led to the success Blizzard had, and that I had as a part of Blizzard. Hopefully, make that something interesting or useful for other people trying to go down the same path.

GamesBeat: Had you ever talked about this before?

Pardo: Not in a retrospective way, which is what I thought would be fun for me and for everyone else. I’ve usually done much more focused talks. I’ve done a couple GDC talks. I’ve done a talk on Blizzard game design values. A lot of my talks are obviously much more design focused. I felt like this crowd, if it does have more of a mix, with people coming from other industries — I wanted to do something more high level. Not just specific to games but hopefully for any creative entertainment endeavors.

GamesBeat: Do you identify with something like the Blizzard way of making games? Do you consider that to be the same as your own view of making games?

Pardo: It’s going to be very hard for me to ever somehow be different from that. I feel like I grew up within Blizzard. So much of my way of thinking about things was forged there with the people that were there. I still continue to believe in a lot of the same tenets as Blizzard, the way they produce games. I don’t know how to separate myself. I’m sure over time, maybe 10 [to] 15 years in the future, I’ll forge some different learnings and lessons at Bonfire, and it’ll be a mix of the two.

GamesBeat: What’s a good way to summarize that? What would you mean by excellence as a journey?

Pardo: Oftentimes, you’re always thinking, “I want to have this great product.” But what I’ve found is that you need to really be focused on the here and now, the people you’re working with, and doing things in the right way. Working with great people and having similar motivations. If you do that well, you’ll end up at a great destination.

What I’ve tried to do is think back through the different projects I’ve been a part of. What are the learnings from each one, some of the high-level learnings? Trying to put that together into my journey of learning how to get better at making games.

GamesBeat: What are some of those that you think built on your previous games? With, say, World of Warcraft, was there something interesting about that, above and beyond what you’d done before?

Pardo: When you start looking at Blizzard or stuff I was involved in at Blizzard, there are a lot of things that come through. Details matter. Nowadays, when you talk about the Blizzard polish, a lot of that comes from being extremely detail oriented throughout. Being really focused on gameplay, caring about the worlds you create, and making them have emotional resonance for the community. Being very player focused.

These are all things I know I’ve talked about or Blizzard’s talked about for years. But just trying to pull out some more specific anecdotes that make those things more real on a day-to-day basis, rather than talking at a high level about things like. “Don’t ship before it’s ready” is another Blizzard tenet, but what does that really mean? How do you decide when it is time to ship? Those are the types of things I’m trying to get into and trying to tell stories from the development of the different games I was a part of.

GamesBeat: How early did you get there at Blizzard?

Pardo: I joined during the original StarCraft development.

GamesBeat: I don’t know if you ever heard this story — it sounds like you weren’t there for this — but on the first game they did for Brian Fargo, Brian wrote them a long note. “Here’s all the things I’ve noticed about the game. These things are good. These things are bad.” [Blizzard cofounder] Allen Adham looked at the note and said, “We have to make these changes. We can’t ship the game the way it is now.” I guess that would have been the first time that they made that kind of decision.

Pardo: It probably was. That’s when they started, working with Brian on those early Interplay games.

GamesBeat: It was basically a finished game as far as they were concerned up until then. I wonder whether that kind of discussion was easy to have or harder to have. Allen has a big personality around standing by that because it served them so well in the beginning.

Pardo: It’s also just continuing to be that detail oriented. My anecdote from the first time I was working with Allen on StarCraft is that I was really focused on giving lots of qualitative feedback and trying to get the game balanced to be as perfect as possible. But Allen was always testing the nuances of the different unit interactions. He would call me up and have me jump into games with him all the time.

We’d do things like, “Let’s take a science vessel and put a defensive matrix on a unit. Now, we’ll put other buffs on the unit and see whether [or] not the matrix acts how you’d intend it to.” At first, that sounds like game testing, but it’s actually game design. You want to make sure you test all the interactions. Maybe they perform correctly in code, but maybe that’s not how you want the interaction to perform once you walk through it. When you’re on the phone with the CEO of the company doing that sort of work, you realize how important it is to the studio.

GamesBeat: There are all kinds of negative things you could think about that, to that style of management. You could say, “Hey, this is second-guessing my design. Why does anybody want to change it?”

Pardo: I don’t know. I guess I never saw it that way. I see it as — you want to make sure that the gameplay is not only how you intend but it’s intuitive to the end user. It’s as bulletproof as you can make it. We didn’t really know it then, but Blizzard games end up lasting for many years past the point of release. It’s because of that maniacal attention to detail. We were always thinking of things that even the players weren’t thinking of at first.

GamesBeat: As far as that kind of actual player feedback, coming from outside the company, how does that figure into your philosophy?

Pardo: It even starts inside the company, though. The thing about a lot of game companies, although certainly, Blizzard is one of them, is that inside of Blizzard is already your core audience. People that work at Blizzard, they love playing the games Blizzard produces. Right there internally, you already have such a great beginning for how to test the game and learn what’s going to be great for players.

What Blizzard does is expand that concentric ring of exposure to the player base as the game gets further along in development. At first, it’s all internally focused, but Blizzard is such a big company that they can now expand that to other teams that maybe haven’t seen the game before, so it’s still a fresh perspective, but it’s from developers and Blizzard players. And then, you keep on expanding that circle. You start having people from outside the company, friends and family, start playing the game. You do some sort of closed alpha test and then a closed beta test.

That whole process is one of the things that allows Blizzard to make games that are great, that last for many years because it is so player focused. It has this gradual exposure to more and more players as you continue to iterate on the game.

GamesBeat: Did you notice a lot of variations on the philosophy among different people at Blizzard? Would your outlook have been somewhat different from, say, [Blizzard cofounder and president] Mike Morhaime’s?

Pardo: Not at a high level? At a strategic level, I think everyone was pretty aligned on what’s important to the game and what has to happen to make a great Blizzard game. Tactically, everyone has different styles and different leadership philosophies, but at the really high level, the product strategy level, there was always a lot of alignment.

GamesBeat: Does it mean that you have to get into the details, even as leader? You have to try to understand the whole game, not just delegate.

Pardo: It depends a bit nowadays because the games are just so much larger in scope, so much more complicated. When we were making the original StarCraft, the entire development team was around 25 people. It’s a much more manageable thing, where you can get as deep into the details as I was talking about. Now, the Blizzard teams are, in some cases, hundreds of people. It’s a lot harder to be microing on every single unit interaction. I don’t think that’s possible anymore.

GamesBeat: If you have to let go of that, what’s the advice you would have for people who want to pay attention but also do their job at the right level?

Pardo: I don’t think there’s a formula to that. It’s very people focused. If you’re a creative or design leader over a much larger team, your number one goal is to have a healthy team with talented folks that you can trust to drive every game. Hopefully then, you can leverage your own abilities and strengths to focus on the areas of the game where you can make an impact. That’s the real high-level theory. The way that actually plays out really depends on a lot of factors. I wish there was a formula to it, but I feel like that’s the closest I can give you.

GamesBeat: Which game do you think forced you to make that transition? If you were talking about those kinds of details with Allen on StarCraft — by World of Warcraft, was that the point where you couldn’t do that anymore?

Pardo: No, not on World of Warcraft. Even on Warcraft III, it got a little challenging to do that exact same level. While I was involved in many aspects of Warcraft III, that game was definitely starting to stretch the scope. We had a level-design team that was doing a lot of great levels on their own. We had a great many units in that game. It was starting to happen there.

By the time you get to World of Warcraft, now there are game systems I’m not able to be as involved in. I’m not able to be involved in every dungeon design. You starting having to have great design leaders driving those things, with high-level feedback and involvement rather than getting to work on every piece of the game every day.

GamesBeat: Are there some concrete examples from your history for people to help them realize what you gain through that extra focus on quality and doing things more carefully?

Pardo: In a lot of ways, quality and excellence — that has to be something that you intrinsically care a lot about. It’s not easy. It can take a lot of work, a lot of heartache. There were many times in my history at Blizzard where we had to kill games, painfully, which is always in that same pursuit. You make it to a place where you have to throw away years of work because it’s just not at that quality standard. That can be really painful. You have to care about finally getting something across the finish line that hits the mark.

GamesBeat: How many times did that happen to you?

Pardo: You know, I don’t know the number, but I would say — we used to jokingly say that our hit rate was probably similar to the rest of the industry. It’s just that we didn’t ship our failures. There were at least as many games killed as there are shipped in Blizzard’s history.