Skip to main content

Data-driven versus intuition-driven game design

Left to right: Philipp Karstaedt, general manager for Europe, Gree International; Adam Telfer, product lead, Wooga; Tammy Levy, director of product, mobile, Kongregate; Christopher Kassulke, CEO and owner, HandyGames; Dean Takahashi, lead writer, GamesBeat at VentureBeat.
Wooga is pronounced "Vooga." It's like saying the old "Sega!" scream.

Above: Wooga is pronounced “Vooga.” It’s like saying the old “Sega!” scream.

Image Credit: Dean Takahashi

GamesBeat: When Wooga started, I imagine your role didn’t exist. Along the way, when did data people become part of design teams?

Telfer: I wasn’t there from the founding, so I don’t exactly know. But Wooga was always a free-to-play company. We started on Facebook. If I were to speculate, I think data was embedded from the beginning. Over my own career, starting in indie games — as soon as you move into free-to-play, you see the value of data. My job as a game lead — part game design, part producer, part product manager — became part data as well. As I gained more knowledge of how to use data, it became a bigger part of my job.

Karstaedt: Going back to what I was saying about what’s hard data and what’s soft data, if you do exit polls, that gives you only so much customer insight. What people tell you they want is not necessarily what they really want, which is why it’s important to back that up with hard data. It doesn’t matter if everyone tells you they love a certain feature, or they’d play your game if it had this feature, when they don’t follow through on that. If the data shows that they’re not acting on this thing they say they love, clearly they don’t love it. That’s the value of hard data.

Sometimes, you have to abandon your own idea of what you’d like to play. A lot of indies, I think, have that problem standing in their way on the path to mass-market success. If you start working on games with a publisher or with a license, at some point, you may have to abandon some of what you personally think is good and go see what the market wants.

Even if you don’t want to design your game based on data or market research, at least … in case you want another company to come in and operate your game or have anything to do with your game, at least put in the data collection triggers. Make sure the game backend collects data, even if you don’t think you want to use it. Give others the opportunity to use it if you want anyone else to be involved with your game. Change things in a way you think is best, but see what your players actually want. With data, you can determine that.

GamesBeat: Chris, you have all kinds of people at your company. You’re the CEO. You have finance people, marketing people, game designers, and now data people as well. You may be the boss of these people, but you have to get them to work together. How do you make sure data people mesh well with designers?

Kassulke: Communication. Drinking beer [laughs]. It’s about understanding where both parties come from. We have very small teams, compared to Zynga or Gree or Wooga and many others. The team decides if they want to use the data. We always say, “Are we asking the right questions?” Data is good, but if you do games like we do — globally, on many different platforms — they have to work on PC, on console, on smartphones, on tablets. Perhaps a title isn’t the best performer in the mobile space, so maybe we should concentrate on another platform. Or, we might concentrate on a particular country. Why is a specific game not performing in this specific region? You can dive deeper and deeper into the analytics, but can you find the right answer?

With us, it’s easy enough. We just try it out and see. That’s what we’ve found is always the best option. If you look at which games are performing in the category — take Zynga for an example. Their latest titles were not huge successes. The problem is that they’re having to invest so much money and time into a new title to get it past a certain point where the analytics will work. When was the last time Wooga released a new game? Release more titles. Try new things.

Data, in my opinion, is currently holding back the game industry. I see a lot of guys who don’t care about data. They’re just trying new, cool stuff. The freshest titles I see are coming from indies who don’t care very much about data. They’ll try out something without data, and then, they’ll analyze from there. That’s a valuable experience for the game designer and the studio.

Karstaedt: In some ways, it’s a bit like building cars. There are cars that look amazing, that are unique, that are hand built, and there are maybe 10 of them in the world. They’re made without data. That’s not what their customer wants. They couldn’t care less. That’s fine. And then, there are the Toyota Camrys and VW Golfs that are maybe a bit more boring, but they sell to a mass market.

It always comes back to this question of video games as an art form versus video games as a business. Mobile free-to-play is more on the business side, but it earns a lot of money for a lot of people. I can understand that they want to make sure what they put out makes money. But in the end, the only proof that data is everything is that you can’t plan for success. You can do all the market research, but most companies that are big now started with one big game that was very successful. Almost no companies have followed the path of something like Supercell and managed to continuously bring out game after game that finds success.

At the core of it, game design is an art — molding an experience for the user, telling a story. Even if it’s a simple story, a story that’s told in five-minute play sessions. If you manage to do that well, you can start looking into data. But like Tammy said before, data can lead you to a dead end. You can overanalyze and over-optimize. With a different approach, you can find a step-function change, and it can perform much better. If you just follow optimization blindly, you get stuck over-optimizing an inferior solution. Only design can go against that.

Telfer: Getting back to the question about data and design people in a co-working space, typically, I don’t see fights breaking out between data and design. Usually, I see more fights between marketing and design. When you set up a team where there are two silos operating completely independently, that’s where you get the finger-pointing.

That’s why it’s important to get data people on the team. Not every team can afford to have a dedicated data person, but design people can take a percentage of their time to become more data driven themselves. That builds their appreciation and makes for a much healthier environment. I’ve worked with great game designers that are way better than me at analyzing data, and it’s made them better at design. They can take learnings from data and apply those numbers.

The interesting thing to me about data driven versus design driven isn’t necessarily around soft launch. When you soft launch a game, you throw up the KPIs and see how you’re doing and say, “OK, are we going to push this or not?” Most people should be doing that. The real question is at the beginning of the game — when the team’s getting together and saying something like, “We’re going to create the greatest mobile MOBA ever because we feel like that’s a game the market’s ready for.” Versus making a decision based on market research and driving the design of a game with market research. That’s a pretty hot debate there — how much of a core game, from the beginning, should be driven by the design team saying, “We think this will do great,” versus analysis and data making that decision.