Greg Broadmore interview — why Weta Workshop spent seven years on its Magic Leap game

Greg Broadmore had never made a video game before, but that didn’t stop him from creating a groundbreaking augmented reality game on the Magic Leap One Creator Edition. It took seven years of conception and two solid years of development with a team of 50 people, but Dr. Grordbort’s Invaders debuted last fall on the Magic Leap platform.

It turned out to be one of my favorite 10 games of 2018, but sadly, very few people have played it. That’s because the Magic Leap One Creator Edition costs $2,300.

It sounds so crazy that Broadmore, who worked on the special effects for The Lord of the Rings movies, dedicated so much time on this AR game. But he and the founders of Weta Workshop — Richard Taylor and Tania Rodger —  were taken with the vision of Rony Abovitz, who wanted to create a new generation of technology capable of “spatial computing,” where animated objects can be overlaid on reality.

Broadmore didn’t know how to build video games or run a game studio. He didn’t know what kind of platform Magic Leap would create. But he said, “F*** it,” and just did it. He talked about this process at the DICE Summit, the elite game event in Las Vegas. I caught up with him afterward.

Here’s an edited transcript of our interview.

GamesBeat: I liked your talk. You have to be patient about something to work on it for seven years.

Greg Broadmore: [Laughs] I’m nothing if not patient.

GamesBeat: You went in without a lot of the usual experience of somebody taking on a game project.

Broadmore: I think about that a lot because people — it holds a lot of people back, the thought that you haven’t done something before, and you don’t know how to do it. People will tell you that you need experts to do that, or you should go and study to do that. You can compare yourself against the world’s best people who’ve done something. Those sorts of thoughts hold a lot of people back. They hold me back for a second as well, but then, I think, “That’s ridiculous. I’m going to be dead one day.”

I love doing stuff. The best way to do something you haven’t done before is just to throw yourself into it and do it, find out the hard way. That’s the fun of it in the first place, to me. If I knew how to do it, it wouldn’t be interesting. I find that when I’ve done something a number of times, when I’m what I suppose people would call a master at a certain discipline, I’m bored of it at that point. I’m not the person who wants to go around in a loop on the same thing. I want to find a new challenge. I wanted to make a video game my whole life. I didn’t let anything stop me.

GamesBeat: How quickly would you have learned the distinction between a game director and a game designer and an executive producer?

Broadmore: Exactly. I didn’t. I said at the start that I would direct this game, so I was the game director. I didn’t even know those two words go together. I just thought, game director, that’s what I’ll do. I think on my contract it says “lead game designer.” At the start, I was doing game design but only of a sort. I realized very quickly that I needed someone to focus on the game mechanics and the game loop. You just realize these things.

Also, I listened to my team. I hired smart people, and I listened to them when they said we should do X, and we should do Y. “It’s good to have someone focus on this.” The most important thing is to listen to your friends and the people around you and the people you work with. Then, you just pick the right way through all that guidance. That’s key. I didn’t force my way through it thinking that I knew the right thing exactly. I drive forward with a goal but then just listen.

GamesBeat: Were there a lot of game veterans that came in?

Broadmore: We had a real mixture. I had never made a game before, but, of course, I lived and breathed games my whole life. I thought about them deeply as long as I can remember. Other artists, like my lead artist Steve Lambert — he’s an amazing artist, but he’d never worked on a game before technically. He’s just utterly brilliant. He’s a machine. He’s one of the most hard-working people. Of course, he’ll be able to do it. He might grind away at the edges a while like me, but he’ll figure it out.

Then, we also had really brilliant people I hired on. Jimmy Beard, who’s a game designer from Australia and New Zealand. James Everett was another game designer I hired. I hired Mike Keith, who’s our technical director for the studio. He goes back to the GameCube days and before. He worked on Rogue Squadron. I had a bunch of people who knew what they were doing. I listened to those people.

GamesBeat: Did you find New Zealand had most of the people you needed?

Broadmore: Most of our crew are from New Zealand, but we had to cast the net wide, yeah. We needed such a wide array of skills. Finding them all in New Zealand, with such a small game industry there — I think I mentioned in my talk that there are 5 million people in New Zealand. Even the film industry, as amazing and influential as it is, is not that big.

GamesBeat: How big did your team get, then?

Broadmore: We have 55 people on, something like that.

GamesBeat: That’s fairly small for a game like this.

Broadmore: I think we did pretty good for it, yeah.

GamesBeat: Is that part of why it stretched out to seven years, or was it for other reasons?

Broadmore: No. The seven years is — to be brutally frank, the first five years aren’t making a game, really. The game actually was crystallized in my mind, at a high level, right from the start. But really, you’re just trying different things and experimenting. We were working with the Magic Leap SDK teams and hardware teams, trying the latest builds.

For instance, the big benchtop I talked about, which is this colossal machine back in Florida, the original bit of light-field tech they developed — it’s this huge thing, and you have a tiny little field of view. You put your head into a very specific place and look through. You couldn’t move, but you would see a bit of a virtual light field. The first displays on that were really simple things. They’d have markers and then focus at different depths.

The first thing I did was, let’s chop out images of ray guns and put them at different depths. One step better than putting little crosses. But then, when we came back a few months later, we managed to get our very first Unity build of Invaders working. It was super primitive. You had a very simple 6DOF controller, and if you brought the controller up high enough, you could see the ray gun in the field of view. You would see the robots walking at you from the top of their heads to their nipples, about, and you’d shoot at them.

It was a miraculous thing. You stick your head into this giant machine and see the future. There’s our game on the other side over there. Of course, I couldn’t turn my head around and look anywhere. But that’s what our development was like for the longest time. There’s a new bit of kit, so let’s bash something into it and see what happens. We’ll learn from that and the hardware teams will learn from that. That ping-pong process shaped the platform and shaped the game.

In the last two years, it started to focus in more. But it honestly probably wasn’t until the last six months that we beat it into the shape of a game.

GamesBeat: How would you describe the degree of risk? Were you capitalized before this started? Did you have money from something else? Did the money come in later? Was there pressure to get something out on a timetable?

Broadmore: I’m not supposed to talk about that stuff. Does that say anything? [Laughs] Very loosely, at the start, it was really me and Steve and a couple of other people. It was very low-risk. Then, it just grew organically. I don’t think there was ever a risk, apart from our personal investiture of time and creative energy. That’s the only risk I ever experienced if I want to get away from finances.

I sunk a huge chunk of my creative life into this. Every now and then, you do have those doubts. This is a crazy thing we’re doing. Are we going to get there? Then, I’d go back to Florida, and we’d meet our team and talk to people and realize that we’re all racing in the same direction. But yeah, there were some aggressive timetables early on, as you have to have if you want to make anything. Sometimes, those stretched out. But we absolutely got there in the end. That’s the crucial thing.

GamesBeat: Was there much second-guessing going on? Did anybody say, “Why don’t we make a console game out of this idea?”

Broadmore: No, no. Not for me. I couldn’t think of anything more fun. Like I said, when I jumped into it — part of it really is the adventure. You want to do something different. I love the jumps in video game technology. I’ve watched them all my life. I really got excited when I’d see these jumps in various areas. I’d be inspired by them. I’m attracted to that shift when things move ahead.

I was excited to make a game, but when Rony postulated Magic Leap, that’s the jump. That’s the literal leap that I’m looking for. I had a choice. I could make a game on a screen, which I love, and I still play all the time to this day. But you’re telling me I could make that game live in my world with me. I’ll make that, even if it’s way harder, and it really was hard. What choice is there? That’s so much more rewarding and satisfying. And nerve-wracking at the same time.

GamesBeat: You’re kind of inventing spatial computing as you go. What was some of that like, where the tech is being invented while you try to make a game on top of it?

Broadmore: Really hard, really fragile. I made an analogy to building this bamboo bridge out to the island of Magic Leap One across the water while behind us, they’re building the actual solid structure that we’ll eventually travel on. It was really like that. We know that to get out to that point, this bit of technology is going to have to exist. It doesn’t exist, and the team in Florida is going to take a year on it. What do we do in the meantime? Well, if we hack this and chop this other thing together and we pretend this thing exists and take this thing for granted, we’ll have a solution.

We did that all the time. That really does feel like building on bamboo scaffolding. It’s rickety. You can climb up and think, “Oh, yeah, this works. This will work.” And then, you remember what’s down there. But you continue on building. Luckily, all those things, or nearly everything we bet on, came to fruition.

It was also pushing back against certain things. I’m not going to get into technical specifics necessarily, but some parameter of the system — whatever it was, we were always saying, “More, please! We need more!” Reality is pushing back the other way. Physics and finances are pushing one way, and we’re creatively pushing back. “We’ll have 12.” “No, you can have 11.” “How about eleven and a half?”

GamesBeat: Can you talk about some ways you made use of spatial computing? If a robot comes at me and there’s some furniture in my room, the robot could actually take cover while I’m shooting.

Broadmore: They won’t literally take cover in that sense. They’re actually very aggressive. But they know the furniture is there. They’ll walk around it. The furniture will absorb shots. It is literal cover to them if they’re behind it. That works more to the player’s advantage, which is nice. You can do things like blind fire. It’s such a great video game convention. In a third-person game, of course, it’s not literally blind. You can see where you’re shooting. But in mixed reality, you just stick your ray gun around the wall and hope you hit a robot.

The world really does become part of it. We did great work on taking the scanned mesh, which the user gathers just by looking around and then extrapolating from that. What are the valid surfaces? Where can robots never go? Where can we place Dr. Grordbort? Where can the caverns be? That’s the interesting part of mixed reality and spatial computing. Half of the equation is the person, and the other half is the world.

You have to go into it with a design mentality thinking that way. Where is this person going to play? What am I going to do with that world data to build something interesting?

GamesBeat: With where the tech is now, did it meet your expectations?

Broadmore: Absolutely. From our point of view, hopefully, we showed that you can make a visually high-quality game with physics and particle effects, something that’s quite sumptuous. Tonandi shows this as well. The device is a powerful little device. It can generate great stuff.

If you focus all that in one direction, you can go even more high powered. We spread things quite thin. We have multiple robots and ray guns and worlds opening and all this stuff happening. If you want to focus all that firepower in one direction, you find that the device is incredibly capable. We’re happy with what it’s able to do, and we’re going to have a lot of fun experimenting with it in the near future. The stuff we have coming out this year is going to be able to go even further with it.

Every capability is sharpening as we go. The software side is getting better and better. There are many things I want to see improved, of course, but that’s the whole job. We knew we were basically planting a flag and saying, “Here we are.” Now, we’ll start looking at all the things that clearly need to be improved and chipping away at those. That’s the job of us as content developers — “We’d like more of X and more of Y” — the audience will tell us that as well.

GamesBeat: Does it bother you some that you can’t sell, say, 5 million copies of this game?

Broadmore: I think we will eventually. [Laughs] It’s just a matter of time. We’ve created something special. Now, it’s there, in the can. That’s the great thing. Early adopters have really been enjoying it. Those people have that, and we know that as new people come to the platform, they’ll be able to find it. In the meantime, we’ll get on with making new stuff and refining what we’re able to do with it.

Like I said, we always knew that we were planting a flag and then continuing to climb. We have a medium to define. We’ve only just cracked open the door, really.

GamesBeat: One company I’ve seen have this strategy, always showing up on a new platform, is Ubisoft. They’re always experimenting. They make enough money from other things to experiment. Then, they show up with a new IP on top of a new platform and everyone says, “Well, you can’t sell 20 million of that. If you had it on another platform, you could.” But their belief was that you establish the IP, and then, the sequels make you all the money. You get a head start on how to do that, test whether the new IP is going to work, and then, the installed base grows enough that the demand for sequels turns out better.

Broadmore: They obviously have a good philosophy. Games are all about iteration, right? Within the game itself you’re working on, but on successive versions of the game as well, being able to shape it with where you’ve gotten to with your thinking and how the audience receives it. It makes sense to dive in on a new medium because you can learn so much early on. You get off to the races.

That’s why I feel so excited that we know so much. We can just start developing. It was seven years for us, and it was excruciating. That’s why I make the analogy of the bridge. It was always one step forward, five steps sideways, and two steps back. Slowly chipping our way forward, getting knocked sideways by the currents every now and then. But through all of that, it’s all learning.

Now, when we start developing, we’re going 100 miles an hour. You can see it in the new stuff we’re doing. Builds just pop up instantly. People are doing cool things. Our guys are putting the juice, the exciting stuff into the projects they’re doing because they’re enabled by all that learning. They’ve smashed themselves against the edges of what spatial computing is and found out the hard way. Now, they’re wiser for it.

GamesBeat: You’re not as crazy as you seem, then.

Broadmore: [Laughs] No, probably not. Crazy is, I don’t know, getting in a yacht and crossing the Pacific or something. The people who came to New Zealand in the first place were crazy.