Microsoft’s Game Stack chief: The next generation of games and game development

AI’s future in gaming

Journalist: The thing I find really interesting is people using AI to remaster games. I think that will be something I see a lot more in 10 years. In 10 years I fully expect that we’ll have AI editing our videos and editing our podcasts. It’ll completely change a lot of how we consume media. It’ll make things easier, but also take away a lot of the specialization. I think you’ll see that in game development as well. AI will be a big part of that. That will play into a lot of what we’re talking about around forwards compatibility.

Journalist: It’s interesting, because people are already doing stuff that’s kind of like that video editing example. There are services that connect to game APIs and Twitch APIs to know you were streaming at this time. You got kills at these times, so here’s your highlight reel for the night of all the people you killed in Fortnite. Which is just bananas. Following you around and making your own personal edit reel.

Journalist: We’ve known for years that game development is becoming more and more procedural, which is wonderful in a lot of ways, because it’s so much more — you have small teams creating incredible worlds. But also, at some point, you’re probably not going to need as many landscape designers. Like No Man’s Sky. There will be new roles, new AI wranglers.

Gwertzman: You were saying that the biggest challenge is acquiring new players. When a three-person team, through procedural generation, can make a game that seems like it’s epic scale, that makes it even harder now. Even more games are coming out that reach that bar.

The power of the cloud

Journalist: One thing that’s interesting to me, and the web people — game people have been doing this for a long time, ahead of traditional apps and web. The decreasing cost of the cloud — you guys talked about that a lot in the last generation at Microsoft. It kind of flopped spectacularly, because I think people didn’t know what to do with it. But I still think that there’s a massive potential for people who are building games to apply, essentially, free compute. Maybe it’s not to — what was the example, the Crackdown building deformation? But to allow people to spin up small instances like you do with Minecraft now. I can have a private world where we can all hop in together and do Minecraft stuff.

Gwertzman: Minecraft Realms does that. Minecraft Realms is actually powered by Playfab. My team helps support that. I love that we can now essentially rent servers to consumers as part of a built-in business model.

Journalist: In-app is the key. You don’t have to go to some weird webpage.

Gwertzman: Right. You just turn it on and boom, you have your own server. I actually wonder about that. One of the things companies like Improbable are talking about — games can have a million players concurrently in the same world. I don’t even know if people want that. I kind of think the 100-player limit in Fortnite feels like it fits a sort of — it’s a magic number. That’s the right number of people to engage with. 10,000 people on a server would actually not be manageable.

Journalist: I often feel like it’s just another one of those things where we have the tools, but we don’t know what to do with them yet. So many people are trying to — I don’t know if you remember Shinra. But trying to reach out and touch the majesty of distributed computing — we can do a 10,000-player game, why not? But that isn’t actually really that good. What kind of game could you make with 10,000 clients? That’s what I’m excited to see in 10 years.

Gwertzman: That’s a question we really need to tackle. Who will tap into — one of the technologies my team is looking at now is serverless multiplayer games. Why do you have to build a server, a game that’s hosted on a physical server, and we have to spin that up? Why not give us the logic and we’ll spin up whatever you need to support that?

Journalist: Because home routers are terrible.

Gwertzman: Yeah, yeah. But we can move a lot of logic — if there’s a demand for that, we can do that. We can say, just give us logic and we’ll magically power it. But then that opens the question of what we do with that. What kind of game experiences will that power?

The cloud is interesting. I first discovered the power of the cloud, ironically, at Playfab, where my engineers were building servers that — we had zero downtime updates. Instead of the old-fashioned model where at midnight on Sunday you take everything down to update, now we would just spin up completely separate instances of everything, slowly move the traffic over, and then take down the old version. It made it easy to spin up things and spin them down. That, to me, was the real power of the cloud.

Now we’re starting to look at things like asset management. If data and assets are the hard, expensive thing, do you move the editing tools to the cloud and do your editing in the cloud, where the data is, versus doing it in a local machine and bringing the data down to the local machine? Especially with massive data sets. The data analysis, obviously. PlayFab today is handling hundreds of thousands of events per second for some games. That all goes in a giant data lake somewhere. Maybe the designers are analyzing it to figure out player behavior and trends and so on.

People are still taking the baby steps as far as what to do with these technologies. That’s one of my big open questions. What can we do to help? Where are the pain points and where do we see the opportunities to add value and help developers, versus simply making them feel even more stressed out by even more things they have to figure out?

Fighting toxicity in games

Gwertzman: We were talking earlier about player data. What are the threats to gaming? The New York Times ran the big article a couple weeks ago about toxicity in games, about grooming and people being harassed. Harassment in games has been acknowledged, but unsolved for a while. Now we’re adding trafficking and other problems on top of it. How do we feel like the game community — are we rising to the challenge? Is it getting worse? Do we feel like that’s an existential threat, or is it just accepted as something that goes along — that anything on the internet has these kinds of challenges?

Journalist: From the game devs I talk to, it really depends on who’s running the forums and who’s designing the game. I’m endlessly surprised by how well teams who practice good community management and really proactive anti-toxicity design can make a big difference in their games.

This is probably a bad example, but I really loved Kind Words last year. The Popcannibal game. That’s a game that has very subtle anti-toxicity systems in the backend that are very hard to get to. They trust the players to not be terrible. So far it seems to be working. That’s a really beautiful — it gives me hope for, in the future, devs being able to build that and continue to cultivate audiences that are not jerks.

Journalist: On the other hand, every major online game has a way to turn off the other player names. You don’t have to see the creative ways that people get past whatever filters might exist.

Journalist: It seems to me that the problems with toxicity in video game online communities are part and parcel of toxicity issues on social media.

Journalist: It’s the internet, right? It doesn’t have to be, but it’s the way the internet has evolved.

Journalist: There was an interesting — it was a thread about an open source project that just said, look, our community has gotten really toxic. We either have to live with having these horrible people representing us to people who come in with problems, or we have to burn it all down and start over. They did that and immediately got five times the engagement they had before. They burned it down and started over, and it dramatically improved the experience. They started over with the rules they should have set. It’s not unfixable.

Gwertzman: Xbox Live, we spend a lot of money policing Xbox Live. We see that as a competitive advantage, so Microsoft puts a lot of resources into it. Entire teams of people moderating and managing it. One of the questions now is, is that one of the services we should be offering? Should we offer moderation or anti-toxicity services that we use internally for Xbox Live and turn that into one of the technologies we expose to the world? They go beyond just profanity filters. It can go into things like Community Sift, a third-party company that we partner with that helps provide some of the technology.

Journalist: Is that some kind of moderation service?

Gwertzman: Community Sift is one of the leaders. They’re a Canadian company that grew out of Club Penguin. They started initially doing advanced bad language filters. Now they’ve gotten pretty sophisticated. They can look for grooming behavior and all sorts of things. They don’t say yes or no. It’s not binary. They score things. As a game developer you can take those scores and use them to create player profiles. Okay, this player is probably someone I should think about banning or segregating or otherwise doing something about.

Journalist: They can put people in buckets. This person uses bad language, but not ethnic slurs.

Gwertzman: Right. As Playfab, we think a lot about that. Do we make it easy to put all your trolls playing with each other? Versus your younger kids who are under 13. Don’t let them play with the trolls. You can do that kind of thing.

The future of games at Microsoft

Journalist: If you want to talk about the larger point of where games are going, I’m going to rewind a bit. I think the question I have for Microsoft is, what is Xbox? There’s been a lot of rumors over time. Everything from, Microsoft is going to get out of games entirely, to Microsoft is going to compete directly with Sony, to Microsoft wants to be a live services provider. My personal view is that Microsoft is trying to get back to its roots, in a way. It’s always been an enterprise software service provider. Creating something like Game Pass is getting back to that. I’m curious what your take is, because I think Microsoft’s main objective will say a lot about how games change in the next two years.

Gwertzman: I can only answer from my perspective, which is, Microsoft is so big that there are a lot of businesses that are equally valid. Gaming and the Xbox is one of Microsoft’s last great consumer businesses. A lot of people are working on Scarlett or things like xCloud who feel almost a sacred responsibility to maintain this consumer business that’s one of the last places we’re doing that.

It’s frankly a very healthy business. We’ve figured out how to make it profitable. We’ve acquired a lot of studios. We’re in gaming for the long term. But independently, where I set is more to your point about Microsoft’s roots. Microsoft started, it’s important to remember this, started out as a language company before we were an operating system company. Microsoft BASIC was our first big product. We licensed it to Apple and Commodore. We were a developer tools company before we were ever a consumer operating system company, and long before we were an office productivity company.

Going back to those developer roots, that’s where I set. How do we help developers? How do we help developers figure out the cloud? How do we help them figure out acquisition and performance marketing? Microsoft has evolved into all these businesses now. I look at things like Dynamics. Dynamics is Microsoft’s competitor to Salesforce and SAP. How do we partner with our Dynamics team internally so that if you’re operating your company with our enterprise ERP package, your games can talk to it seamlessly and you have better correlation? There are interesting opportunities there.

One of my questions, let me open this to the whole table. One of the things Microsoft can do, we’re such a big company. We can provide the leadership in areas where it’s needed. So where is leadership needed? Where can Microsoft help the industry by either taking a stance or creating a standard or open sourcing key technology? What are some examples of where that would be helpful to the industry?

We have this thing called Game Stack. Game Stack is our umbrella marketing term for all of Microsoft’s game technologies. It’s more marketing right now than product reality, but a lot of people are working hard to make it a reality. How do we provide a bunch of technologies that work together seamlessly to help operate your game?

Acquisition is not — we don’t really have a thing in the acquisition space. We don’t have Adjust or Appsflyer or one of these acquisition tracking companies. We also don’t have Facebook or Twitter. We do have LinkedIn. We do have Bing, for what it’s worth. Are there ways we can leverage those to help with acquisition?

Developing games for the next generation machines

Journalist: How hard is game development going to get for the next generation? For PlayStation 5 and Xbox Series X? The big problem in the past was when you had to switch to a new chip, like the Cell. It was a disaster. PlayStation 3 development was painful and slow. It took years and drove up costs. But since you’re on x86, it shouldn’t happen, right? A lot of those painful things go away because it’s just another faster PC. But what’s going to be hard? What’s the next bar that everybody is going to shoot for that’s going to give them a lot of pain, because they’re trying to shoot too high?

Gwertzman: You were talking about machine learning and content generation. I think that’s going to be interesting. One of the studios inside Microsoft has been experimenting with using ML models for asset generation. It’s working scarily well. To the point where we’re looking at shipping really low-res textures and having ML models uprez the textures in real time. You can’t tell the difference between the hand-authored high-res texture and the machine-scaled-up low-res texture, to the point that you may as well ship the low-res texture and let the machine do it.

Journalist: Can you do that on the hardware without install time?

Gwertzman: Not even install time. Run time.

Journalist: To clarify, you’re talking about real time, moving around the 3D space, level of detail style?

Gwertzman: Like literally not having to ship massive 2K by 2K textures. You can ship tiny textures.

Journalist: Are you saying they’re generated on the fly as you move around the scene, or they’re generated ahead of time?

Gwertzman: The textures are being uprezzed in real time.

Journalist: So you can fit on one blu-ray.

Gwertzman: The download is way smaller, but there’s no appreciable difference in game quality. Think of it more like a magical compression technology. That’s really magical. It takes a huge R&D budget. I look at things like that and say — either this is the next hard thing to compete on, hiring data scientists for a game studio, or it’s a product opportunity. We could be providing technologies like this to everyone to level the playing field again.

Journalist: Where does the source data set for that come from? Do you take every texture from every game that ships under Microsoft Game Studios?

Gwertzman: In this case, it only works by training the models on very specific sets. One genre of game. There’s no universal texture map. That would be kind of magical. It’s more like, if you train it on specific textures and then you — it works with those, but it wouldn’t work with a whole different set.

Journalist: So you still need an artist to create the original set.

Journalist: Are there any legal considerations around what you feed into the model?

Gwertzman: It’s especially good for photorealism, because that adds tons of data. It may not work so well for a fantasy art style. But my point is that I think the fact that that’s a technology now — game development has always been hard in terms of the sheer number of disciplines you have to master. Art, physics, geography, UI, psychology, operant conditioning. All these things we have to master. Then we add backend services and latency and multiplayer, and that’s hard enough. Then we added microtransactions and economy management and running your own retail store inside your game. Now we’re adding data science and machine learning. The barrier seems to be getting higher and higher.

That’s where I come in. At heart, Microsoft is a productivity company. Our employee badge says on the back, the company mission is to help people achieve more. How do we help developers achieve more? That’s what we’re trying to figure out.

Making next generation art

Journalist: You touched on this, but are you projecting that studios are going to increase art staffs to meet the demands of the new consoles?

Gwertzman: I don’t know. I know that every previous generation of consoles, everyone bemoans the fact that art staffs have to go up and the economics just don’t support that anymore. I don’t know that I’m hearing a lot of complaints about that now.

Journalist: A lot of people have been generating stuff for 4K on PC for a while now.

Gwertzman: I’m not hearing a large — are you guys hearing a hue and cry about concerns of producing next-gen games? I’m not.

Journalist: Maybe if you’re CD Projekt.

Journalist: Ubisoft explicitly said….

Journalist: Yeah, but they have 50 teams around the world.

Takahashi: You have Rockstar.

Gwertzman: The new Xbox console, if you look at where the investments went for our new console — we’ve talked publicly about this. Nothing I’m sharing here is private. But the actual graphics performance is not that crazy an increase. It’s 4X or something. Its hard drive speed, and the fact that we have an SSD now, means that data access is 10X faster. The biggest differences are things like data access, and maybe network latency. It’s not about the raw graphics pipeline anymore. I do think the emphasis is shifting a bit. Raytracing is another thing now. That’s more about rendering path. It’s not necessarily about art production.

Journalist: It accelerates the way devs are already going into live games — the default is live games now. Or I shouldn’t say that. For a lot of devs, it’s whatever their dream game happens to be, But for a lot of teams, the way the business is going is live games. Once you have a broad customer base that can pick something whenever they feel like it, grab it and try it, then the new updates, new whatever, become so much more important. You don’t have to worry about having the game installed. Oh, yeah, I love that game, I’ll check it out again. It just steamrolls that whole process.

Gwertzman: That’s something I think about a lot, which is player engagement. The magic word these days is we’re entering the experiential economy. Every business I talk to — Playfab often gets pulled into conversations with hotel chains or travel companies, where they’re looking at how to apply lessons from live games to the retail economy or something like that. It’s not just gamification, which has sort of had its day. This is more about experiences from these always-on live games.

There’s this interesting idea that, if you have — a live game is essentially about player engagement. How do you continually engage your players in a very active way? That’s no different than the “digital feedback loop” you hear about in the enterprise space from Microsoft. How do we gather data, analyze it, and change your experience on the fly based on what you’re doing? If every game is competing with Disney+ or Netflix or Peloton or something, if we’re all competing for people’s attention, you have to connect that feedback loop tighter and tighter.