With no preamble and no warning, Respawn Entertainment launched one of the most popular games of all time: Apex Legends.
Let’s not underplay this. Scaling from zero to 10-million players in 72 hours, and then to 50-million unique players in four weeks, is wild. For context; Fortnite, widely regarded as the biggest title in the world right now, took 16 weeks to hit 45-million.
In this blog, we’ll explore the game server tech and all of the hard work that made it happen, before looking at where we go next.
The Apex of scaling technology
While at Multiplay, we don’t mind boasting (we’ve been doing awesome stuff for 20 years now, folks), there’s no denying that what Apex Legends achieved has only become possible in recent years due to innovations in cloud technology.
The main reason? Capacity planning. Five years ago, you’d have had to predict and plan in your server needs months in advance, and there’s no way Respawn or Multiplay could have foreseen that Apex Legends would explode in popularity in quite the fashion it did.
Thankfully, when you combine the flexibility of multiple cloud providers with a vast network of highly performant bare metal data centers, capacity planning is less of a concern.
Multiplay’s purpose-built Hybrid Scaling platform, backed by its unique orchestration platform, allowed Apex Legends to not only scale with the demands of its players, but soar above the competition.
During the first few weeks, Multiplay was scaling:
- Well over 2 million concurrent players
- 6,500 Virtual Machines (VMs) on cloud alone
- Across 54 locations
Why do Respawn use Multiplay? The answer is simple: They wanted their game launch to be boring. Not for the player of course but operationally; Respawn didn’t want to worry about players struggling to find a server to play on. For that, they needed a specialist.
Lead Programmer for Respawn, Jon Shiring, said: “For the launch of Apex Legends, it was critical both that we could scale our servers up to handle whatever size audience showed up, and that those servers were reliable.”
He added: “We continued our partnership with Multiplay to leverage their multi-cloud auto-scaler, so we can focus our developer time on the game itself and not on keeping the game servers running.”
With support from Multiplay, Respawn were free to focus on making Apex Legends the incredible game it is today, and didn’t have the stress of worrying about servers!
One less worry during crunch time is always helpful. Utilizing Multiplay also helps with budgets and headcount. With the Multiplay team on hand, Respawn didn’t have to hire a team of network engineers (with expertise in data center infrastructure and/or cloud), or spend the thousands of hours it’d take to get everything in place for a smooth launch.
Why was the Apex Legends launch such a success?
There are three key reasons the Apex Legends launch was so successful:
- Multiplay’s Hybrid Scaler and orchestration tools took away the risk of capacity planning by utilizing both bare metal data centers and the cloud.
- Planning and preparation, because even with the Hybrid Scaling platform, video game launches are a tangled web of unknown, unknowns.
- And, of course, Respawn’s awesome backend infrastructure, working in harmony with Multiplay’s tech.
Let’s look at each in detail.
Multiplay’s Hybrid Scaler and orchestration tools
Multiplay’s Hybrid Scaling is in a league of its own when it comes to hosting a game with unknown launch day numbers (which is almost all of them). With Apex Legends, capacity scaled with player demand, bursting through enterprise-grade bare metal into cloud when needed.
It offers the best of both worlds:
- Multi-cloud, for peak times. The main reason we love cloud is that it’s flexible; it’s your safety net. With our tools we can spin up cloud servers at incredible speed, allowing you to stay ahead of that insatiable player demand and, importantly, scale back down just as quickly! Indeed, at peak, we were spinning up 3000 vCPU a minute on Google Cloud for Apex Legends. In addition to Google Cloud, Multiplay’s hybrid system has been designed to take advantage of AWS EC2 as well, so capacity and coverage will never be an issue.
- Managed infrastructure, utilizing Multiplay’s network of 150 data centers around the world. These machines are performant, yet cost-effective. We can take the challenge out of capacity planning by starting small and growing alongside your game, with hardware dedicated to you (no multi-tenancy here!).
As Multiplay’s Larry Rye said during his talk at GDC on Hybrid Scaling: We basically created a super cloud.
Hybrid Scaling spreads the risk of bare metal deployments across multiple data centers, even within the same geographical region. Couple that with a multi-cloud implementation, you have the most scalable, the most resilient and the most cost-effective hosting platform ever designed. All you have to do to use it is make a couple of API calls.
The secret sauce that controls the Hybrid Scaling? We call it our orchestration toolkit. This handles everything from server allocation, to patching, to monitoring. You can learn more about this here.
Planning and preparation
Multiplay’s deployment teams worked tirelessly to ensure every eventuality was handled, keeping Respawn informed at every step of the way, pre and post-launch.
We won’t go into the planning side too much because you can read about it in our previous blog but the TLDR is:
- Respawn knew the game was going to be popular but they didn’t know how popular, so we ran a scaling test at the end of 2018.
- Multiplay’s Hybrid Scaler requested around 12,000 VMs, enough capacity for 14-million concurrent players, from Google Cloud, in under 90 minutes.
- The test was critical for us to not only push our platform further than we had done before, but to give peace of mind to Respawn. We test everything end-to-end, to provide a flawless launch experience.
This level of preparation is essential and gave us (and Respawn!) the confidence that every element of the complex platform which supports Apex Legends was battle-ready. When it comes to mic drop launches like Apex Legends, you have to be ready for anything, especially when your game is free to play!
Respawn and Multiplay, working in harmony
We’ve talked a lot in this piece about what Multiplay does to ensure games like Apex Legends can scale up and down freely, but that is of course only part of the picture.
We work closely with the team at Respawn, headed up by the venerable Jon Shiring, to ensure our systems are compatible. This is true of any of Multiplay’s customers – the solution has to be, to a degree, bespoke.
A simple overview of Respawn’s backend infrastructure, taken from the Multiplay/Respawn talk at GDC 19.
The perfect match
So how does the integration outlined above lead to the perfect gaming experience? Put simply, the answer lies in the synergy between Respawn’s backend requesting servers and Multiplay delivering them.
Respawn’s incredible team developed a fantastic matchmaker called Stryder. It’s responsible for receiving players, grouping them, requesting servers from us and giving them back when it’s done with them.
Multiplay’s primary role is getting players into matches – The physical machines needed to connect customers to bare metal and cloud. On top of this we run additional services, such as those needed to help connect players to the right geographical location and voice servers.
We worked closely with Respawn to ensure the varying templates for servers are setup and ready for use. These servers can then be requested via our API. This includes Match servers for the gameplay itself, Party servers for grouping players together and Training servers to show players the ropes.
Outside of the hardware itself, Multiplay’s world-class support team monitors the game 24/7/365. Anything that causes even the slightest degradation in service is investigated and we keep in constant contact with Respawn to stay on top of the latest issues.
Using this information we make sure that we’re always one step ahead and there’s always somewhere for you to play.
Multiplay doesn’t simply focus on hosting, but the whole solution, helping to find potential bottlenecks when and if your game scales.
What next for Apex Legends?
With Apex Legends, Respawn Entertainment are taking a different approach to previous titles, and most other games. The free-to-play model is only financially viable if gamers are committed to playing the game longterm, buying aesthetic content, maps, loot boxes, etc.
As per the image above, it is clear that Respawn (as you’d expect) have nailed their roadmap, but what does this mean from a hosting perspective?
While DLC launches aren’t as unpredictable as new game launches, they’re still a challenge. All titles experience a drop in player numbers at some point and can counter that with new content, but predicting how successful that new content will be in attracting old players back or new players, is difficult.
Thankfully, as with launches, Multiplay’s Hybrid Scaling is the solution.
When new content is released and there’s a spike in player numbers, Multiplay’s Hybrid Scaler will kick in once again to spin up more cloud machines faster than Octane can sprint.
This allows the awesome team at Respawn more time to focus on what’s important: Building on the huge success of the Apex Legends launch and maintaining momentum.
Is your game the next Apex Legends?
Whether your game hits 2,000 or 2-million CCU, we can help. Our aim at Multiplay is to take the stress out of game server hosting – be that launch day, DLC-drop day or any other day of the week. Get in touch today and let’s start talking about your next project.
Further reading
- Mastering the Apex of Scaling Game Servers – Multiplay and Respawn at GDC (video).
- Read more about Apex Legends and download the game (web).
- Learn more about the scaling test Multiplay did for Apex (blog).