One of the main advantages of Microsoft’s gaming console like Xbox One is the Xbox Live gaming service. Millions of players use Xbox LIVE service to enjoy their gaming everyday and it is the largest gaming service in the world. Xbox Live Operations Centre is the team at Microsoft which handles the day-to-day operations of Xbox LIVE service to keep it online all the time. Located in a windowless room in Microsoft’s Redmond campus, it is manned 24 hours a day, 365 days a year.
And looming ominously above it all is a massive red digital timer. “Any time there’s a major incident going on that’s either user-impacting or we think it might be user-impacting, we start that clock,” warns Microsoft Gaming Ninja (seriously, that’s actually his title) Eric Neustadter. “We use that to make sure we’re doing the right things at each point in our process. We have playbooks for every kind of major incident, which tell us what to do at each point in the process. There’s actually someone in the room during an incident whose job is incident commander. They don’t troubleshoot. Their job is to watch the clock and to follow the process and make sure we don’t skip anything.”
As if on cue, the clock suddenly starts ticking. Neustadter laughs; apparently this isn’t a rare occurrence. “What started that clock could have been one of our monitoring systems alerting – the equivalent of a big red light,” he explains. “We’ve got alerts on essentially anything a customer could do where it’s going to tell us if it runs into an error. So when a console tries to do something and fails, it lets us know instantly.”
Read about it in detail here.