QuebraRegra wrote: »

Then I guess the lesson here is to pay for and ensure diversely routed/provided carrier services? What about server virtualization at a remote or offsite?

Overall, considering that we're talking about a videogame here, I guess the service metrics are acceptable. Wouldn't fly on my network.

nobertpaulb16_ESO wrote: »

This may not be totally over yet; do be surprised if the servers go down again sometime in the near future.

The system is still weak...according to...https://downdetector.com/status/level3/map/

Read the comments at the bottom....

ghostwise wrote: »

Yeah me and my buddies were experiencing heavy lag a short while ago. Very similar before it crashed last night.

Knaxia wrote: »

Everyone had major lag spikes in Cyro (and not the usual ones) including people getting disconnected earlier, I didn't get anything but even our newly crowned emp had issues doing anything.

cjthibs wrote: »

I am pretty sure the total unplanned downtime experienced does not exceed 3.65 days in the past year, but even if it were 3.65 entire days would still put us at 99% uptime. Which is respectable for an operation this large.

stevenbennett_ESO wrote: »

Oh, and for all those posting their confusion about how server X could be affected, etc… In this day an age, network topology is extremely complex even when you're talking about small systems. Databases often run on separate instances in the cloud than the servers which need to access them, and that communication happens through the internet. When you start getting into multiple interacting servers, the communications between each server happens through the internet. Often just the communications channels are complex, with server group A communicating to a bridge which is tunneling through the internet to server group B, C, D, etc. And that happens even with fairly simple websites and the like. There's a LOT of communication across the net which needs to happen or things stop working. And often, if those communications fail for more than just a few seconds, it causes program crashes or abnormal terminations, which then cascades to other failures.

It's often possible (and common) to have redundant backup communications, but when the problem happens in a major internet backbone, there's a decent chance your backups go through the same backbone -- it's out of your control if that happens. Usually the backbones have some redundancy, but the redundant circuits can get overloaded and drop a lot of traffic -- something which has low traffic needs might work fine with a bit of a slowdown, and if you've got a sufficiently distributed system, you might still work because your clients can go to alternate data servers, but when you get into MMOs, that's another animal.

MMO network topology, in particular, is a massive spiderweb of cross communications, some between local servers, some across the net to remote servers, some to databases which may be local or remote, some to admin / login servers which may be local / remote, with status monitoring, and synchronization issues which make normal network administrators shy away in terror at the complexity of it all. And unlike, say, a streaming server, or your average website, ALL of that communication needs to run at a fairly consistent high speed or the MMO fails.

Fortunately, an MMO is a game, not something critical. People will survive an 8 hour outage. OTOH, I'm sure there are a lot of companies who are dealing with some very serious issues this morning due to the outage last night.