Why no rolling updates?

Funhammy

sarahthes wrote: »

show previous quotes

Funhammy wrote: »

It’s more because you references a cloud system with thousands of servers of which any one could go down and the cloud would at most have some performance issues but not go fully down.

As for the mega server spaghetti code they use for ESO; it’s just that, one server per server… no extra servers.

Which means when the server requires maintenance the entire server must go down to do it.

They don’t have a backend set up in order to allow hot fix maintenance because that requires modulating server load between multiple severs which they don’t have since they use a mega server.

Plus hotfixes are only so capable of fixing things, take RuneScape which hotfixes all the time, when they’ve had to do rollbacks they also have to turn all of their servers offline to do it.

You can’t modulate around the server load of major changes, one server would say that your account is in one state and another would say it’s in a different state and when they try to merge your save data who knows which it would pick (what happened with the PTS) so they have to turn servers off for anything beyond a small patch.

And as far as I’ve ever seen ESO doesn’t even have the capability to do a hotfix anyways.

They can, but I think the scope of what they can do is incredibly limited.

Yeah idk if they can or not but I don’t think I’ve ever seen a hotfix in this game.

Trejgon

Rolling updates to be possible, require a very specific infrastructure architecture to support it. Since ESO was designed from the beginning for "classic" server infrastructure (even megaservers were late development) it was not designed to be doing these.

As for AN somehow patenting trying to base your servers on distributed architecture for an MMO, I call bullcrap on that, that practice has been already stardard thing outside of gaming by the time AN started developpign either of GW games there is no way they could just patent that. It is just much easier to build your servers to not support that.

sarahthes

Funhammy wrote: »

show previous quotes

sarahthes wrote: »

Funhammy wrote: »

It’s more because you references a cloud system with thousands of servers of which any one could go down and the cloud would at most have some performance issues but not go fully down.

As for the mega server spaghetti code they use for ESO; it’s just that, one server per server… no extra servers.

Which means when the server requires maintenance the entire server must go down to do it.

They don’t have a backend set up in order to allow hot fix maintenance because that requires modulating server load between multiple severs which they don’t have since they use a mega server.

Plus hotfixes are only so capable of fixing things, take RuneScape which hotfixes all the time, when they’ve had to do rollbacks they also have to turn all of their servers offline to do it.

You can’t modulate around the server load of major changes, one server would say that your account is in one state and another would say it’s in a different state and when they try to merge your save data who knows which it would pick (what happened with the PTS) so they have to turn servers off for anything beyond a small patch.

And as far as I’ve ever seen ESO doesn’t even have the capability to do a hotfix anyways.

They can, but I think the scope of what they can do is incredibly limited.

Yeah idk if they can or not but I don’t think I’ve ever seen a hotfix in this game.

I've seen plenty, but they are for minor issues.

zaria

TKo_ROUSE wrote: »

show previous quotes

Nerouyn wrote: »

TKo_ROUSE wrote: »

Just for frame of reference: The last Guild Wars 2 downtime was August 13 of 2016.

As far as how they can patent something like that I do not know. I am not an expert in copy right law.

According to this article, they used Amazon Web Services.

https://www.guildwars2.com/en/news/inside-arenanet-live-game-outage-analysis/

Yes and several other MMO's use AWS and have down time. Remember AWS is just renting the server infrastructure of another company. The MMO's are an application that runs on top of some one else's infrastructure. So AWS has very little downtime typically but these companies still have to write their applications in a way that could allow updates to happen without taking the system offline.

ArenaNet accomplished this with Guild Wars 2. They probably spin up new servers with the update in AWS, transfer those clients between those servers and then take down the old servers. I probably am making that sound more simple than it actually is. I am sure it gets convoluted with things like the clients need to update, and how does a running instance of the application handle a version difference when they are handed off. I was under the impression that it was under a Patent but someone else pointed out I may be wrong. I really don't know for sure as I only know based on what others have told me. The only thing that might speak for patent is I would think other MMO's would have adopted these technologies, but maybe they are just cost prohibitive?

Also I did forget EU had a downtime that NA did not. There NA server has been up for over 8 years though.

Now say Cyrodil, BG, dungeon finder and trials was down during update it would still be much better than today.
I think GW 2 is less of an open world than ESO.

Danikat

zaria wrote: »

I think GW 2 is less of an open world than ESO.

It's the same.

Guild Wars 1 worked differently, basically all the maps outside towns were instanced, like dungeons in ESO. But in Guild Wars 2 it's the same as ESO, the world is split into different zones, but they're persistent and shared between players and groups. There can be multiple copies of the same map on the same server, but that's the case on ESO as well, it's why we never have to queue to enter a map even when an event is sending everyone to the same place.

moo_2021

DigiAngel wrote: »

You don't see Azure going down for maintenance....or Oracle or Google. Why ESO?

Their services do go down. Amazon doesn't even guarantee your virtual machines could be restarted.

Their supposed guarantee is for you to pay for redundant systems that can be switched to any time, not free of course. I doubt game companies would do that.

Sluggy

DigiAngel wrote: »

2 billion....I'll just leave that there

That's not even a drop in the bucket. Microsoft earns ten times more per year just from Azure alone (in profit btw, not total revenue).

There is absolutely no way you're going to justify ripping out the guts of the entire server architecture in a multi-year project that brings basically no additional income, has a high chance of failure, all just to avoid a couple extra hours of downtime every other week.

HatchetHaro

sarahthes wrote: »

show previous quotes

DigiAngel wrote: »

From the thread:

Now that all scheduled maintenance has concluded on the PC, PTS and console megaservers, we’d like to outline next steps and what to expect in the coming days:

• Tomorrow, April 25 beginning at 6am EDT, we’ll be taking the PC NA megaserver offline for maintenance. During this time, we’ll be restoring affected PC NA accounts back to their state from Monday, April 15 at 9:30am EDT.

You don't see Azure going down for maintenance....or Oracle or Google. Why ESO?

I mean... they only have the one mega server. What is it supposed to be rolling with?

They already have PCNA, PCEU, XboxNA, XboxEU, PSNA, PSEU, and PTS, not to mention (I'm guessing) at least two production servers for internal builds.

They've also done a beautiful demonstration of writing to one database from multiple servers at the same time.

All they need is one additional server in each region for the rolling updates, one for NA and one for EU. When maintenance rolls around, just put the new game build into Server B, and when players update their clients it will connect them to Server B. Give the players still on the previous game build an announcement to update their game, and once updated they will connect to Server B. Server A now becomes the new server to put the next build on, whether it be Xbox or PS's turn.