Why do maintenances take so long?

siddique

Estimated time to finish is about six hours and I have a feeling it would take even more. That's like almost a full day of work.

Siohwenoeht

In a very basic way, they have to clean up the servers like you would clean up your hdd or sdd on your home computer. It can take a while due to the amount of usage those servers get over the course of a week. They've been good recently about finishing before the estimated time, and this maintenance isn't adding a big patch so I'd expect it to be up before the six hour time frame.

siddique

Aah, I see. Well let's hope so, I dread playing other games. :P

Tasear

Adding new assets for the Russians to datamine

Ravereth

siddique wrote: »

Why do maintenances take so long?

XxMissAlycexX

Where are u getting an estimated time from?

Mariusghost84

Siohwenoeht wrote: »

In a very basic way, they have to clean up the servers like you would clean up your hdd or sdd on your home computer. It can take a while due to the amount of usage those servers get over the course of a week. They've been good recently about finishing before the estimated time, and this maintenance isn't adding a big patch so I'd expect it to be up before the six hour time frame.

What do you mean by cleaning up? What specifically needs cleaning? I would guess they are applying patches to the servers and restarting them, followed by a confirmation of uptime. I really dont think deleting any sort of files is part of the maitnance.

zyk

Mariusghost84 wrote: »

What do you mean by cleaning up? What specifically needs cleaning? I would guess they are applying patches to the servers and restarting them, followed by a confirmation of uptime. I really dont think deleting any sort of files is part of the maitnance.

ZOS has never explained exactly what happens, nor should they be expected to.

If I had to guess, I'd say ZOS probably updates the server-side code and systems far more often than we realize and each time that happens there is an extensive testing regiment that follows. There's probably a lot of database maintenance as well.

TempPlayer

It is their way of saying you had been playing ESO for too long. They are giving you the chance to clear up you Steam/PS4/Xbox backlog.
I did just that during the last maintaince and now I'm speeding less time in ESO and more time on other game cause I want to clear it. Guess that will reduce the server load so it is kind of help keeping the server healthy, hence maintaince.

Siohwenoeht

Mariusghost84 wrote: »

Siohwenoeht wrote: »

In a very basic way, they have to clean up the servers like you would clean up your hdd or sdd on your home computer. It can take a while due to the amount of usage those servers get over the course of a week. They've been good recently about finishing before the estimated time, and this maintenance isn't adding a big patch so I'd expect it to be up before the six hour time frame.

What do you mean by cleaning up? What specifically needs cleaning? I would guess they are applying patches to the servers and restarting them, followed by a confirmation of uptime. I really dont think deleting any sort of files is part of the maitnance.

On some level they are most certainly performing basic maintenance tasks that any computer system needs to run as well as it can. I'm sure after a week of running those servers have a helluva lot of cached data that it no longer needed. Afaik, there was not a large patch announced for this week and zos has been reluctant to hotfix, preferring to add those types of fixes in the larger updates.

siddique

XxMissAlycexX wrote: »

Where are u getting an estimated time from?

At the top of the forum, where there is the announcement of maintenance, it says 4 a.m. to 10 a.m.

zaria

siddique wrote: »

XxMissAlycexX wrote: »

Where are u getting an estimated time from?

At the top of the forum, where there is the announcement of maintenance, it says 4 a.m. to 10 a.m.

Yes and giving estimated time is pretty new.
Pretty sure its an conservative estimate since if they said 3 hours but used 4 people would complain all over but if they say 6 and uses 5 people are happy.

Elsonso

They have to cleanly shut down all the databases and server applications. Then, I would imagine that they perform a backup of critical data. If they have any hardware or data center infrastructure that they want to add, swap out, or repair this gets done, and new hardware is configured, provisioned, and made ready to be used. Then they update the backend server code, mainly the operating system and backend support software. If they have a game patch to install, they install the patch. They start up all of the server programs and wait for them to get all warm and comfy. If QA has anything to do, QA will step in and do whatever testing is necessary before they open the doors. Then, they allow the players in.

Cously

They probably need 2 hours but have to be on the safe side and say 6, else if goes beyond 2 hours people will start to whine.

AlienSlof

I'm fine with it. I'd much rather they maintain the servers regularly than leave them with possible problems. Loading screens anyone?

I just find something else to do - housework for me today. 3 Ragdoll cats means a LOT of fluff on my carpets to sweep up!

Onigar

I have worked in the Software Industry most of my life.

One of the main factors in deciding how to progress a project though subsequent releases is the anticipated expectations of customers.

Integrating incremental changes into a released project where the customers are very demanding is very hard to manage in a way to achieve the company goals as well as keeping the customer base sated.

For instance in Banking software there are generally no more than 3 releases of a new production build within a year and believe me as I have worked on Banking software that I say this game has software comparable in complexity to a large bank but they are able to release new updates on a weekly or bi-weekly time frame with no or little recovery action needed.

This says a lot for the professionalism and dedication of the software engineering team as well as the management.

The forums have asked for so long for an update time frame and now they are giving one but instead of being thankful so you can get on with life, or find a life if that is the case, for some there is moaning about the time frame.

There is no way a software supplier can win these days with a mass market of customers as some will always use their voice to demand "I Want It Now". No don't get me wrong, I support this but please look at what you have already been given and let your demands have a productive element too

"Parent starts to wrap a present, Child walks in,
Dad, whats that,
Oh Umm, its for your Birthday,
Oh great, can I have it now,
No, you must wait,
But, but, my birthday is next week and I've seen it already,
What did you see?
The silver blue box, its the Iceman right?
Hmm, I guess
Thanks, Dad, you know I wanted that, your the best,
I still need to wrap it
That's OK, I understand, but it is the Blue Iceman right?
What do you mean?
Well the White one is like see through and looks like a skeleton
Oh, I'm sure its Blue ("Man, why didn't I know there was a Blue and a White one")"

Even the best plans fall foul of incomplete knowledge and assumptions.

Do not criticise the giver for giving when you have not clearly and unambiguously stated what you want

Alcast

phairdon

Now 12.40am in this part of the world. Maintenance in my time zone is always late in the evening through to the early hours of the morning. Work evenings, so its no big deal for me.

siddique

Onigar wrote: »

I have worked in the Software Industry most of my life.

One of the main factors in deciding how to progress a project though subsequent releases is the anticipated expectations of customers.

Integrating incremental changes into a released project where the customers are very demanding is very hard to manage in a way to achieve the company goals as well as keeping the customer base sated.

For instance in Banking software there are generally no more than 3 releases of a new production build within a year and believe me as I have worked on Banking software that I say this game has software comparable in complexity to a large bank but they are able to release new updates on a weekly or bi-weekly time frame with no or little recovery action needed.

This says a lot for the professionalism and dedication of the software engineering team as well as the management.

The forums have asked for so long for an update time frame and now they are giving one but instead of being thankful so you can get on with life, or find a life if that is the case, for some there is moaning about the time frame.

There is no way a software supplier can win these days with a mass market of customers as some will always use their voice to demand "I Want It Now". No don't get me wrong, I support this but please look at what you have already been given and let your demands have a productive element too

"Parent starts to wrap a present, Child walks in,
Dad, whats that,
Oh Umm, its for your Birthday,
Oh great, can I have it now,
No, you must wait,
But, but, my birthday is next week and I've seen it already,
What did you see?
The silver blue box, its the Iceman right?
Hmm, I guess
Thanks, Dad, you know I wanted that, your the best,
I still need to wrap it
That's OK, I understand, but it is the Blue Iceman right?
What do you mean?
Well the White one is like see through and looks like a skeleton
Oh, I'm sure its Blue ("Man, why didn't I know there was a Blue and a White one")"

Even the best plans fall foul of incomplete knowledge and assumptions.

Do not criticise the giver for giving when you have not clearly and unambiguously stated what you want

I think you misunderstood me. I just wanted to know what is it that takes so long. I love this game and have been playing since 2014.

I also appreciate all the hard work and timely maintenances. So yeah.

ccfeeling

Only ESO maintenance that much and long in my Mmorpg life.

One more thing, this is 2019.

They really need a better team.

Androconium

There is no technical reasin for any data centre to be offline for six hours.

Fault tolerant hardware has been available since the early 1990s. Shadowed/mirrored partitioning has been around for the same amount of time, i.e. thirty years.

This is risk management.

They don't see consumer backlash as a risk that will have catastrophic consequences.

Keep subscribing and maintaining the status quo.

method__01

i understand the maintenance,even the many hours
what i dont understand is why they brought down ps4,xbox amd pc the same time
on top,Fallout servers are down too!!

guess i spend my day over F5 key

Chaos2088

They can take as long as they need, clean up those severs....-imagines hamsters with little brooms-

Cloudless

https://www.youtube.com/watch?v=xK37Dbxr1OA

Androconium

Onigar wrote: »

I have worked in the Software Industry most of my life.

One of the main factors in deciding how to progress a project though subsequent releases is the anticipated expectations of customers.

Integrating incremental changes into a released project where the customers are very demanding is very hard to manage in a way to achieve the company goals as well as keeping the customer base sated.

For instance in Banking software there are generally no more than 3 releases of a new production build within a year and believe me as I have worked on Banking software that I say this game has software comparable in complexity to a large bank but they are able to release new updates on a weekly or bi-weekly time frame with no or little recovery action needed.

This says a lot for the professionalism and dedication of the software engineering team as well as the management.

The forums have asked for so long for an update time frame and now they are giving one but instead of being thankful so you can get on with life, or find a life if that is the case, for some there is moaning about the time frame.

There is no way a software supplier can win these days with a mass market of customers as some will always use their voice to demand "I Want It Now". No don't get me wrong, I support this but please look at what you have already been given and let your demands have a productive element too

"Parent starts to wrap a present, Child walks in,
Dad, whats that,
Oh Umm, its for your Birthday,
Oh great, can I have it now,
No, you must wait,
But, but, my birthday is next week and I've seen it already,
What did you see?
The silver blue box, its the Iceman right?
Hmm, I guess
Thanks, Dad, you know I wanted that, your the best,
I still need to wrap it
That's OK, I understand, but it is the Blue Iceman right?
What do you mean?
Well the White one is like see through and looks like a skeleton
Oh, I'm sure its Blue ("Man, why didn't I know there was a Blue and a White one")"

Even the best plans fall foul of incomplete knowledge and assumptions.

Do not criticise the giver for giving when you have not clearly and unambiguously stated what you want

You are living in a waterfall.
They use agile development.

Too bad that, like banks, this company sees it's customers as the owners; and the end users as fiscal resources.

Elsonso

Androconium wrote: »

There is no technical reasin for any data centre to be offline for six hours.

Fault tolerant hardware has been available since the early 1990s. Shadowed/mirrored partitioning has been around for the same amount of time, i.e. thirty years.

This is risk management.

They don't see consumer backlash as a risk that will have catastrophic consequences.

Keep subscribing and maintaining the status quo.

Actually, this is a game, and fault tolerant, high availability, redundant systems that can remain up while being patched are more expensive. Yes, it can be done, but does it really need to be done? If ESO was a banking application, or an airplane flight control system, I would see the justification. This is just a game.

Zorgon_The_Revenged

Takes time to get things synchronised....

Bam_Bam

AlienSlof wrote: »

I'm fine with it. I'd much rather they maintain the servers regularly than leave them with possible problems. Loading screens anyone?

I get what you're saying but....

...do you or anyone else feel like the maintenances ever resolve ANY existing problems?

They have problems that haven't been fixed in almost FIVE years...

Androconium

lordrichter wrote: »

Androconium wrote: »

There is no technical reasin for any data centre to be offline for six hours.

Fault tolerant hardware has been available since the early 1990s. Shadowed/mirrored partitioning has been around for the same amount of time, i.e. thirty years.

This is risk management.

They don't see consumer backlash as a risk that will have catastrophic consequences.

Keep subscribing and maintaining the status quo.

Actually, this is a game, and fault tolerant, high availability, redundant systems that can remain up while being patched are more expensive. Yes, it can be done, but does it really need to be done? If ESO was a banking application, or an airplane flight control system, I would see the justification. This is just a game.

No technical reason.

Onigar

siddique wrote: »

Onigar wrote: »

I have worked in the Software Industry most of my life.

One of the main factors in deciding how to progress a project though subsequent releases is the anticipated expectations of customers.

Integrating incremental changes into a released project where the customers are very demanding is very hard to manage in a way to achieve the company goals as well as keeping the customer base sated.

For instance in Banking software there are generally no more than 3 releases of a new production build within a year and believe me as I have worked on Banking software that I say this game has software comparable in complexity to a large bank but they are able to release new updates on a weekly or bi-weekly time frame with no or little recovery action needed.

This says a lot for the professionalism and dedication of the software engineering team as well as the management.

The forums have asked for so long for an update time frame and now they are giving one but instead of being thankful so you can get on with life, or find a life if that is the case, for some there is moaning about the time frame.

There is no way a software supplier can win these days with a mass market of customers as some will always use their voice to demand "I Want It Now". No don't get me wrong, I support this but please look at what you have already been given and let your demands have a productive element too

"Parent starts to wrap a present, Child walks in,
Dad, whats that,
Oh Umm, its for your Birthday,
Oh great, can I have it now,
No, you must wait,
But, but, my birthday is next week and I've seen it already,
What did you see?
The silver blue box, its the Iceman right?
Hmm, I guess
Thanks, Dad, you know I wanted that, your the best,
I still need to wrap it
That's OK, I understand, but it is the Blue Iceman right?
What do you mean?
Well the White one is like see through and looks like a skeleton
Oh, I'm sure its Blue ("Man, why didn't I know there was a Blue and a White one")"

Even the best plans fall foul of incomplete knowledge and assumptions.

Do not criticise the giver for giving when you have not clearly and unambiguously stated what you want

I think you misunderstood me. I just wanted to know what is it that takes so long. I love this game and have been playing since 2014.

I also appreciate all the hard work and timely maintenances. So yeah.

While I do not work for ZOS I do not imagine the following development concept is far away from what they use,

For a production release,

Each developer would normally have an instantiation of the game that they can modify. When they are happy with their mods then these are merged into a Dev test version combining all the developers mods.

After successful testing of this development game version it is designated as a release candidate.

At the start of a maintenance period,

• The existing game image including databases are preserved in case of rollback
• the new changes are installed and verified
• The physical computing environment is generally different to the development test area are so basic computing functional and performance testing is done to be sure nothing has been compromised.
• A series of standard tests are run to establish the main game software functionality has not been affected. These can involve manual dexterity tests as well as automated tests.
• Now you have established the baseline for the game is unaffected by the integration of the new code it is time to run the tests for this new code. The tests will already be planned in advance but unexpected events are always a possibility and if they occur and are sufficiently understood then they can be corrected in real time.
• After testing is complete the software is released or if the update is withdrawn then a rollback is needed.

.
Hope this helps with your understanding

ps. I have also been playing for 4+ years now

RobZha

It could be faster, but they have a private game of pvp in cyro before maint is over and the lag extends the entire thing by 1 hour or so.