Downtime....inexcusable

Berenhir · October 2016

Well, the server downtimes exist because ZOS decided to implement a maintenance system that creates them. It's not a general law of nature as some believe here.

Probably a different system would just be more expensive. So as long as there are no significant numbers of people leaving the game because of maintenance periods...

human151 · October 2016

Qyrk wrote: »

human151 wrote: »

Bryanonymous wrote: »

This isn't a phone company.

Some businesses do actually close for one or two days a week.

This is not a car.

You are comparing products that have almost nothing in common, and now you have lost ALL credibility with your faulty logic.

When you bought ESO was there any information stating theyre closed 5 or more hours every week, one one particular day? Companies do tell you their business hours. Your faulty logic has made you lose all credibility.

I never could understand why some people feel compelled to defend corporations...

Umm look into their terms and conditions that YOU accepted. ZOS CAN have their downtime (I.e. lose of service) at no notification to you

this is your response ....lol

Taternater · October 2016

Qyrk wrote: »

human151 wrote: »

Bryanonymous wrote: »

This isn't a phone company.

Some businesses do actually close for one or two days a week.

This is not a car.

You are comparing products that have almost nothing in common, and now you have lost ALL credibility with your faulty logic.

When you bought ESO was there any information stating theyre closed 5 or more hours every week, one one particular day? Companies do tell you their business hours. Your faulty logic has made you lose all credibility.

I never could understand why some people feel compelled to defend corporations...

Umm look into their terms and conditions that YOU accepted. ZOS CAN have their downtime (I.e. lose of service) at no notification to you

Wait, you stopped and read the terms instead of just clicking accept?

human151 · October 2016

Bryanonymous wrote: »

ElBiggus wrote: »

Bryanonymous wrote: »

This isn't a phone company. Some businesses do actually close for one or two days a week. This is not a car. You are comparing products that have almost nothing in common, and now you have lost ALL credibility with your faulty logic.

No, this is a business whose primary, and indeed only, function is to run an online game, and the fact that they can't manage to do that for more than 95% of the time on a good week suggests that they're doing it wrong. I've already explained the incredibly simple process (which every server admin on the face of the planet learns pretty much on day one) that allows you to update a server-side service without requiring huge downtime, and still people like you insist that there's no other way of doing it. Even businesses whose online presence is merely a portion of their business will probably do it the no-downtime way, so for a company that relies entirely on online services to make such a mess of it is inexcusable and (in case I haven't said it enough times) EASILY AVOIDABLE.

You think you can do it better? Who's stopping you?

You feel entitled? Then sue.

You don't like it? Then leave.

Or deal with it like the rest of us. Seriously. You probably have no clue how their programs are setup on their servers, and why they need to do a weekly maintenance. For all we know, the servers need a weekly reboot because they suffer from the same memory leak as our personal games. And as Gena said before, rebooting the servers takes about an hour. They also do micro patches often after big updates. You think you know how to do that? Well, make your own game and do it, or apply for a position with them. Just for the love of everything, the entitlement is driving all of us crazy. You are not special. Get over yourself.

Patching and rebooting a server takes an hour? I doubt you even know what a server looks like.

Qyrk · October 2016

Bryanonymous, I rarely take these kind of (clueless) people seriously. If it makes them feel good about themselves on their high horse for a few minutes, I just sit and laugh

. It is the internet after all,

Asardes · October 2016

I've been playing this game for a year. There is usually maintenance for 4-6 hours every Monday morning/afternoon (EU), late night/early morning (NA). When a new patch comes out it takes 8-10 hours and the server is up by evening. I can live with that, just like most users, who are at work/school during the maintenance time. In every business you have to have partial or total downtime for regular maintenance and various fault fixes. Which is chosen at a time when the least amount of users is affected. Phone companies have that too, and it's scheduled late at night, just like ESO maintenance relative to NA.

The problem here is that now they have just one client, and probably a single maintenance client, so they can't do the EU maintenance 6-8 hours before NA. So they have to do them together, picking the time when the aggregate number of connected users is lowest on both servers. Most people play in the evening during weekdays. Had a day off at some point and decided to connect. It was about 10 a.m GMT and Mournhold was basically deserted and couldn't find anyone to do the pledges at that hour. It was just me and the NPCs from the guild kiosks. During prime time it's so full that you often get timeout DC when porting there because there are so many PCs there.

Universe · October 2016

human151 wrote: »

Lord wrote: »

I presume that ZOS employees are actually checking the bugs on Live after the implementation of the patch.
Only this step can take hours.

I would rather lose a few gameplay hours to have a more stable server with less bugs.
The average time of maintenance is about 6 hours(sometimes 7-8 hours or a bit more).
It's only 6 hours out of 168 hours in a week.
That's acceptable.

one might presume that an online gaming company might have a testing environment with a clone of the live server that they could use for testing before they roll out an update...unless you presume theyre amateurs.

They clearly stated that confirming a fix on the test server is different from confirming it on Live server due to different software and other limitations.
I can't find the quote for that but I remember they mentioned this.
They do have a private test server but they are not using it for all fix confirmations.

Mickey_Ox · October 2016

skywarnmc27 wrote: »

Mickey_Ox wrote: »

Get a new job with more sociable hours maybe??

Or perhaps next time you go to the store you should not be allowed to buy anything, since 90% of them are stocked by 3rd shift workers...Or Police will stop after 8pm, same with fire and EMS.

Not quiet sure why your attacking me, I often work nightshift myself but I'm not the person on here giving out how their choice to work nights is clashing with the server downtime. For whatever reason Z$S's servers require downtime every week, its always been that way, its that way in countless other MMO's. OP claims Z$S do it wrong because he works in data centers and he knows better but the fact is he does not know the ins & outs of Z$S's servers or coding so can't make such claims. So if the MANDATORY downtime is that much of an annoyance to him he should get a new job or quit crying about it

purple-magicb16_ESO · October 2016

If ZoS had the resources, they could maintain alternate live servers (NA-2/EU-2) with opposing downtimes, so players in these types of situations would have a more convenient option to move to.

Oh well, we can still dream...

P3ZZL3 · October 2016

These threads are fun to read tbh

So many untruths and "lack of information" responses

gard · October 2016

ElBiggus wrote: »

I've said it before, and I'll say it again. Anyone who says downtime "needs to happen" to run maintenance or that it's ZOS "doing their job" has plainly never worked anywhere with a large server infrastructure. There are many ways of doing maintenance without requiring that everything be shut down for eight+ hours a week, and if they were "doing their job" then things would never get so bad that it requires 5% weekly downtime.

As for the "well it's been like it forever, so there's no point complaining" argument, that's just dumb -- the fact that even after all this time they've not got a better system in place is more reason to complain.

You wouldn't put up with your internet only working 95% of the time or if your car didn't work on Mondays every week and the dealer wasn't able to fix it after two years of trying you'd kick up a stink, but game developers are apparently blameless if their products are so unreliable, and I find it confusing...

Show me in the tos where it says anything about uptime guarantee.

In fact the tos says there isn't one, iirc.

Siphoneer · October 2016

AzraelKrieg wrote: »

Oh look. Another "woe is me, maintenance is during my gaming time/day off" topic. Get over yourself. I finished work for the day a few hours ago, got a little time to log in, check mail, craft for a guild mate and logged off. I knew maintenance was coming, and I accept it. I don't come on here and complain because of entitlement. I do something else. I just let the launcher sit in the background while do something else like watch some stuff that I've been mean to watch or play another game. So seriously, get off your high horse. I want to be playing right now, I know I can't but I don't complain.

Lol you're probably American

Tryxus · October 2016

P3ZZL3 wrote: »

These threads are fun to read tbh

So many untruths and "lack of information" responses

Well we gotta have smth to do during downtime

Laughing at other ppl's stupidity will do

Bryanonymous · October 2016

human151 wrote: »

Bryanonymous wrote: »

ElBiggus wrote: »

Bryanonymous wrote: »

This isn't a phone company. Some businesses do actually close for one or two days a week. This is not a car. You are comparing products that have almost nothing in common, and now you have lost ALL credibility with your faulty logic.

No, this is a business whose primary, and indeed only, function is to run an online game, and the fact that they can't manage to do that for more than 95% of the time on a good week suggests that they're doing it wrong. I've already explained the incredibly simple process (which every server admin on the face of the planet learns pretty much on day one) that allows you to update a server-side service without requiring huge downtime, and still people like you insist that there's no other way of doing it. Even businesses whose online presence is merely a portion of their business will probably do it the no-downtime way, so for a company that relies entirely on online services to make such a mess of it is inexcusable and (in case I haven't said it enough times) EASILY AVOIDABLE.

You think you can do it better? Who's stopping you?

You feel entitled? Then sue.

You don't like it? Then leave.

Or deal with it like the rest of us. Seriously. You probably have no clue how their programs are setup on their servers, and why they need to do a weekly maintenance. For all we know, the servers need a weekly reboot because they suffer from the same memory leak as our personal games. And as Gena said before, rebooting the servers takes about an hour. They also do micro patches often after big updates. You think you know how to do that? Well, make your own game and do it, or apply for a position with them. Just for the love of everything, the entitlement is driving all of us crazy. You are not special. Get over yourself.

Patching and rebooting a server takes an hour? I doubt you even know what a server looks like.

I said rebooting their servers takes an hour. That's what their public relations rep has said. L2R. And fyi, they've also posted pictures of their server room, so a lot of us know exactly what they look like. And I've been inside corporate server rooms before. They keep the very cold like the freezer of a 7-Eleven or Walgreens'. Having doubts about someone you don't even know just shows your arrogance in making general assumptions and reflects your own lack of reason. Keep crying. Won't change anything. Your parents should have taught you that.

AzraelKrieg · October 2016

Siphoneer wrote: »

AzraelKrieg wrote: »

Oh look. Another "woe is me, maintenance is during my gaming time/day off" topic. Get over yourself. I finished work for the day a few hours ago, got a little time to log in, check mail, craft for a guild mate and logged off. I knew maintenance was coming, and I accept it. I don't come on here and complain because of entitlement. I do something else. I just let the launcher sit in the background while do something else like watch some stuff that I've been mean to watch or play another game. So seriously, get off your high horse. I want to be playing right now, I know I can't but I don't complain.

Lol you're probably American

No, I'm Australian.

BFT88 · October 2016

human151 wrote: »

AzraelKrieg wrote: »

Oh look. Another "woe is me, maintenance is during my gaming time/day off" topic. Get over yourself. I finished work for the day a few hours ago, got a little time to log in, check mail, craft for a guild mate and logged off. I knew maintenance was coming, and I accept it. I don't come on here and complain because of entitlement. I do something else. I just let the launcher sit in the background while do something else like watch some stuff that I've been mean to watch or play another game. So seriously, get off your high horse. I want to be playing right now, I know I can't but I don't complain.

So youre putting down customers who are not happy with being inconvenienced by a company they are giving money to for a service or product? I subbed for over a year to this game and regularly buy stuff out of the crown store. I am a customer.

Are you telling me I dont have a right to be upset that I cannot use the product I paid for and continue to pay for?

That little thing you agreed to called the terms of service says otherwise. So tired of people using the "I paid for such and such and deserve blah blah." While that may hold up to paid services to say, put new carpet in your house, fix your car, etc, it doesn't hold up here.

Every single person signed that right away when you agreed to the TOS that specifically states they can bring down the servers whenever the heck they please.

ElBiggus · October 2016

Bryanonymous wrote: »

Or deal with it like the rest of us. Seriously. You probably have no clue how their programs are setup on their servers, and why they need to do a weekly maintenance. For all we know, the servers need a weekly reboot because they suffer from the same memory leak as our personal games. And as Gena said before, rebooting the servers takes about an hour. They also do micro patches often after big updates. You think you know how to do that? Well, make your own game and do it, or apply for a position with them. Just for the love of everything, the entitlement is driving all of us crazy. You are not special. Get over yourself.

Why are you so insistent that I'm not allowed to complain about what I perceive as a needlessly poor service? It's nothing to do with being "entitled", and if me complaining about it is so upsetting to you then don't read it.

Seriously, it does not make one iota of difference "how their data is stored" and whether it takes two seconds or two months to reboot the servers is neither here nor there. I used to work for IBM's business systems departments dealing with creaky old AS/400s, and those things could take several hours to come back to life when you shut them down, particularly if you did it in a hurry. Pretty much all of our customers had at least two (but usually more; basically enough that even in the worst case of losing a huge chunk of their data cantre they'd still have capacity to take the load), and when one required maintenance they'd make sure another one was running so that when they took the faulty one offline they wouldn't interrupt service.

Honestly, whether you feel like I'm a whiny entitled scrote who doesn't deserve to play the game or not, ZOS need to sort out their infrastructure and rethink their maintenance policies. It is not a technically difficult challenge, it is one that has been solved many times over, and I suspect they only reason they do it the way they do is for cost reasons -- doing it properly would require at least one more server, but that's why some of us pay our ESO+ money, so they can afford stuff like that...

Asardes · October 2016

Reboot is one thing, patch is a software change that usually ends with a reboot. There have been emergency reboots during prime time, and planned reboots during maintenance window before.

ADarklore · October 2016

ElBiggus wrote: »

Tabbycat wrote: »

I'm sure ZOS would be absolutely thrilled if you were to design for them for free a workable server system that never has be taken down for maintenance or patches ever.

But (and here is the point I keep trying to make) I don't need to design it -- being able to do this is common industry practice. Still, if they're unsure how it works, here's the basic rundown.
Test the patches on a test server.

Once you're sure it works as intended, push the updates to a deployment server, leaving the production (i.e. live) server running.

Once the update has been pushed, switch the point of entry -- the deployment server now becomes the production server. New users logging on will automatically connect to the updated server, and you can push out a message to users still on the old server that they should log out and back in again when it's convenient (maybe giving a deadline of a couple of hours before forcibly ejecting them).

Once the old server is empty it becomes your deployment server for the next update.

Well look at that. You've pushed out a new update, and the only inconvenience is that some people have had to log out and back in again. No massive downtime or anything. I wonder why nobody thought of that befo... oh, wait, they did. I remember learning about it at college nearly 30 years ago...

So exactly 'which' INDUSTRY are you talking about?!? Because I know almost ALL MMOs have server downtimes for maintenance, DCUO goes down EVERY MORNING for 1/2 hour to an hour for maintenance, other MMOs do it less frequently but also have less players and less massive updates.

human151 · October 2016

Bryanonymous wrote: »

human151 wrote: »

Bryanonymous wrote: »

ElBiggus wrote: »

Bryanonymous wrote: »

This isn't a phone company. Some businesses do actually close for one or two days a week. This is not a car. You are comparing products that have almost nothing in common, and now you have lost ALL credibility with your faulty logic.

No, this is a business whose primary, and indeed only, function is to run an online game, and the fact that they can't manage to do that for more than 95% of the time on a good week suggests that they're doing it wrong. I've already explained the incredibly simple process (which every server admin on the face of the planet learns pretty much on day one) that allows you to update a server-side service without requiring huge downtime, and still people like you insist that there's no other way of doing it. Even businesses whose online presence is merely a portion of their business will probably do it the no-downtime way, so for a company that relies entirely on online services to make such a mess of it is inexcusable and (in case I haven't said it enough times) EASILY AVOIDABLE.

You think you can do it better? Who's stopping you?

You feel entitled? Then sue.

You don't like it? Then leave.

Or deal with it like the rest of us. Seriously. You probably have no clue how their programs are setup on their servers, and why they need to do a weekly maintenance. For all we know, the servers need a weekly reboot because they suffer from the same memory leak as our personal games. And as Gena said before, rebooting the servers takes about an hour. They also do micro patches often after big updates. You think you know how to do that? Well, make your own game and do it, or apply for a position with them. Just for the love of everything, the entitlement is driving all of us crazy. You are not special. Get over yourself.

Patching and rebooting a server takes an hour? I doubt you even know what a server looks like.

I said rebooting their servers takes an hour. That's what their public relations rep has said. L2R. And fyi, they've also posted pictures of their server room, so a lot of us know exactly what they look like.

Not to sound condescending but rebooting a server usually take no more than 5 minutes, for the modern servers. The company I work for does monthly patching a reboots to every server and rolling out the patch and rebooting is about 30 minutes (it can take a few minutes for the patches to be applied upon reboot),

I understand how non IT people can be like "its a necessary thing, it has to happen every week" etc. I do this for a living which is why I cannot understand why it takes 5 hours or more every week. The Corporation I work for would NEVER allow this to happen every week. Theyre livid if something is down for more than an hour. Change Management at my place of employment would never allow open ended maintenance windows, much less allow them to go on week in and week out for years without improvement.

What you people also fail to grasp is the concept of server clustering. They can take down one server, or a group of servers to do maintenance on them and the only thing that would happen is that the servers in the same cluster pick up the load.

Big companies also have a DR site, which in theory is an exact mirror of the main data center, they could run the game from the DR site while the main systems are being patched.

you see, no need to think we're idiots just because we know it could be done another way.

Garrick · October 2016

I just want to say I agree with the OP. Good defense of the (invariably missed) point. It never ceases to amaze me what some people will blindly support.

I actually thought that this was going to be people posting about how annoying it is to have the servers down on a Holiday.

Reykice · October 2016

Most games i played had minimal downtime unless unexpected problems, some like War Thunder used to patch while you played then asked you to restart the game so you can download the patch...

And do not fool yourself, ANY game can be made to patch like that if the developers want to. If they care enough about their clients to do it. They clearly do not, one of the reasons i spent so much more in other games vs ESO where i just keep the sub running and that is it, never needed the cosmetic stuff there or here but when i could see the devs caring to make the downtimes smaller or fix bugs faster i would just buy stuff to support them.

The maintenance also starts in the middle of the day for EU players... because apparently ZOS thinks anyone non american is second rate i guess.

ADarklore · October 2016

human151 wrote: »

Not to sound condescending but rebooting a server usually take no more than 5 minutes, for the modern servers. The company I work for does monthly patching a reboots to every server and rolling out the patch and rebooting is about 30 minutes (it can take a few minutes for the patches to be applied upon reboot),

I understand how non IT people can be like "its a necessary thing, it has to happen every week" etc. I do this for a living which is why I cannot understand why it takes 5 hours or more every week. The Corporation I work for would NEVER allow this to happen every week. Theyre livid if something is down for more than an hour. Change Management at my place of employment would never allow open ended maintenance windows, much less allow them to go on week in and week out for years without improvement.

What you people also fail to grasp is the concept of server clustering. They can take down one server, or a group of servers to do maintenance on them and the only thing that would happen is that the servers in the same cluster pick up the load.

Big companies also have a DR site, which in theory is an exact mirror of the main data center, they could run the game from the DR site while the main systems are being patched.

you see, no need to think we're idiots just because we know it could be done another way.

So the company you work for is a global MMO?!? Because I'm betting there are major differences between a inter-company sever system and a global MMO that connects hundreds of thousands of players from outside.

Asardes · October 2016

The PTS and the live server have quite different capabilities. The latter is much stronger because it has to support hundreds of thousands of connected players. If ZoS maintained two equally capable server clusters, they will spend much more money than they do now because they will have to rent at least 40% more server capacity.

Bryanonymous · October 2016

ElBiggus wrote: »

Why are you so insistent that I'm not allowed to complain about what I perceive as a needlessly poor service?

Probably because instead of asking in an AMA why they need so much down time, you instead chose to come make demands and insist that you are an expert who should be listened to.

Attitude and tone go a long way, as well as phrasing. Your whiny OP comes off as entitled demands and sobbing exaggeration.

It's like if we were all on a hot crowded bus, where the AC was broken, and your were yelling at the driver. The rest of us are just trying to wait out the heat, and your attitude makes it worse.

human151 · October 2016

ADarklore wrote: »

human151 wrote: »

Not to sound condescending but rebooting a server usually take no more than 5 minutes, for the modern servers. The company I work for does monthly patching a reboots to every server and rolling out the patch and rebooting is about 30 minutes (it can take a few minutes for the patches to be applied upon reboot),

I understand how non IT people can be like "its a necessary thing, it has to happen every week" etc. I do this for a living which is why I cannot understand why it takes 5 hours or more every week. The Corporation I work for would NEVER allow this to happen every week. Theyre livid if something is down for more than an hour. Change Management at my place of employment would never allow open ended maintenance windows, much less allow them to go on week in and week out for years without improvement.

What you people also fail to grasp is the concept of server clustering. They can take down one server, or a group of servers to do maintenance on them and the only thing that would happen is that the servers in the same cluster pick up the load.

Big companies also have a DR site, which in theory is an exact mirror of the main data center, they could run the game from the DR site while the main systems are being patched.

you see, no need to think we're idiots just because we know it could be done another way.

So the company you work for is a global MMO?!? Because I'm betting there are major differences between a inter-company sever system and a global MMO that connects hundreds of thousands of players from outside.

at any one time there are literally hundreds of thousands of transactions occurring between the Corporation I work for and our customers. Its a bit more important than a game, in the grand scheme of things. That said, this game is extremely important to ZOS in the same way my companies products are to it. Same concept though.

ElBiggus · October 2016

ADarklore wrote: »

So exactly 'which' INDUSTRY are you talking about?!? Because I know almost ALL MMOs have server downtimes for maintenance, DCUO goes down EVERY MORNING for 1/2 hour to an hour for maintenance, other MMOs do it less frequently but also have less players and less massive updates.

Every industry apart from gaming, apparently, where faulty products, poor customer service, and consumer-hostile practices are order of the day, and will continue to be so as long as people keep defending developers and publishers regardless of the quality of their work.

Not sure which video it was in, but Jim Sterling once made a point that some gamers seem to confuse "criticism of a game they like" with "attack on them personally" and will defend their beloved product beyond all sense of reason, even if the complaints are about something that, were it addressed, would benefit them too. If you're happy with server downtime, great, well done, but you'd presumably rather be playing it now than wasting time on here telling me how great ZOS are and how there's absolutely nothing that could possibly be conceivably done to avoid the regular and tedious periods of extended downtime (because somehow video games are "special" kinds of computer programs that function in some magical way that makes them immune to practices that work on everything else from the mom-and-pop hardware store up the road to the multi-billion dollar banking institutions that rely on realtime data so precisely and in such large quantities that it makes ESO look like a microscopic snail in comparison.)

Chickenstein · October 2016

Bryanonymous wrote: »

Or deal with it like the rest of us.

...

You are not special. Get over yourself.

Oh, we have a spokesperson for the free world here telling others not only how "all of us" feel, but that as a complaining individual you're not special (which they obviously are by that logic).

Seriously, what you're saying is "we all have to buy that bread although it's moldy; if you don't like it be a baker.". While this may be an evolutionary correct response, this is enterprising and not mother nature, so the correct answer would be "if you don't like it, leave", or, in response "if you don't like my complaints, ignore them, you're not the recipient".

On a side note, what frustration entitled you to insulting other users out of friggin nowhere? "Your whiny OP comes off as entitled demands and sobbing exaggeration." "Keep crying. Won't change anything. Your parents should have taught you that."
Well I know what parents should have taught *you*.

idk · October 2016

human151 wrote: »

Yeah, Downtimes are inconvenient and since I work nights im always affected by them since Sunday, Monday, Tuesday are my days off (I stay up nights and sleep days on my days off). A few hours.....for what? To run some patches? I work in a Data Center and for those of you who do not know, the "Server" is not just one server, it is hundreds or even thousands of servers clustered together (meaning all the servers behave as one, in simple terms) . I imagine on Sundays the load is low (amount of people online as compared to other nights) so why cant ZOS just move the resources off of a group of servers and bring down those servers and do maintenance. Once complete with that group, bring them back online and move the resources onto them and then bring down the other group? THen there would be no need for downtime.

THe answer is, this would double the time maintenance would take which is why they wont do it. I just find it laughable that they would rather inconvenience their customers than lengthen the DT. I find 3 or 4 hours every week unacceptable (in one chunk) . ANd no, this is not my first MMO, Other MMOs have downtime too but it is usually only an hour.

@human151

Sorry to hear about your inconvenience.

Since you work in a data center and understand the "server" is actually a matrix of servers then you would understand it is an increased technical challenge updating the server while it is online. This is why most games bring the "server" down to perform maintenance when delivering a patch.

Additionally, they actually bring the server up, without giving us access to make sure it actually starts and do some initial testing.

While there is a very small number of games that have patched while servers are live, it is certainly not the norm.

human151 · October 2016

Asardes wrote: »

The PTS and the live server have quite different capabilities. The latter is much stronger because it has to support hundreds of thousands of connected players. If ZoS maintained two equally capable server clusters, they will spend much more money than they do now because they will have to rent at least 40% more server capacity.

they can do adequate testing on a smaller scale. Their testing servers do not even have to be in a Data Center, Data Center space is really expensive, they can have them in some office inside their building. Surely Bethesda could figure something out.