SERVER CRASH AGAIN?!

AzraelKrieg · August 2015

ZOS_GinaBruno wrote: »

Ok guys, finally heading to bed to catch some zzz's. I'm passing the torch to Mr Servotte who will keep you all updated. It was great hanging out with you all tonight, and thanks again for your endless patience! Hope you see you all on ESO Live later on.

Night Gina. Be sure drink lots of coffee before ESO Live cause you might need it xD

LadyDestiny · August 2015

jonjees wrote: »

SLy_Kyti wrote: »

jonjees wrote: »

for eso plus members, it is still a subscription.
for whatever reasons eso is down, compensation needs to be given to subscription users.
in this era where redundancy, high availability and business continuity are all viable solutions, this should not be happening.

likewise down for patching, crashes and what nots, the subs needs to be compensated, because they have paid for the service.

Wait what?
You want compensation for the hour or two the servers are down? even during maintenance? I can't even..I just...nm.

its been 5hours since i couldnt login.
add to the maintenance that always happens during asia/oceania peak hours.

most games will provide some form of good will compensation, as it is their business to stay online as an online game.
if you read further i am not asking for a million dollars in compensation.

but there needs to be a form of compensation or penalty else eso can decide to crash or go offline as and when they have to.
*shrugs* different of opinions.

Lol compensation for a few hours? Go read your terms you agreed to when you installed the game.

Starwhite · August 2015

What can I say .....

jonjees · August 2015

Spottswoode wrote: »

jonjees wrote: »

couple of hours to reboot servers .....

A statement from someone who has no experience with business networking. If Google had to reboot all of their servers, it could take a few days. (That is to say, ALL of them including the datacenters and backup datacenters.) As ZOS probably doesn't have a backup center to operate from (or backup network) this kind of reset is a lot of work.

wow really a few days?
thats when you just showed how much u know about redundancy and high availability.
maybe you need to read a paper on google hadoop cluster, hdfs just doesnt work this way.

also in terms of networking, u do not reboot ur servers if you have a network failure.

but ya i really got no experience.

SLy_Kyti · August 2015

Night Gina!

Hello Night Crew! @ZOS_MichaelServotte
c682c4cceb81e90337e4a8d1db6cbf805c63b3c27d74270e1a86c123ff17987d.jpg

Batavia · August 2015

ZOS_GinaBruno wrote: »

Ok guys, finally heading to bed to catch some zzz's. I'm passing the torch to Mr Servotte who will keep you all updated. It was great hanging out with you all tonight, and thanks again for your endless patience! Hope you see you all on ESO Live later on.

good night and thanks for the updates.

Eshelmen · August 2015

jonjees wrote: »

Spottswoode wrote: »

jonjees wrote: »

couple of hours to reboot servers .....

A statement from someone who has no experience with business networking. If Google had to reboot all of their servers, it could take a few days. (That is to say, ALL of them including the datacenters and backup datacenters.) As ZOS probably doesn't have a backup center to operate from (or backup network) this kind of reset is a lot of work.

wow really a few days?
thats when you just showed how much u know about redundancy and high availability.
maybe you need to read a paper on google hadoop cluster, hdfs just doesnt work this way.

also in terms of networking, u do not reboot ur servers if you have a network failure.

but ya i really got no experience.

Network failure is a broad term, how about your server ISP crashed instead?

Think of Zeni as the middle man on this one.
Of course they will reboot to make sure everything is in order.

ZOS_MichaelServotte · August 2015

@SLy_Kyti it's already the morning around here.

Gina will get some well deserved sleep, and I'll be there to keep you updated as soon as I got news for you. Meanwhile I'm reading this thread and I just found the best picture ever. This is disturbing.

jonjees · August 2015

Greeniewolfub17_ESO wrote: »

jonjees wrote: »

wormfudge wrote: »

Lol. Add a couple hours of maintenance to the mix. Just to reboot servers?

heh, two hours really isn't that much. Before they switched to rolling restarts, it used to take Linden Lab the better part of a day to restart all their servers. Aeria takes a good several hours as well for server restarts for most of their games.

keep in mind they have to backup each server, verify the backup, actually restart the thing, and then verify it doesn't need to be restored from backup. And they have dozens of servers to go through this process.

thing is it was really a network outage, unless its really servers crashing or some issues with the patches.
backup the servers? doubt that can even complete in 6hrs if they actually still doing it conventionally.

either way, all there is to do is wait 5hrs 30mins and counting.

DaffyDilly · August 2015

ZOS_GinaBruno wrote: »

Ok guys, maintenance on our end is underway. As we're rebooting all megaservers for all platforms, it may be a couple hours to complete everything. Thanks again for your patience tonight.

Just happy to have ANY update on an ETA for servers.Power nap time.

keep-calm-because-highly-trained-monkeys-are-fixing-the-error.png

Rainuu · August 2015

so could u plz tell me why theres no zos in german thread

Xyrqes · August 2015

LOL the only reason they carried out any maintenance was to add a login timer to the log-in screen
so you can now only attempt to log in once per minute

this isn't a bad thing, since it will relieve the login server from silly people constantly clicking 'login' when the servers are finally online but struggling to cope with the masses trying to actually play

but suggesting it is 'maintenance' hmmm

some have said you can't blame zenimax for the problem
but of course you can
any system should be prepared, and have contingency options in place
failure to do so is unprofessional
Z should have procured an alternative backup feed, in the case of this eventuallity

kaithuzar · August 2015

I just woke up to this -__________-

Greeniewolfub17_ESO · August 2015

Rainuu wrote: »

so could u plz tell me why theres no zos in german thread

guessing that no one even thought about it to be honest. Gina was doing a great job, especially given that she had been about to go to sleep when this went down but even she didn't think to check fb or twitter till others mentioned them. I suspect French, German, and Spanish threads all got overlooked.

Xyrqes · August 2015

LOL the only reason they carried out any maintenance was to add a login timer to the log-in screen
so you can now only attempt to log in once per minute

this isn't a bad thing, since it will relieve the login server from silly people constantly clicking 'login' when the servers are finally online but struggling to cope with the masses trying to actually play

but suggesting it is 'maintenance' hmmm

some have said you can't blame zenimax for the problem
but of course you can
any system should be prepared, and have contingency options in place
failure to do so is unprofessional
Z should have procured an alternative backup feed, in the case of this eventuallity

jonjees · August 2015

Xyrqes wrote: »

LOL the only reason they carried out any maintenance was to add a login timer to the log-in screen
so you can now only attempt to log in once per minute

this isn't a bad thing, since it will relieve the login server from silly people constantly clicking 'login' when the servers are finally online but struggling to cope with the masses trying to actually play

but suggesting it is 'maintenance' hmmm

some have said you can't blame zenimax for the problem
but of course you can
any system should be prepared, and have contingency options in place
failure to do so is unprofessional
Z should have procured an alternative backup feed, in the case of this eventuallity

finally someone that understands this concept.

Eshelmen · August 2015

Xyrqes wrote: »

LOL the only reason they carried out any maintenance was to add a login timer to the log-in screen
so you can now only attempt to log in once per minute

this isn't a bad thing, since it will relieve the login server from silly people constantly clicking 'login' when the servers are finally online but struggling to cope with the masses trying to actually play

but suggesting it is 'maintenance' hmmm

some have said you can't blame zenimax for the problem
but of course you can
any system should be prepared, and have contingency options in place
failure to do so is unprofessional
Z should have procured an alternative backup feed, in the case of this eventuallity

Yeah..............just like we all have generators hooked up to our houses if the electric goes out.

SLy_Kyti · August 2015

kaithuzar wrote: »

I just woke up to this -__________-

Go back to sleep!?!

@Xyrqes we heard you the first time buddy.

Pennthuis · August 2015

Its Friday night! & yay the week end, I'll be online on ESO on Sunday as my Khajiit, seeing were I start off after getting booted today part way through a quest lol oh the excitement.

Xyrqes · August 2015

Yeah..............just like we all have generators hooked up to our houses if the electric goes out.

well yes I do
but I also don't have 100,000s of users depending on my services

kaithuzar · August 2015

Is there going to be an announcement made when we can log back in? I show realm status as online even though it says the server's are offline in a message box pop up at the start?
Also, is PTS affected by this too? (I'm assuming so but figured I would still ask)

Epona222 · August 2015

jonjees wrote: »

Xyrqes wrote: »

LOL the only reason they carried out any maintenance was to add a login timer to the log-in screen
so you can now only attempt to log in once per minute

this isn't a bad thing, since it will relieve the login server from silly people constantly clicking 'login' when the servers are finally online but struggling to cope with the masses trying to actually play

but suggesting it is 'maintenance' hmmm

some have said you can't blame zenimax for the problem
but of course you can
any system should be prepared, and have contingency options in place
failure to do so is unprofessional
Z should have procured an alternative backup feed, in the case of this eventuallity

finally someone that understands this concept.

Yep I may have mentioned it earlier, but I once worked in an office where half of the walls were taken out by a carbomb. Due to contingency planning, the office was back up and running in a different location within a couple of hours.

I know there will be some folks on here in a minute to say how that was not possible in this case, blah blah blah

Spottswoode · August 2015

jonjees wrote: »

Spottswoode wrote: »

jonjees wrote: »

couple of hours to reboot servers .....

A statement from someone who has no experience with business networking. If Google had to reboot all of their servers, it could take a few days. (That is to say, ALL of them including the datacenters and backup datacenters.) As ZOS probably doesn't have a backup center to operate from (or backup network) this kind of reset is a lot of work.

wow really a few days?
thats when you just showed how much u know about redundancy and high availability.
maybe you need to read a paper on google hadoop cluster, hdfs just doesnt work this way.

also in terms of networking, u do not reboot ur servers if you have a network failure.

but ya i really got no experience.

uhh....datacenters and backups are down? We're talking global EMP attacks and universal communications blackouts apocalypse scenario. (I mean if you want to go through the whole emergency plan we could....) Google does have backups and redundancy out the wazoo to prevent that from happening in the fist place, but IF it did happen it's still gonna take a few days to test all that. But still, the point being ZOS probably doesn't have anywhere near the amount of backup systems that any data company has.

Depends on the source of the failure, size of the server, and amount of data loss on whether or not you'd reboot the server in the event of network failure. Also depends on whether or not you're loading a full backup, the type of server, etc. etc. Generally speaking, though, you are correct.
You also might be forgetting what time it is over here in the US.
Still a silly statement.

I'll still apologize though. I'm sorry.

Belidos · August 2015

Xyrqes wrote: »

some have said you can't blame zenimax for the problem
but of course you can
any system should be prepared, and have contingency options in place
failure to do so is unprofessional
Z should have procured an alternative backup feed, in the case of this eventuallity

Except in this case you are entirely missing the point.

It is impossible for ZOS to have had a backup contingency for this because it wasn't just their ISP that went down, it was the network backbone that went down, you know the whole network for the state that every ISP goes through, even if they had another ISP to fall back on that one would have been down too.

bareheiny · August 2015

If it makes anybody feel better, it's almost 9pm Friday night here...prime gaming time for me.

Now I have to slum it with an inferior game

jonjees · August 2015

players just dont understand that this is a business as well.
eso being an online game needs to stay ONLINE.
15$ a pop for sub and at 10k subs, thats easily 150k a month and im sure theres more than 10k subs.

if this is a free mmo, where i just download and play free with cash shops, i wouldnt give a hoot if its down this much.
also for the console players, we just want to turn it on and play hassle free, thats the entire concept of going console.

6hrs of downtime and a few more hrs for maintenance ..... if i bought eso today as a console playerand tried playing during this time, i am so returning it.

ZOS_MichaelServotte · August 2015

kaithuzar wrote: »

Is there going to be an announcement made when we can log back in?

We'll let you know as soon as possible, in the meantime we ask you not to try log into the game. Our teams are working on the servers and it may be a couple hours to complete everything. Thanks again for your patience.

Eshelmen · August 2015

Well, it's a good thing you're not new eh?

k2blader · August 2015

I wouldn't have been so irked if one of the ZOS people hadn't posted the problem was fixed and for anyone still experiencing issues to check with his/her ISP. The problem wasn't/isn't my ISP, [snip]. Just totally irresponsible to post something like that.
[edited for profanity bypass]

Techlisp · August 2015

A naughty Khajiit took a leak in the server room I hear.