SERVER DOWN?

BetweenMidgets

If some people DONT get banned for this, its going to be such a bad look for ZOS.

There is no way to NOT KNOW that a PTS character, with all the CP and gold on the LIVE server isn't a bug. If they don't come down hard on those bad actors, it gives them an even worse look than they already have.

This update has been a real [snip]

[edited for profanity bypass]

Sakiri

Bluepetunia wrote: »

No need to panic, relax, get up out of your chair........

Take a shower
Eat something besides microwaved food
Go brush your teeth
Walk the dog
Spend time with spouse, kids, Mom, Dad (at least let them know you are still among the living)

Maybe put on some sunglasses and venture outside. I promise no dragons, werewolves, harpy's (well maybe the wife........j/k), vampires will get you'

Dude.

It's almost 2 am. I'm not doing any of that, since I've either done it already, don't need to, or can't.

And my bff's due on in two hours and we were gonna farm me some tank gear tonight.

KKolly

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

BetweenMidgets

KKolly wrote: »

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

Not in the budget.

Darquer

Help. Help, I need ESO now! LOL, Seriously though, why? What might I lose? I've been doing this for 10 years and now 14 characters, so it gets messy,

Loralai_907

Well....I can't say I've seen this be an issue before....

ragnarok6644b14_ESO

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

baratron

kyle.wilson wrote: »

I'm curious as to why the pts server going live affected the live server.
They should be completely separate with no intermingling.

"Should" is the word, and indeed explains what the issue is.

I have no idea how this could have happened and I feel very bad for the person who made the mistake, as well as everyone who has to try to clean it up.

ragnarok6644b14_ESO

It's a database dickup, it happens all the time - that's why backups exist in the first place. Databases are almost never flawless.

Now how specifically this database dickup happened is not something I can describe, but as someone who works for large entities with lots of data, it's distressingly common how often someone explodes some critical database or something.

I got locked out of my own office by the electronic keypad because two separate professionals exploded two separate databases (that they were used to working with every day) with bad inputs within 15 seconds of each other. It's just humans and data failing to co-exist.

KKolly

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

I would like the testing to remain on the testing server. I don't pay a monthly subscription to test their mistakes on a stable release

Seriously though, this isn't a bug in code that slipped into a stable release. This was catastrophic databasing error. Its unacceptable. How can I trust them with my private information if this is how they handle their databases?

shadyjane62

Just tried playing on EU.

Not worth my time.

virtus753

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

A PTS for the PTS!

PTSception.

ragnarok6644b14_ESO

KKolly wrote: »

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

I would like the testing to remain on the testing server. I don't pay a monthly subscription to test their mistakes on a stable release

Seriously though, this isn't a bug in code that slipped into a stable release. This was catastrophic databasing error. Its unacceptable. How can I trust them with my private information if this is how they handle their databases?

Presumably the people that make the PTS go and the people handling your private information have different standards for productivity and safety; I doubt they are trying to move and shake your private information as much as they are the character databases for the server.

I will say, though, that if you believe no one has ever made a similar database error in some other database this very week, boy have I got a deal on a bridge in Brooklyn for you.

Hopelessly fragging your database at some point is like a coming-of-age moment for every database manager.

Jimbru

I remember many years ago in the early days of EverQuest 1, there was a bug after a patch that glitched loot tables throughout the game. Suddenly every mob in the game, all the way down to the level 1 rats in the starting areas, began dropping level 50 end game boss loot, and of course people went on a feeding frenzy. When Sony realized what was happening, they literally pulled the plugs on the servers and instantly downed the game, and it stayed down for a long time while they fixed the bug and rolled back everything to get rid of all the ill-gotten loot. This error in ESO tonight strikes me as falling into the same category of "Big Trouble Needs Big Fix" in more ways than one, and I wouldn't bet on us getting to play any more in US PM time tonight. So it goes, I guess.

fireshine6623

kyle.wilson wrote: »

I'm curious as to why the pts server going live affected the live server.
They should be completely separate with no intermingling.

This must be the doing of Morlag Bal, trying to pull the worlds together lol. Either way I'm rooting for all the staff, and @ZOS_GinaBruno thank you for keeping us posted.

I recommended eso to a few potential new players, and I got a text over this past weekend that they got the base game and LOVED it. It truly is an excellent game, but things happen. I'm not disturbed by it in the least, I have every confidence in our beloved ZOS guardian angels of the servers. After all, they preserve the world we love to play in.

Immortal_Dark410

Dam R.I.P NA

SilverBride wrote: »

Bluepetunia wrote: »

No need to panic, relax, get up out of your chair........

Take a shower
Eat something besides microwaved food
Go brush your teeth
Walk the dog
Spend time with spouse, kids, Mom, Dad (at least let them know you are still among the living)

Maybe put on some sunglasses and venture outside. I promise no dragons, werewolves, harpy's (well maybe the wife........j/k), vampires will get you'

That is assuming that we spend every second of our day at our computers and never interact with other people, or take care of our hygiene, or get fresh air, or eat healthy. That is not a fair assessment.

ClowdyAllDay

This one wonders how many people will be perma banned from this screw up that they caused.

Oh hey look a template for gold gear woo! let me play with that on pc na so i can get perma banned!

shadyjane62

Why was the PTS even up now? During the so-called event of the year? They should have been fixing the abysmal drop rate on the style pages instead.

HeavyMetalHippie420

BetweenMidgets wrote: »

KKolly wrote: »

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

Not in the budget.

But they can afford to travel the world and host large events.

Glorious_Platypus

Y'all got any more of that (scratches neck)... ESO?

ragnarok6644b14_ESO

shadyjane62 wrote: »

Why was the PTS even up now? During the so-called event of the year? They should have been fixing the abysmal drop rate on the style pages instead.

I agree, they should fix something that isn't broken instead of testing the next major update to the game for bugs or issues. I mean really.

(/s)

Alaztor91

fireshine6623 wrote: »

This must be the doing of Morlag Bal, trying to pull the worlds together lol.

It took a decade, but the Planemeld is now complete. PTSharbour and Liveriel are now one.

KKolly

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

I would like the testing to remain on the testing server. I don't pay a monthly subscription to test their mistakes on a stable release

Seriously though, this isn't a bug in code that slipped into a stable release. This was catastrophic databasing error. Its unacceptable. How can I trust them with my private information if this is how they handle their databases?

Presumably the people that make the PTS go and the people handling your private information have different standards for productivity and safety; I doubt they are trying to move and shake your private information as much as they are the character databases for the server.

I will say, though, that if you believe no one has ever made a similar database error in some other database this very week, boy have I got a deal on a bridge in Brooklyn for you.

Hopelessly fragging your database at some point is like a coming-of-age moment for every database manager.

Databases get ***. Making them interact *** them even more. Its in their nature. But this is why you test something before pushing it out. Its clear they did not.

That speaks to bad structural design in how they do their work, something that is only made more obvious by the nearly constant slew of gameplay altering bugs. They have no Q&A it seems.

Dating to last monday, we've had to cancel 3 of our last 4 raids because the game simply wasn't playable. Thats not considering the week before where PvP was broken with tarnished nightmare inheriting other skills modifiers (like proxy). That was unfortunate..but acceptable. It happens. But what happened today should not.

JoeCapricorn

The entire Elder Scrolls series is a fever dream of a Soulshriven living in the post Planemeld world.

dcrush

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Obviously this is a bigger bug, but it would be nice if ZOS afforded a bit more urgency to issues like the Cyrodiil rollbacks to at least give the impression of caring about their customers. There is too big of a difference between the approach to both issues.

Path

Really? No one has mentioned the hamsters?

Feed them!

HeavyMetalHippie420

ZOS_GinaBruno wrote: »

Everyone, just popping in to let you know we are working on things and will update as soon as we can.

Is there an estimate on how long this will take? Also, will the rollbacks impact players who don't use pts?

virtus753

Path wrote: »

Really? No one has mentioned the hamsters?

Feed them!

They fed the hamsters, all right. They gave the PTS hamsters Live food.

ragnarok6644b14_ESO

KKolly wrote: »

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

I would like the testing to remain on the testing server. I don't pay a monthly subscription to test their mistakes on a stable release

Seriously though, this isn't a bug in code that slipped into a stable release. This was catastrophic databasing error. Its unacceptable. How can I trust them with my private information if this is how they handle their databases?

Presumably the people that make the PTS go and the people handling your private information have different standards for productivity and safety; I doubt they are trying to move and shake your private information as much as they are the character databases for the server.

I will say, though, that if you believe no one has ever made a similar database error in some other database this very week, boy have I got a deal on a bridge in Brooklyn for you.

Hopelessly fragging your database at some point is like a coming-of-age moment for every database manager.

Databases get ***. Making them interact *** them even more. Its in their nature. But this is why you test something before pushing it out. Its clear they did not.

That speaks to bad structural design in how they do their work, something that is only made more obvious by the nearly constant slew of gameplay altering bugs. They have no Q&A it seems.

Dating to last monday, we've had to cancel 3 of our last 4 raids because the game simply wasn't playable. Thats not considering the week before where PvP was broken with tarnished nightmare inheriting other skills modifiers (like proxy). That was unfortunate..but acceptable. It happens. But what happened today should not.

How do you test if a live server is attached to the wrong database before attaching it to the wrong database?

It's like asking to test if a screw is screwed in tightly before you screw it in...

And that's interesting, my groups have been doing their scheduled raids just fine; the only cancellations in the last month or so has been because folks were unavailable.

kyle.wilson

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

ragnarok6644b14_ESO wrote: »

KKolly wrote: »

dcrush wrote: »

Rollbacks in Cyrodiil and players losing items - 2 business days to shut down the servers

Players gaining things because of a server error - panic button

Typical.

I exclusively PvP and this is a **MUCH** bigger issue than the rollback bug. In this case servers needed to be taken down ASAP to avoid further exploit.

The real question is:

WHERE IS ZOS' QUALITY AND ASSURANCE

This is becoming downright embarrassing on their end.

Right here, fixing the problem less than an hour after it was detected?

What, do you want them to minority-report fix things before they happen?

How exactly would you catch this error in advance, considering the error was caused by doing the thing that permits testing for issues in advance in the first place?

I would like the testing to remain on the testing server. I don't pay a monthly subscription to test their mistakes on a stable release

Seriously though, this isn't a bug in code that slipped into a stable release. This was catastrophic databasing error. Its unacceptable. How can I trust them with my private information if this is how they handle their databases?

Presumably the people that make the PTS go and the people handling your private information have different standards for productivity and safety; I doubt they are trying to move and shake your private information as much as they are the character databases for the server.

I will say, though, that if you believe no one has ever made a similar database error in some other database this very week, boy have I got a deal on a bridge in Brooklyn for you.

Hopelessly fragging your database at some point is like a coming-of-age moment for every database manager.

Databases get ***. Making them interact *** them even more. Its in their nature. But this is why you test something before pushing it out. Its clear they did not.

That speaks to bad structural design in how they do their work, something that is only made more obvious by the nearly constant slew of gameplay altering bugs. They have no Q&A it seems.

Dating to last monday, we've had to cancel 3 of our last 4 raids because the game simply wasn't playable. Thats not considering the week before where PvP was broken with tarnished nightmare inheriting other skills modifiers (like proxy). That was unfortunate..but acceptable. It happens. But what happened today should not.

How do you test if a live server is attached to the wrong database before attaching it to the wrong database?

It's like asking to test if a screw is screwed in tightly before you screw it in...

And that's interesting, my groups have been doing their scheduled raids just fine; the only cancellations in the last month or so has been because folks were unavailable.

by ensuring that none of the test servers have any connection to the live servers. The PTS servers should be 100% segregated from the other servers.