Update 33 PC Launch Postmortem

Tandor

Thanks @ZOS_MattFiror for the clear explanation of the performance issues, and for engaging with us pretty promptly over this aspect of the concerns many have expressed in the last week or two.

Could we now please have a similarly clear explanation and engagement to address the other concerns many have expressed in the last month or two over the way acount-wide achievements have been implemented? Specifically, what issues that were highlighted on the PTS and Live forums do you acknowledge and to what extent are you working on them or simply dismissing them as "working as intended"?

I'm thinking of things like broken NPC interactions and quests, routine NPC dialogue, and broken zone maps and guides. Your thoughts on how these things are working would be very much appreciated.

EF321

Very nice info, thanks.

mrsninetales

Really appreciate the open communication on the issue. Thank you to the entire team for working so hard to resolve the issue as quickly as possible and for all the after hours work it may have involved.

Looking forward to the upcoming events and High Isle in June!

ectoplasmicninja

Thank you for the explanation of what happened, and for the compensation. At 1800+CP I personally am not as enthused about the XP scrolls (although plenty of folks will be!) but the tripled seals will be awesome and in good time to use on the new crate season...I have a Maormer-themed sorcerer who needs all the ocean things.

tomfant

Very welcome communication, for sure .

The serious FPS drops making the game in certain places unplayabke, however, are still there. I guess this is then not related to the hardware failure...

Zezin

Thank you for the postmortem, very insightful and exactly the kind of communication we want.

Branovices

I really appreciated a transparent, behind the scenes explanation like this. Helps me understand the problem, and appreciate the effort it took to fix it.

gariondavey

Thank you for the update and owning the situation.

More communication and ownership is very important, so please continue to do that.

Thank you as well for the compensation.

Lastly PLEASE separate the bg queues into dm + objective (backfills into dm after 10-15 minutes if necessary) for next patch. PLEASE...this is so desperately needed.

@ZOS_MattFiror @ZOS_Kevin @ZOS_GinaBruno

sarahthes

gariondavey wrote: »

Lastly PLEASE separate the bg queues into dm + objective (backfills into dm after 10-15 minutes if necessary) for next patch. PLEASE...this is so desperately needed.

@ZOS_MattFiror @ZOS_Kevin @ZOS_GinaBruno

I'm pretty sure the backfill timer was around that before when there were separate queues. It resulted in everything being a deathmatch because 1 person picking dm would force the 11 who queued random into their dm queue.

Starlight_Knight

Wonderful insightful explination, thankyou for sharing this info, cant wait for the extra seals

Chaquinho89

Normally I would give them a lot of [snip] for this, but I'm an IT/NOC Manager myself and I totally understand this struggle.
Gotta cut it for ZOS this time...

[edited for profanity bypass]

FeedbackOnly

Arunei wrote: »

I appreciate the honesty here, but I do have a few questions.

First, why are beta accounts and characters still being held? We don't have access to those accounts or characters, there should be no reason to keep them, at least none I can think of. Unless those are kept on a completely separate, inactive server, why waste database space on them, especially when your answer to database issues was to implement AwA in the way you did?

As for the 150% Exp Scrolls, that's 10 hours of increased experience. Which is a fraction of time compared to the five days a lot of people lost. You really should be giving people 20 or so of these things, it's not like it would cost you anything, and that would still only be 40 hours of increased experience, not even a full two days. Daily Seals should also be increased for five days, not just the weekly ones.

@ZOS_MattFiror

Edit because I was wrong, a friend informed me that the 150% scrolls are only one hour long, not two as I had thought. That's even less time people are being compensated for. We really should be getting a lot more of these if this is how you want to handle the compensation.

I actually joined ESO because I had a beta account over another game. I logined into such an account too.

These accounts are now in cold storage though. So don't worry.

Raevenglass

@ZOS_MattFiror Thank you for the update, the candidness, and the scrolls/seals.

Honestly, just seeing the update and then reading it made me feel a lot better than I have over the past month where it felt like radio silence and a team that didn't care about the player base.

This is the kind of communication we want and, frankly, deserve.

Thanks again, and I'm glad you found the problem!

Raevenglass

FeedbackOnly

sarahthes wrote: »

gariondavey wrote: »

Lastly PLEASE separate the bg queues into dm + objective (backfills into dm after 10-15 minutes if necessary) for next patch. PLEASE...this is so desperately needed.

@ZOS_MattFiror @ZOS_Kevin @ZOS_GinaBruno

I'm pretty sure the backfill timer was around that before when there were separate queues. It resulted in everything being a deathmatch because 1 person picking dm would force the 11 who queued random into their dm queue.

That was because drop down menu was poor way to implement it. Everyone would just press bottom button.

It was a design issue.

JoeCapricorn

Tandor wrote: »

Could we now please have a similarly clear explanation and engagement to address the other concerns many have expressed in the last month or two over the way acount-wide achievements have been implemented? Specifically, what issues that were highlighted on the PTS and Live forums do you acknowledge and to what extent are you working on them or simply dismissing them as "working as intended"?

I'm thinking of things like broken NPC interactions and quests, routine NPC dialogue, and broken zone maps and guides. Your thoughts on how these things are working would be very much appreciated.

Seconding this.

However, in one of the threads, there was a list of NPC interactions that may be affected by AWA and a player went through and tested them - only one was affected, the others seemed to work properly.

I haven't checked if the Verandis epilogue is affected.

I would still strongly prefer if delves, world-bosses, dungeons, public dungeons and world-events remained character specific in terms of completion.

Various NPC interactions that might currently pull from achievements should also be character specific. You have to make the entire ESO narrative experience replayable in full.


But this post was a good acknowledgement and explanation.

VaranisArano

Thanks for the explanation!

Crown_of_Antlers

Arunei wrote: »

I appreciate the honesty here, but I do have a few questions.

First, why are beta accounts and characters still being held? We don't have access to those accounts or characters, there should be no reason to keep them, at least none I can think of. Unless those are kept on a completely separate, inactive server, why waste database space on them, especially when your answer to database issues was to implement AwA in the way you did?

This right here. Why weren't these accounts taken care of first before taking individual character achievements away from us?

StevieKingslayer

Thank you for admitting mistakes. We needed to hear that. I logged in yesterday, and apart from my annoyance at AWA, the server seemed stable.

Admitting mistakes, earns trust, as does the detailed explanation. Continue to update us like this in future.

QuietPanda_22

Really appreciate the transparency.

Tenthirty2

@ZOS_MattFiror

Thank you.
As someone who has been involved with several all-hands type IT issues over the years, THIS is the kind of communication I was looking for.
A clear and concise explanation and a no-excuses ownership of the issue and steps to recover reputation.
As of last night in another post I was starting to lose faith, but your post has restored it.
It also explains the complexity of the compounding challenges and why they took the shape, and time, they did.

@ZOS_GinaBruno (cc @ZOS_Kevin )
You have asked us awhile back what better communication looked like to the player base.
This thread from Matt, this is it.
Compensation is nice and all, but to be totally honest I would have been happy with just Matt's explanation.
It makes me feel like my time and money as a customer is respected, and well spent.

FeedbackOnly

JoeCapricorn wrote: »

Tandor wrote: »

Could we now please have a similarly clear explanation and engagement to address the other concerns many have expressed in the last month or two over the way acount-wide achievements have been implemented? Specifically, what issues that were highlighted on the PTS and Live forums do you acknowledge and to what extent are you working on them or simply dismissing them as "working as intended"?

I'm thinking of things like broken NPC interactions and quests, routine NPC dialogue, and broken zone maps and guides. Your thoughts on how these things are working would be very much appreciated.

Seconding this.

However, in one of the threads, there was a list of NPC interactions that may be affected by AWA and a player went through and tested them - only one was affected, the others seemed to work properly.

I haven't checked if the Verandis epilogue is affected.

I would still strongly prefer if delves, world-bosses, dungeons, public dungeons and world-events remained character specific in terms of completion.

Various NPC interactions that might currently pull from achievements should also be character specific. You have to make the entire ESO narrative experience replayable in full.


But this post was a good acknowledgement and explanation.

It's carelessness, why would you move on to data storage issues while extreme latency issues still existed, along with group finder? The same reason you would would implement Awa while lots of bugs and problems exist.

"We will fix it later" attitude. That's just unprofessional in my opinion. We still have long term we will fix it later issues like sugar skulls ui error? How many years do we wait.

I seem to remember another company crying saying we have to push things back because we want to provide you guys with best game possible. A big name mmo too so same genre.

The thing is we had 8 days instability. I truly love eso, but this was not acceptable experience. I would like to see better QA before and after patch days.

We should see a list of ongoing issues that occur after each patch and plans to fix them. This would be far more organized instead of praying someone sees bugs.

I apologize if this came out rude. I am just disappointed at how this all went down. It ruined my hype for new dungeon dlc.

Xandreia_

what about real money that was wasted on days of unplayable eso plus time?

Storm27Stars

Hope I don't get in trouble for this but..

There always seems to be a word missing from ZoS communications and that's 'sorry!!!'.

The message was very detailed, but yeah, could have started with 'sorry'.

duagloth

Thanks,
This kind of communication needs to happen more often.

tmbrinks

While I appreciate the candor with this post... it's low hanging fruit. Who are you going to upset when you get to explain that the issue was a "hardware" issue? Nothing that ZoS did wrong, just a hardware failure caused it.

Yet, the things that are in the purview of things that are caused by shoddy coding, unpopular decisions made for "performance" (which have pretty much never materialized), and the ever dwindling quality of updates, are never commented on.

This gets an almost immediate stickied post from the studio head. Other changes are still wallowing in everlasting silence.

The fact this post exists actually makes me angrier about the lack of communication on the other major problems.

adirondack

“Postmortem” is such a dark dreary negative feeling term.

You’ve solved a problem and made good progress forward. It’s good to look back and investigate what happened so that steps can be taken to ensure there are no repeat issues.

I prefer the military phrase “After Action Review”. Sounds so much more positive and upbeat. Good AAR report.


And thanks also for the communication. How’s the investigation going into the combat bug? I think I have been in combat since 2014.

Kesstryl

@ZOS_MattFiror Detailed communication like this is most welcome and exactly the kind of response your players need. I would never be upset because of hardware failures and server problems, and it was obvious your team was ready working on the issue. All during my 8 years when there was lag and login issues, those were not the things that upset me because I trusted things would get fixed, and they did get fixed. That being said, communication is key, and you can see by all the responses how very positively honest communication from ZOS is received.

What some of us now need is communication about incorrect dialogue, out of order NPC chatter, map problems, and lost quests that are all plaguing our alts with Account Wide Achievements. When will we see fixes for those, or even acknowledgement that these issues exist and are being looked into? Many of us who did not want to lose individual character tracking would be fine moving on if the game world still treated our alts individually like we had before Update 33. Losing individual character identity AND losing the game world treating our alts as individuals is a double whammy and was not necessary.

AwA does not need to intrude into gameplay so much. By intrusive I mean it's way way way too intrusive! Achievements should be there for me to look at and be proud when I open the tab, allow me to buy achievement furnishings, and have titles, and that's it!!! Achievements should not effect how NPCs react to alts or chatter about them as if they are the player and not their own individuals in the game world, lock out quests as one time per account, or mess up the map and zone tracking. Please look at the feedback on both PTS and in official live feedback threads to see how the game world interacts with alts post update. We just want our game and immersion back for our alts.


Edited for clarity because I was on my phone at the time and it was my lunch break.

Lauranae

Thank you, i appreciate the fact you admit your errors. We all do. Communication is one of the most important in any situation. Please dont just post when this is either to late or in middle in a crisis. Do it more often.

We all work, or have worked, and we understand and know how its stressing to deal with those times but on our side is also stressing and sad.

Now, if i take my case, up to U33 I was playing with no lag since the game started, at full graphics.

As of today, each time i cross players on the road, or in town, my computer game freeze. And the more i play during the day the more lag is accumulating.
- i went from maximum graphics to medium, this does nothing
- i am playing on a new account with 1 character
- i tested some days without any addons, no change
- i removed some addons when adding them again (those for achievements, for pricing which seemed to be more demanding)
- all drivers are up to date
- i dedicated ports to Eso

I am now without options. Of course i can play, its not like those who cant log in. But the fluidity of the game is gone.

So are you still working on this lag and group finder problem or do you consider its all ok ?

Thank you

Epona222

Thank you for letting us know in some detail what had gone wrong - that really is much appreciated and it is great to have some honest communication about things when they go wrong.

Abigail

Oops, the ship sank because we used cast-iron rivets on the steel plating. To compensate your loss we'll be giving away complimentary life savers.