Greymoor & Update 26 Launch – Game Director Update

wills43b14_ESO

@ZOS_MattFiror as players we tend to rib y'all as devs a lot for not communicating with us as much. As a somewhat longer-term player who raids competitively and sometimes PvPs I really appreciate this update.

I'm sure there are plenty who will continue to just talk about the problems you haven't solved here. However, a lot of us in our trials guild (in Discord) are very supportive of you taking time to make this update. If this is something you, or another dev (not the community managers if possible), could do once a month or whenever something relevant happens we would all be VERY appreciative and much more understanding than we have been the last few months.

I can say with confidence the lack of communication angered many long-term players and caused many to take a break/quit (nearly myself included). This really does go a long ways towards alleviating our frustrations when you acknowledge what the problem is and what's being done to fix it. If you don't know immediately, feel free to tell US as players what to do to/what feedback to submit to help y'all diagnose the issues. We want to help make the game better and perform well. Even if the fix is complicated and might take awhile, just letting us know what it is or what the plans are really does help to alleviate our frustrations since it shows y'all are working on it and it's not simple like we sometimes think it is.

Many thanks for the update!

wills43b14_ESO

@volkeswagon I think I can speak for many when I say the lack of communication wasn't great. It frustrated the crap outta me because it makes it seem like ZOS doesn't care. Or, when the community managers do all the communicating, it feels second rate comparatively compared to the devs. When the problems have been this bad, a direct dev update goes a long way. I am in major support of these updates from devs on major issues!

Relladen

Appreciate the detailed feedback but the continual omission of the state Cyrodiil and PvP is concerning to say the least, yet not surprising. Did you guys really just give up on PvP or what?

Hotdog_23

Thanks for the update.

Very glad to see the current fixes are going to be implemented for the console release. Condolences to our PC brother and sisters who had to experience these problems and thanks as well for highlighting them and ZOS getting the fixes in time for the console release.

Firstmep

Hotdog_23 wrote: »

Thanks for the update.

Very glad to see the current fixes are going to be implemented for the console release. Condolences to our PC brother and sisters who had to experience these problems and thanks as well for highlighting them and ZOS getting the fixes in time for the console release.

But they didn't, I don't know who told Firor, that skill delay was better or fixed, it's not.
On PC EU it can still lag during primetime, not just in cyro.
Even the dungeon finder have been bugging out lately etc.

Thoragaal

wills43b14_ESO wrote: »

@ZOS_MattFiror as players we tend to rib y'all as devs a lot for not communicating with us as much. As a somewhat longer-term player who raids competitively and sometimes PvPs I really appreciate this update.

I'm sure there are plenty who will continue to just talk about the problems you haven't solved here. However, a lot of us in our trials guild (in Discord) are very supportive of you taking time to make this update. If this is something you, or another dev (not the community managers if possible), could do once a month or whenever something relevant happens we would all be VERY appreciative and much more understanding than we have been the last few months.

I can say with confidence the lack of communication angered many long-term players and caused many to take a break/quit (nearly myself included). This really does go a long ways towards alleviating our frustrations when you acknowledge what the problem is and what's being done to fix it. If you don't know immediately, feel free to tell US as players what to do to/what feedback to submit to help y'all diagnose the issues. We want to help make the game better and perform well. Even if the fix is complicated and might take awhile, just letting us know what it is or what the plans are really does help to alleviate our frustrations since it shows y'all are working on it and it's not simple like we sometimes think it is.

Many thanks for the update!

wills43b14_ESO wrote: »

@volkeswagon I think I can speak for many when I say the lack of communication wasn't great. It frustrated the crap outta me because it makes it seem like ZOS doesn't care. Or, when the community managers do all the communicating, it feels second rate comparatively compared to the devs. When the problems have been this bad, a direct dev update goes a long way. I am in major support of these updates from devs on major issues!

I couldn't have said it better myself!

Artorias24

Any Update on Cyro Performance? Didnt saw much of an improvement to be honest. Normally Weekend at about 1pm was the best time to PvP. All High Pop, lag was really minimal. Now even this time we have lag, delays everything.

What does your reports say about PvP Performance?

SmukkeHeks

Thanks for the update.

I’m glad to hear that you are all taking care of yourselves, your families and co-workers. Just be safe. Both from the pandemic and the protests (✊🏿❤️)

Game wise, I was back on Monday to a smooth gameplay. Skills firing when asked, ultimates that didn’t let me down, none sudden loading screens. Tuesday was the same. Wednesday, the stutter slowly came back. Yesterday, prime time Eu, the missing skills came back. The server can’t handle the pressure, it seems.

But Monday was the best day in a looooooooooong time

Bucky_13

Sarousse wrote: »

@ZOS_MattFiror

Performance in cyrodiil as soon as there is 3 bars or organized groups is awful, and skill delay is unbrearable.

No need to monitor. Just go to cyrodiil on eu server during prime time, and play.

This. I do appreciate Matt's post a lot since I do think ZOS needs to communicate more with their players, I can also understand the challenges of working from home on a game like this.

But Cyrodiil is rarely mentioned by ZOS when it comes to performance. It would be nice to hear about a serious plan to improve performance as the high latency have returned to Cyrodiil again. It's one of the best PvP mode of any MMO I've played, if not the best. Giving us some love and improving performance would be welcome, or at least talk to us about it for a start instead of ignoring it.

Cinbri

And how about problem that people who didn't bought Chapter but ported there due to bug, gained access to new lines and farmed new Mythic sets before zos fully closed access, so now those people playing with Mythic sets without having Chapter. How bout fix that too.

GlorphNoldorin

Thanks for the update, it is appreciated.

If the pc NA and Eu servers are so populated, I assume that means the game is very profitable for Zenimax.

If the server load is such a problem....and it is- as clearly can be seen from game performance in trials, arenas and pvp, can Zenimax please do something about it?

The lack of communication and the counter intuitive business model (that a game's performance becomes worse as it gets more popular) is a major slap in the face for those that have supported the game for many years.

For those that are new to the game it should sound alarm bells.

gepe87

Input delay soon to be solved??? I didnt noticed!

Elsonso

Bucky_13 wrote: »

But Cyrodiil is rarely mentioned by ZOS when it comes to performance. It would be nice to hear about a serious plan to improve performance as the high latency have returned to Cyrodiil again. It's one of the best PvP mode of any MMO I've played, if not the best. Giving us some love and improving performance would be welcome, or at least talk to us about it for a start instead of ignoring it.

They have barely commented on Cyrodiil for years now. The last comment was from Lambert last month, where he said it was a priority, but that Cyrodiil performance was basically tied to other performance work being done.

As for input lag, not solved but better is the best description I have for it. That means I still see it, but not as much.

Lysette

i still think that zones are too crowded at or near prime time and servers need an upgrade - less players per instance.

Otherwise I appreciate the input from Matt and just wish, it would be more often.

Idinuse

Princess_Ciri

MissBizz wrote: »

That seemed to be random as per the instance so I'm not convinced it's for sure fixed. I only really noticed it in trials and have only been to a handful since the update

I raid about 4 times a week and my guild were experiencing issues (as well as speaking to others who could confirm the same) that it happened to us in about 80% of instances.

I haven't seen it once since the update so I would guess they did fix it.

Destai

ZOS_MattFiror wrote: »

Now that the dust is settling from the launch of Update 26, I'd like to take a moment give everyone an update on what's been happening in the last week. Obviously it was one of the more bumpy releases we've had, and it's important that you all know the issues we found, what we did to fix them, and in general how things are going.

First and foremost, we remain extremely grateful for the support we get from all of you. As I've said more than a few times, ESO as we know it would not exist without the community that plays it every day. We have been experiencing extremely high player populations, especially PC EU and PC NA, for a few months now. ESO is as popular as it has ever been, and we could not have done it without your participation. Thank you so much, and we'll continue to work diligently every day to make Tamriel your go-to virtual world.

We are still working from home and have been since mid-March. We are taking the pandemic very seriously and are doing what we need to do to ensure that everyone at ZOS and our families are safe. Working from home has presented some challenges, especially considering our large user base, and especially taking into account we launched Update 26 while working remotely.

The first issue we encountered on Tuesday morning was about two minutes after we opened the servers: we had the PTS flag set for live accounts, so everyone had access to Greymoor, whether they purchased it or not. So we had to take servers down and set everything straight – we had some lingering effects all week as some users were caught in limbo having purchased it but not being able to travel there. We have addressed almost all of these cases, but please contact Customer Support if you are still having issues.

The second problem we encountered was much more serious. Update 26 includes many new achievements, especially with a new Chapter and the new Antiquities system. This then combined with an issue where character records saved far more often than intended. These two factors together meant that save times grew over time, backing up all character loads and saves, which led to infinite load screens, timeouts while zoning from one place to another, etc. It took us a while to find the root cause and fix this one, but by Wednesday afternoon (after another maintenance that morning), it was fixed.

Next up was the "zone lag" issue that lasted until Friday where each instance of a zone became less and less responsive the more people entered that instance. This was a bug with combat Synergies – the more often you were in large combat situations (PvP, Dungeons, Trials, Harrowstorms, etc.) the more it bogged down the entire zone, becoming "laggy". This bug was especially disheartening to us, because we included a bunch of performance improvement fixes in U26, but this Synergy bug masked all of them, rendering them moot. After much investigation and analysis, we found and fixed the problem. Friday morning's maintenance contained the fix and finally, by the end of Friday, things were in a better place.

Now, as of Monday afternoon, we're down to smaller and more edge case issues (Guild Traders didn't process correctly on PC EU Sunday evening for example). We will fix these as they arise. It’s important to note that all of the fixes listed so far will be included in the Console and Stadia versions when they launch on June 9 and June 16 respectively, and we will continue to monitor for new problems and address them as they appear.

Now that we’ve fixed the Synergy bug, we are starting to run our metrics analysis on how performance has improved in Update 26. Overall server performance should be improved in all zones, and the controller/keyboard "input lag" issue first seen in Update 25 should be much alleviated. I am not saying that everything is fixed, but it should now be better, smoother, and more responsive overall. We are continuing to work on performance improvements, and we will keep giving you regular updates, although as you saw from our recent post, we are looking to adjust how we communicate our performance improvements to ensure you are getting the information you need.

Thanks everyone. Seriously. We are doing the best we can under the circumstances, and we are very proud that we got Update 26 to launch only one week later than scheduled. Launch could have been smoother, but we will work hard to ensure that ESO is stable and fun for years to come.

Stay safe and healthy, and wash your damn hands.

Matt

Thanks for posting this and going into some details. Thanks for working through the mountain of issues during this time.

But...it REALLY sounds like you guys rushed Greymoor here. It shouldn't have launched. Especially when you guys are dealing with the hurdles of deploying an MMO from home. I think more people would understand delaying a launch in this time than launching it in this state. It shouldn't have launched new software on top of software that's undergoing massive performance updates. Supposedly.

And it looks bad that you're not continuing the monthly performance updates. People aren't going to be happy about that. You have to be aware that this is issue #1 for a lot of people. You say you're looking to adjust how you communicate with us - based on what feedback? Marketing? Website analytics? Do an AMA, get on these forums, and engage people for how we'd like to be given updates.

Take a look at how this looks from a customer:
1. We're charged $100+ for yearly content - DLC and Chapters. Most games include dungeons in their expansions. ZOS has chosen not to and charges us each. step. of. the. way.
2. Items in the crown store frequently cost more than the game itself. No explanation given as to why a house should cost more than a year's worth of content. BUY CROWNS is plastered all over this site. Crown store pops up soon as I log in the game.
3. Items in the crown store are rolled out instead of them being earned in game.
4. The combat desync has plagued us for months, with no resolution in sight. A core feature of the game is broken.
5. ZOS employee attitudes - won't name names - are often completely tone-deaf and counter-productive. Especially when it comes to the perfected weapon launch. There's got to be some self-awareness on that issue.
6. We're told silly things like "The servers are holding up well." when CLEARLY they aren't.
7. Greymoor launches with loads of issues, which have been communicated fairly well, on top of the long-standing performance issue.
8. Performance updates are slated to stop being monthly amidst all of this.

Darkstorne

ZOS_MattFiror wrote: »

Now that we’ve fixed the Synergy bug, we are starting to run our metrics analysis on how performance has improved in Update 26. Overall server performance should be improved in all zones, and the controller/keyboard "input lag" issue first seen in Update 25 should be much alleviated.

An entire year of "performance fixes" including making everyone redownload the entire game and performance is genuinely in a worse place than it was this time last year. You know your game is in an embarrassing state when ping below 100 is considered good. All your competition - WoW, SWTOR, FFXIV, GW2 etc - provide consistent ping of 20-50 for the vast majority of their playerbase. Your servers are not up to this. How much is too much? When will you acknowledge the servers need to change? How many traceback examples does your playerbase need to provide?

What you've really been saying with all these "performance updates" leading up to Greymoor:

And how so many of us are feeling these days:

Orpheaus

Good to see that you're confirming to everyone you do not give a *** about PvP. Claiming that you improved performance, and the only problems are outliers now is a blatant slap in the face to the PvP community seeing as anytime Cyrodiil gets more than 2 bars on any faction it becomes immediately unplayable. It's reflected in population too. Main campaign used to never drop below 2-3 bars no matter what time of day or what day of the week. Now it rarely gets over 3 bars in non-primetime.

You've made your priorities clear. And obviously PvPers are not a priority.

dhboy123

I fail to see any optimisation in game performance. If anything it seems worse.

ArchMikem

So all these fixes will be put into the June 9th Console update. Wow, PC actually did Beta Test this time.

I really liked this post, and hope all those demanding communication appreciate it as well.

wolfbone

instead of postponing grey moor launch on consoles, you could have actually helped by donating money. but, y'all care too much about money.

volkeswagon

PVPers are a rough crowd

twitch_zero

I just want to reiterate what others have said about Cyrodiil performance. Today in the Gray Host NA campaign with all 3 factions locked performance was worse than ever. There has been no improvement. Skill lag, desync and huge numbers of load screens.

TheValkyn

The remote work excuse is used far too much.

TheValkyn

ArchMikem wrote: »

So all these fixes will be put into the June 9th Console update. Wow, PC actually did Beta Test this time.

I really liked this post, and hope all those demanding communication appreciate it as well.

I play on both environments. Console and PC. The recent performance dip story is the same but they are trying to blame the new patch.

The worst part is that you can’t monitor your latency through the console ui.

technohic

volkeswagon wrote: »

PVPers are a rough crowd

We have some loud ones, but if you sit down and talk to most; generally pretty easy going. Most I know appreciate the post, but just want them to know that it is not that much smoother. Its somewhere between Update 25 and prior. Its like its actually better in delay and actual lag than prior to update 25, but it has all that position desync update 25 brought.

seerevaloc

This is the long missed communication. Thanks Matt.

Shantu

You know, it's amazing what good will and understanding come from just some good old fashioned, honest communication. Thank you.

Rianai

Cinbri wrote: »

And how about problem that people who didn't bought Chapter but ported there due to bug, gained access to new lines and farmed new Mythic sets before zos fully closed access, so now those people playing with Mythic sets without having Chapter. How bout fix that too.

Yes, we can't have players who don't keep shoveling money at a broken game on even terms. Gotta enforce P2W.