Now that the dust is settling from the launch of Update 26, I'd like to take a moment give everyone an update on what's been happening in the last week. Obviously it was one of the more bumpy releases we've had, and it's important that you all know the issues we found, what we did to fix them, and in general how things are going.
Edited by ZOS_MattFiror on June 4, 2020 3:01PM
First and foremost, we remain extremely grateful for the support we get from all of you. As I've said more than a few times, ESO as we know it would not exist without the community that plays it every day. We have been experiencing extremely high player populations, especially PC EU and PC NA, for a few months now. ESO is as popular as it has ever been, and we could not have done it without your participation. Thank you so much, and we'll continue to work diligently every day to make Tamriel your go-to virtual world.
We are still working from home and have been since mid-March. We are taking the pandemic very seriously and are doing what we need to do to ensure that everyone at ZOS and our families are safe. Working from home has presented some challenges, especially considering our large user base, and especially taking into account we launched Update 26 while working remotely.
The first issue we encountered on Tuesday morning was about two minutes after we opened the servers: we had the PTS flag set for live accounts, so everyone had access to Greymoor, whether they purchased it or not. So we had to take servers down and set everything straight – we had some lingering effects all week as some users were caught in limbo having purchased it but not being able to travel there. We have addressed almost all of these cases, but please contact Customer Support if you are still having issues.
The second problem we encountered was much more serious. Update 26 includes many new achievements, especially with a new Chapter and the new Antiquities system. This then combined with an issue where character records saved far more often than intended. These two factors together meant that save times grew over time, backing up all character loads and saves, which led to infinite load screens, timeouts while zoning from one place to another, etc. It took us a while to find the root cause and fix this one, but by Wednesday afternoon (after another maintenance that morning), it was fixed.
Next up was the "zone lag" issue that lasted until Friday where each instance of a zone became less and less responsive the more people entered that instance. This was a bug with combat Synergies – the more often you were in large combat situations (PvP, Dungeons, Trials, Harrowstorms, etc.) the more it bogged down the entire zone, becoming "laggy". This bug was especially disheartening to us, because we included a bunch of performance improvement fixes in U26, but this Synergy bug masked all of them, rendering them moot. After much investigation and analysis, we found and fixed the problem. Friday morning's maintenance contained the fix and finally, by the end of Friday, things were in a better place.
Now, as of Monday afternoon, we're down to smaller and more edge case issues (Guild Traders didn't process correctly on PC EU Sunday evening for example). We will fix these as they arise. It’s important to note that all of the fixes listed so far will be included in the Console and Stadia versions when they launch on June 9 and June 16 respectively, and we will continue to monitor for new problems and address them as they appear.
Now that we’ve fixed the Synergy bug, we are starting to run our metrics analysis on how performance has improved in Update 26. Overall server performance should be improved in all zones, and the controller/keyboard "input lag" issue first seen in Update 25 should be much alleviated. I am not saying that everything is fixed, but it should now be better, smoother, and more responsive overall. We are continuing to work on performance improvements, and we will keep giving you regular updates, although as you saw from our recent post, we are looking to adjust how we communicate our performance improvements to ensure you are getting the information you need.
Thanks everyone. Seriously. We are doing the best we can under the circumstances, and we are very proud that we got Update 26 to launch only one week later than scheduled. Launch could have been smoother, but we will work hard to ensure that ESO is stable and fun for years to come.
Stay safe and healthy, and wash your damn hands.
ESO Game Director