What's also really concerning is that we haven't had an update since the initial aknowledgement. Of course they don't have to inform us of every step in the process, but since this bug makes a lot of fights in the game unplayable, it would be nice to have some information or a projected timeline, even if we don't have a concerete fix yet.

Vitaely wrote: »

p00tx wrote: »

Vitaely wrote: »

I have been watching this thread rather impatiently waiting for ZOS to finally come out and try to explain this situation. @ZOS_GinaBruno @ZOS_KaiSchober @ZOS_Finn Alas, that's still not the case.

To re-iterate, I -for one- would like to hear, why we have never seen ZOS rollback a faulty patch.

I think everyone understands that you have included many small combat and in-game fixes in the last few patches, but none of them were VITAL to PVE gameplay as a whole as this enemy position desync bug. If you don't see why that is, you don't play your own game. Because this bug has ****ed with every single mob/boss/elite animation in the game. If that doesn't sound important to you, I don't know what is an important bug anymore. It affects every aspect of the game, except purely PVP environments.

Getting back to my original question, if ZOS don't do rollbacks, is this because they wouldn't or they couldn't do a rollback? This is a serious issue. If you don't use a version control system in this day and age, that is unheard of! If you do, then it must be cleared to you that this version of the game is ****ed.

If ZOS is able to do rollbacks but they wouldn't do it because it somehow messes with your "crown store calendar", us consumers require you to get your priorities right. No one experiencing these problems should pay you for your terrible support and lack of professionalism.

I hope we can get some explanation on how and why this bug is let rampant after a whole month of release.


edit: Forgot to add, a rollback would also save the increasing amount of people reporting illegible text/UI scaling issues since this week's incremental patch.

It absolutely affects PvP. We've been getting desyncs and misfires like this for years now. The PvE community just didn't notice it or care until now and actually told us to shut up because we were being "whiny and dramatic" when we dared to complain about it. Welcome to our world. It's funny too, because I actually came over to PvE to escape these kinds of bugs, and now it's spread to there too. Even though it's a little late in the game, I'm glad the entire community is (mostly) finally on the same page in terms of demanding appropriate fixes.

I have also experienced PVP bugs with people being desynced from where they are standing, it's not exactly the same problem. In PVP if you're or your enemy is experiencing a lag spike, the server gets unsynced information from both sides, so even if it looks like the animation fired on your screen, the server also has the information that your enemy has moved since you have fired it. So sometimes it will resolve the desync by not counting some of the input, and completely ignoring one of the contradicting input. So it's not exactly a miss in the game sense, but you miss hitting abilities that seemed to connect.

The PVE issue is caused by the game code allowing mobs to move mid-mechanic, where they shouldn't be allowed to move. So on your monitor the enemy is moving, but in reality the enemy is standing where it should have been. So ALL of the damage indicators when this happens are NOT where they seem to be, but elsewhere. Open world you will experience this and have a bad time, but endgame raiding doing trials or arenas, this is entirely game-breaking.

That is not to say, the PVP lag spike desync is not game-breaking, it really is. However there's much less the servers can do to accomodate that. The PVE issue is completely fabricated by the coders, introduced to the game by error trying to fix other visual indicator bugs, and probably other issues.

This might explain what I saw in SCP last patch. I posted something (which of course got completely ignored) in the bugs portion of the forums, but I actually saw the Ice Simulacrums start to walk around the arena and charge at us on the final boss. It happened in two separate instances, and I wasn't the only one to see it.

As far as the PvE bugs, I've seen them all too. HRC final boss shooting his shield out of his butt when the AoE is clearly targeted in the opposite direction. Lothis in vAS spinning his poison cone like a fidget spinner. Players in vSS who see themselves in stack, but are visible to everyone else as being in the back of the arena, usually right in the boss's fire cone AoE. They show they are getting the same heals as the groups and the same buffs, but they also take damage that is going out in the other spot. World shaper mechs slapping people who are clearly not standing in it.

There are other issues too. The poison dmg in vSO feels like it's hitting waaaaay harder than normal. That is purely anecdotal, but I'm not the only one who has noticed (yes, we all had the right CP, no one was a werewolf). Players are STILL being shot up into the air in vAA final boss fight, only to crash back down, die, then bounce off into another outer space free fall. We're STILL getting an overabundance of storm atros spawning on hard mode (8-10 sometimes, even in a really fast burn). Players are STILL spawning in dead into the first split on the right side during vAA because the agro table extends over the spawn point and we're getting overcharged and killed before we load in.

They need some serious work on their content...

ZOS_GinaBruno wrote: »

Thanks for the reports on this, everyone. We have a fix for this issue going into Update 23.

https://forums.elderscrollsonline.com/en/discussion/478583/enemies-teleport-when-killed-making-them-unlootable/p3

Apparently as it is not a game breaking bug there is no rush to get the fix in place.

But not to worry they are working on content for the Witches Festival so Yay

It's pretty much official now that this bug won't be fixed until Update 23. So... virtually an entire patch with a bug as glaringly severe as this.

It's disgraceful, frankly.

I'm also a software dev. I think a bit more nuance is needed here, especially when you're analyzing a project you're not working on.

code65536 wrote: »

A little bit of history...

• 2019-06-03: The 5.0.7 incremental patch introduces a bug that causes enemy positions to become desynced and AoE telegraphs to appear in the wrong location. One of the fixes/changes in 5.0.7 had some sort of unintended side effect.
• 2019-06-10: The 5.0.8 incremental patch makes an attempt to fix this bug. The patch notes read, "Fixed an issue where some monsters that were supposed to be locked in place would instead turn to face your character. This fixes many gameplay issues, particularly with cone and Area of Effect based attacks, most evident in Trials but found throughout the world." Either this fix was not actually included, or it just didn't work, as none of the problems were fixed or improved.
• 2019-06-17: No mention of a fix in the 5.0.9 patch.
• 2019-06-24: No maintenance, no incremental patch.
• 2019-07-01: No mention of a fix in the 5.0.10 patch.

I distinctly remember NPC charge path aoe indicators not reflecting their actual charges when I was leveling three years ago. So, I doubt 5.0.7 directly caused the issue. Instead, I believe the whole game engine architecture plays fast and loose with concurrency, leading to intermittent, difficult to fix bugs. I absolutely can't say for sure that's the case. But multi-core CPUs are popular, per-thread perf is stagnant, and everyone complains about performance. That sounds like a combo that'll back devs into threading things out without respecting the challenge that brings.

I also remember trying to figure out what made it perform so badly (after suffering at 10 fps on vhof twins), and looking at a hot section with lots of lock-prefixed instructions in what looked like spinlocks. When I get some time I'll also look at how often data's being passed between CPU cores. Intel's offcore_response counters can indicate how often a cross-core snoop finds a cache line in modified state, which may or may not be interesting...

Now let's address the other points.

What is infuriating about this process is why the change that caused this bug wasn't simply backed out and reverted in 5.0.9. After the initial attempt at a fix in 5.0.8 didn't work, indicating that the problem was complicated, then you should have immediately reverted the fix from 5.0.7 that caused this bug.

(snip) Reverting the the 5.0.7 change while going back to the drawing board is the standard--and common-sense--way to address this. (snip)

A revert is not easy. There are likely many changes that go out with each release. Identifying a culprit is not straightforward. One line in patch notes could correspond to several changes across several components. A change may also affect database schema, creating a really problematic situation where rolling back would make things worse. At best, players can't properly interact with items they got after the patch (assuming graceful forward/backward compatibility between releases). At worst, you get catastrophic crashes and undefined behavior. That explains their 'roll forwards' approach.

Tacking on to "identifying a culprit is not straightforward", I'm sure you've dealt with concurrency related bugs before - they're little packages of hell where slightly changing the timing of something might make it happen 0.1% of the time, down from 50% or something. That would also explain them saying they fixed an issue, followed by people still hitting it.

Frankly, from my perspective as a software developer, the way that ZOS has handled this bug is appallingly, disgracefully amateurish.

Yeah, we have the luxury of working in places with very well enforced engineering best practices, release pipelines with tons of testing/monitoring that catch bugs before they get close to production, and aren't pushed to release pretty substantial DLCs every quarter or so. And I bet our respective companies have ironed this stuff out over many years of experience.

But that doesn't help ZOS. To fix engineering debt and return to best practices, you need time. With other products, you can buy this time because the demand for new content isn't as urgent. MMOs can get seriously hurt by devastating, game breaking bugs. At the same time, letting content get stale will also lead people to leave, because they don't have anything else to do in the game. So ZOS dug themselves into a very deep hole, and the realities of keeping a MMO relevant prevent them from digging themselves out.

And to salt the wound, today's patch notes contains this precious gem: "Fixed an issue where you were able to swap out abilities during combat."

Instead of fixing a glaring pain point that has been around for a month, you go and fix an issue that nobody is complaining about. An issue that serves to ameliorate the effects of the common stuck-in-combat bugs in this game. What a splendid display of priorities.

With a team, you have a list of bugs/features, and different people work on different bugs/features. A bug that's easy to tackle will likely be fixed much faster than a difficult bug, even if the difficult bug is causing more user pain.

Agree, every incremental patch has seen lag and overall performance get worse ever since the launch of Tameriel one, i know if i had f00kd up for 3 and a half years straight i wouldn't have a job. Which means you can only conclude that the priority is not how the game performs but maybe something else, who knows what it is but Im sure it rhymes with micro-transactions.