Cyrodiil Chronicle: Where did it go wrong?

Pinja

So I think this thread could help a lot if we could identify in patch by patch detail where Cyrodiil got so messed up. I've seen threads were people speculate what's the best patch, but I want to discover the problem patches. A definitive answer to some of Cyrodiil lag issues exists somewhere in the past. A search to find what went wrong.

First up we'll look a this game from console launch from someone playing on a toaster. Xbox Patch v1.0.0.5 or v1.0.0.6
https://www.youtube.com/watch?v=45hZiNJ-tkA
In this primal video every thing is working smooth. Not much is happening in terms of combat because it's a month after launch and everyone's a noob, so there's not an intense server strain. However all the siege is firing live, all the audio is in sequence, and the frame rate is good. (I remember well my surprise the first time my siege went on strike and told me it was busy. As a matter I clipped it from Xbox Patch v1.9.1.0)

Now lets compare it to Fengrush's gameplay on 6/4/2020 PC Patch v6.0.6
Tune to 6:45:00 you can stop or continue watching after the following battle
https://www.twitch.tv/videos/641090158?t=06h44m58s
https://www.twitch.tv/fengrush/clip/ThankfulDifferentParrotWoofer
The latter video is a glitch which hasn't been fixed since the release of Undo, but the prime focus on this video is the repetitious input and queuing of abilities that the commentators are talking about. Not long into the video you see his Orc break free 3 times. In older patches you could break free even after being dead... A much preferred outcome. You also see the new siege take several seconds to complete its firing sequence making it look virtually useless, compared to the siege in the first video.

Next up we have cross platform content from a video posted in 2016. Since the patches would be speculative I will only state it is before the date of the video, Jan 24, 2016.
https://www.youtube.com/watch?v=Fs9vClV3cJE
In this video we see zergs, some small, some big, some zerg on zerg, but we all see good gameplay. Even on the toasters animations are firing, people are quick casting, canceling animations, and doing the works we expect... Around this time I think load screens where an issue but not crashing, a deeper analysis would have to occur as to when chronic crashes started to happen. But don't let that take away how clean cut the gameplay is... Snipe glitch may or may not have existed during these patches.

Now we dive to the evolution of a problem with latency. Around PS4 Patch v1.23
https://www.youtube.com/watch?v=igu2Gl1iyzs
This video shows an issue developed with player positioning updating that still exists today.

Next we have early commentary on a problem that may've compounded with the Harrowstorm patch giving us insane input lag. Around PC Patch v4.2.10 or v4.2.11
https://www.youtube.com/watch?v=JgbaotKCFmQ
No where near as bad as the current patch Nerfmire was named after balance, but did we miss a slight input delay?

In current patches I haven't seen the old dysync were players can be flattened by abilities stacking, but we do see the reverse desync were abilities wont fire. I remember fighting through old lag, I think back in 2017, where you could keep firing abilities and have them register in the worst of it but it'd be delayed.

Pretty much if you have a time stamped video of good, laggy, or bugged Cyrodiil gameplay I'd like to see it, than correlate it to a patch. This will be easy for streamers and anyone who plays on console. If your on Xbox check Game DVR for clips that highlight bugs or good gameplay.

If you know any patch details such as when certain aspects of the game were rotated server side also share.

Pinja

One important thing to note about what this thread exposes is the lag we see now is not a fundamental part of core gameplay. Proven by the earlier variants of the game that work better. Meaning block changes were unnecessary, roll dodge changes were unnecessary, braking the combat system is also unnecessary.

geonsocal

as soon as the decision was made to *** with the ayleid wells - gameplay was headed to oblivion...

Kadoin

Honestly, performance really tanked in Homestead for me.

At least server-side.

It was also around then keep walls started bugging all the time, terrain bugs destroying eyeballs, keep texture reloading over and over, loading screens, etc.

What a coincidence, right? Right?

VaranisArano

One major thing that went wrong is that players decided to abuse ZOS' client-side functions in order to cheat. CE, meteor spam, all that.

ZOS had to move functions to server side, which had a pretty severe impact on performance. This wasn't Cyrodiil's original design.

People. We're why we can't have nice things.

KoultouraS

VaranisArano wrote: »

One major thing that went wrong is that players decided to abuse ZOS' client-side functions in order to cheat. CE, meteor spam, all that.

ZOS had to move functions to server side, which had a pretty severe impact on performance. This wasn't Cyrodiil's original design.

People. We're why we can't have nice things.

Cheating is a plague of modern online gaming.
An unfortunate standard of our days.
No matter what, cheaters always seem to be one step ahead.
However, no matter what, ZOS is not the 1st to deal with such a problem nor will it be the last.
We have seen better solutions by companies with far less the assets ZOS has, or at least solutions that haven't had rendered one vital mode of the game unplayable.

And to be honest I really don't buy this "anti-cheating measures kills PvP performance"
cause lately we have been obviously having terrible performance migrating to PvE as well!
So what is the excuse for vMA being so fkn laggy lately?
How come can A SOLO play be so laggy/buggy?
Cheaters?

stefan.gustavsonb16_ESO

The "bad patch" you are looking for is 1.2.

The initial design for ESO was to have essential computations regarding movement, collision and combat performed client-side. This was very efficient and made for smooth gameplay with hundreds of players in the same area, and this is what we saw in the first few months after release. However, that design was also very vulnerable to exploits, I would say to the point of being stupidly naive. If there is a way to cheat, there will be cheaters. It's human nature, sadly. Because cheating was so easy, it got out of hand. It started with teleporting farming bots zapping between nodes while staying hidden under ground, and before long we had invulnerable flying players spamming meteors in Cyrodiil. The cheating got to the point where it was wide-spread and obvious, and threatened to completely ruin the game.

In the infamous "interior lighting patch" 1.2, the lighting changes as such were really of very little consequence. In that patch, they also made far-reaching changes to the client-server protocol and moved most decisions on movement and combat to the server side. Instead of having some checks being applied here and there to catch cheaters, and putting a special watch on suspicious clients, they just moved everything wholesale to the server. While this did its intended job of closing the gaping security holes, it put a lot more stress on the servers, both in terms of computational workload and network traffic. The infrastructure was not designed to handle this, performance tanked as a result, and it has never recovered.

In the following patches, they applied some insufficient band-aids to the problem, like removing friendly wildlife in Cyrodiil, but their main remedy was to lower the population caps for campaigns. How large they were initially and how large they are now is not public information, but it's very clear that the population cap was large at launch, and now it's quite small.

Izanagi.Xiiib16_ESO

KoultouraS wrote: »

VaranisArano wrote: »

One major thing that went wrong is that players decided to abuse ZOS' client-side functions in order to cheat. CE, meteor spam, all that.

ZOS had to move functions to server side, which had a pretty severe impact on performance. This wasn't Cyrodiil's original design.

People. We're why we can't have nice things.

Cheating is a plague of modern online gaming.
An unfortunate standard of our days.
No matter what, cheaters always seem to be one step ahead.
However, no matter what, ZOS is not the 1st to deal with such a problem nor will it be the last.
We have seen better solutions by companies with far less the assets ZOS has, or at least solutions that haven't had rendered one vital mode of the game unplayable.

And to be honest I really don't buy this "anti-cheating measures kills PvP performance"
cause lately we have been obviously having terrible performance migrating to PvE as well!
So what is the excuse for vMA being so fkn laggy lately?
How come can A SOLO play be so laggy/buggy?
Cheaters?

People were using cheat engine in PVE too. Especially score pushing in VMA. I'm sure ZoS has tried to combat some of it further with the public showing of these problems last year again and this year too which is why it might have gotten worse.

Pinja

VaranisArano wrote: »

One major thing that went wrong is that players decided to abuse ZOS' client-side functions in order to cheat. CE, meteor spam, all that.

ZOS had to move functions to server side, which had a pretty severe impact on performance. This wasn't Cyrodiil's original design.

People. We're why we can't have nice things.

I've heard this quite the bit. But one thing with console is you can't mod it so we never had a problem with flying Meteor spams.
Rather then move anything server side they should have adopted software integrity patches and practices used by Microsoft. Simple concept of it is if you change the software in any way you can't access online. If they were to install a client check that looks for foreign batch files or modified batch files every time you loaded and updated the game you wouldn't have to worry about most hackers. Sure you'd have to register and account for acceptable add-ons, but if it means you get to get things working do it.
The solution to Cyrodiil's lag may be simple, practical, convenient, and cheap. As the game loads batch files on start up have it broadcast the name of the files to the server making it so you can't start/load the game unless your already connected to the server. This may require updating the launcher. You shouldn't need to require players be connected to the server to start the launcher only the game.

Pinja

It would appear the exploit was happening right before dark brother hood after Patch v2.3.11
(Patch coincidence of exploit, not actually a cause)
https://www.youtube.com/watch?v=KbEtN24VS4E

There's another video that shows the glitch, and despite it raining meteors there's no lag just a bit of graphic frameyness.

Like in the above post if ESO implements software checks instead of server side checks it may make things smoother.