Seriously the lag is ridiculous now

Volrion

Daviiid_ESO wrote: »

works fine in housing and roleplay, stop complaining :DDDDDDDDD

lol

Joy_Division

This is the only game in which I have upgraded my computer, software, and ISP and yet the performance has deteriorated.

Pinja

Joy_Division wrote: »

This is the only game in which I have upgraded my computer, software, and ISP and yet the performance has deteriorated.

I still really don't see why they feel it's the combat team's job to fix performance.
What are we to do about it, noting. What's the combat team to do with it, @idk ...

idk

Pinja wrote: »

Joy_Division wrote: »

This is the only game in which I have upgraded my computer, software, and ISP and yet the performance has deteriorated.

I still really don't see why they feel it's the combat team's job to fix performance.
What are we to do about it, noting. What's the combat team to do with it, ...

Not sure why I was tagged on asking what the combat team is to do about the lag. It really is not the combat team but the overall management of the game.

Joy's statement is correct. This is the only game I have played where performance continues to get worse outside of the initial launch of this game. Matt Firor tries to claim they are "killing it" as though they are actually making improvements. The decay of the game's performance we have experienced speaks loudly to the failures and poor decisions management of this game have made over the years and the only thing getting killed by Zos is the performance.

But no, it is not the fault of the combat team. The issues leading to the decay in performance are managed by different people and the management of Zos such as the game's director. In reality, all major issues lay at the feet of the game's director as they have the final say in every major aspect of this game.

Pinja

idk wrote: »

Pinja wrote: »

Joy_Division wrote: »

This is the only game in which I have upgraded my computer, software, and ISP and yet the performance has deteriorated.

I still really don't see why they feel it's the combat team's job to fix performance.
What are we to do about it, noting. What's the combat team to do with it, ...

Not sure why I was tagged on asking what the combat team is to do about the lag. It really is not the combat team but the overall management of the game.

Joy's statement is correct. This is the only game I have played where performance continues to get worse outside of the initial launch of this game. Matt Firor tries to claim they are "killing it" as though they are actually making improvements. The decay of the game's performance we have experienced speaks loudly to the failures and poor decisions management of this game have made over the years and the only thing getting killed by Zos is the performance.

But no, it is not the fault of the combat team. The issues leading to the decay in performance are managed by different people and the management of Zos such as the game's director. In reality, all major issues lay at the feet of the game's director as they have the final say in every major aspect of this game.

Thank you for your insights, idk. You usually have a good perception of what's what.
Yes, it looks like it's a mess everywhere, and the game director is responsible for cleaning it up. I am glad they added weekly testing as their pass to a solution. However, not only did they only now just start to really try and fix things late into the year, but after all these years they've yet to pinpoint the problem. I'm watching them experiment with nerfs, instead of performance. Like why not try different exchange patterns between clients and servers? It's like I watching them play the excavation game and they used they eye, but started digging in the red. I don't want to say it's a gimmick, but they're probably trying the cheapest solution first.
Like from what I seen over the years, cross examining content, while an AoE may be the most complex server calculation. It's not the root of the problem. I was a purge spammer when it hit everyone, game ran fine 20-50 people getting hit with purge on a breach, and this was on original xbox one.
Maybe there's problems client side with memory leaks, there was a thread around not too long ago showing the usage statistics of the game and they where high.
People lag where ball groups aren't around. Empty campaigns, IC, battlegrounds. Different people have different types of lag. Fengrush on his moderately high end PC, feels lag when he's targeted by multiple opponents. Some people have a bad frame rate and get hit with a load screens incrementally.

I guess AoEs a start, but it could be a month or months wasted.

Now, I know plans can change. Some say that's a reason Devs don't post much, but what happened to this:

Behind-the-scenes Combat Ability Improvements Phase 3

AoE Ability Performance on the Server: This work focuses on making these ability types more specialized so that they are more efficient on server performance. The code work is complete, and data work has been shifted to Update 27. The current focus is on tracking down current issues on the live servers. Note: The overall player experience for these abilities (damage/functionality) should not change.

Maybe it's a hardware problem. I have an old PC that I occasionally had to bang while it was running it because the processor or something was misaligned. It would make whole bunch of noise, not from the fan but from the circuitry going haywire. And would freeze, until you hit it. Maybe it was a short circuit from a dust build up I'm not really sure. Either way they should make a check list and run it.

Elsonso

idk wrote: »

But no, it is not the fault of the combat team. The issues leading to the decay in performance are managed by different people and the management of Zos such as the game's director. In reality, all major issues lay at the feet of the game's director as they have the final say in every major aspect of this game.

While it might not be the fault of the combat team, up through the various directors and producers responsible for the combat team, they certainly did not help the situation. If you drill down on what they said, a lot of what they admitted was causing problems was because they made game play decisions, mainly combat and itemization, that contributed to the problem.

If they blame too much sustain, then it is them that made that possible. If they blame too much AoE, then it is them that made that possible. If they blame too many people in a tight area, with too much sustain, doing too much AoE, then it is them that made that possible.

While I agree that, at the core, it is not the combat team, they certainly drove the party bus to where we are today.

Pinja wrote: »

Maybe it's a hardware problem.

Lambert pretty much laid it bare, assuming he used proper language to describe it.

"we reached a point where casting so many continuous AOE abilities in such a small area started to overwhelm the server process for that area, leading to situations where the "lag meter" spikes and the server becomes unresponsive for a period of time."

In a nutshell, the server process that handles this combat was not designed for the load that it is being asked to handle. It is simply failing to perform as it needs to, and that failure cascades until it shows up as lag, desync, and whatever else.

Sadly, if that is the case, this is not a "moar hardware" fix. They could fill the room to the rafters with server hardware and the only change people would notice is that the lights would dim in Frankfurt when they powered it up.

They could use faster CPUs, but this is a dead end road. Here, we hit the limit of how much performance a company like Intel can put into a "server process". It is easily possible that they are already at that limit.

There are only two basic solution paths, and neither of them are hardware.

1. Reduce the demand being placed on the server process. This means less combat calculation over the same of time. It is where they have spent most of their time. This is where the "3 second cooldown" comes in and why they are thinking about applying it to more than just a single person. Personally, I like the idea of eliminating stacking of similar effects, or at least limiting how many can be stacked until the server process can handle it better.

2. Re-architect the combat calculation process from the ground up to increase the load that it can handle and make it more resilient against failure due to high demand. I feel that this is the core solution, but if they have taken any steps down this path, they have not talked about it enough.

Pinja

Elsonso wrote: »

idk wrote: »

But no, it is not the fault of the combat team. The issues leading to the decay in performance are managed by different people and the management of Zos such as the game's director. In reality, all major issues lay at the feet of the game's director as they have the final say in every major aspect of this game.

While it might not be the fault of the combat team, up through the various directors and producers responsible for the combat team, they certainly did not help the situation. If you drill down on what they said, a lot of what they admitted was causing problems was because they made game play decisions, mainly combat and itemization, that contributed to the problem.

If they blame too much sustain, then it is them that made that possible. If they blame too much AoE, then it is them that made that possible. If they blame too many people in a tight area, with too much sustain, doing too much AoE, then it is them that made that possible.

While I agree that, at the core, it is not the combat team, they certainly drove the party bus to where we are today.

Pinja wrote: »

Maybe it's a hardware problem.

Lambert pretty much laid it bare, assuming he used proper language to describe it.

"we reached a point where casting so many continuous AOE abilities in such a small area started to overwhelm the server process for that area, leading to situations where the "lag meter" spikes and the server becomes unresponsive for a period of time."

In a nutshell, the server process that handles this combat was not designed for the load that it is being asked to handle. It is simply failing to perform as it needs to, and that failure cascades until it shows up as lag, desync, and whatever else.

Sadly, if that is the case, this is not a "moar hardware" fix. They could fill the room to the rafters with server hardware and the only change people would notice is that the lights would dim in Frankfurt when they powered it up.

They could use faster CPUs, but this is a dead end road. Here, we hit the limit of how much performance a company like Intel can put into a "server process". It is easily possible that they are already at that limit.

There are only two basic solution paths, and neither of them are hardware.

1. Reduce the demand being placed on the server process. This means less combat calculation over the same of time. It is where they have spent most of their time. This is where the "3 second cooldown" comes in and why they are thinking about applying it to more than just a single person. Personally, I like the idea of eliminating stacking of similar effects, or at least limiting how many can be stacked until the server process can handle it better.

2. Re-architect the combat calculation process from the ground up to increase the load that it can handle and make it more resilient against failure due to high demand. I feel that this is the core solution, but if they have taken any steps down this path, they have not talked about it enough.

Good answer.
It is true I'm a "moar hardware" guy, but that solution isn't necessarily a better hardware solution. The more hardware solution doesn't test the limits of hardware and Intel. What is does is disperse server load so that one server has to handle less AoEs and calculations. That way you won't see a spike. Unless your going to tell me Glademist grounds and Fort Glademist zones are on two different servers (unlikely as there would have to be a combat transfer between servers,) then the partition Cyrodiil idea will work. Take into account that although IC does lag lower level clients because of all the renderings, it probably has the best server performance out of battlegrounds and Cyrodiil. It has low population on an unstrained server. I'm not going to state that as a sure fact, as I have seen videos from out of IC with positioning lag. But positioning lag, can sometimes be a clients delayed internet, not necessarily pinpointed at the servers. Which is why I asked a few important questions in another thread.

Now I put in bold a point of yours that I want to talk about. It goes along with Re-architecting the servers calculation process. If what you are saying about the servers is that even a low level amount of AoEs and abilities are causing the servers to go haywire, then that is a problem "moar hardware" wont fix. However, that is a fundamental problem that needs to addressed directly through recoding the servers. AoE cooldowns may be a bandaid, for a much bigger, however simpler problem. It's like trying to put a giant cork in a braking Dam, it's still going to brake if abilities are incompatible with the servers code. Now I was going to say that not even the AoE cooldown thing would work as your essentially doing the same thing as "moar hardware," and just trying to lower server load. But you do, do something a little different, and no lie I may have pinpointed the problem without even looking at the code. There's a corruption stack around an array or variable. It's an error I found while coding my last project for a class and it makes a lot of sense. Pretty much the servers have an overlooked and undesigned que for abilities. A variable is only supposed to store one set of data. However their is no hardwire for this, and they can become corrupted if programed incorrectly. The result usually is that they don't work. I'm thinking more so thinking it's a corruption around an array or string. Where someone tried to set the null character (\0) as the escape/sentinel character for filling the array. I learned that when you fill an array you can over wright this character if you over populate the entry to a string all at once. This results in an overpopulated array, but unlike a variable when corrupted it still runs.
Enough of the background info. What this means and why I think this applies, is because the server is designed to store and execute transmitted abilities, but was is purposefully or contingently designed to que them? And is the array for the que big enough for all the back log? The thing is with an array, it still will execute all the unassigned data that exceeds the size of the array, depending on the escape character for the loop that runs it. Worse yet all the unassigned data will stick around after execution unless a similar loop structure that wrote it clears it. But this will cause spikes in memory while it's there as you have rouge elements. I'm not sure how they tried to fix macro slicing but chances are they didn't fix the core of it, which was probably an array overstacking.

They might be able to fix some problems by designing the server side current ability string, to hold a massive amount of abilities so it doesn't get a corruption stack.
Interestingly enough I didn't get an error in visual studios till after I exited the program.

Why I say the AoE cooldown might be a bandaid is because it lowers the chance of abilities stacking in an array of abilities waiting to be executed. However, my skepticism doesn't fully go away as any chain of abilities can stack.

I say it again here though, do you really want to fix the five second skill delay, by adding a five second skill delay?

Radiance

It seems to get increasingly worse when partaking in Greymoor content in addition to the multitude of bugs I've encountered...

Elsonso

Pinja wrote: »

Good answer.
It is true I'm a "moar hardware" guy, but that solution isn't necessarily a better hardware solution. The more hardware solution doesn't test the limits of hardware and Intel. What is does is disperse server load so that one server has to handle less AoEs and calculations. That way you won't see a spike. Unless your going to tell me Glademist grounds and Fort Glademist zones are on two different servers (unlikely as there would have to be a combat transfer between servers,) then the partition Cyrodiil idea will work. Take into account that although IC does lag lower level clients because of all the renderings, it probably has the best server performance out of battlegrounds and Cyrodiil. It has low population on an unstrained server. I'm not going to state that as a sure fact, as I have seen videos from out of IC with positioning lag. But positioning lag, can sometimes be a clients delayed internet, not necessarily pinpointed at the servers. Which is why I asked a few important questions in another thread.

It is already known that they "partition" Cyrodiil. That was a topic of discussion around the time they explained why they needed to remove all the deer, and other related Cyrodiil things, a few years back. My thinking is that this is what the "server process" is, and there are more than one of them. This concept has probably been around since launch. It is also known that Cyrodiil is broken into many "cells", since they made reference to this already, and these "server processes" probably cover a series of contiguous cells.

Pinja wrote: »

Now I put in bold a point of yours that I want to talk about. It goes along with Re-architecting the servers calculation process. If what you are saying about the servers is that even a low level amount of AoEs and abilities are causing the servers to go haywire, then that is a problem "moar hardware" wont fix. However, that is a fundamental problem that needs to addressed directly through recoding the servers. AoE cooldowns may be a bandaid, for a much bigger, however simpler problem. It's like trying to put a giant cork in a braking Dam, it's still going to brake if abilities are incompatible with the servers code. Now I was going to say that not even the AoE cooldown thing would work as your essentially doing the same thing as "moar hardware," and just trying to lower server load. But you do, do something a little different, and no lie I may have pinpointed the problem without even looking at the code. There's a corruption stack around an array or variable. It's an error I found while coding my last project for a class and it makes a lot of sense. Pretty much the servers have an overlooked and undesigned que for abilities. A variable is only supposed to store one set of data. However their is no hardwire for this, and they can become corrupted if programed incorrectly. The result usually is that they don't work. I'm thinking more so thinking it's a corruption around an array or string. Where someone tried to set the null character (\0) as the escape/sentinel character for filling the array. I learned that when you fill an array you can over wright this character if you over populate the entry to a string all at once. This results in an overpopulated array, but unlike a variable when corrupted it still runs.
Enough of the background info. What this means and why I think this applies, is because the server is designed to store and execute transmitted abilities, but was is purposefully or contingently designed to que them? And is the array for the que big enough for all the back log? The thing is with an array, it still will execute all the unassigned data that exceeds the size of the array, depending on the escape character for the loop that runs it. Worse yet all the unassigned data will stick around after execution unless a similar loop structure that wrote it clears it. But this will cause spikes in memory while it's there as you have rouge elements. I'm not sure how they tried to fix macro slicing but chances are they didn't fix the core of it, which was probably an array overstacking.

They might be able to fix some problems by designing the server side current ability string, to hold a massive amount of abilities so it doesn't get a corruption stack.
Interestingly enough I didn't get an error in visual studios till after I exited the program.

Why I say the AoE cooldown might be a bandaid is because it lowers the chance of abilities stacking in an array of abilities waiting to be executed. However, my skepticism doesn't fully go away as any chain of abilities can stack.

I say it again here though, do you really want to fix the five second skill delay, by adding a five second skill delay?

It is impossible to tell, without looking at the code, how to fix it. One thing that I am pretty certain about is that they have looked at the code in quite a bit of detail. Could there be a bug? Can't rule it out. I don't think that is the answer, though. This is simply situation that is inherent in the design, probably through a decision that was made a decade ago.

Reducing the server load is equivalent to "moar hardware". It suffers from the same fatal flaw as "moar hardware". As soon as the server process load gets too high, the problem returns. The symptoms are being managed, not the disease. Maybe that is enough, though. It isn't like this game is going to be actively developed by ZOS a decade from now.

Vapirko

Y'all are still barking up this tree?

Pinja

Vapirko wrote: »

Y'all are still barking up this tree?

Yes

Elsonso wrote: »

Pinja wrote: »

Good answer.
It is true I'm a "moar hardware" guy, but that solution isn't necessarily a better hardware solution. The more hardware solution doesn't test the limits of hardware and Intel. What is does is disperse server load so that one server has to handle less AoEs and calculations. That way you won't see a spike. Unless your going to tell me Glademist grounds and Fort Glademist zones are on two different servers (unlikely as there would have to be a combat transfer between servers,) then the partition Cyrodiil idea will work. Take into account that although IC does lag lower level clients because of all the renderings, it probably has the best server performance out of battlegrounds and Cyrodiil. It has low population on an unstrained server. I'm not going to state that as a sure fact, as I have seen videos from out of IC with positioning lag. But positioning lag, can sometimes be a clients delayed internet, not necessarily pinpointed at the servers. Which is why I asked a few important questions in another thread.

It is already known that they "partition" Cyrodiil. That was a topic of discussion around the time they explained why they needed to remove all the deer, and other related Cyrodiil things, a few years back. My thinking is that this is what the "server process" is, and there are more than one of them. This concept has probably been around since launch. It is also known that Cyrodiil is broken into many "cells", since they made reference to this already, and these "server processes" probably cover a series of contiguous cells.

Pinja wrote: »

Now I put in bold a point of yours that I want to talk about. It goes along with Re-architecting the servers calculation process. If what you are saying about the servers is that even a low level amount of AoEs and abilities are causing the servers to go haywire, then that is a problem "moar hardware" wont fix. However, that is a fundamental problem that needs to addressed directly through recoding the servers. AoE cooldowns may be a bandaid, for a much bigger, however simpler problem. It's like trying to put a giant cork in a braking Dam, it's still going to brake if abilities are incompatible with the servers code. Now I was going to say that not even the AoE cooldown thing would work as your essentially doing the same thing as "moar hardware," and just trying to lower server load. But you do, do something a little different, and no lie I may have pinpointed the problem without even looking at the code. There's a corruption stack around an array or variable. It's an error I found while coding my last project for a class and it makes a lot of sense. Pretty much the servers have an overlooked and undesigned que for abilities. A variable is only supposed to store one set of data. However their is no hardwire for this, and they can become corrupted if programed incorrectly. The result usually is that they don't work. I'm thinking more so thinking it's a corruption around an array or string. Where someone tried to set the null character (\0) as the escape/sentinel character for filling the array. I learned that when you fill an array you can over wright this character if you over populate the entry to a string all at once. This results in an overpopulated array, but unlike a variable when corrupted it still runs.
Enough of the background info. What this means and why I think this applies, is because the server is designed to store and execute transmitted abilities, but was is purposefully or contingently designed to que them? And is the array for the que big enough for all the back log? The thing is with an array, it still will execute all the unassigned data that exceeds the size of the array, depending on the escape character for the loop that runs it. Worse yet all the unassigned data will stick around after execution unless a similar loop structure that wrote it clears it. But this will cause spikes in memory while it's there as you have rouge elements. I'm not sure how they tried to fix macro slicing but chances are they didn't fix the core of it, which was probably an array overstacking.

They might be able to fix some problems by designing the server side current ability string, to hold a massive amount of abilities so it doesn't get a corruption stack.
Interestingly enough I didn't get an error in visual studios till after I exited the program.

Why I say the AoE cooldown might be a bandaid is because it lowers the chance of abilities stacking in an array of abilities waiting to be executed. However, my skepticism doesn't fully go away as any chain of abilities can stack.

I say it again here though, do you really want to fix the five second skill delay, by adding a five second skill delay?

It is impossible to tell, without looking at the code, how to fix it. One thing that I am pretty certain about is that they have looked at the code in quite a bit of detail. Could there be a bug? Can't rule it out. I don't think that is the answer, though. This is simply situation that is inherent in the design, probably through a decision that was made a decade ago.

Reducing the server load is equivalent to "moar hardware". It suffers from the same fatal flaw as "moar hardware". As soon as the server process load gets too high, the problem returns. The symptoms are being managed, not the disease. Maybe that is enough, though. It isn't like this game is going to be actively developed by ZOS a decade from now.

Honestly that's not much for me to go on here. Your going to turn me into a member of the mages guild going back through ancient text on the forums, just to try and guess and interpret the architecture. Let's discuss though.
I would think that the larger maps are divided into smaller regions, "cells," for client rendering purposes.
Speaking hypothetically...
In order for combat to work continuously with multiple "server cells" on the same map as players and combat moves across them. You'd need some sort of "server process" that designated which server "cell" handled an instance of combat, (with a cell being it's own server and independent CPU in order for it to be the equivalent of the partition idea.) This creates problems as you could have people standing next to each other with combat on different servers and when they engage each other, they'd swap server "cell" designations in the middle of combat. This would create it's own lag, and maybe even mistiming of buffs as all that active information of what buff's are up would have to be passed between "cells." Fundamental problems would be a doubling up or disappearing of buffs as they may fail to expire or pass between server "cells."
Because I haven't really noticed combat lag between map regions, as a server process would naturally have passing combat between "cells," I not fully sure what your saying exists. Like there would be a whole world of bugs and weirdness, if these cells where separate servers. For instance if the server process that managed designation lags or fails you get two separate results for combat returned to a client.
Exiting the hypothetical.
To add to the confusion, there's a lot of terminology like megaserver which is made of smaller servers with their own instances. These look fairly strait forward each instance of a zone is it's own server. Whenever you cross one you get a long load screen as the server loads data.
What further contradicts Cyrodiil already being partitioned between multiple servers, is the fact that a 3v2v1 at Vlastarus still lags. If it were to already be on it's own server, something is terrible, terrible wrong.

As far as that array stacking thing goes, I'm just throwing it out there because the code could work and look right but still contain that fundamental glitch.

As far as curing the disease(s), we have to make is plural. There's a multitude of lag at different levels that needs to be sourced and identified. A good portion of the people that follow the topic say it was moving certain process server side. I say it's gotten worse since than and that, that is not the only issue. I blame a theoretical concept called the reverse dysync. The masses as you know say just upgrade the servers, with speculation of better resource management and tech during Mid year Mayham. I can't confirm that it got better during MYM, not to say that it didn't. For whatever reason the Devs look at what was once a world famous combat system, and say because it's a fundamental part of the game, it's a fundamental part of the problem. Problem is it's a fundamental part of the game and it would be terrible to brake it. Not to mention performance is down hill, not an even slope of the same slop that's been hear since launch. Sustain was better pre-Morrowind. So what they say about changes in player behavior doesn't fully equate. Especially back when abilities and set's weren't group restricted.

There's also was the issue with crashing people out of keeps, but that was mainly a client issue.

I think they need to contemplate the potential to trace and replace certain band aids that had negative repercussions on performance.

The process is soo slow though. It's really important to pinpoint the right subject area. Where are we going to be one year from now? I wouldn't want to see the game in a state where players are still lagging and the Devs fumbling trying to rebalance abilities instead of working on performance.
It would be frustrating to say the least.

EdmondDontes

Joy_Division wrote: »

This is the only game in which I have upgraded my computer, software, and ISP and yet the performance has deteriorated.

Same.

Volrion

I feel for you guys, I really do. But this game is done. It's been 6 years and performance has been garbage the whole time. We all know what this game could be, but they blew it.

ESO is a single player game now.

Palidon

Volrion wrote: »

I feel for you guys, I really do. But this game is done. It's been 6 years and performance has been garbage the whole time. We all know what this game could be, but they blew it.

ESO is a single player game now.

I agree. NoCP 8/22/20 was completely unplayable with the massive lag. Playing on NA server. The lag in Cyrodiil is pathetic and ZOS' has continually ignored the problem for years.

Naglaami

The game is not playable on ps4 Europe server anymore.. You have to hit the buttons 4-5 times before maybe something happens. And I have no problems with my internet 100Mbit wired (to the ps4) connection since every other online game works just flawless. Come on ZOS you have to fix this. I can't play the game....

vamp_emily

Just imagine what this game would be like if they fixed it.

I know in the under level 50 BwB days we had a blast with very little lag.

Lag = I don't want to play
No Lag = fun