@Winnamine These are not solutions at all. They haven't said these are solutions, they have said this is a data gathering exercise so that they can formulate solutions.
Part of what they are doing is seeing what the players can afford to ignore which won't help at all.
A great example is what is players literally just stack into MOAR REGEN to keep their AOE spam. Then the cost increase is pointless and won't help at all.
I imagine stage 1 is literally to enforce less aoe's and to see if performance is "fixed". It's the sledge hammer to confirm the hypothesis and to set a benchmark for things working properly.
After we complete the above tests, we may try other combinations of cooldown, cost, and regen values on AOE abilities. However, we need to run these tests first and then assess the data. We will then let everyone know what we found and how we will move forward. We will be very upfront, but please be aware that if these tests confirm our hypothesis, then chain-casting AOE abilities will no longer form the core of the ESO PvP experience in the way it has for the last few years. We would then go through each class and ensure that there are viable builds for each and make adjustments as necessary.
ZOS_RichLambert wrote: »Test 1... For example, as a Templar, if I cast Ritual of Retribution, I wouldn’t be able to cast Puncturing Sweep for 3 seconds...
Test 2... For example – as a Templar, if I cast Puncturing Sweep, I wouldn’t be able to cast Puncturing Sweep again for 3 seconds.
Test 3... Similar to how streak or roll dodge works, where when you cast an AOE you receive a debuff for 5 seconds, each stack of the debuff increases the cost of any AOE cast.
...
After we complete the above tests, we may try other combinations of cooldown, cost, and regen values on AOE abilities... please be aware that if these tests confirm our hypothesis, then chain-casting AOE abilities will no longer form the core of the ESO PvP experience in the way it has for the last few years. We would then go through each class and ensure that there are viable builds for each and make adjustments as necessary.
Dottzgaming wrote: »This unfortunately will probably get a "positive" performance result. Combat is going to slow down to a grinding hault, so performance will improve since people will be playing the game at a significantly slower speed.
ESO's hallmark feature is it's combat speed and fluidity, especially for PvP. Cooldowns shouldn't be introduced into a system that was fundamentally designed to not have them. Period.
it's not working, bombblades here for a decade and the only thing they achieved - records of AP gain.itscompton wrote: »If the real issue is people stacking in ball groups which allows them to endlessly spam AOE's why not just go all in on ball group busting sets/abilities to strongly discourage that type of play? The buff to VD is a nice start but how about increasing the damage and range of Proxy/Inevitable Det explosions significantly? Put a long cooldown and or incredibly large cost increase on consecutive casting to keep it from being abused while making it a strong enough burst to truly wipe a ball group.
rprice1819_ESO wrote: »Why not make it a turn based game and eliminate all server side lag. Each player can have up to an hour to decide what ability to use and where, then after the hour all abilities go off. Everyone can use up to 24 abilities per day
rprice1819_ESO wrote: »Why not make it a turn based game and eliminate all server side lag. Each player can have up to an hour to decide what ability to use and where, then after the hour all abilities go off. Everyone can use up to 24 abilities per day
Can we make it text based?
> go west
> siege keep
> cast jabs
ZOS_RichLambert wrote: »As part of our ongoing performance optimizations, we have been spending a lot of time analyzing and addressing problems in Cyrodiil. I'm going to give a summary of what has happened over the years, what we have been doing to alleviate the problems, and our future plans.
When talking about ESO server performance, it is important to understand that all abilities in the game (with some exceptions) have a "soft" limit on the number of times they can be executed. An example of a "hard" limit on an ability is an ability that can be executed one time every two seconds, enforced by the server. ESO doesn't have many of those. Instead, we designed the game to have soft limits, which mean you can execute as many abilities as you want as long as you have the resources - Stamina or Magicka - to cast them. There is a global "cool down" timer on all abilities, which is set to 1000 milliseconds.
The design goal of soft limits in any game is to allow players to create builds that let them execute abilities more often depending on build choices made, and not to have hard ceilings on damage or healing per second. This gives a lot of control to the player, which makes for a really fun and interesting system, but it can lead to situations where players cast too many abilities too quickly (and continuously) if strict limits on resources are not enforced.
Over the years, player power has grown considerably. With the addition of the Champion System, various armor/weapons sets, and changes to abilities, we have reached the stage where players - with the right build - can cast near-infinite numbers of abilities. If you add in a properly managed group, with some focused on damage and some focused on healing and regen boosts, you have a perpetually running never-ending stream of abilities.
With that in mind, consider how Area of Effect (AOE) abilities work: when cast, they look at a specific target area - almost always the area directly around the caster - find targets, and perform the ability. Each of those steps requires server calculations.
At launch, Cyrodiil’s processes were able to keep up with the number of AOEs cast, because most players couldn’t cast that many of them: they ran out of Magicka or Stamina, so they had to use AOEs judiciously. Over time, as player knowledge grew and regen builds grew in power, more players could cast more and more AOEs before running out of resources.
During these years, we found and fixed many performance issues in Cyrodiil, and we continue to do so. We've fixed significant issues, uncovered more, and continue to find and fix with every Update.
However, one foundational issue remains. At some point, we crossed a threshold where most players in PvP were able to cast endless AOE abilities, without ever running out of resources. This is done through player knowledge, builds and group mechanics – resulting in a constant stream of AOEs with many players never using any other type of ability.
This is not what we intended, but part of the fun of Elder Scrolls games is designing a build that has unexpected and powerful results, and we allowed it. However, as this behavior grew more prevalent, we reached a point where casting so many continuous AOE abilities in such a small area started to overwhelm the server process for that area, leading to situations where the "lag meter" spikes and the server becomes unresponsive for a period of time.
Our initial response to this problem – starting with Update 22 – was to find (and fix) more than a few problems with AOE ability calculations and make them more performant - and we stayed (mostly) ahead of the problem. But as more players reached maximum Champion Level and more players started utilizing this particular method of "AOE spamming", we have reached the point where we cannot fix this issue around the edges: we need to address the core problem, which we will be starting with Update 27.
To do this, we will need to first do some analysis, and we can only do this on the live servers. As much as we try to avoid running tests on live servers, they are the only place where the combination of player behavior, specific builds, and mass battles happen. So, starting the week of August 24, we will be running a series of tests on the live PC servers – both NA & EU, only in Cyrodiil. Please note that we will not be running these tests on any Console servers. Each test should take about a week, but if needed, we will extend the testing time.
The first round of tests we are planning will focus on Area of Effect (AOE) abilities in Cyrodiil and will make it more difficult for AOE abilities to be the only abilities used, adjusting cooldown, cost and regen values of all AOEs (damage and healing).
Specific details on the tests we will be running in Cyrodiil are as follows:During the times that any of these tests are active, we will be awarding double Alliance Points for anyone active in Cyrodiil.
- Test 1 – Shared global AOE cooldown - 3 second timer. This test adds a global 3 second shared cooldown to any AOE ability. This means that when you cast an AOE, you will not be able to cast another for 3 seconds. For example, as a Templar, if I cast Ritual of Retribution, I wouldn’t be able to cast Puncturing Sweep for 3 seconds.
- Test 2 – Individual AOE cooldown - 3 second timer. This test adds an individual AOE cooldown to each AOE ability. This means that when you cast an AOE, you will not be able to cast that same AOE ability for 3 seconds. For example – as a Templar, if I cast Puncturing Sweep, I wouldn’t be able to cast Puncturing Sweep again for 3 seconds.
- Test 3 – No cooldown, global ramping AOE cost. This test adds a global ramping AOE cost for each AOE cast. Similar to how streak or roll dodge works, where when you cast an AOE you receive a debuff for 5 seconds, each stack of the debuff increases the cost of any AOE cast.
- Test 4 – Individual AOE cooldown – 3 second timer, global ramping AOE cost. This test adds an individual AOE cooldown to each ability as in test 2, but also combines that with a global ramping cost from test 3.
After we complete the above tests, we may try other combinations of cooldown, cost, and regen values on AOE abilities. However, we need to run these tests first and then assess the data. We will then let everyone know what we found and how we will move forward. We will be very upfront, but please be aware that if these tests confirm our hypothesis, then chain-casting AOE abilities will no longer form the core of the ESO PvP experience in the way it has for the last few years. We would then go through each class and ensure that there are viable builds for each and make adjustments as necessary.
The code for these changes will be going into today’s PTS patch, and we will be running some basic tests throughout the day on Tuesday in Cyrodiil to ensure we’re able to make the above changes without requiring any maintenance or downtime. Once we launch Update 27 for PC on August 24, we will announce when one of the tests are beginning through an in-game announcement and will have a forum thread detailing which test is currently being performed. Remember, these tests will be limited to Cyrodiil, so AOE abilities in other PVP spaces (Imperial City and Battlegrounds) and will remain unchanged for now. We will evaluate how these tests go, and let everyone know next steps.
Thanks for being patient and understanding with these tests. They are extremely important for the team and will help us work towards improving Cyrodiil performance.
silky_soft wrote: »ZOS_RichLambert wrote: »As part of our ongoing performance optimizations, we have been spending a lot of time analyzing and addressing problems in Cyrodiil. I'm going to give a summary of what has happened over the years, what we have been doing to alleviate the problems, and our future plans.
When talking about ESO server performance, it is important to understand that all abilities in the game (with some exceptions) have a "soft" limit on the number of times they can be executed. An example of a "hard" limit on an ability is an ability that can be executed one time every two seconds, enforced by the server. ESO doesn't have many of those. Instead, we designed the game to have soft limits, which mean you can execute as many abilities as you want as long as you have the resources - Stamina or Magicka - to cast them. There is a global "cool down" timer on all abilities, which is set to 1000 milliseconds.
The design goal of soft limits in any game is to allow players to create builds that let them execute abilities more often depending on build choices made, and not to have hard ceilings on damage or healing per second. This gives a lot of control to the player, which makes for a really fun and interesting system, but it can lead to situations where players cast too many abilities too quickly (and continuously) if strict limits on resources are not enforced.
Over the years, player power has grown considerably. With the addition of the Champion System, various armor/weapons sets, and changes to abilities, we have reached the stage where players - with the right build - can cast near-infinite numbers of abilities. If you add in a properly managed group, with some focused on damage and some focused on healing and regen boosts, you have a perpetually running never-ending stream of abilities.
With that in mind, consider how Area of Effect (AOE) abilities work: when cast, they look at a specific target area - almost always the area directly around the caster - find targets, and perform the ability. Each of those steps requires server calculations.
At launch, Cyrodiil’s processes were able to keep up with the number of AOEs cast, because most players couldn’t cast that many of them: they ran out of Magicka or Stamina, so they had to use AOEs judiciously. Over time, as player knowledge grew and regen builds grew in power, more players could cast more and more AOEs before running out of resources.
During these years, we found and fixed many performance issues in Cyrodiil, and we continue to do so. We've fixed significant issues, uncovered more, and continue to find and fix with every Update.
However, one foundational issue remains. At some point, we crossed a threshold where most players in PvP were able to cast endless AOE abilities, without ever running out of resources. This is done through player knowledge, builds and group mechanics – resulting in a constant stream of AOEs with many players never using any other type of ability.
This is not what we intended, but part of the fun of Elder Scrolls games is designing a build that has unexpected and powerful results, and we allowed it. However, as this behavior grew more prevalent, we reached a point where casting so many continuous AOE abilities in such a small area started to overwhelm the server process for that area, leading to situations where the "lag meter" spikes and the server becomes unresponsive for a period of time.
Our initial response to this problem – starting with Update 22 – was to find (and fix) more than a few problems with AOE ability calculations and make them more performant - and we stayed (mostly) ahead of the problem. But as more players reached maximum Champion Level and more players started utilizing this particular method of "AOE spamming", we have reached the point where we cannot fix this issue around the edges: we need to address the core problem, which we will be starting with Update 27.
To do this, we will need to first do some analysis, and we can only do this on the live servers. As much as we try to avoid running tests on live servers, they are the only place where the combination of player behavior, specific builds, and mass battles happen. So, starting the week of August 24, we will be running a series of tests on the live PC servers – both NA & EU, only in Cyrodiil. Please note that we will not be running these tests on any Console servers. Each test should take about a week, but if needed, we will extend the testing time.
The first round of tests we are planning will focus on Area of Effect (AOE) abilities in Cyrodiil and will make it more difficult for AOE abilities to be the only abilities used, adjusting cooldown, cost and regen values of all AOEs (damage and healing).
Specific details on the tests we will be running in Cyrodiil are as follows:During the times that any of these tests are active, we will be awarding double Alliance Points for anyone active in Cyrodiil.
- Test 1 – Shared global AOE cooldown - 3 second timer. This test adds a global 3 second shared cooldown to any AOE ability. This means that when you cast an AOE, you will not be able to cast another for 3 seconds. For example, as a Templar, if I cast Ritual of Retribution, I wouldn’t be able to cast Puncturing Sweep for 3 seconds.
- Test 2 – Individual AOE cooldown - 3 second timer. This test adds an individual AOE cooldown to each AOE ability. This means that when you cast an AOE, you will not be able to cast that same AOE ability for 3 seconds. For example – as a Templar, if I cast Puncturing Sweep, I wouldn’t be able to cast Puncturing Sweep again for 3 seconds.
- Test 3 – No cooldown, global ramping AOE cost. This test adds a global ramping AOE cost for each AOE cast. Similar to how streak or roll dodge works, where when you cast an AOE you receive a debuff for 5 seconds, each stack of the debuff increases the cost of any AOE cast.
- Test 4 – Individual AOE cooldown – 3 second timer, global ramping AOE cost. This test adds an individual AOE cooldown to each ability as in test 2, but also combines that with a global ramping cost from test 3.
After we complete the above tests, we may try other combinations of cooldown, cost, and regen values on AOE abilities. However, we need to run these tests first and then assess the data. We will then let everyone know what we found and how we will move forward. We will be very upfront, but please be aware that if these tests confirm our hypothesis, then chain-casting AOE abilities will no longer form the core of the ESO PvP experience in the way it has for the last few years. We would then go through each class and ensure that there are viable builds for each and make adjustments as necessary.
The code for these changes will be going into today’s PTS patch, and we will be running some basic tests throughout the day on Tuesday in Cyrodiil to ensure we’re able to make the above changes without requiring any maintenance or downtime. Once we launch Update 27 for PC on August 24, we will announce when one of the tests are beginning through an in-game announcement and will have a forum thread detailing which test is currently being performed. Remember, these tests will be limited to Cyrodiil, so AOE abilities in other PVP spaces (Imperial City and Battlegrounds) and will remain unchanged for now. We will evaluate how these tests go, and let everyone know next steps.
Thanks for being patient and understanding with these tests. They are extremely important for the team and will help us work towards improving Cyrodiil performance.
Upgrade your servers, drop those 6 year old xeon golds and get some rome going with decent thread count. Drop some things on the crownstore for the whales to buy then you can afford it.
rprice1819_ESO wrote: »Why not make it a turn based game and eliminate all server side lag. Each player can have up to an hour to decide what ability to use and where, then after the hour all abilities go off. Everyone can use up to 24 abilities per day
Can we make it text based?
> go west
> siege keep
> cast jabs
"You are standing next to a large rock. You see a Nightblade to the east."
> go east
"You are standing in a large field. There is a Nightblade here."
"NIghtblade attacks you for 23,930 damage."
"You are dead."
There we go....
Meme of the month Love it, had a good laugh.. almost as good as the changes they wanted to implement with heavy/light attacks.
ZOS finally decided to remove the only good thing going for them(their dynamic and intuitive combat) .. and made their game turn based, to "fix" performance .. what a joke.
I guess their PUBLIC TEST SERVER can't hadle these kind of TESTS, so they have to put it on live.
And they dare to say that they fixed many performance issues in cyrodiil, while that may be true ... they might've fixed 2problems, but added another 10..
Removed animation canceling to fix dsyncs, which didn't happen .. just made performance worse .. but no instead of reverting these changes they keep the trainwreck going, making combat even slower..
Say bye bye to your endgame community, dead game.. let's move on.
Actual 2Heads working at ZOS.
After reviewing what I said ..
I actually hope that ZOS will get some good data from the testing, actually fixing some of the performance issues..
Also these changes will surely not go trough like this in the future, it's just a matter of gathering data on what specifically is causing the lag when using aoe abilities.
ESO's "hallmark feature" of no cooldowns could be great, but it requires careful balance that ZOS doesn't seem to want to implement. You can't have overtuned abilities that have little/no counterplay sprinkled in with abilities of a few classes and expect things to just work, either everyone gets them, or no one does, and right now balance is stuck in the middle of some classes getting them, and others getting the shaft.
The unrestricted builds is at the root of a lot of issues in PvP, not simply performance.ZOS_RichLambert wrote: »Over the years, player power has grown considerably. With the addition of the Champion System, various armor/weapons sets, and changes to abilities, we have reached the stage where players - with the right build - can cast near-infinite numbers of abilities. If you add in a properly managed group, with some focused on damage and some focused on healing and regen boosts, you have a perpetually running never-ending stream of abilities.
Sugaroverdose wrote: »There was no Xeon Scalable in 2014, it was Xeon E3/5/7v3 but eso most definetly runs on E5v2 if ZoS does hold their own hardware
I have a few questions mixed in with my commentary to try to understand the rationale here, and while I am a software developer I'm not a game programmer, so bear with me.
From what little I'm able to find on the matter, it seems that AoE in a gridless combat game is calculated with a point-in-polygon algorithm, where every point would be a player. Thus, for every AoE ability cast, it sounds like you have to check against every player on the server who could be affected. Is that correct?
If so, does that mean every hit of a multi-hit ability like Hurricane is causing such a check? I can't see any way it wouldn't. Then moves such as Biting Jabs, which appear to hit multiple times per second, sound especially troublesome, which is awful because that's literally their spammable.
How many other mitigation strategies have been attempted? For example, pursuant to the above assumption, is it possible to have a temporary no-check table for characters far enough outside an AoE effect? If someone is on the exact opposite side of a keep to me and I start spamming steel tornado, for example, it's an incredibly safe assumption that we are not going to be able to close the distance between us for a certain amount of time based on distance from my first cast. If every potential target is evaluated for being in the polygon of a caster's AoE, could there be a gradated number of seconds that they remain on such a table for the casting player's AoEs based on their distance from the source? You could math it out to just be a function of max move speed if two players are assumed to be full sprinting at each other on mounts with max speed buffs, idk. If the cost of maintaining that table is less than having to do the AoE calculation every time for every player, that may be a good avenue to pursue.
Another assumption I have is that Cyrodiil is not actually a single server, but multiple servers with overlapping transitions zones to enable continuous movement. That might be totally wrong, but it would partially explain some phenomenon we experience, such as "render lines" and the ability to have fights at different parts of the map wind up more stable than one big fight with everyone showing up. If this is the case, is it possible to increase the number of these servers so that the map is more finely subdivided? Or has it been mathed out already to be at optimal placement given the inter-server communication needs? Is this even a real thing to begin with?
How many other redesign options are on the table besides the cost increases and GCD? There are a lot of continuously-firing AoE abilities going off every second that maybe don't need to and whose redesign would be considerably more palatable. For example, going back to Hurricane again, right now it looks like it deals damage every 1 second. That's a check every second just for having one of your main sorc buffs up, on top of other AoE abilities you might want to cast. Could it not instead tic less often (every 2 or three seconds) and instead deal more burst damage? Or how about Caltrops? Why do a check every second when you could just apply the snare and a bleed on the first hit and then have that player enter the aforementioned no-check table? Naturally any ability you're doing this for would need an extensive balance pass based on your change, but players will probably find "Mandatory 3 second AoE GCD" a lot less palatable than "Maybe Caltrops works differently a little".
How about a stacking AoE lockout, either per skill or generally, as per the testing schedule? Very coordinated groups (who are admittedly the main culprits for our issues, I imagine) tend to only have a window of a handful of seconds to do their damage when fighting each other. We proxy up, cast our shalk, then push in and cast our AoE spammable while the first two (plus our ults) go off. If a character is not dead within a few seconds of all that going off, odds are they will just heal up to full health and further uncoordinated AoE spam won't do you any good beyond that regardless. We seem to really like that type of engagement, but we also are all different in terms of how disciplined we are to stop casting AoE past the point of its usefulness. If instead of a GCD we build stacks, say with a max of 5 or so, and then got locked out of AoE upon getting the full stack, we would still have that dynamic gameplay we know and love, while simultaneously saving ourselves from our worst impulses, or worst Impulse spams, as it were. Naturally that might not actually be enough to mitigate the issue, but it might be worth its own testing bracket if and when the GCD test succeeds.
I would also like to express some caution regarding the tests. As always, anything that affects the balance of healing and purges is going to risk polluting the results with the potential heavy siege meta. Siege is already too strong for the amount of effort involved, and if we can't even heal or purge it consistently on top of all the other dots, debuffs, and damage that we're taking, then its presence on the battlefield is directly antithetical to the purpose of the tests. We cannot reliably reproduce the same hard-hitting fights causing the server performance problems if we can't even do open field fights without pugs dropping a handful of siege, never mind actually going to a keep and trying to take it. This already happens on live, but is more or less manageable by good groups who run heals, purges, and siege shields, so believe me when I say that this will severely handicap your ability to gather truthful data that has parity with normal live performance.
That said, I look forward to these tests and appreciate the honesty about the state of the game and the challenges we're facing. To five more years of Cyrodiil!! 🍺🍺🍺