Update on Cyrodiil Performance & Upcoming AOE Tests

Ralamil

ZOS_GinaBruno wrote: »

Hi all,

Just posted an update in the main stickied thread, but wanted to cross-post here as well for extra visibility.

As we approach the end of our scheduled tests, we’ve determined that we’d like to run at least two more tests to gather additional data, which are outlined below. Our goal here is to see how a combination of changes from previous tests affect the performance, as we’ve found they have not had a significant impact individually.

• October 19 - 26:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
• October 26 – November 2:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
  • Global ramping cost per successive AoE cast by 50%/100%/150%
  • Regen decrease per successive AoE cast by 33%/66%/99%

Our current plan is to disable all tests and double AP once Update 28 launches for PC on November 2. At that time, we’ll need to take some time to analyze the data collected from all the tests and determine next steps. Once we have a better idea of what that entails, we’ll let everyone know.

@ZOS_GinaBruno

While I have been particularly unhappy with most of the tests the team has chosen to run, I have (admittedly, with MANY salty bug reports filed) endured and continued to play, to help contribute to whatever data the team thinks its collecting.

However, it has become quite apparent that the team is just throwing whatever it can at a wall and hoping that something sticks. These tests are, as a friend of mine eloquently put it, an invitation to go find a different game to play.

SpiderKnight

ZOS_GinaBruno wrote: »

Hi all,

Just posted an update in the main stickied thread, but wanted to cross-post here as well for extra visibility.

As we approach the end of our scheduled tests, we’ve determined that we’d like to run at least two more tests to gather additional data, which are outlined below. Our goal here is to see how a combination of changes from previous tests affect the performance, as we’ve found they have not had a significant impact individually.

• October 19 - 26:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
• October 26 – November 2:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
  • Global ramping cost per successive AoE cast by 50%/100%/150%
  • Regen decrease per successive AoE cast by 33%/66%/99%

Our current plan is to disable all tests and double AP once Update 28 launches for PC on November 2. At that time, we’ll need to take some time to analyze the data collected from all the tests and determine next steps. Once we have a better idea of what that entails, we’ll let everyone know.

This tells me they will ultimately decide solo/zerg healers will be killed off if they're bringing this back for two more tests.

Aztlan

I've tried to remain positive, but I'm tired of this. The AoE cooldowns, global and single, are not fun.

Recremen

It's cool that y'all are still trying to fix lag and clearly putting a lot of effort into obtaining the data needed to help get us to a good state of performance, but especially with the latest changes it seems more like you're testing your actual fix proposals more than you're testing weaknesses in the code that can be improved to make Cyrodiil more performant without sacrificing the gameplay we've come to expect.

It also feels like, instead of doing a pass on how abilities/item sets work to reduce total AoE checks/disincentivize spamming them, you're throwing your hands up and saying "No, it is the people using abilities who are wrong". This is not the desired look. Though it is nice that we're getting more open and frequent communication about what's going on, at least.

adirondack

I'm so sick of the testing... just end it already. Geez.

Squeaky_Clean

Ok, another two weeks without PVP, please tell me when the madness is over, and I can rejoin PVP.

Ackwalan

Looks like global cool down and 12 man groups are going to be here to stay. No matter what the data shows.

silvereyes

I think you may be missing the forest for the trees here. You are so laser focused on getting your server frame time metrics down that you forgot the entire point of the exercise, which was to make Cyrodiil less frustrating for players.

Personally, I'd almost prefer to just leave things be at this point. I'd much prefer to have the only times my skills don't fire be when two zerg balls collide, rather than enshrine skills not firing as an official gameplay mechanic that affects 100% of my play time in Cyrodiil.

RMerlin

adirondack wrote: »

I'm so sick of the testing... just end it already. Geez.

Agreed. After having gone through all these tests where it still wasn't anywhere close to acceptable performance, I'm about to give up on ZOS ever fixing the performance issue. They just had a whole year of "performance improvements", and yet the game still remains unplayable during weekend peak times (and even some week nights nowadays, which is a step back).

The fact they are doing more tests around AoE leads me to believe their intent is to ultimately deploy a version of one of these tests into live. Otherwise, they would have stopped after the first series of tests, and moved on to something else.

I've been playing this game without a break since beta. I give up. I changed my 6-month ESO+ sub into a single month sub last month as they were doing these tests, and now seeing the results, I will not be renewing again once it expires next month. It makes me sad because I truly believed in this game. Like many here I have been more patient and forgiving than a normal customer would have if this had been any other game (or product) because of my love for Elder Scrolls content in general. But the fact is, too often I find myself simply unable to play at all when I want to play. I can no longer enjoy this game on Friday nights like I used to years ago, as the game is simply unplayable in PvP. Some game mechanics made things even worse because things like having a cast delay added to ultimates while the lag is present means that ultimates can no longer be used reliably in PvP. And all of the user community complaints about this change fell on deaf ears.

After that whole year of "optimizations", I don't see any real fix in sight anytime soon, so I must reluctantly start looking for an alternative to this game.

NotTaylorSwift

It's just painfully obvious how disconnected the developers are from the game and the community. If you do not actually play Cyrodiil and also do not listen to the community of people who do, how can you come up with a good solution to the problem? Looking at data is all well good but the game wasn't designed to be played with cooldowns and whatever other additions are being added. Sure the global cooldown may reduce lag (which is pretty obvious seeing as more than half the skills in the game were counted as aoe and most classes could only use 1 skill every 3 seconds...) but it wasn't designed to be played this way in the first place and because of that most classes are unplayable this way. Especially magicka specs which for the most part don't work at all.

I would LOVE to see the developers go into Cyro and try to actually play, and have fun at the same time, while these tests are active. Not just run in a zerg of 40 people light attacking, and then come back and try to say these are good ideas for the game. We see streams all the time of them showing off the new content they keep bringing but why not of them actually playing the game they are trying to fix?

Beaverton

Oh, the worst of the worst from the tests. A s**t pie for everyone. Very few will be in Cyrodiil to test these things. I bet the lag will be non-existent.

edit: I think the Greyhost PC/NA campaign finishes on the 20th? 21st? So you might get two days of data on the first test as some people care enough about the campaign score to suffer through it.

Capsaica

More tests? While I appreciate an effort to pick apart the causes of the lag, I agree with my friend Men'do wholeheartedly that these new approaches seem to border on kitchen-sink-esque fixes rather than a careful analysis to pull apart the source problem. Being a scientist/engineer by profession, I understand very well the concepts of problem solving and testing hypotheses to get at the root cause of the problem. I also understand that rarely is a problem so simple that it has a single root cause, but testing requires some level of consistency and limits on variables.

Your double AP and doing this on the live server is an attempt to get enough population and groups to continue playing to get valid data. The sheer level of burnout and disheartening inability to play the game in Cyrodiil - to "play the way we want"- is reducing numbers, driving players away from the game altogether and reducing that population you so desperately need for the testing. Have you even tried to play a magplar in Cyrodiil during these tests? Practically anything you do has a cooldown. Spammable? Nope. Self heal? Nope. Other self heal? Nope. Can I heal myself at all? Yes - take a potion.

The primary PvP groups I play with have made honest efforts to participate in these tests in good faith that your efforts are aimed at improving our Cyrodiil experience. That being said, we haven't really had fun in Cyrodiil for 6 weeks and now we have two more weeks of ever-increasing misery to push through. Even with our efforts we have been losing players left and right to sheer frustration and burnout. As I've told many of our folks, this is a game and it is supposed to be fun. If you aren't having fun, then you shouldn't be "playing" while miserable. It is called playing for a reason, isn't it?

We need a light at the end of the tunnel. Right now, to quote others much better at speeches than myself, I'm pretty sure that light at the end of this particular tunnel right now is an oncoming train. We need some incentive to keep slogging through these tests, and I don't mean double AP or something inconsequential like that (most of us avid PvPers do not need more AP, tbh, although we take it). We need hope that you have a plan or even without a plan, we need reassurances that you are not planning to take the very thing about ESO PvP which makes it superior to every comparable game out there (fast, engaging, exciting combat) away from us.

I have played continuously since Beta. I have subscribed continuously as well. The reason I keep coming back is the PvP end game (although I do all the other PvE things too) and the fast friends I have made there. I met my now-spouse across enemy lines in Chillrend. I have met game friends in real life, have sent flowers and ice cream to game friends who have lost mothers, fathers, have received flowers from those same friends when experiencing that loss myself. I've been invited to weddings and get-togethers and so on. I am, to understate this tremendously, dedicated to the game and especially the PvP aspect of it. Please give us some hope.

~Caps

Ragnaroek93

The tests won't have any meaningful results tbh. How do you evalute the gained data when the test was performed so far away from the reality, even if everybody would take part in it (which I highly doubt because most classes won't work during these tests). At least give some AoE's an exception (like Jabs, Hardened Ward etc for example).

Maybe try disabling AoE proc sets for a week instead.

Starshadw

ZOS_GinaBruno wrote: »

Hi all,

Just posted an update in the main stickied thread, but wanted to cross-post here as well for extra visibility.

As we approach the end of our scheduled tests, we’ve determined that we’d like to run at least two more tests to gather additional data, which are outlined below. Our goal here is to see how a combination of changes from previous tests affect the performance, as we’ve found they have not had a significant impact individually.

• October 19 - 26:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
• October 26 – November 2:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
  • Global ramping cost per successive AoE cast by 50%/100%/150%
  • Regen decrease per successive AoE cast by 33%/66%/99%

Our current plan is to disable all tests and double AP once Update 28 launches for PC on November 2. At that time, we’ll need to take some time to analyze the data collected from all the tests and determine next steps. Once we have a better idea of what that entails, we’ll let everyone know.

.... I'm speechless. If the desire here is to squelch any enthusiasm to PvP in Cyrodiil in this game, well, you're succeeding. I've done my best to be out there with DIG every week in order to help test, but this is getting ridiculous.

mav1234

This... is just too much. Its like a random grab bag of tests.

renne

ZOS_GinaBruno wrote: »

Hi all,

Just posted an update in the main stickied thread, but wanted to cross-post here as well for extra visibility.

As we approach the end of our scheduled tests, we’ve determined that we’d like to run at least two more tests to gather additional data, which are outlined below. Our goal here is to see how a combination of changes from previous tests affect the performance, as we’ve found they have not had a significant impact individually.

• October 19 - 26:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
• October 26 – November 2:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
  • Global ramping cost per successive AoE cast by 50%/100%/150%
  • Regen decrease per successive AoE cast by 33%/66%/99%

Our current plan is to disable all tests and double AP once Update 28 launches for PC on November 2. At that time, we’ll need to take some time to analyze the data collected from all the tests and determine next steps. Once we have a better idea of what that entails, we’ll let everyone know.

So... I assume Templar mains are going to get a complimentary class change? Since you're so insistant on killing the class by keeping the global AOE cooldown?

Also sucks to be one of the poor folk on console who're in the vast amount just going to get these MASSIVE changes dumped on them without any explanation, people'll rock up to Cyro to play some PvP suddenly every AOE is in a global cooldown and it's not a bug it's a """"""""feature""""""""".

SodanTok

Ragnaroek93 wrote: »

The tests won't have any meaningful results tbh. How do you evalute the gained data when the test was performed so far away from the reality, even if everybody would take part in it (which I highly doubt because most classes won't work during these tests). At least give some AoE's an exception (like Jabs, Hardened Ward etc for example).

Maybe try disabling AoE proc sets for a week instead.

Disagree. The odds of useful data gained are quite high, just their last post mentioned it. I am half surprised their last test doesnt obliterate AoE from existence completely as its looking more and more like AoE spamming, while impactful, is just one of many reasons for poor cyrodiil performance and such big test could prove that.

Only way such test would produce no meaningful data if it actually resulted in great performance improvement by lack of participants.

Ezhh

I am mostly a PvE player, but I always enjoyed PvP as well. Specifically as a solo player or in very small groups. I poked away at the first couple of rounds of testing before I lost willpower. I wanted to believe the tests were a good sign - that things would get better. But the tests just weren't fun.

So back to only PvE I went, in which I recently stopped only playing DD and started making healers because I want to be more flexible when it comes to roles I can play. I was looking forward to bringing those healers into Cyro, for fun, as well as to level alliance skill-lines and get skills I need for PvE.

Now I see there is yet more testing in which those characters won't be able to do anything useful at all unless I try join some huge group... which removes most of the fun for me.

Cyro lag was never as bad as what these tests have done to my ability to play the game the way I enjoy it.

GorraShatan

This is very disheartening.

You can make it up to us by having Mendo voice a Senche-Rat mount.

Thanos7895

ZOS_GinaBruno wrote: »

Hi all,

Just posted an update in the main stickied thread, but wanted to cross-post here as well for extra visibility.

As we approach the end of our scheduled tests, we’ve determined that we’d like to run at least two more tests to gather additional data, which are outlined below. Our goal here is to see how a combination of changes from previous tests affect the performance, as we’ve found they have not had a significant impact individually.

• October 19 - 26:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
• October 26 – November 2:
  • Shared global AoE cooldown with a 3 second timer
  • Group size limited to 12 in Cyrodiil
  • Ally-targeted abilities only applied to group members
  • Global ramping cost per successive AoE cast by 50%/100%/150%
  • Regen decrease per successive AoE cast by 33%/66%/99%

Our current plan is to disable all tests and double AP once Update 28 launches for PC on November 2. At that time, we’ll need to take some time to analyze the data collected from all the tests and determine next steps. Once we have a better idea of what that entails, we’ll let everyone know.

Will ZOS be providing refunds for ESO Plus for the 2 months of playtime where Cyro has been under tests? I mean since you are putting the 7 year old game back to a beta state with these tests it seems only fair...

Thanos7895

Instead of the horrible last test - why not try disabling addons to see if that improves performance?

Spiritrush

I started ESO in Beta with the intent to explore PvE and I thoroughly enjoy all aspects of PvE. As a former coder and experienced executive in media entertainment, I appreciate that PvE is the monetary emphasis for the longevity and growth of the game.

But - does anyone remember - the original promise of ESO at Beta was PvP. ESO offered an extraordinary 24-person experience that really cannot be matched in any other game, even to this day. The small-man and solo experiences can be exceptional too. The original GvG experiences were truly unforgettable and, if encouraged and supported, provide for long-term, passionate, loyal game advocates. Despite the many, many significant issues with lag and performance over the years, without question PvP remains the single most under-appreciated asset of the game.

As Caps wrote above, these tests appear to be (and I can't say for sure) somewhat superficial, meaning the developers may have given up on the underlying code (or, perhaps, simply cannot afford to properly analyze/tackle any meaningful refactoring) and are focused on limiting traffic above the underlying code. Not to be discouraging, but if that is the purpose these tests may simply be "rearranging the chairs on the deck of the Titanic" (double AP = the string quartet). To the executives, PvP is the most significantly under-appreciated asset of the game that has the greatest ability to generate the most valuable advocates for the game, attracting new PvP and PvE players alike.

My appeal would be that ZoS (a) seriously re-consider investing in proper code refactoring (even consider justifying part of the expense off balance sheet in concert with very early SFO/ESO2 development), and (b) endeavour to benefit from Microsoft's leadership and resources in designing quality server performance (e.g. a complete overhaul). Good luck, and if you are already doing these things, please let us know!

Ralamil

Thanos7895 wrote: »

Will ZOS be providing refunds for ESO Plus for the 2 months of playtime where Cyro has been under tests? I mean since you are putting the 7 year old game back to a beta state with these tests it seems only fair...

I'm actually in a situation where my ESO Plus sub was running out during these tests, and until they can prove they're actually fixing the issues without neutering what makes PVP in this game fun (and it IS fun), I am not planning on renewing that sub. First time intentionally letting it lapse since beta.

techyeshic

@ZOS_GinaBruno This is oppressive. Just disable rapid/radiating regen from stacking and limit purge then large groups can die rather than continue a continuous spam of abilities. Just try that. No need to keep smothering everyone.

Vizirith

I'm not liking the direction these tests are indicating pvp is heading towards. Make note of the fact that it was said that none of the individual tests provided enough impact, and as such these new ones can be inferred to be closer to the end result than the individual tests.

First week:
Shared global AoE cooldown with a 3 second timer
Group size limited to 12 in Cyrodiil
Ally-targeted abilities only applied to group members

And then the even more stringent first new week tests and:

Global ramping cost per successive AoE cast by 50%/100%/150%
Regen decrease per successive AoE cast by 33%/66%/99%

It looks like there is a decent chance the shared global cooldown and reduction is group size is here to stay. Remember the individual test of: Group size is limited to 12 in Cyrodiil, and if a group contains 6 or more players, everyone’s AoE abilities will go on individual cooldowns and have escalating cost was not enough.

merevie

Today we played 6hrs.

It was the WORST performance in my 6yr history.

Siege -6-7 times being busy before it would fire.

Group members constantly being thrown out of the game, logging in, being crashed, logging in, being crashed.

No abilities working.

We kept finding ourselves on different floors and sides of the keep.

Enemies appeared from nowhere.

Outside siege was magically reaching places it never had before indoors.

People were going from full health to being dead.

I really look forward to the next stage of ESO ownership. We are paying clients. Thousands of us are now enduring months long wrecking of our game. I am not sure why this is allowed as a business model -we all know perfectly well how things ran with extra server management during the PvP event this year. Perfectly fine. Is no-one accountable?

And no, it is not 'here to stay'. None of us is going to accept the game being wrecked because the current team are not able to find a solution.

Mr_Nobody

If anyone from ZoS read it ~ @ZOS_GinaBruno

Your latency issues are not going to be fixed even if all of these changes are implemented. Fights last too long. Cooldowns on AOEs don't help that but rather kill fun gameplay and counter-gameplay. (Hello Templars?)

Just analyze which spells are cast the most during huge battles or when a ball group arrives ~

PURGE.

1 and only spell that allows ball groups and stacked zergs to exist and last. Change it, or nerf it heavily - fights will last way shorter thus decreasing the lag. It should have never been an aoe cleanse. It should have never given the bonuses it gives now. You think it's huge cost matters? It doesn't. Every ball group or huge zerg is just blinking like a Christmas tree, and has been for a very long time. It's time to change.

If it was my decision - if it stayed the same it should be an ultimate ability on a very high cost.

Joy_Division

ZOS is going about this the wrong way. The problem is not so much that players are casting too many abilities or the number of calculations are over-burdening the servers, rather it's the limited capacity of the severs and what they can handle in the first place.

For 6 years I have read dozens and dozens of patch notes that x change will reduce server calculation, altering Y means a more efficient processing that should improve performance, saw just about every ability in the game streamlined to dramatically reduce the amount of calculations, the removal of many assets in Cyrodiil (RIP deer) to again reduce the server calculations, etc., and here we are. Now 6 weeks of testing have gone by that have further reduced the burden by preventing us from casting skills and restricting grouping and

ZOS_GinaBruno wrote: »

we’ve found they have not had a significant impact individually.

Of course they didn't. What the server can handle is way too small. With every reduced server calculation, ZoS is doing the equivalent of tearing one page out of War and Peace. Sure, there's less to read, but the book is still over 1000 pages!

Everything I use in my daily life involving technology and computers have improved/become more efficient by quite a bit expect ZoS's servers which are still stuck circa 2014. Amazing. Tell the cooperate suits to stop wasting the devs talents (not to mention sanity) and just chalk up the cost if they are genuinely interested in Cyrodiil's gameplay performance.

********

Here's the thing about these tests.

I don't like them. But as much as I don't like them, I'd be totally OK with doing them if I had reason to believe they might lead to a better Cyrodiil experience.

But therein lies the problem. How could things possibly be better?

Let's say even with these very extreme changes that will make the game unplayable still sees "no significant impact." That just leads us back to square one and all but confirms that the 6 years of attempts to reduce calculations have been a dead end.

Or, suppose when combining all these restrictions does make the game perform better, what then?

I actually dread this more because that means ZOS is going to have to implement these sorts of radical changes on a permanent basis that all but makes the Cyrodiil experience unrecognizable and quite frankly, not fun. It really isn't fun Gina. It's not an inconvenience or a matter of adapting, it's not fun. I've never heard more "cant play tonight, sorry RL stuff," "wife is horny tonight," "kid needs help with homework" and other assorted "can't play, sorrys" than this past month. To use the War and Peace analogy above, Zos would have tore so many pages out of it that the book would be unrecognizable and incoherent.

I'd rather play in lag.

*********

And what is the future of Cyrodiil anyway? Has ZoS hired a PvP dev to take the place of Brian? Or are we supposed to believe he can somehow juggle combat, Cyrodiil, and Battlegrounds? There's no possible way.

It's all very discouraging.

• No ZOS dev whose job is PvP
• Proc sets out of control (again)
• Servers somehow stuck in 2014
• Performance won;t increase without onerous restrictions in how we play

So many things are stuck in 2014. Do you know the same person has been Emperor every night I've logged into Ravenwatch? Every night we dethrone him and then it's Groundhog Day the next evening as we get to see more of that Soul Assault build (though recently he's seems to be bored of that, go figure). How is allowing one person a monopoly over Emperorship for an entire campaign remotely healthy? It isn't. We all know it, yet nothing gets done about it because there's nobody at ZOS whose job is to reform Cyrodiil's problems and so much attention is focused on how to reduce server calculations instead of how to make PvP more interesting.

In short (too late), we already learned what we needed to for these tests: even with restricting the number of calculations we put on the server to the point of not wanting to play, nothing changed. We're already at the point where the cure is worse than the disease. Invest the time, resources, and the talent the devs have elsewhere.

Tigor

Everyone should be happy with the extra effort and opportunities to contribute to improve pvp in Cyrodiil.

Obtaining additional data is, nevertheless, good, and the proposed two added tests are interesting enough. Just do it.....

keto3000

I like that the testing is happening, however, Week #1 should be repeated when we are NOT in an event & there should NOT, IMO, be any tests during upcoming events. The IC event kept many out of Cyrodiil and that week's testing results are particularly skewed, as a result.
I'd like to see ZOS ( @ZOS_GinaBruno @ZOS_BrianWheeler ) redo that particular week of testing.