Maintenance for the week of April 6:
• PC/Mac: No maintenance – April 6

How is the AOE test being evaluated? Cyrodiil is a ghost town

pauld1_ESO
pauld1_ESO
✭✭✭✭
First of all starting this during the IC event was a HUGE mistake. Second, due to the drastic changes many people have opted out of going to Cyrodiil altogether because their builds are not viable. This is why things like this belong on a TEST SERVER. People who want to test will test, people who do not will not. I don't understand how any of this data is going to be useful unless you will only be going off of player feedback.

But if your goal was less Cyrodiil lag....GRATS! You fixed that. Me and the other 20 people in there thank you.
  • Luke_Flamesword
    Luke_Flamesword
    ✭✭✭✭✭
    ✭✭
    On test servers you can only have much less people. If you think that now we have no useful data, than on test servers it can be only worse, so what's you point?
    PC | EU | DC |Stam Dk Breton
  • Recremen
    Recremen
    ✭✭✭✭✭
    ✭✭✭✭✭
    On test servers you can only have much less people. If you think that now we have no useful data, than on test servers it can be only worse, so what's you point?

    I think most of us are worried that their analysis won't account for the fact that we don't actually have poplocked campaigns right now, and they'll end up going with the solution that "decreases lag the most", which of course will be the one from the test where there's hardly anyone out there fighting. Since this is also one of the worst solutions, people don't like that prospect.
    Men'Do PC NA AD Khajiit
    Grand High Illustrious Mid-Tier PvP/PvE Bussmunster
  • Sarannah
    Sarannah
    ✭✭✭✭✭
    ✭✭✭✭
    ZOS should have added a 19th character slot to everyone's account(for testing purposes), on which a person can make an instant level 50 with character creation, with all maximum skillpoints and all skill lines fully available. Allow them to make any free character-bound level 50 sets, and monster helm/shoulder set. Just for this testing phase. As barely anyone would re-spec their main completely, re-gear completely, and then try the test. Just remove that character after the test is done.
    Edited by Sarannah on September 10, 2020 2:15PM
  • Luke_Flamesword
    Luke_Flamesword
    ✭✭✭✭✭
    ✭✭
    Just don't make ZOS a fools :) They know how many people are there, they can compare perfomance with similiar population numbers in past. Do you really think that they won't take into account obvious parameters of tests? Also just wait for weekend - there will be full campaigns with no problem - only difference will be with shorter queues :)
    PC | EU | DC |Stam Dk Breton
  • Jaraal
    Jaraal
    ✭✭✭✭✭
    ✭✭✭✭✭
    Who in their right mind would willingly play with a three second cool down on AOEs? Especially with the skills not working bug. How could they get a realistic test when some skills are disabled?

    I'm not surprised at all.


    Edited by Jaraal on September 10, 2020 2:31PM
  • ealdwin
    ealdwin
    ✭✭✭✭✭
    On test servers you can only have much less people. If you think that now we have no useful data, than on test servers it can be only worse, so what's you point?

    During one pts cycle, I was messing around in Cyrodiil and noticed one of AD's gates was open, and then proceeded to go capture the scroll and run it all the way back to one of DC's gate keeps without even seeing another player.

    For very good reasons, if the tests are going to be remotely useful, they have to be done on live servers.
  • Cheezits94
    Cheezits94
    ✭✭✭✭✭
    As a Magplar Healer main, i am staying the hell away from cyrodiil this week, and i enjoy the action in IC instead.
    My build is not viable with that cooldown, literally ALL my skills are AoE (as I said, magplar HEALER) and I am not willing to completely respec my char into a DD.
    If you can't even spell sets, locations and items, you probably have no clue what you even are talking about.

    Tamriel, not Tamerial, Temerial or Tamériál
    Alkosh, not Alkoash
    Dolmen, not Dolman
    Olorime, not Oloramie
    Sorcerer, not Sorceror
  • idk
    idk
    ✭✭✭✭✭
    ✭✭✭✭✭
    pauld1_ESO wrote: »
    This is why things like this belong on a TEST SERVER. People who want to test will test, people who do not will not.

    If this was a good idea Zos would have already done it on the PTS. The issue, and Zos has stated this before, the PTS does not draw enough players into Cyrodiil to get load data which is what is required for such a test. Considering the smaller number of players that actually download the PTS and that we do not spend much time there this should be obvious making it also obvious that Zos would not be able to get good load data.

    That would mean even less data for Zos to work with making the entire test pointless and this testing on the live server that much more effective regardless of some players avoiding Cyrodiil.

    As for your concerns about how the data will be used, Zos will analyze the load data to see how these changes affect performance. It is really that simple. What changes Zos makes, as a result, cannot be provided right now because they have not been determined, hence the need for testing to see how these changes affect Cyrodiil. Last test Zos did on live Cyrodiil did not lead to changes directly related to the test but probably lead to indirect changes behind the scenes.

    It is unfortunate that some players do not want to be part of the solution.
  • Inspiral808
    Inspiral808
    ✭✭✭
    While the timing is unfortunate with the IC event going on, the biggest reason for Cyrodiil being quiet right now is because the double AP incentive for testing is not working.

    Fix that and Cyrodiil will be heaving with people.
  • Rungar
    Rungar
    ✭✭✭✭✭
    imo the right thing to do is have a live test campaign for cyrodiil they can turn off and on when they want. They can use double ap and other incentives to get people to join there when needed.

    they have shown that they can put up another campaign on the fly so i dont see why they didnt go this route. That being said, if you dont bother to test it out, dont bother to complain either.
    Edited by Rungar on September 10, 2020 3:21PM
  • Damnationie
    Damnationie
    ✭✭
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.
  • BigBragg
    BigBragg
    ✭✭✭✭✭
    ✭✭✭✭
    I just want to know who thought running a Cyrodiil test overlapping an Imperial City even was the right call.
  • redlink1979
    redlink1979
    ✭✭✭✭✭
    ✭✭✭
    Tests like these must be done on live server, during periods of high population such as events.

    Bots that ZOS can create on internal/test servers can't replicate all the human behaviors/gear/skill combos players have while playing.
    "Sweet Mother, sweet Mother, send your child unto me, for the sins of the unworthy must be baptized in blood and fear"
    • Sons of the Night Mother | VforVendetta | Grownups Gaming EU | English Elders [PS][EU] 2500 CP
    • Daggerfall's Mightiest | Eternal Champions | Legacy | Tamriel Melting Pot [PS][NA] 2300 CP
    • SweetTrolls | Spring Rose | Daggerfall Royal Legion | Tinnitus Delux [PC][EU] 2525 CP
    • Bacon Rats | Silverlight Brotherhood | Canis Root Tea Party | Vincula Doloris [PC][NA] 2300 CP
  • kojou
    kojou
    ✭✭✭✭✭
    ✭✭
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.

    I think it boils down to level of effort required to do testing like you say. I agree that what you are suggesting would be ideal.

    Even if they just spawned numbers of characters in a test instance in a keep and had macros that cast skills over and over again it would still be a lot of effort to set up the tests.

    Configuring the number of "bots" required to put an appropriate amount of load would be even more labor intensive then having stationary characters with macros firing off skills.

    In a data entry type application transactions can be simplified easily with randomized variables etc, but in a game with concepts of 3D movement, social interactions, and actions that affect other players it is not so easy to simulate. It can be done, but I think it would be very labor intensive to set up.

    Playing since beta...
  • Kosef
    Kosef
    ✭✭✭✭
    All this lag and stuff would be fixed if they just allowed pvp all over with a flag system. Then it wouldn't be centralized to cyrodiil.


    OR MY FAVORITE IDEA:

    a SEPERATE server for cyrodiil only. Where the entire point of logging into that server is for realm pvp. You would be able to copy a character from live servers over, and interchange them on a weekly basis if you wish. Once that character is selected, it's locked to cyrodiil for that week.
    Server: PC-NA

    Vincent Jurard - Breton - DK
    Aronil Elsinwatch - Altmer - Templar
    Ulfgar Stormbringer - Nord - Sorc
    Ra'Jhera Mahrabi - Khajiit - NB
    Kur Zahir - Imperial - Necro
    Khur - Redguard - Warden
    Lives-By-Sword - Argonian - DK
    Sara Willowthorn - Bosmer - Templar
    Elsa Bjariksen - Nord - NB
    Soliril Elsinthar - Altmer - Sorc
    Khargol Ogdum - Orc - Warden
    Veros Sarvani - Dunmer - Necro
    Vedran The Toxic - Dunmer - DK
    Hafnar Ironbark - Nord Warden
    Low - Level: Arcturus The Loyal - Templar
    Guild:
    [Ninja Squirrels]
  • alainjbrennanb16_ESO
    alainjbrennanb16_ESO
    ✭✭✭✭✭
    strange was in ravenwatch eu and there was loads of peeps
    Main character dk - Vanikifar whitestrike
  • techyeshic
    techyeshic
    ✭✭✭✭✭
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.

    They are being honest. They could have just patched this stuff in as "balance" and let it run for 3 months per as a patch cycle and called it a "feature." Lord knows enough software developers do that.
  • idk
    idk
    ✭✭✭✭✭
    ✭✭✭✭✭
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.

    This is correct for most things. However, for Cyrodiil server load issues and similar, it is very much incorrect and I would not expect most IT professionals to be aware since most work on simpler systems.

    First off, we are talking about a server load issue that requires an actual load. Second, Zos is fully aware there are systems that artificially simulate a server load and they use it that this person has brought. However, such systems do not properly account for the variability of player actions.

    BTW, Zos has said this before. While I am not a fan of how Zos has manged this game I am pretty sure they are more aware of how various aspects of this game can be tested effectively than anyone in these forums. In other words, Zos has already spoken to this and said it is not that simple.
  • Damnationie
    Damnationie
    ✭✭
    Tests like these must be done on live server, during periods of high population such as events.

    Bots that ZOS can create on internal/test servers can't replicate all the human behaviors/gear/skill combos players have while playing.

    If they can't replicate them then humans can't do them in the first place. Is there a massive variety in things people can do, of course. But they are done within the rules of the game engine, and anything a player can do can be simulated.

    At the end of the day they don't need to replicate everything. You don't need that to find your problems. You put basic monitoring on the server to identify what people are doing like, oh, a combat log ( https://forums.elderscrollsonline.com/en/discussion/467949/encounter-logging/ ).

    You can then select interesting player(s) referred to in the log at the points in time your servers started complaining and run a script to convert the log into bot/ test automation commands. Setting up sucha a script is an overhead but like most well done test automation it will pay you back in spades over time. But you have to be preperred as a business to invest in this. Over the long run it saves you time, effort and money but non-IT people, especially middle managers always seem to have a hard time getting that.

    Then you just debug what the player(s) were doing at the time and see what the test run throws up. It can be that simple IF you put the up front effort in. It also helps if your development process and standard work towards supporting test automation and problem investigation, something I suspect has not happened with ESO.
  • Ei8htba11
    Ei8htba11
    ✭✭✭✭
    Tests like these must be done on live server, during periods of high population such as events.

    Bots that ZOS can create on internal/test servers can't replicate all the human behaviors/gear/skill combos players have while playing.

    If people are avoiding the test area because they dislike the changes/outcomes, you're not going to get reliable test data anyway.
  • TwinLamps
    TwinLamps
    ✭✭✭✭✭
    how is lag now that cyro is emptier?
    Awake, but at what cost
  • TineaCruris
    TineaCruris
    ✭✭✭✭✭
    ....and those that are in cyrodiil have all changed their builds and bars to not use any aoe's.

    This aoe test is a nightmare for reasons other than just it is an attempt to throw shortcomings on the companies part onto the shoulders of paying customers.

    My primary PvP guild announced today we are shutting down until the test is over.

    Edited by TineaCruris on September 10, 2020 5:15PM
  • Jeremy
    Jeremy
    ✭✭✭✭✭
    ✭✭✭✭✭
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.

    I don't understand it either and made a similar post awhile back.

    Like you say, all that would be required to test this AoE theory of their's would be a simple simulation of said amount of players spamming AoEs. I also know they already know this (any basic programmer who has ever worked with game engines does). So I've concluded there must be another reason for this test besides the one described.

    I suspect the combat team is considering transitioning into a cool down based system (instead of the current APM system) in order to narrow the skill gap they want to fix. So I think that's the real reason behind this test. They want to judge how their player base reacts to longer cool downs on their abilities and have cleverly disguised the ploy as an experiment to reduce lag.
  • TineaCruris
    TineaCruris
    ✭✭✭✭✭
    Jeremy wrote: »
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.

    I don't understand it either and made a similar post awhile back.

    Like you say, all that would be required to test this AoE theory of their's would be a simple simulation of said amount of players spamming AoEs. I also know they already know this (any basic programmer who has ever worked with game engines does). So I've concluded there must be another reason for this test besides the one described.

    I suspect the combat team is considering transitioning into a cool down based system (instead of the current APM system) in order to narrow the skill gap they want to fix. So I think that's the real reason behind this test. They want to judge how their player base reacts to longer cool downs on their abilities and have cleverly disguised the ploy as an experiment to reduce lag.

    This IS NOT their motivation.
  • Jeremy
    Jeremy
    ✭✭✭✭✭
    ✭✭✭✭✭
    Jeremy wrote: »
    Can we just put this "we have to do this on live" nonsense to bed.

    Outside the gaming industry, in the more general IT world, if you are doing performance investigations, live is the last place you go to do performance testing. I've over two decades in programming, a lot of spent as a troubleshooter of misbehaving systems and this idea you can't find performance problems on the test system is a warning sign I've come across a number of times. What it normally translates too is
    • We don't want to spend money on a proper test system that is a proper scaled down replica of our live system.
    • We don't know enough about how the system works to set up a proper test system.
    • Proper testing automation and load testing tools have not been purchased for use by the developers.
    • Trying to get people without the experience / expertise in trouble shooting to investigate complex issues because they won't spend for specialist help.

    None of what ZOS is doing makes logical sense if they actually want to solve their issues. For a performance test to be any good you need to have control over the inputs and be able to repeat the exact same activity every time you adjust the code.

    If the conditions are not identical than you cannot properly gauge the impact of a change. For example, currently there is a global GCD cool down in Cyrodiil. As a result a lot of people seem to be just removing those skills off their skill bars, maybe leaving just one. So the test is not actually seeing the impact of the changes to performance, rather the impact of people using different skills altogether. As they try the different scenarios they have suggested people will again alter their characters to suit. So each test will not be comparable to each other. Any conclusions they draw will be flawed, and no, you can't compensate for that. Been there, watched as a lot of people made bad decisions with confidence resulting in a major disaster.

    What they should be doing is going and getting a load of bot scripts (If they don't have test automation tooling they'd work just as fine for this) and running them on their internal test system to replicate standard player activity. With monitoring you can see the impact each bot has and the impact of each of the skills. You can enable debug level tracing of the code and properly determine where the code bottlenecks are. Debug level logging enabled on most production applications would crash them. Its why you don't do this type of testing in live.

    You can start with one simulated player and just ramp up the numbers each run watching performance. As you add simulated players you'd start to see the pain points and bottlenecks in the code. They may not get as dramatic as what is happening on live but it should be detectable. And from that you can figure where to look and once you make changes you can get a reliable read on whether they had any impact or not. What they are currently doing is hoping they can find a solution.

    What they are currently doing falls into the less then professional end of the IT world.

    I don't understand it either and made a similar post awhile back.

    Like you say, all that would be required to test this AoE theory of their's would be a simple simulation of said amount of players spamming AoEs. I also know they already know this (any basic programmer who has ever worked with game engines does). So I've concluded there must be another reason for this test besides the one described.

    I suspect the combat team is considering transitioning into a cool down based system (instead of the current APM system) in order to narrow the skill gap they want to fix. So I think that's the real reason behind this test. They want to judge how their player base reacts to longer cool downs on their abilities and have cleverly disguised the ploy as an experiment to reduce lag.

    This IS NOT their motivation.

    I've seen the developers say that was a motivation of their's and something they were interested in fixing. They believe there is too wide a gap between high and low "APM" players and want to narrow that. Adding cool downs would definitely be one way to narrow that gap.
    Edited by Jeremy on September 10, 2020 5:24PM
  • techyeshic
    techyeshic
    ✭✭✭✭✭
    ....and those that are in cyrodiil have all changed their builds and bars to not use any aoe's.


    I'm pretty sure thats the idea behind the tests. They are saying they think it is the AOEs doing calculations to determine who it hit in an area and by how much so they want to see if less AOEs are used, if performance gets better. I don't think they care much about how well it works on abilities as they have said they would have to review abilities once they coclude which tests work at reducing the lag.

    The unfortunate thing is; this really came to a head at Update 25 with degrading performance leading up to that and for whatever reason; they can't seem to go back. That also was the patch where they took out a huge chunk of client file size
  • idk
    idk
    ✭✭✭✭✭
    ✭✭✭✭✭
    Tests like these must be done on live server, during periods of high population such as events.

    Bots that ZOS can create on internal/test servers can't replicate all the human behaviors/gear/skill combos players have while playing.

    If they can't replicate them then humans can't do them in the first place.

    Of course humans can test this. The scripts scrips are a simulation, not the real deal. Which means it is quite the opposite by definition.
  • idk
    idk
    ✭✭✭✭✭
    ✭✭✭✭✭
    techyeshic wrote: »
    ....and those that are in cyrodiil have all changed their builds and bars to not use any aoe's.


    I'm pretty sure thats the idea behind the tests. They are saying they think it is the AOEs doing calculations to determine who it hit in an area and by how much so they want to see if less AOEs are used, if performance gets better. I don't think they care much about how well it works on abilities as they have said they would have to review abilities once they coclude which tests work at reducing the lag.

    The unfortunate thing is; this really came to a head at Update 25 with degrading performance leading up to that and for whatever reason; they can't seem to go back. That also was the patch where they took out a huge chunk of client file size

    Correct. However, Zos is saying that it is that we can lay down AoE at a faster pace than intended and we used to do. Our sustain has improved over time. I guess Zos has grown tired of crimping our sustain. In the end, Zos is really just collecting information here. They are not testing to see what design will work best going forward as much as they are working to collect information and then they will devise a solution.
  • Oreyn_Bearclaw
    Oreyn_Bearclaw
    ✭✭✭✭✭
    ✭✭✭✭✭
    Not gonna lie, I was pretty impressed with everything on Tuesday Primetime. It wasnt jammed, but there was a small queue and things seemed pretty crowded. Performance was great, gameplay was Meh, but expected. I felt like it was going to be a solid test.

    By Wednesday, well as stated, it was a ghost town. Cant remember the last time EP was at 2-3 bars during primetime. I cant imagine they are getting useful data at this point.

    They really need to fix the double AP. With no incentive, people aren't going to show up for this.
  • vestahls
    vestahls
    ✭✭✭✭✭
    At this point, why even test? I step for 5 minutes in IC, get sniped once and my FPS drops from 59 to 11, can't do a single thing and just die. That was enough ESO for the day. The whole game is broken, no amount of testing will help.
    “He is even worse than a n'wah. He is - may Vivec forgive me for uttering this word - a Hlaalu.”
    luv Abnur
    luv Rigurt
    luv Stibbons

    'ate Ayrenn
    'ate Razum-dar
    'ate Khamira

    simple as
Sign In or Register to comment.