Maintenance for the week of May 27:
• PC/Mac: No maintenance – May 27

The true history of Cyrodiil's server problems and hope for the future.

Lowbei
Lowbei
✭✭✭✭✭
@ZOS_GinaBruno @ZOS_JessicaFolsom @ZOS_BrianWheeler @ZOS_RichLambert

This discussion is not meant to cause drama, bait any responses, or disrupt in any way. It is meant to clear up some misunderstandings and state hope for the future.

While discussing Cyrodiils performance in guild last night, I found that the majority of players version of history seemed to be unfortunately incorrect, obscured by time and lack of communication. At this point, fixing Cyrodiil requires a good understanding of how we got here.

During development, it was determined that adding PvP would bring in a vast crowd of players from the other side of the market. The most successful PvP game to date, Dark Age of Camelot, which at one point had hundreds of thousands of accounts, was determined to be the model, due to its design around 3 sided realm combat, which could be easily fit into the ESO universe. Members of DAoCs team were brought in to work on it. The design was worked out, and Cyrodiil was chosen as the location, mainly due to its centralized map geography, and being centered around the Imperial City, which would serve as this games Darkness Falls, a pve/pvp location open to only whichever side was winning.

The marketing campaign to court PvPers was extensive, and worked very well. Tens of thousands of PvP focused players flooded the beta server at launch. During this beta, hundreds of players on screen at the same time fought with very little lag, all skills fired when pressed, bars swapped, no health desyncs, etc.

What happened next, was unfortunate.

During this time period, small skilled groups coming from DAoC and another pvp games, dominated the large zergs, as can be expected. This lead to the usual spam reports of statements like "we had 40 people, how did we lose to 4 people?!?" and "they must be hacking, theres no way..." and "if this isnt changed, me and my massive zerg guild will cancel our subscriptions," etc.

Now a quick history lesson on the subject of zerging.

DAoC devs were use to the flood of angry reports from zergers dying, and knew to disregard it, because logically you cannot die to 4 players AoE abilities unless you are stacked like a blob and not paying attention. In truth, most of these threats to quit were just salty nonsense, as people hate losing in anything competitive, and would rather blame the system itself than their own failures. In DAoC, the answer given to them, and adopted by the good groups, was "spread on incoming" and "pay attention to your surroundings” aka pan your camera. This resulted in a strong pvp community with dozens of groups 8v8ing every night, as well as zergs still roaming with 100+ people. Due to a high skill ceiling, the pvp community was very healthy and lasted many many years.

Fast forward to Warhammer Online. During its beta, zergs were getting dominated by small skilled groups, and made the usual demands and threats to quit. Unfortunately, the dev team had changed quite a bit since DAoC, and they were not experienced enough to know to ignore the flood of reports. I can understand it from a customer service reps view (being a non game player) that if the majority demand it change, it should change. Thus, a massive change was wedged into the launch patch notes to say that AoE caps were now enabled, so that you could only hit a few people with your AoE abilities, allowing zergs to win via sheer numbers. Unfortunately, and as expected by me and other experienced PvPers, the result was a mass exodus of all the skilled groups within a month from launch, which left only zergs. With no actual competition going on (zerg v zerg isnt competition, its an unorganized mess), the pvp community died, and thus the game died quickly.

The exact same thing happened in Guild Wars 2, so I wont go into detail there.

Now back to ESO. During the week before launch, due to the mass reports from zergers dying to skilled small groups, an internal discussion was had on the subject, resulting in somehow the same quick decision being made, to add AoE caps to the launch build. it was wedged into the patch notes claiming it had always been there, despite that being both incorrect and clearly a very strange thing to say in patch notes. Patch notes are to update you to changes, not to say "this is how this works FYI" for no apparent reason. This had two major consequences. One being the mass exodus of the skilled groups for obvious reasons, and the introduction of blob groups. In a blob zerg, due to the AoE cap, you could be content to be nearly unkillable, since any AoE would likely hit a different 6 people each time due to movement. The combination of a loss of skilled smallman groups, and introduction of blob zergs, drove away the majority of the tens of thousands of PvP only players who came to play ESO.

The massive loss of the pvp crowd, resulted in a removal of resources from the cyrodiil servers, which combined with blob zergs, caused massive server calculation issues. The movement of dozens of players tight together spamming AoE abilities was/is tough on the server, as it has to determine which players receive the effects of AoE abilities (again limited to 6) on constantly moving blobs based on the individuals x y z coordinates, not to mention the lighting effects involved as well. This caused constant server crashes and performance loss to all involved, resulting in even further exodus of pvp players.

The unfortunate solution to all this, was to first lower the population caps from the thousands boasted before launch to much lower numbers, to make the population bars look higher. Then to address the players, by stating the creation of a new spell, magika detonation, which would by design, only destroy zergs. The concept of removing AoE caps, which had been repeatedly stated over and over to be the only real solution, was once again pushed into the light by the few skilled PvP groups left, because magika detonation wouldnt work unless AoE caps were removed as well. Unfortunately, once again the zergers immediately began their angry campaign against the changes, and once again due to sheer numbers of them, magika detonation was nerfed to increase only 25% per player, up to a cap of 250%. This meant zero change to blob groups, and the server instabilities continued. Eventually, after the second major population cap drop (now only a few hundred players) to make the bars look populated again, somebody got the message that AoE caps were the cause of Cyrodiils problems. The statement was made that they may be removed, once again followed by the zergers angry demands and threats against what would destroy their only playstyle (stack on crown and spam 11121), thus AoE caps were only nerfed, to where now the first 6 players take full damage, then 6 more take 50%, then 25% for the next 6, etc... Thus the blobbings continued, because that was meaningless to a 24+ player blob group.

The end result. Finally, many years too late, the decision was made to remove AoE caps completely. Most of these blob groups then either left the game, or started playing smaller groups to avoid embarrassment at losing to 4 people. Cyrodiil finally had the chance to be playable. Unfortunately, due to most of the server resources having been moved away from Cyrodiil in the years prior, it was now unplayable with 20+ people in one area at once, with massive health desyncs, skills and bar swaps not happening, nobody taking damage, and ping jumping around between 300 and 999+. The loss of both the skilled groups and the majority of zergers, lead to the tiny Cyrodiil population we have now, with not even enough players to fill two campaigns.

So here we are today. The big reason for this thread, is because going forward, a correct analysis of what happened is important. Its not that the MMO market consists of only 1-2% pvp players now, its that they all left for good reason, because no games are offering them a competitive environment with servers that can handle a fight.

My conclusion, and the big takeaway here, is that Cyrodiil CAN be popular. It was hindered from its potential for so long that it lost its following, but the system itself is still viable. As stated by numerous players, during PvP events you see spikes of server performance that are unseen the rest of the year, which are clearly a resource shift, which then disappears at the end of the event. If ZOS adds resources (a lot of resources) and creates a new marketing campaign to announce this, they will bring in a massive influx of thousands of new and old players, excited to see Cyrodiil at the potential they saw in beta. These players are still out there, waiting for any MMO with a good combat system to make the correct environment available.

We want to like your system.
We want to give you our money.
We want to play your game.
We cannot if you dont allow the environment required for PvP.

Thank you for your time.
Edited by Lowbei on March 29, 2020 10:14AM
  • TelvanniWizard
    TelvanniWizard
    ✭✭✭✭✭
    ✭✭✭
    Very good post.
    Options
  • WilliamESO
    WilliamESO
    ✭✭✭
    I will give thousands € for a kickstarter new cyrodill or for new servers...
    Options
  • Neophyte
    Neophyte
    ✭✭✭
    Amazing post Sir.
    Options
  • IndianaJames7
    IndianaJames7
    ✭✭✭✭✭
    Very good post.

    ^
    Options
  • ryh
    ryh
    ✭✭✭
    No money from PvP = no point to care about PvP.

    What MMO is now worth to play in similar fast paced action combat style with massive battles?
    Options
  • ValueDrift
    ValueDrift
    ✭✭✭
    Interesting info, though I wonder about something. I started playing after AOE cap was removed, only Cyrodiil campaign I play is NoCP/PCEU, and I have never seen a zerg get killed by a small group of skilled players. It takes either a bigger zerg, or a big PvP guild. If we want to give a small organized group a fighting chance against a zerg, then the game probably needs some more item sets and skills dedicated to the purpose.
    Options
  • Kebabkruncher
    Kebabkruncher
    Soul Shriven
    Thanks for some rational info, instead of the demented rants that populate these forums
    Been here since day 1, and still have lots to learn!
    Options
  • Raudgrani
    Raudgrani
    ✭✭✭✭✭
    Would Insightful, Agree and Awesome it - only if I could.

    Just still in 2016, the Cyrodiil was still so awesome - I think my "feelings" towards Cyrodiil are still mainly based on that, and not the current state of the game. I don't believe I would have become a diehard PVP'er if I entered Cyrodiil for the first time today.
    Options
  • Lowbei
    Lowbei
    ✭✭✭✭✭
    ValueDrift wrote: »
    Interesting info, though I wonder about something. I started playing after AOE cap was removed, only Cyrodiil campaign I play is NoCP/PCEU, and I have never seen a zerg get killed by a small group of skilled players. It takes either a bigger zerg, or a big PvP guild. If we want to give a small organized group a fighting chance against a zerg, then the game probably needs some more item sets and skills dedicated to the purpose.

    The combination of Vicious Death and Magika Detonation (as well as other misc abilities), wielded by a small group of skilled players can bring down a blob zerg of any size. It is unfortunately rare to see nowadays now that very few skilled groups are left, which is not an insult to current groups, but nevertheless accurate.

    I would link some videos of my smallman wiping zergs, but in the past they were deemed "naming and shaming" and I dont want to derail the thread.
    Edited by Lowbei on March 29, 2020 9:22AM
    Options
  • Thevampirenight
    Thevampirenight
    ✭✭✭✭✭
    ✭✭
    It was indeed a very informative post and very interesting to read by the way.
    PC NA
    Please add Fangs to Vampires.
    Options
  • GeorgeBlack
    GeorgeBlack
    ✭✭✭✭✭
    ✭✭
    All I read is "we the 'skilled players' were killing zergnoobs. Zergnoobs complained and we the 'skilled players' left the game.
    Since we the 'skilled players' left the game, Cyro was empty and ZOS stopped supporting PVP (facts ye? facts) and now we lag.

    And then I see some ppl that I wont describe as I would like, in an attempt to affiliate with the "skilled players", liking this topic, which is titled ".......... Cyrodiils server problems.........." in which the only reference to performance within this self pat on the back of a "skilled player" is that ZOS pulled resources from PvP.

    In no other game have I seen 1vX and your so called 4 vs 40.
    Do you know why that is? It is not due to your imaginary skill.

    It's because Cyrodiil is a battleground in which all sorts of people say "I will pvp now!" and queue. Any level any build with no idea that they are about to face builds that turtle and burst.
    These so called skilled players defeat people who though "oh! I can play as I want with my bow wielding tank nightblade. "Dont forget ZOS said massive pool of abilities to play as you want.

    Do you want to see skilled combat? Go play an open world mmorpg. You will see some guys alone kill 2-3 enemies and they all know how to pvp thanks to:
    1)Time spend leveling up against challenging overland
    2)The constant possibility that another player might attack you
    3)Real mastery of a real class, and not fotm meta gear and rotation
    4)Real guilds fightin meaningful sieges to gain control of a castle/town against other real guilds and their alliances.
    5)Open world raid bosses who are contested by rival players. Pvp and PvE at the same time (challenging stuff, not the senches and the goblins you come across in cyrodiil)
    6)again real mastery of a class. Not a pre determined turtle rotation followed by a pte determinted burst rotation that works against all enemies:
    Healer
    Tank
    Warrior
    Archer
    Summoner
    Assassin
    Mage
    Bard


    As for the technical issues of Cyrodiil I dont have any response because all you said was: us skilled PvP players left the game and Zos pulled away the resources. What??
    *** it Ill give you an answer.
    Eso was based on open client end calculations to allow usage of addons because they didnt do a good enough job in having essential Gameplay Settings and UI customization + features like Dressroom.
    And then they also taxed their system with anti cheats.

    And then content after content after content every 3 months to sell a story questline to all those single player people has created problems in the code.





    But please... dont come out and say "let me tell you why cyrodiil lags..... It all started when me and 3 other skilled players killed 40 noobs that didnt build for 1 button spam function on the crown....."
    Edited by GeorgeBlack on March 29, 2020 7:53PM
    Options
  • oxygen_thief
    oxygen_thief
    ✭✭✭✭✭
    ValueDrift wrote: »
    Interesting info, though I wonder about something. I started playing after AOE cap was removed, only Cyrodiil campaign I play is NoCP/PCEU, and I have never seen a zerg get killed by a small group of skilled players. It takes either a bigger zerg, or a big PvP guild. If we want to give a small organized group a fighting chance against a zerg, then the game probably needs some more item sets and skills dedicated to the purpose.

    i think this talks about zergs vs small scale groups are full of hypocrisy. from my personal experience so called small scale people just camp at resources and farm loners 6vs1. they attack pvers zerg to feed their ego when they know they will win and run away or die of experienced ball groups to come here and whine about official support of the filthy casuals. why should player be able to survive if 50 people simultaneously land light attack on him? is it just because he feels some kind of superiority? numbers matters
    Options
  • ItsMeToo
    ItsMeToo
    ✭✭✭✭✭
    The OP forgot about a lot of those so called skilled people used CE.
    FYI - There is no such thing as 'night capping' in a world wide MMO.
    FYI - There was no paid Beta. When they launched the game the Beta was over, even if you don't think it was.
    FYI - It's B2P not F2P. There is a difference.
    FYI - It doesn't take any player skill to mash keys or buttons in this game. The ones that stay alive longer have the better internet connection and speed.
    FYI - The game is not broken, it still works. It just has 'bugs' that need to be fixed.
    Balance is a "Bad" thing.

    Example: There were hundreds of Jedi and only two Sith in Star Wars. The Jedi wanted, "Balance in the Force" and they got it. Now there are only two Jedi and two Sith.

    Balance is a "Bad" thing.
    Is the glass half full or half empty?
    I say, "Get a smaller glass."
    Options
  • GerStyler
    GerStyler
    ✭✭✭
    Amazing Post.

    Btw. zergling trash are a shame and i will bag everyone from you in cyro !
    Options
  • Starlock
    Starlock
    ✭✭✭✭✭
    ✭✭✭✭✭
    References?

    Not at all saying the OP is being untruthful, but why should we believe everything in their post at face value without references? There’s no reason not to add them if there’s time - it would elevate what they say with evidence and thus carry even more weight.
    Options
  • Sergykid
    Sergykid
    ✭✭✭✭✭
    nobody needs any history or to know about any history to be like it is now. When i first joined i knew nothing about anything and i was still playing zergs. Anybody today can imagine a zerg playstyle, be it history in other games or not.

    and yes it's not fair for 4 people to kill 20. I don't care how skilled you are, skill is seen in 2v2/3v3 rated arenas. Not everyone is a mathematician to calculate every bit from every build to come up with something that can 1vX, so you can't expect anyone to accept that his enemy's ability hits 2-3 times higher damage while taking 2-3 times less damage so he can 1vX, or playing a busted class or a cheese build.
    -PC EU- / battlegrounds on my youtube
    Options
  • linoge63
    linoge63
    ✭✭✭
    If ever there was a post that should be perma stickied--THIS is it
    Options
  • Sanctum74
    Sanctum74
    ✭✭✭✭✭
    ✭✭✭
    Sounds like some conspiracy theory stuff to me. In reality performance decreased dramatically with the lighting patch which transferred many calculations to the server side to combat cheating. From that point on each patch added more spaghetti code causing problems due to rushing content. They also started adding more proc sets, costumes, flashy mounts and skill animations. Everybody’s cp also started increasing causing much more server calculations as well.

    To say resources were taking away seems unfounded since the performance issues can be traced back to the lighting patch and then each patch afterwards. That would also suggest they were taking resources away each patch which would make no sense. The problem is their coding and there is no amount of resources or improved servers that can change that. They need to take the time to fix their code or the performance will continue to degrade with each patch.
    Options
  • Gilvoth
    Gilvoth
    ✭✭✭✭✭
    ✭✭✭✭✭
    please fix game
    i cannot pvp with constant extrteme high ping now 300+ in red constantly unable to even do aNY DAMAGE
    very angerring
    Options
  • flubber77
    flubber77
    ✭✭✭✭
    @ZOS_GinaBruno @ZOS_JessicaFolsom @ZOS_BrianWheeler @ZOS_RichLambert

    The problem for this is that each time they add something they ruin 10 other things. and to fix that the broke ofc they break a new one. and the biggest problem is that they dont care to admit and keep adding new *** before fixing the old issue.

    If they ever gonna fix this they need to start at the beginning where it all started, Lightning patch and how the game looks when using skills. but also the latest "nerf" to blockcancel and and animation cancel in general with lightweave etc.

    Go back in time and fix this. and ofc upgrade your hamsters serverside.
    Still a grudge, only to see false what u want and nothing less.
    Options
  • VaranisArano
    VaranisArano
    ✭✭✭✭✭
    ✭✭✭✭✭
    Can you provide your definition of "blob groups"?

    Because there's a distinct difference between naturally occuring faction stacks, stacked PUG raids of multiple groups of 24 players, large guild raids of 16-24 players, smaller guild raids of 8-16, and small scale. Short of small scale, I've seen all of those come under fire for blobbing, zerging, and causing performance problems.

    I generally agree with your conclusions, but I think it's worth being precise about which playstyles are being complained about.
    Edited by VaranisArano on March 29, 2020 2:55PM
    Options
  • Varana
    Varana
    ✭✭✭✭✭
    ✭✭
    Next time I should pull stuff out of my own ... as well, about how bloody awesome I was, and get tons of Insightfuls. ;D

    What this post for whatever reason fails to mention, is the widespread use of cheats, esp. Cheat Engine. Not a single word about that, and I find that ... fascinating.
    Options
  • VaranisArano
    VaranisArano
    ✭✭✭✭✭
    ✭✭✭✭✭
    Varana wrote: »
    Next time I should pull stuff out of my own ... as well, about how bloody awesome I was, and get tons of Insightfuls. ;D

    What this post for whatever reason fails to mention, is the widespread use of cheats, esp. Cheat Engine. Not a single word about that, and I find that ... fascinating.

    Yeah, good point. ZOS did originally have a lot more of PVP handled client-side, but thanks to cheaters and particularly Cheat Engine they had to move many functions server-side with a noticeable impact on performance.

    People. We're why we can't have nice things.
    Options
  • yodased
    yodased
    ✭✭✭✭✭
    ✭✭✭✭✭
    The lack of any sort of real consequence for exploitative behavior and the allowance of straight up cheating put the playerbase into a perception of what this game is.

    Ask 100 people who pvp about cheating,exploits,overperforming sets or skills and see what those answers are.

    The devs lackadaisical approach to curtailing cheating and exploits in the first 2 years of the pvp game set the tone and its put us where we are now.

    People cheated hard, no one did crap about it so it continued and everyone either left, joined or dealt with the fact that there are players that cant be killed" player that had unlimited resources, players who increase lag and latency by being near them, players who move faster in combat than fully sprint mount, players who can fly etc etc.

    This got to a boiling point where we had a social media push with flying players dropping unlimited meteors.

    This publicity caused them to overcorrect and add an exponential increase of server side variables that were not in the initial server design specifications.

    Over taxing the server with all these calculations is what started the problems and every patch afterword has been trying to balance anti cheat and performance.

    Tl;Dr People cheated ZOS over corrected causing server hiccups.
    Tl;dr really weigh the fun you have in game vs the business practices you are supporting.
    Options
  • Urvoth
    Urvoth
    ✭✭✭✭✭
    ryh wrote: »
    No money from PvP = no point to care about PvP.

    What MMO is now worth to play in similar fast paced action combat style with massive battles?

    Where does this false narrative come from? They don’t make PvP content that people could buy anyway, and most regular PvPers buy all the chapters and have ESO Plus. The incorrectly perpetuated myth that PvPers don’t spend money is just straight up wrong. If anything, I’d say it’s the other way around. All you have to do is go mount up on a resource during prime time to see that nearly everyone is running around on apex or radiant apex mounts.
    Options
  • Lowbei
    Lowbei
    ✭✭✭✭✭
    In no other game have I seen 1vX and your so called 4 vs 40.
    Do you know why that is? It is not due to your imaginary skill.
    i think this talks about zergs vs small scale groups are full of hypocrisy. from my personal experience so called small scale people just camp at resources and farm loners 6vs1. they attack pvers zerg to feed their ego when they know they will win and run away or die of experienced ball groups to come here and whine about official support of the filthy casuals. why should player be able to survive if 50 people simultaneously land light attack on him? is it just because he feels some kind of superiority? numbers matters
    Sergykid wrote: »

    and yes it's not fair for 4 people to kill 20. I don't care how skilled you are, skill is seen in 2v2/3v3 rated arenas. Not everyone is a mathematician to calculate every bit from every build to come up with something that can 1vX, so you can't expect anyone to accept that his enemy's ability hits 2-3 times higher damage while taking 2-3 times less damage so he can 1vX, or playing a busted class or a cheese build.

    I will provide some videos that may help you understand then. Note that these videos are not meant to name and shame anyone, but to simply show examples of zerg bombing. Plus I would bet that 99% of the people in them dont play anymore.

    Firstly, to the statement that you have never seen any other game with 1vX and so called 4v40. Allow me to introduce you to DAoC, the game that Cyrodiil was modeled after.

    *note that the music may be extremely dated*

    Dem Hibbies - https://youtu.be/PFZ7NeSgYwY?t=74
    Skip to 1:14

    Eulogy - https://youtu.be/-BxyNKATs2s?t=32
    Skip to :32

    Danish Hibcarls - https://youtu.be/lh9jISkiVA8?t=197
    Skip to 3:17

    You can youtube the words "zerg bombing" to find a LOT more, in a lot of games.

    Now for ESO... because I have been called self-congratulatory in this thread, I will link only *one* video, and skip past the smallman fights.

    *note that we are not the same Dem Hibbies group from the above video*
    Dem Hibbies - https://youtu.be/z80NOqSLx94?t=812
    Skip to 13:32 and the rest of the video is zerg bombing I believe.
    ItsMeToo wrote: »
    The OP forgot about a lot of those so called skilled people used CE.
    Varana wrote: »
    Next time I should pull stuff out of my own ... as well, about how bloody awesome I was, and get tons of Insightfuls. ;D

    What this post for whatever reason fails to mention, is the widespread use of cheats, esp. Cheat Engine. Not a single word about that, and I find that ... fascinating.
    yodased wrote: »
    Tl;Dr People cheated ZOS over corrected causing server hiccups.

    There were less than 20 total players using Cheat Engine in Cyrodiil. Most of them were banned. The result however was that zergers had new ammo in their argument that they died unfairly to smaller groups, because like I said above, "people hate losing in anything competitive, and would rather blame the system itself than their own failures." Less than .1% of players were cheating, and for a short period of time, yet I see that excuse thrown around here. It is incorrect.
    Edited by Lowbei on March 29, 2020 6:54PM
    Options
  • GeorgeBlack
    GeorgeBlack
    ✭✭✭✭✭
    ✭✭
    Not gonna watch any of that.
    Want skilled pvp? Go see Line][Age 2003-2008 and Tera Online 2014.

    Eso combat has 0 skill.
    Options
  • VaranisArano
    VaranisArano
    ✭✭✭✭✭
    ✭✭✭✭✭
    The fact that relatively few players were cheating does not change the facts that ZOS significantly altered how Cyrodiil was run server-side in response to the public embarrassment of cheating.
    Options
  • Lowbei
    Lowbei
    ✭✭✭✭✭
    Not gonna watch any of that.
    Want skilled pvp? Go see Line][Age 2003-2008 and Tera Online 2014.

    Eso combat has 0 skill.

    Having played both of those games as well, I disagree with your opinion.

    Have a nice day.
    Edited by Lowbei on March 29, 2020 7:10PM
    Options
  • GeorgeBlack
    GeorgeBlack
    ✭✭✭✭✭
    ✭✭
    Lowbei wrote: »
    Not gonna watch any of that.
    Want skilled pvp? Go see Line][Age 2003-2008 and Tera Online 2014.

    Eso combat has 0 skill.

    So you made a false statement, which was proven wrong via video evidence, and now refuse to watch it...

    I understand. Have a nice day.

    The false statemate was that because 'skilled players' left the zergodiil, zos lowered the server performance.
    And if people expect Zos to sticky a topic that says "We lowered our games capabilities" I rly dont know how people think that thinking works.
    Have a nice day Im going to sleep
    Edited by GeorgeBlack on March 29, 2020 7:02PM
    Options
This discussion has been closed.