Update on Cyrodiil Performance

deleted221205-002626 · March 2015

I still say reduce the max raid size from 20 - 16 or 12.. this will also help reduce the size of group movements and larger tight coordination.

Shave a little lag off here and there and we just might smooth it out

reddog1948 · March 2015

Since patch 2.0 one new problem has shown up - toon status bars remain empty/or fixed after reviving in Cyrodiil - /reloadui does fix the issue. But doing a /reloadui after every resurrection in Cyrodiil gets old.
Second problem is game crashing - it is worse in heavy populated Cyrodiil areas - and from the comments of my fellow guild members - not a "lone' problem but fairly wide spread. Even just standing still for a while occasionally results in a crash. The game crashes started about patch 1.3 and have just gotten more frequent with each major patch.
Tech support recommendations to "turn off" everything else in the computer (including things like firewall and malware checkers, all add-on, etc) does not fix the issue. There must be some kind of indication as to why this problem is happening more and more frequently.
It is enough of a problem that the game is quickly becoming unplayable. i.e. Game crash while in a POP Locked Cyrodiil instance - means you are effectively locked out of the game until the POP Lock drops. - This can be quite some period of time. I waited over 30 minutes one evening and eventually just shut off the game. There is no feedback as to where you might be in a queue - or the game just can not re-establish a proper connection - the circle spins and spins but you get no where.
I have 32GB RAM with a normal 4.5GB base and under 8% CPU load - with game about an additional 3+GB usually bringing me up to just over 8GB total RAM usage and CPU usage up to around 20% maximum. (figures from Task Manager.)

ThinkerOfThings · March 2015

Thank you to the folks that responded to my idea! Always appreciate constructive feedback, like I read somewhere: "Discussing a problem without proposing a solution is called whining." Pretty sure a Nord said that, maybe a Breton, I'm certain there was a mustache.

Rune_Relic wrote: »

Put these at the end of an impossible quest against inpossible odds.
Anyone who actually does manage to get to them....well.... they earnt them well and truly.
After one day they are withdrawn from nirn and returned to their sanctum.

I agree that some if not all of these skill sets and temporary power ups should definitely be based around known Daedric Artifacts, however personally I feel like there is not enough Risk Vs Reward for the above scenario. Can we come up with fun ways of incorporating these nearly impossible quests into PvP so that it keeps the flow of battle going? If the Daedric Artifact was so hard to get, would having it for such a short time be truly worth it, if not what fun ways can we incorporate that without having players running around all the time in Uber Mode? Taking it a step further, would these proposed artifacts be available outside of Cyrodiil? Something like that sounds like a dangerous recipe for PvE leaderboard disasters.

Part of the reason I constructed the idea the way I did, was so that more players would have access to that feel of super greatness that comes with Emporership, although fleeting, and without bonuses that could effect the balance of PvE as they would be lost upon leaving Cyrodiil or their faction losing the shrine. Even a little taste of that kind of awesome can keep people excited for PvP and wanting to come back for more.

stefan.gustavsonb16_ESO wrote: »

I'm not even sure the band-aid would work at all. After the Daedric artifact was acquired, everyone would probably clump on the current Daedric Champion, friend and foe. The inability of the game to handle large groups in one location would still be a big problem, perhaps even bigger than now.

This too was a concern of mine, do you have any ideas on ways of spreading players out as I did, with diminish returns on the buffs and other factions having their own daedric shrines, one of which becomes active closer to their territory, becoming a new point of defense. What mechanics do you prepose that could help prevent this scenario, either through incentive's not to group up, or by penalties to large groups, such as the current increased damage of siege weapons?

Just some food for thought.

firewatch · March 2015

reddog1948 wrote: »

Since patch 2.0 one new problem has shown up - toon status bars remain empty/or fixed after reviving in Cyrodiil - /reloadui does fix the issue. But doing a /reloadui after every resurrection in Cyrodiil gets old.
Second problem is game crashing - it is worse in heavy populated Cyrodiil areas - and from the comments of my fellow guild members - not a "lone' problem but fairly wide spread. Even just standing still for a while occasionally results in a crash. The game crashes started about patch 1.3 and have just gotten more frequent with each major patch.
Tech support recommendations to "turn off" everything else in the computer (including things like firewall and malware checkers, all add-on, etc) does not fix the issue. There must be some kind of indication as to why this problem is happening more and more frequently.
It is enough of a problem that the game is quickly becoming unplayable. i.e. Game crash while in a POP Locked Cyrodiil instance - means you are effectively locked out of the game until the POP Lock drops. - This can be quite some period of time. I waited over 30 minutes one evening and eventually just shut off the game. There is no feedback as to where you might be in a queue - or the game just can not re-establish a proper connection - the circle spins and spins but you get no where.
I have 32GB RAM with a normal 4.5GB base and under 8% CPU load - with game about an additional 3+GB usually bringing me up to just over 8GB total RAM usage and CPU usage up to around 20% maximum. (figures from Task Manager.)

I have been experiencing numerous crashes while in Cyrodiil since 2.0. I had no problems with crashing prior to that. It's really been a game breaker for me and ZOS does not seem particularly concerned with the issue. I have been playing non-stop since beta, but this is the last straw for me. I am going to unsub until ZOS gets this fixed (if ever) and look for something else to play.

stefan.gustavsonb16_ESO · March 2015

ThinkerOfThings wrote: »

What mechanics do you prepose that could help prevent this scenario, either through incentive's not to group up, or by penalties to large groups, such as the current increased damage of siege weapons?

Patch 2.0.3 added incentives for players to spawn off some small scale and temporary PvP to nearby delves. Still a band-aid solution, still treating the effects of the problem and not the causes, but your suggestion might have a similar and more wide-spread effect to create diversions if the temporary Daedric buff was not an Emperor-like superpower, but a less dramatic buff that didn't make everyone want to hang on to the current Champion of Mehrunes Dagon, or equivalent.

Perhaps the "good" and "bad" Daedric followers could be pitted against each other instead of following the general zerg? Followers of Mehrunes Dagon being able to choose to seek out and battling followers of Azura, separate from the alliance war, regardless of faction? It would add to the complexity, but it would also introduce the kind of smaller scale PvP and cross-faction grouping that people have been asking for. Followers of one Prince could choose to be be allies with each other, but still legal targets for everyone else? Nah. Probably too complex, like a whole different game added on top of what's already in Cyrodiil.

The bottom line for me is that creating diversions to make players not try to fight the epic battles is not really a solution. It's an admission of failure.

JTorus · March 2015

Ninrod86 wrote: »

[snip]

If you're speaking about the Havok game engine, it was clarified long ago that this game does not use it. They used it for some preliminary stuff, but afterwards abandoned it for something custom built. The only reason the logo shows at the splash screen is some legal mumbo jumbo crediting Havok, as it was part of the game's development.

[edited to remove quote]

stefan.gustavsonb16_ESO · March 2015

JTorus wrote: »

The only reason the logo shows at the splash screen is some legal mumbo jumbo crediting Havok, as it was part of the game's development.

They might still be using Havok's physics simulation engine. Not that they are doing a lot with physics, or indeed anything, but still, some remnants of licensed Havok code might still be in there somewhere to handle collision detection.

But yes, the game engine is custom built by ZOS developers. They did not choose an engine badly suited for the task. They created it for this game, so if it's bad for the job, it's nobody's fault but their own.

Sacadon · March 2015

Rune_Relic wrote: »

Sacadon wrote: »

C3N50R3D wrote: »

stefan.gustavsonb16_ESO wrote: »

If things work like I think they might (and I'm only guessing, but hopefully in an educated way), the console platforms might do just fine without all the server side safety checks to defend against hacked clients. Consoles do not allow running unsigned code, and that could be the salvation for ESO PvP. Without all the extra CPU load on the server side, without the increased network traffic and without the increasing sensitivity to latency that we have seen since update 1.2.3, their original "naive trust" networking model might just be able to handle the kind of large scale battles we had back then, and which they are still advertising.

Has anyone confirmed it is the server safety checks that are causing the slowdowns? Have any devs commented on this? If this unlikely scenario is true then I would rather put up with a few cheaters that can be pointed out and banned by the community than a nearly unplayable PvP experience.

No, I've seen nothing to this and doubt they are as a significant contributor as suspected by many.

The security checks that affetc peformance are not what you think they are.
They are normally subroutines that are run on client instead of run on server.
The more the client can do, the less network packets and tasks the server has to do.

If you have the client recording its own location, action, damage, heals etc...then the system is rapid. But then the client can dictate the gameplay and exploit the damage ect it does to other clients.
By "running on server" ZOS keeps control of all damage calcs and just tells the client what it gets.
So there can be no exploits and hacks to make people godlike.

Apparently before the 1.3 patch some have said ZOS was running client code.
When they fixed this security hole it killed the performance.
At came at the same time as the lighting patch so many assumed it was this instead.

No one can be 100% certain of this...but it makes sense how massive combat suddenly became a problem for ZOS. ZOS already confirmed there is no issues with the lighting patch and they have been through it with a fine tooth comb.

Not exactly sure how you know what I think they are, but I'm probably misinterpreting your words... The scenario as you described it, could certainly explain additional server-side load. My stance on this is still that they are not as significant as believed. Instead, I think the design itself is the root cause. I do think if your point is correct, then both of these are at play (design and security change). Maybe the initial design didn't take into account the vulnerabilities and the scramble to address them was either not aligned with or simply amplified and/or exposed issues with the design.

Who is the "some have said" and would like to hear your thoughts on why you think this source is credible.

FadedJeans · March 2015

What is the actual problem?

Is traffic bottlenecked somewhere?

Are hardware resources being tapped?

Are there clustering inefficiencies?

Are there application inefficiencies?

Surely someone is staring at a graph right now, looking at a spike and saying "well, THAT's not good", yes?

Rune_Relic · March 2015

Sacadon wrote: »

Rune_Relic wrote: »

Sacadon wrote: »

C3N50R3D wrote: »

stefan.gustavsonb16_ESO wrote: »

If things work like I think they might (and I'm only guessing, but hopefully in an educated way), the console platforms might do just fine without all the server side safety checks to defend against hacked clients. Consoles do not allow running unsigned code, and that could be the salvation for ESO PvP. Without all the extra CPU load on the server side, without the increased network traffic and without the increasing sensitivity to latency that we have seen since update 1.2.3, their original "naive trust" networking model might just be able to handle the kind of large scale battles we had back then, and which they are still advertising.

Has anyone confirmed it is the server safety checks that are causing the slowdowns? Have any devs commented on this? If this unlikely scenario is true then I would rather put up with a few cheaters that can be pointed out and banned by the community than a nearly unplayable PvP experience.

No, I've seen nothing to this and doubt they are as a significant contributor as suspected by many.

The security checks that affetc peformance are not what you think they are.
They are normally subroutines that are run on client instead of run on server.
The more the client can do, the less network packets and tasks the server has to do.

If you have the client recording its own location, action, damage, heals etc...then the system is rapid. But then the client can dictate the gameplay and exploit the damage ect it does to other clients.
By "running on server" ZOS keeps control of all damage calcs and just tells the client what it gets.
So there can be no exploits and hacks to make people godlike.

Apparently before the 1.3 patch some have said ZOS was running client code.
When they fixed this security hole it killed the performance.
At came at the same time as the lighting patch so many assumed it was this instead.

No one can be 100% certain of this...but it makes sense how massive combat suddenly became a problem for ZOS. ZOS already confirmed there is no issues with the lighting patch and they have been through it with a fine tooth comb.

Not exactly sure how you know what I think they are, but I'm probably misinterpreting your words... The scenario as you described it, could certainly explain additional server-side load. My stance on this is still that they are not as significant as believed. Instead, I think the design itself is the root cause. I do think if your point is correct, then both of these are at play (design and security change). Maybe the initial design didn't take into account the vulnerabilities and the scramble to address them was either not aligned with or simply amplified and/or exposed issues with the design.

Who is the "some have said" and would like to hear your thoughts on why you think this source is credible.

Yes..sorry there was quotes of quotes so I was making a generic reply.. with regard misinterpreting words.

There have been 3-4 mentions of security code updates before and leading to the 1.3 patch to prevent client code exploits.
I could try and look through the 100s of posts that I have read....but I frankly dont have time.
They are no more or less credible than anyone else.
Until ZOS mans up and says this is the issue....we are left searching through scraps.

I believe more has been mentioned on the zos videos.

Our Cyrodiil performance is something we are very aware of. Performance drags when there are numerous players in the same place at the same time. This is why performance in Cyrodiil is fine for much of the day, but gets worse during more popular times. We are currently investigating ways in which we can reduce the spike of performance loss. We added in some features for Update 6 which we hoped would help, but ultimately did not. This is not a situation where we can just add more hardware. Player population in a given area hurts the performance and the more people that are in one area, the more performance is going to be hurt.

What did they bring into 6 to help performance that failed ? The 50 person AoE cap I assume.
There answer was to increase siege damage and use...knowing this is heavy damage but very slow AoE..so minimum damage calcs out of all the AoE available.
While everyone was using siege, there wasnt such an issue with lag.
Compare to comets/meteors firing off multiple times per second hit upto 50 people a time.

As they say it is not population that is the issue....its population density = maximum damage calcs/second.

Bouvin · March 2015

Also... I love the idea of giving players a 60-minute 20% AP bonus for killing Delve bosses.

Then locking them out of Cyrodil for > 60 minutes (while the buff wears off) because they have to re-enter the stupid long Queue to get back into overland.

Well played ZOS well played. Unfortunately, players have caught on to your red herring and are just ignoring the delves at this point.

Bouvin · March 2015

Rune_Relic wrote: »

What did they bring into 6 to help performance that failed ? The 50 person AoE cap I assume.
There answer was to increase siege damage and use...knowing this is heavy damage but very slow AoE..so minimum damage calcs out of all the AoE available.
While everyone was using siege, there wasnt such an issue with lag.
Compare to comets/meteors firing off multiple times per second hit upto 50 people a time.

As they say it is not population that is the issue....its population density = maximum damage calcs/second.

So what they really need to do to fix lag is hire someone to write a better algorithm for calculating damage.

Too bad they aren't getting all of the subscription money any more or they could hire a decent CS Phd to crank out some algorithms for them.

Bouvin · March 2015

stefan.gustavsonb16_ESO wrote: »

The bottom line for me is that creating diversions to make players not try to fight the epic battles is not really a solution. It's an admission of failure.

THIS ^^^^^^^

Rune_Relic · March 2015

Bouvin wrote: »

Rune_Relic wrote: »

What did they bring into 6 to help performance that failed ? The 50 person AoE cap I assume.
There answer was to increase siege damage and use...knowing this is heavy damage but very slow AoE..so minimum damage calcs out of all the AoE available.
While everyone was using siege, there wasnt such an issue with lag.
Compare to comets/meteors firing off multiple times per second hit upto 50 people a time.

As they say it is not population that is the issue....its population density = maximum damage calcs/second.

So what they really need to do to fix lag is hire someone to write a better algorithm for calculating damage.

Too bad they aren't getting all of the subscription money any more or they could hire a decent CS Phd to crank out some algorithms for them.

The algorythm probably works exactly as intended.
IF the problem was moving from client side to server side calcs....then they were on a hiding to nothing.
Nothing wrong with that as long as they accept they cant do the same kind of performance with the same numbers without much tougher managment.
Client side could be lossless damage calcs without issue.
Server side I dont think they can cope with anything like the numbers...so it has to be a lossy system.
I dont see any other option anyway that isnt anything other than a bandaid on a severed artery.

wraith808 · March 2015

stefan.gustavsonb16_ESO wrote: »

The bottom line for me is that creating diversions to make players not try to fight the epic battles is not really a solution. It's an admission of failure.

Or maybe an interim short term fix? There is still that possibility.

Roechacca · March 2015

Anything new ? No ? Kthnx bye

stefan.gustavsonb16_ESO · March 2015

wraith808 wrote: »

stefan.gustavsonb16_ESO wrote: »

The bottom line for me is that creating diversions to make players not try to fight the epic battles is not really a solution. It's an admission of failure.

Or maybe an interim short term fix? There is still that possibility.

OK, let's give them that much and hope for the best. But while I'm waiting, I'm not giving them any more of my money.

wraith808 · March 2015

stefan.gustavsonb16_ESO wrote: »

wraith808 wrote: »

stefan.gustavsonb16_ESO wrote: »

The bottom line for me is that creating diversions to make players not try to fight the epic battles is not really a solution. It's an admission of failure.

Or maybe an interim short term fix? There is still that possibility.

OK, let's give them that much and hope for the best. But while I'm waiting, I'm not giving them any more of my money.

And that's the right response if you're not satisfied. I'm satisfied with what the game gives me, so I'm still in.

stefan.gustavsonb16_ESO · April 2015

wraith808 wrote: »

And that's the right response if you're not satisfied. I'm satisfied with what the game gives me, so I'm still in.

Don't get me wrong, I certainly don't consider my money wasted this far. I have had a lot of fun with the game for the past year, but now I'm running out of things to do. I find myself logging in to the game just to hang around in chat, farm some rare drops, take a seemingly infinite amount of riding lessons and help low level guild mates out, and while I will probably keep doing that, it's not enough to keep me subscribed. When they release new content and/or fix PvP so it's enjoyable again, I will happily send some more money their way.

wOOOOt_of_SD · April 2015

ZOS_PaulSage wrote: »

Actively, we are looking at changing the behavior of the players to remove incentives for large groups to stay in the same area. We want to do this by providing larger incentives for Alliances to split up and take on multiple-challenges in Cyrodiil. We’ll continue to work on this. We are also asked by players if there is anything they can do to help. In this situation, the best thing you can do is split off to different objectives when you notice performance going down. Cyrodiil is a big place with lots of different things to do. And thank you for asking.

Do you even listen to yourselves here?? CHANGING PLAYER BEHAVIOUR??
We came here to eso to play MASSIVE PVP - We came here to be in big battles,
and the nature of the common PVP player in an 3 way faction PVP setting is TO FOLLOW THE BIG BATTLES.

You are not solving any problem with "spreading the crowd" - The crowd DONT WANT TO BE SPREAD..

What are you going to do? Make us get a buff in some dungeons? who cares about PVE - We are there to PVP!
What next - are you going to remove the map indicators of where there is battle, so we cant find each other to fight?
LOL - that would just result in factions communicating and arrange places to fight.

What im trying to point out, is that the nature of this kind of PVP is to have zerg vs zerg. Cyradiil was bloody hell designed for that. And now you try to change player behaviour?? LOL... Get real - you can't!
And you should not! - what you are doing is treating smaller symptoms with bandaids - when you should get back and find and fix the real problem.

And for the record, i play on EU Thornblade, which is full, pop locked 90% of the time.
After the newest patch, the lag feezes got really bad. BUT it was actually good the week before.
So how about you undo some of the f*ckups you did there?

Kupoking · April 2015

http://forums.elderscrollsonline.com/en/discussion/162672/video-of-scroll-steal-during-lagspike#latest

report of lag in cyrodiil

LudovicVendi · April 2015

REMOVE METEOR!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

untill u fix it pls!

LudovicVendi · April 2015

Just look at what dc is doing on thornblade , lag exploiting 12'yearolds.

Sarousse · April 2015

Game has become totally unplayable in AvA, even in small/medium scale battles.

It wasn't like this with the 1.5 version.

We have no update since you posted here, and people will soon stop the game if you don't fix the lag.

Cancelling my subscription until you fix the lag.

kaithuzar · April 2015

@ZOS_PaulSage
Can you verify that certain players or certain guilds are not exploiting/causing the lag; because many of us in the community think otherwise. Some of us even contain what we feel to be "proof". Would you be interested in a one on one private discussion? There are those of us that believe we may be able to offer assistance in fixing the problem, suggestions at least, but prefer to speak on a less public venue than forums.

Fenrlr · April 2015

LudovicVendi wrote: »

REMOVE METEOR!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

untill u fix it pls!

+

Elicit · April 2015

Though I don't care much for PVP. I am a returning player, after 4 months of not playing at all, it's very discouraging to see existing problems not fixed. Playing another extremely popular MMO that I wont mention the name of, their pvp runs so smooth it's unbelievable. Having 3 factions with 24 people on each. Also adding other forms of smaller level PVP could essentially help for the people who are just wanting to just battle other players and not do all the extra stuff. At first people would complain to such restrictions, but after it's running smoothly and with new added ways to PVP. I'm sure they'd get over it quickly.

Yasha · April 2015

I know this is totally anecdotal, but I played intensively on the weekend with two people playing on the same line in the non-vet campaign, and it ran completely smoothly with no unusual lag for most of the time. There was a window of about 30 minutes where we got some strange lag (there was not a particularly large amount of players around or anything) but that went away.

This was a completely different experience to when I was playing the game last year, I don't know if anything has changed, or if certain campaigns have worse lag, but I thought it was fantastic to finally play ESO's pvp without slideshows, rollbacks, huge lag spikes. Note that I didn't join a large group so I don't know if there is still a problem with players dropping offline when in large parties/groups.

jelliedsoup · April 2015

i first saw a skeleton close knit ad zerg many months ago. I'm on AD as well so it's not an anti AD thing just a statement of fact. i don't believe they spammed aoes then.

Im anti aoes anyway if people talk about l2p andskill yet spam aoes this seems to be the height of hypocrisy to me.

TheRealMcKoy · April 2015

Darklord_Tiberius wrote: »

PvP also needs a reboot and a restructure and honestly Cyrodiil feels like it has nothing to do with the rest of the zones. Which is odd because isn't this games idea based around an Alliance War? Incorporating Cyrodiil into the rest of the game would be nice to see, other than those garbage "buffs".

I agree it would be nice to see some impact of the Cyrodiil War in the rest of Tamriel. It would be interesting (although not sure how feasible) that perhaps if one alliance is dominating in Cyrodil that it starts to spill out into the other areas and players could see the war creeping into their zones and have some sort of game mechanism to push it back into Cyrodiil. Make the players feel like there is something at stake. Impose tariffs to travel on roads occupied by another alliance, see trade routes cut off or somehow restricted. Supply shortages in the remaining non-Cyrodiil areas. Encourage small raids to open up supply/trade routes? Think of The Revolution: Citizens having enough of their liberties restricted that they start forming small groups to harass and fight against the dominating power. Small skirmishes that the average player can take part in and not have to feel like they can't make a difference because they are not PVPers or min/maxers. Make these effects lesser as you move away from the borders of Cyrodiil as to not totally ruin the experience for the non-PVPers. I am not sure what the solution is, but definitely something to make the War aspect felt more in the non-Cyrodiil areas.