List of things that Zenimax could do improve Performance. Please share any you can think of.

Thevampirenight

The performance is getting worked on but the stuff they are doing will only go so far. However the stuff right here, should be sacrificed before maybe animation canceling and fast paced combat play people like. Anyways the biggest one that would fix a lot of it I feel would be number one on the list.
1. Percentage Based Calculations converted to flat values.
From what I understand, Percentage based calculations are more extensive. Basically I think either using a dividing or multiplication and adding onto the base of what its increasing.
Actually someone else posted an example of this on the forums I looked it up on google. A class rep posted this on a much older thread. Copying and pasting that here. This was several years ago. Did this because I'm not as great as explaining this concept.

Percentage would be, let's say
Base value of 100, 20% potion on top
100*(1.2)=120
Flat value of 20 would be
100+20=120
If you get for example another 10% buff (and 10 flat to show advantage of percentages) on the percentages, the percentages stack on more than flat values:
Percentage double buff:
100*1.2*1.1=132
Flat double buff:
100+20+10=130
So depending on the situation, percentages buff your stats more than flat values and vice-versa.
Usually, the higher the Base value, the more effective percentages become, with a Base value of 500 we get:
500*1.2*1.1=660
500+20+10=530
Thank you Masel.
Basically its the way things are calculated by converting most if not all percentage based values to a flat based value.
For example

This ability does 50% extra damage, by itself would be like 500 damage increase and say an ability does 1000 damage by default proc it right if its the condition for the bonus and then 50% gets calculated. By changing to flat, basically going by my understanding you would get a flat 500 bonus damage that would proc it would just be added it wouldn't have that complex calculation to it. Though the percentages do seem to have this ability to give you more damage. Flat Isn't as great. However by converting to flat based they can remove many of the complex calculations the game server has to run for every single person in a zerg when having a zerg on zerg fight. That would be a huge increase if they converted every ability every passive every armor set every food passive in the game and all Cp perks to flat based values.

So yes complex calculations need to be replaced by Simple ones aka convert them all to flat and the performance might go up by well a lot.

2. Crafting Bag Limit. The Bag is supposed to be an unlimited bag that you can place millions of crafting things inside. Saddly since they tell us that to much increased inventory space might hurt performance from what I understand and also the housing limit is there for a reason. This right here is needed badly. The crafting bag should have a max limit of 5000 crafting materials. So there has to be a limit to this feature because well if having to many items decreases performance well then having an unlimited bag won't help at all. I like the unlimited bag functionality. But that needs to have a cap.

3. Removal of unneeded Systems such as the Durability system and repair armor system. What the durability system does is decrease the armor effectiveness forcing you to have to repair it. So its actually removing a lot of unneeded code here, one it removes the need to repair gear, but it also removes a lot of coding and processes that determine the durability and effectiveness of the gear. Then they could remove traits or rework traits that decrease this effect. So that would be more code that could be removed. Its an obsolete system that only serves to annoy players. Heck your gear doesn't even look broken so why bother with having it if it looks as good as new even if broke?

4. Decreasing the size of Cyrodiil reducing mob sizes, make the pvp map smaller then it is now. Here is why the map is just to big, the bigger it is the more that has to be calculated on it. For Cyrodiil having it be a bit more scaled down could help with performance in a way there. Though the biggest fix I think will be fix number one turning all percentage based calculations into flat based.
5. Removing the keep upgrading/downgrading system.
6. Reducing the group size you are allowed to have in Cyrodiil to either twelve or eight. Reasoning because the bigger the zerg the worse the performance group play has a big impact on server performance its a big issue in Cyrodiil. Reducing the Number of people that can be grouped up to a reasonable smaller number. Is needed to help with the performance and help reduce the lag sinks. Was going to add this but forgot.

So gist, of what we have the list will keep on growing.
1. Converting all complex percentage calculations to simple flat based calculations.(( Sorry Theory Crafters rather have a working game. Then a complex mess that hinders it. Percentage calculations are just to complex for Eso servers to handle. Its a sad reality ))
2. Crafting Bag Limit
3. Removal of the Durability System/Repair System.
4. Decreasing the Size of Cyrodiil and reducing the number of pve mobs.
5. Removing the keep upgrading/downgrading system.
6. Reducing the group size you are allowed to have in Cyrodiil to either twelve or eight.

Anything you feel should be added to this list please say down below. I will add it to it as people make suggestions. When I get the chance of course.

oXI_Viper_IXo

Make the bridges and milegates indestructible again and get rid of the keep upgrading/downgrading system.

Thevampirenight

oXI_Viper_IXo wrote: »

Make the bridges and milegates indestructible again and get rid of the keep upgrading/downgrading system.

I will add that second part to the list, as it seems more complex.

colossalvoids

Only first point makes sense and it's already started with wrathstone racials i guess, goals just shifted direction for now but it was discussed already. Bet it's tied closely with cp rework as it's probably main offender here. Same with bringing spell and phys defence (penetration probably on a list also, probably more in the future) to a one stat which should not only benefit diversity but will cut unnecessary stats from calculations.

relentless_turnip

I agree, with what you are saying completely.

I think in terms of cyrodiil reducing group sizes and keeping just all buffs(i.e. heals etc...) only effective within groups would be helpful.

Also just growing on the degrading of armor, why not say epic(purple) and above is indestructable... this keeps an element of rp to the game or just remove armor degradation from all pvp zones. I believe at the moment some npc's cause degradation and other don't...

The way cp is calculated in cp campaigns is absolute drain on the server too. A lot could be done there I feel.

Thevampirenight

relentless_turnip wrote: »

I agree, with what you are saying completely.

I think in terms of cyrodiil reducing group sizes and keeping just all buffs(i.e. heals etc...) only effective within groups would be helpful.

Also just growing on the degrading of armor, why not say epic(purple) and above is indestructable... this keeps an element of rp to the game or just remove armor degradation from all pvp zones. I believe at the moment some npc's cause degradation and other don't...

The way cp is calculated in cp campaigns is absolute drain on the server too. A lot could be done there I feel.

I was going to add group sizes being reduced but forgot about adding it. Its on the list for sure as that was what I was going to add anyways but forgot it was on my mind before starting this thread. Thank you for reminding me about that.

Mayrael

Hire full time coders not just some "mercenaries", spend just a bit of what they earn on servers upgrades, you know the stuff that does work, not some shady fixes to some niche stuff - like we all know that both stuck in combat and increasing lags are connected to ball groups or in other words, large amount of players grouped in a tight space. Start with this. AoE skills cause exponential growth of calculations, they need to focus on this not on some ridiculous stuff. Issues we experience most are related to server side, our clients are responsible mostly for distribution of recieved data from servers where all calculations are done (to prevent cheating).

Thevampirenight

colossalvoids wrote: »

Only first point makes sense and it's already started with wrathstone racials i guess, goals just shifted direction for now but it was discussed already. Bet it's tied closely with cp rework as it's probably main offender here. Same with bringing spell and phys defence (penetration probably on a list also, probably more in the future) to a one stat which should not only benefit diversity but will cut unnecessary stats from calculations.

Well Cp work shouldn't be when they do all that they need to start doing it over time and not waiting to do it. They can convert the abilties to flat and releasing them in small patches like your topical small bug fix patch we might get once in a while on pc. For consoles they can group them all together in one bigger patch. There is many abilties and passives that still use percentage based values. All they have to do is make the changes and then just add them in. All they should do is test them internally and then add them to the live game.

Instead of releasing these changes in a dlc/chapter patch they should be proactively adding changes like this over time and not waiting to do so. The more they convert the better off the game will be. I think they should start with some of the class passives that are still percentage based and then onto the class abilties and other abilties and doing small changes to the cp system reworking the worst offenders and adding those into the patches they do. The other stuff is there because they could be possible contributors and their fix or removal helping reduce the strain of it all.

Thevampirenight

Mayrael wrote: »

Hire full time coders not just some "mercenaries", spend just a bit of what they earn on servers upgrades, you know the stuff that does work, not some shady fixes to some niche stuff - like we all know that both stuck in combat and increasing lags are connected to ball groups or in other words, large amount of players grouped in a tight space. Start with this. AoE skills cause exponential growth of calculations, they need to focus on this not on some ridiculous stuff. Issues we experience most are related to server side, our clients are responsible mostly for distribution of recieved data from servers where all calculations are done (to prevent cheating).

They are reworking how the Aoe works with the servers that will be going live with Greymoor from their performance improvement plan. Its something they have confirmed they are working on so they do know about that one and are working to fix it.

Aendruu

Here's a (probably extremely unpopular) suggestion: how about restricting pets and familiars in populated areas? Obviously, familiars are essential in combat, but they're certainly not needed when visiting a bank or while crafting writs; similarly, vanity pets are completely unnecessary in combat. And perhaps a restriction on duelling near wayshrines might also reduce lag in those areas?

For anyone who would like to be considerate to their fellow players, but don't want to have to mess around within their character menu, I recommend the Pet Dismiss addon for a simple key-bound dismissal.

relentless_turnip

Rhypon wrote: »

Here's a (probably extremely unpopular) suggestion: how about restricting pets and familiars in populated areas? Obviously, familiars are essential in combat, but they're certainly not needed when visiting a bank or while crafting writs; similarly, vanity pets are completely unnecessary in combat. And perhaps a restriction on duelling near wayshrines might also reduce lag in those areas?

For anyone who would like to be considerate to their fellow players, but don't want to have to mess around within their character menu, I recommend the Pet Dismiss addon for a simple key-bound dismissal.

I would like them to add a synergy to pets that occurs in towns... You press them and they blow the *** up!

I am joking, but instead of having to apply more conditional code to certain area's, something like this might incentive's people to put the bloody pets away in populated area's...

Thevampirenight

Guys please be serious about this. This thread is to help narrow down things Zenimax, can do to help the game. Not down them. They have done a very good job with making a very great game. When it comes to story telling lore and other things. So I wouldn't want anyone else getting the game because it could be a lot worse say like if perfect world got ahold of it so lets not go into that.
Lets please keep this thread constructive please.

Pauls

Servers hardware upgrade? Looks like recompiling game code isnt working for years.

justaquickword

Mudcrabs are obviously esponsible for all performance issues. They should be dealt with swiftly and without mercy.

Czekoludek

Moving part of the calculations from server side to client side (that was the original design imo, they changed that because of cheat engine). At this point it is easier to fight with cheaters then improve performance as they clearly fail at that. Also improve database queries as they seems to be really inefficient. Even moving only cp calculations would improve performance greatly. Also moving part of addon functionality (like mm price checks) to the base game would allow them to write it more efficiently (modders have limited options). Of course this functionality should be toggle one as not everyone are using things like mm.

technohic

Has anyone mentioned getting rid of Volendong? It shows up, factions stack, lag spikes. It's the opposite of what they did before of putting bonuses in dungeons trying to spread people out

RinaldoGandolphi

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

kylewwefan

Back a long time ago when things worked kind of decently, it was noted the server/client relationship had a “trust” kind of thing going.

I heard this was changed at some point to combat rising issues. (Like unlimited meteors and invincible characters) The game has never been the same since.

I don’t know they could go back to a trusted environment without a lot of heartache and pain; but I think it would give the performance enhancement everyone is really looking for.

precambria

Remove CP PVP from the game. It's super dumb anyway.

REMOVE WEATHER EFFECTS FROM ALL PVP ZONES MY GOD

Targeted heals, the server would have a much easier time figuring out what is happening with heals if they were queued up or channeled on to targets rather than all just be floating around. As much as I think cast times are a BAD MOVE for this game damn it sure would making healing more fun also.

Make animations more flexible and rethink the ones which are clearly not working, AKA fear, hidden cast times on spells due to animations especially wardens, there needs to be a way to compensate for the interruption that they cause basically make animations cancelling more fluid by allowing animations to quickly change from one position to another without snapping or having to use block. We can already see what this would look like a bit with light attack weaving with a staff especially, make it so attacks that would likely be chained are all just variations of the same position and have transition animations for them. That way the roleplay justifications for changing ani cancel can be satisfied and we can keep skill as a part of ESO.

Change the auctions system, it basically forces people into using addons which than proceed to decimate the server, lol try playing the game when a MM update just happened you can literally feel the impact.

Synergy contributes to lag, before they removed the option you could tell when a harmony bomb group was coming near you in BG because the game just slows right down, it's still like that with magwardens sadly, is the northern storm made of frozen memory leaks or something cause it sure feels like it. I don't think there is a real solution to that those things are essential part of gameplay, they just need to optimize the way aoe is dealt with but hell I'm sure that is obvious and they are working on it.

precambria

Oh yeah and consolidate the PVP rewards system, WHY in the blue blazes are you filling up databases with utterly pointless mail spam when you have a currency for PVP already in the game, it's called AP it exists I cannot for the life of me figure out why it's not been fully implemented. If people want RNG rewards they can just buy containers, which BTW already exist too and having the Golden vendor be a maybe AP is not worthless this week is more than enough RNG imo

Lintashi

Several games have this button, that makes graphic settings go to minimum instantly. Press it again, graphics go back as you like. Really useful in events, raids, etc. Also I bet, all these glittery, sparkling mounts, also add to sever strain, together with calculating tons of different sets, with many of them having similar effects. But we will get more and more sets and shining mounts with complex animations in every next expansion.

RinaldoGandolphi

Please note i am not bashing the devs.

ESO was really Bethesda/ZOS 1st real online game. I am sure they had reasons when the game was being made why they used TCP. Perhaps they didn't have the in-house expertise to use UDP. Perhaps the game engine they have wouldn't support it without major re-writes that they just couldn't fit into the timetable dictated by management to release the game. At the end of the day all of us usually have deadlines.

At this point, All we can do is hope they get things sorted out and be respectful about it. thats my plan anyways.

Mayrael

RinaldoGandolphi wrote: »

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

I wasn't even aware ESO is based on TCP, lol, it was obvious for me it should work on UDP protocol... Looool. So that's why we have turn based combat now lol again... Lazy design at it's best... TCP is not designed for this type of usage.

Inaya

RinaldoGandolphi wrote: »

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

This is the BEST, most thorough explanation I've ever read. Thank you for sharing your knowledge. This is the first thing I've ever read that made any sense!

Joy_Division

RinaldoGandolphi wrote: »

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

I'd pay real money to see a ZOS executive address this post.

relentless_turnip

RinaldoGandolphi wrote: »

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

Nicely done buddy! Would really like to hear a developer's response to this @ZOS_GinaBruno @ZOS_RichLambert @ZOS_BrianWheeler @ZOS_Gilliam

Olauron

To better understand what can be done (note: not should be done) to improve performance it is better to compare the initial state of the game (when performance was better) and current state of the game. I want to stress one more time that solutions I will name here are not necessary the only possible ones, there may be another solutions. Actually, I think developers already know these reasons and obvious solutions but they can't find solutions that will not change combat so drastically.

First is the number of zones. Each new zone with its mirrors decreases performance at least a bit. From personal experience I can say that every time new campaigns were added for MYM event overland zones and delves outside Cyrodiil got performance decrease. Last Murkmire event likely ensured more mirrors for Murkmire zone and performance there was good. On the other hand, that resulted in decreased performance in Summerset zone and others that, it seems so, were restricted in creating all usual mirrors. Since there were less mirrors, there were more that usual players in each mirror, performance for those players was worse than before.
On the other other hand, there is a problem of empty mirrors. When a zone is underpopulated it uses more resources than when population is distributed across zones evenly. An 'empty' mirror for the zone will be created even if there is only one player in it. Now we have plenty of underpopulated (old DLC, chapters) zones. I will add that this was likely one of the reasons of Vet zones removal (it is better to have one populated zone than one almost populated and two almost empty zones).

Not adding new zones is likely not an option (at least for chapters). Solution is distributing players across zones more evenly so there will be as less mirrors overall as possible. Psijic skill line and treasure hunter skill line are likely parts of this solution.

Second is combat itself, or number of calculations for combat. We may compare initial combat and current combat from different sides. First side is player effectiveness. Initially most of the players were new to the game. Most were simply doing attacks and skills once a second. Some players were better but they were not numerous. Now most players at least try to be effective, try to increase dps. Players use weaving, players use block-cancel, players use dot rotations... The thing is increasing dps increases server-side calculations and as a result decreases performance. Simply compare one or two (if with weaving) calculations in the first case and 7+ calculations in the second case. There are obvious solutions (including implementing buff and debuff instead of hot and dot) but those will change combat a lot.
Second side is itemisation. Plain constant buffs to attributes or damage and complex conditional 'proc' sets need completely different number of calculations. Solution is decreasing or removing conditions as the first step.
Third side is complexity of character attributes. CP does not decrease calculations compared to no CP. CP passives implement additional conditional effects that further decrease performance compared to initial state of the game.

ZOS_Ragnar

We have removed some bashing comments from this thread that were not adding value to the discussion. Please help keep this thread constructive by considering how you are posting your criticism. When providing feedback, consider whether or not what you’re posting will be something the Dev Team can use to turn into real change. It is possible to be negative, respectful, and constructive at the same time. Rember, we do want to hear your thoughts and feedback, both the positive and the negative.

Muizer

relentless_turnip wrote: »

RinaldoGandolphi wrote: »

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

Nicely done buddy! Would really like to hear a developer's response to this @ZOS_GinaBruno @ZOS_RichLambert @ZOS_BrianWheeler @ZOS_Gilliam

Seconded!

kylewwefan

https://help.elderscrollsonline.com/app/answers/detail/a_id/1133/~/what-ports-do-i-need-to-open-for-the-elder-scrolls-online?


Well, this guy sounds like he knows what he’s talking about. But here you already have to have some UDP ports open. So IDK?

Would rewriting the whole netcode be something feasible at this point? I kind of doubt it. ESO is a potential 10 year project right? We’re 6 years in now.

PrimusNephilim

RinaldoGandolphi wrote: »

there is only ONE thing they need to do:

RE-WRITE THE GAMES NETCODE

That is where the bottleneck is. Their reliance on TCP is unfeasible for how the game is designed. I raised these concerns during the beta numerous times but it fell on deaf ears.

The problem is TCP requires every single packet or group of packets send to be ACK'ed, before the next packet is sent. All packets must be received in order. Any packet that is no received before the TTL value expires is assume to be lost and TCP will actually dial back(throttle the connection) and re-transmit. the times you see 999+ ping, the server simply can't respond to anymore packets so they are discarded. In most applications not being an online real-time game this would be ok, this happens everyday when your watching youtube videos, netflix, surfing the internet, etc...TCP is very resilient in this regard. In the case of ESO though this sorta stuff causes the problems we have today.

There is a reason TCP is no longer used in real-time combat games. Devs learned the hard way in the late 90's early 2000's that it just doesn't work.

They need to re-write the netcode in UDP and TCP should only be used as a control channel to authenicate your account, not for actual game play data.

When you got 50 people on screen, the game is sending TCP requests with location data, skill use, block status, weapon swap, damage numbers, resisitance, etc, etc...all these packets have to be Ack, this is where the overhead and bottleneck comes in.

With UDP, you have to do more of the heavy lifting yourself, BUT it doesn't require every packet to be delivered in order, packets that expire the TTL value are no big deal, as each packet contains all the updated information so no need to re-send, no need to Ack everything. You can write your own custom error handling into the code.

Nothing they do to the client will matter much because the client isn't the problem, the netcode is. As long as that netcode relies on TCP large scale stuff is never going to work right. The problems with the game are not related to framerate or hitching, its related to ping and latency and that is directly tied to their use of TCP.

I 100% guarantee you if they would just spend the money, hire someone to come in and re-write that netcode to UDP it would fix all their issues with Cyrodiil. You would see maybe a max ping of 150 and 30 fps during faction stacks for Emp at the games worse conditions which would be completely playable.

ESO is/was an ambitious project, in many ways the game was ahead of its time. TCP is holding it back though, TCP just doesn't work well for real-time combat games, all your real-time combat and FPS games moved away from TCP due to its limitations.

Furthermore, Zenimax owns the UDP Netcode written by John Carmack for Quake. So they already own the core code to make this happen. All they gotta do is hire someone to write the abstraction layers to put it on the server and client. Do this and watch performance increase 500% overnight.

"Quake uses the UDP Internet protocol as its networking backbone. A slightly older version of the commonly used TCP/IP protocol, UDP offers several advantages for game play. With UDP, each packet is transmitted one time, but there is some packet loss associated with the protocol. Conversely, TCP/IP retransmits packets to ensure reliability -- a key consideration in mission-critical networking. What UDP lacks in reliability, it gains in performance, which is certainly a major issue for gamers. With a lot of packet loss on the network, UDP is more effective for game play because it does not try to resend stale data.

https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

https://www.gamasutra.com/view/news/197460/How_Quake_came_to_one_of_the_worlds_first_online_game_services.php

One thing which bears mentioning is ensuring that UDP-based game stack implementations can’t be leveraged for UDP reflection/amplification DDoS attacks – using nonces, and the like (the NTP monlist fix does this).

If measures aren’t taken to eliminate this problem, not only will your service be abused and perhaps blackholed by a lot of ISPs, but your game servers will end up being DDoSed by miscreants bouncing DDoS attacks through them.