Why Challenge Difficulty isn't split into instances (Post from Finn)

Arunei · June 14

So because it hasn't been brought up officially over here in General by any of the devs and I keep seeing people bringing up instancing Challenge Difficulty for people on higher settings, I figured I'd just go ahead and quote the post that got made over in the U50 Feedback Thread for Challenge Difficulty:

ZOS_Finn wrote: »

So, to answer some questions on Challenge Difficulty and instancing combat. This is going to be a little bit technical to hopefully explain the limitations from that perspective.

Everything in the game has a "weight" to it. This includes monsters (very weight heavy), interactable objects (these very but can be weight heavy), etc. Weight is determined by how much information is stored in a given object. I mention this because we have formulas for how much "weight" we can attribute to a given zone or instance of a zone and that is based on the intended player cap. So, for instance, Overland zones generally have no weight restrictions because we expect a lot of players in those zones so the weight attributed to the zone is justified and won't adversely affect the server. It gets much more tricky in situations where we expect reduced players in a given zone. The most restrictive are solo instances or fight spaces. We have a lot of development tools we use to make sure we do not exceed weight capacities in these spaces so it should feel seamless to players.

Each zone is controlled on the backend by something that determines which of our servers will spin up which zones when new copies are needed. In Greymoor, we saw the effects of the zones being spun up were not correctly attributing weight. Trials would spin up on weight heavy controllers which would result in adverse conditions in any zone connected to that controller. Our engineers took great pains to reorganize the code in these controllers to make sure the zones were even and weight was correctly distributed.

I bring this up to highlight the technical aspects of how the game operates from a monster and zone perspective to further illustrate that spinning up new zones based on difficulty without weight concerns and attribution would be extremely detrimental to the entire game. Needing to essentially double all of the available zones in the game without regard to population and weight would mean the entire server would have some pretty major issues regardless of where you would be playing. Even in Delves and Public Dungeons, locations that were not created with these concerns in mind, would potentially be breaking for the server.

We want Challenge Difficulty to be a fun experience for players and, while most of the feedback is regarding the desire for us not to split players from a design standpoint, the technical concerns are most likely bigger than that. This is not said to dismiss feedback but more provide context.

Please DO keep providing feedback. We have some fixes in the works coming for CD. Mainly around the calculations we are making on the backend so that player damage is more in line with expectations but that is not exhaustive and we continue to monitor things. We are taking a look at the Master difficulty in particular as thats where I think most folks will find their groove and we want to make that as smooth as possible.

tl;dr: It would basically be a huge detriment to the game to have have two separate instances (one of Adventurer and one for the other three difficulty settings) for ALL Overland content going at once, and would as such clearly make things even worse if they tried to do a separate instance for each setting.

DestroyerPewnack · June 14

I still don't get it. Doesn't the game already use sharding/layering systems? What would be the issue with having 4 separated layers running at all times, one for each difficulty type, and adding players into those layers depending on their difficulty setting of choice? If you change your difficulty, you get a prompt that asks you if you're sure, and if you say yes, you get teleported into that instance, much like you would, if you joined a group, and your group leader was in a different instance of the same zone.
And they still wouldn't have to change anything in the zones themselves. As far as I understand it, these difficulty settings only affect your character, by making them take more and deal less damage.
I'm probably missing something, as I don't fully understand the technical aspect of this, but on paper at least, it doesn't sound impossible to me.

Toanis · June 14

DestroyerPewnack wrote: »

I still don't get it. Doesn't the game already use sharding/layering systems? What would be the issue with having 4 separated layers running at all times, one for each difficulty type, and adding players into those layers depending on their difficulty setting of choice?

Where other MMOs are open about instancing and have no qualms about dropping you as the only player in a new instance when the others were full, ESO pretty much hides the fact that there is sharding and spreads the players behind the scenes to maintain a healthy population for both player experience and server strain. Adding a new instancing factor with player choice will mess up the whole process that has been fine-tuned over the past decade.

When suddenly an empty zone has 4 zones at 1% load and a highly populated zone that used to have 4 instances at 60% population load now has 3 normal instances at 50% and 3 difficulty instances at 50, 30 and 10%, the network engineers would have to start from scratch and re-adjust the dedicated (i.e. paid for) processing power per zone and difficulty.

Best case scenario: the usage data of the current difficulty implementation can serve as base for a discussion whether that investment, in time, money and temporary player dissatisfaction due to wonky servers would be worth it.

Gabriel_H · June 14

DestroyerPewnack wrote: »

I still don't get it. Doesn't the game already use sharding/layering systems? What would be the issue with having 4 separated layers running at all times, one for each difficulty type, and adding players into those layers depending on their difficulty setting of choice? If you change your difficulty, you get a prompt that asks you if you're sure, and if you say yes, you get teleported into that instance, much like you would, if you joined a group, and your group leader was in a different instance of the same zone.
And they still wouldn't have to change anything in the zones themselves. As far as I understand it, these difficulty settings only affect your character, by making them take more and deal less damage.
I'm probably missing something, as I don't fully understand the technical aspect of this, but on paper at least, it doesn't sound impossible to me.

You have 1 instance of Deshaan running. It can hold a maximum of 150 players (number made up for illustrative purposes only), but you don't know how many players are going to be going to Deshaan. It takes time to spin up an instance.

So, how many instances do you spin up that allows for the seamless port into the zone? At what point do you direct players from the existing zone to the new one so that you don't have 150 players in one, and 2 players in another? Probably around 75 and split them as evenly as you can.

So now you have 2 instances of Deshaan both with 75 players each, but you have a system that allows those players to change difficulty anytime they like. So, now you need a zone for each of those difficulties, and because you don't know how many will choose what difficulty, and because you don't want the zone either empty or too full you have to spin up 6 more instances.

On paper, it is possible, but there would have to be some changes, such as changing difficulty moves you to a wayshrine instead of to your current location, and you'd have to have separate servers to pull it off, each dedicated to a single difficulty. And of course there is then the other issue: How are you going to pay for it?

Sylosi · June 14

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

Gabriel_H · June 14

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

So, how much more are you willing to pay? The real world requires money. Microsoft require a very high return. Are you going to fund ESO instead?

Sylosi · June 14

Gabriel_H wrote: »

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

So, how much more are you willing to pay? The real world requires money. Microsoft require a very high return. Are you going to fund ESO instead?

Players already pay, this is not a free game, ESO is already more than adequately funded, it had an average revenue of $15 million a month (up to 2024 IIRC). It is merely a question of what is the level of their contempt for their customers. Which is apparently quite high, given how they try to pretend it's a technical issue rather than a question of resources.

As for me, I don't even have the game installed at the moment and strangely enough I don't pay for products I am not using. And that is precisely in part due to their implementation of overland difficulty, because it is useless as a feature if I am going to be stuck in the same instance as people on the normal setting who will simply blow up what I am fighting.

Just typical Zenimax these days, they come up with what could be good ideas, yet implement them terribly (overland difficulty, subclassing, 2 team BG's, vengeance, etc).

No wonder this May was the worst May on Steamcharts since 2017 (which is even more laughable when you consider far fewer PC gamers used Steam in 2017).

BardokRedSnow · June 14

When crossplay is enabled, I'd maybe consider this a good idea. As it stands now though it doesnt do the game any good to split instances even further when there's already dead zones with very little player count.

If there was more of a reason to do harder difficulties than rp and the title, I'd see this as a problem. As it is, its a tacked on feature for the few of us that asked for harder fights in quests and overland content to breathe new life in the game. I don't know who wanted this feature more, people who enjoy soloing occasionally, always, or also people that group, but for me questing is a one, maybe two people thing. That others come in and make a bossfight easy because they're not on vestige mode isnt really much of a problem, its not like you wont get the chance to fight the boss again and questing is already your groups own instance.

In other words, I doubt making more instances for the small percentage I assume of players that even wanted this feature is worth it for them. The addition of crossplay may change that.

SkaiFaith · June 14

Aside from technicalities... Who wants the "instance split" should consider that some players may prefer not to have it: as pointed out, split instances would mean that any time you swap difficulty you get a reload of the zone and you get located to a different instance. While this could be fine for some it could also be an annoyance for others.

Maybe I'm having a fight and since I keep dying I decide to reduce difficulty.
I swap, get a reload screen, get placed in another spot from where I was, and if I was in a group now I don't see them anymore.
That doesn't really sound optimal to me, just annoying.

How the system works right now is a much less stressful experience, both for players and server.

SeaGtGruff · June 14

It's ironic that some players think it's all just a matter of ZOS buying and adding more servers to run the game on, yet some players are very quick to loudly announce that they have, or are planning to, cancel their subs, and to urge other players to "vote with your wallets." Where do they think ZOS is going to get the money needed to buy all of these servers that they want ZOS to buy?

Warhawke_80 · June 14

And yet other MMO's have been doing it for decades......

Soarora · June 14

Honestly, so far I like playing with other people on mixed difficulty. People still playing on adventurer aren’t necessarily overpowered, they can be bad players, so it balances out. It also lessens the pain of being 1-shot when I can res and get back into the fight since everyone else isn’t getting 1-shot. My only pain point right now is that group finder isn’t usable for overland— nobody joins. But I can’t solo group overland content on Vestige.

DoofusMax · June 14

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

Khajiit translates the translation: we don't want to pay; either it's free or we complain (and we complain even when it is free because we're ESO)

Sylosi · June 14

DoofusMax wrote: »

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

Khajiit translates the translation: we don't want to pay; either it's free or we complain (and we complain even when it is free because we're ESO)

Khajiit what? Do you need medical assistance?

Soarora · June 14

Sylosi wrote: »

DoofusMax wrote: »

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

Khajiit translates the translation: we don't want to pay; either it's free or we complain (and we complain even when it is free because we're ESO)

Khajiit what? Do you need medical assistance?

Some people talk exclusively or near exclusively in Khajiiti grammar. I think it’s pretty awesome (pawsome?).

DestroyerPewnack · June 14

SkaiFaith wrote: »

Aside from technicalities... Who wants the "instance split" should consider that some players may prefer not to have it: as pointed out, split instances would mean that any time you swap difficulty you get a reload of the zone and you get located to a different instance. While this could be fine for some it could also be an annoyance for others.

Maybe I'm having a fight and since I keep dying I decide to reduce difficulty.
I swap, get a reload screen, get placed in another spot from where I was, and if I was in a group now I don't see them anymore.
That doesn't really sound optimal to me, just annoying.

How the system works right now is a much less stressful experience, both for players and server.

I don't know, this seems a little exaggerated to me. Like you said, technicalities aside, we already sometimes have to port to a different instance of the same zone when we join a group. And if they changed it to where the group leader is the one who sets the difficulty for the entire group, it would be like queuing into Cyrodiil or IC as a group. You get a pop up message, saying you are ready to be teleported in, everyone accepts, everyone teleports.

I don't see how any of that is annoying, especially when it's something we already do in game, for other things. And really, how often do you join a group anyway? You do it once at the beginning of pledges or to go into Cyrodiil, and then you're with that group for an hour or two. It won't be something that you have to deal with too frequently.

SkaiFaith · June 14

DestroyerPewnack wrote: »

SkaiFaith wrote: »

Aside from technicalities... Who wants the "instance split" should consider that some players may prefer not to have it: as pointed out, split instances would mean that any time you swap difficulty you get a reload of the zone and you get located to a different instance. While this could be fine for some it could also be an annoyance for others.

Maybe I'm having a fight and since I keep dying I decide to reduce difficulty.
I swap, get a reload screen, get placed in another spot from where I was, and if I was in a group now I don't see them anymore.
That doesn't really sound optimal to me, just annoying.

How the system works right now is a much less stressful experience, both for players and server.

I don't know, this seems a little exaggerated to me. Like you said, technicalities aside, we already sometimes have to port to a different instance of the same zone when we join a group. And if they changed it to where the group leader is the one who sets the difficulty for the entire group, it would be like queuing into Cyrodiil or IC as a group. You get a pop up message, saying you are ready to be teleported in, everyone accepts, everyone teleports.

I don't see how any of that is annoying, especially when it's something we already do in game, for other things. And really, how often do you join a group anyway? You do it once at the beginning of pledges or to go into Cyrodiil, and then you're with that group for an hour or two. It won't be something that you have to deal with too frequently.

Let's say the group wipes 200 times at a Dragon for the achievement. And they decide "please Tank swap to Adventurer".

And before someone says "then they shouldn't get the achievement" - that would go against ESO phylosophy of "play how you want". We already have plenty of dedicated Veteran and HM content/achievements/rewards in Group Dungeons, Trials and now Night Market. Overland is expected to be free of those requirements, reason why every Challenge Difficulty achievement smartly only requires Seasoned; only one for Vestige and many players are already saying (if you value their feedback) they'll never touch Vestige again after getting that one.

Doesn't feel a good idea to have an "only Vestige" instance. At least to me. And I use Vestige.

Apollosipod · June 14

Warhawke_80 wrote: »

And yet other MMO's have been doing it for decades......

I've made this argument many times. I love ESO, but compared to other MMOs/engines of the same age it's astounding how poorly it works. It's being held together by skooma and hope. When I play it's struggling with sound, input lag, etc.

I guess what I'm saying is that yes, other MMOs of similar age do this well and function without issue, but the in-house ESO engine just straight.uo can't handle equivalent levels of stress that others can. I wish it could, but it just can't keep up.

Orbital78 · June 14

If it could get me a less crowded instance of some areas, I would use it.

Attorneyatlawl · June 15

We used to have three difficulties in their own instances of the zone in beta and for around a year post launch. Other than the game world being bigger now (and more players to support filling these zones now), I don't get why they can't do it again. The game was much more sparsely populated back then and it worked fine. Plus we don't have faction separation in pve zones now anyway as far as population goes.

Gabriel_H · June 15

Sylosi wrote: »

Gabriel_H wrote: »

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

So, how much more are you willing to pay? The real world requires money. Microsoft require a very high return. Are you going to fund ESO instead?

Players already pay, this is not a free game, ESO is already more than adequately funded, it had an average revenue of $15 million a month (up to 2024 IIRC). It is merely a question of what is the level of their contempt for their customers. Which is apparently quite high, given how they try to pretend it's a technical issue rather than a question of resources.

As for me, I don't even have the game installed at the moment and strangely enough I don't pay for products I am not using. And that is precisely in part due to their implementation of overland difficulty, because it is useless as a feature if I am going to be stuck in the same instance as people on the normal setting who will simply blow up what I am fighting.

Just typical Zenimax these days, they come up with what could be good ideas, yet implement them terribly (overland difficulty, subclassing, 2 team BG's, vengeance, etc).

No wonder this May was the worst May on Steamcharts since 2017 (which is even more laughable when you consider far fewer PC gamers used Steam in 2017).

Let me repeat as you missed the context:

So, how much more are you willing to pay? The real world requires money. Microsoft require a very high return. Are you going to fund ESO instead?

SerafinaWaterstar · June 15

Sylosi wrote: »

DoofusMax wrote: »

Sylosi wrote: »

Translated - Zenimax don't want to spend the money on the extra servers it would take to accommodate more instances.

Khajiit translates the translation: we don't want to pay; either it's free or we complain (and we complain even when it is free because we're ESO)

Khajiit what? Do you need medical assistance?

That you question this makes me wonder if you actually play the game!!!🤣🤣

katanagirl1 · June 15

So I would have thought that the overall number of instances should be about the same for everyone in the same difficulty versus mixed difficulties. That tells me either there aren’t a lot of players using the new difficulties, or that with having one difficulty, even with multiple instances, there is some sharing of the resources (enemies, containers, etc.) and with multiple difficulties they have to be duplicated. I guess they are duplicated within the same instance with mixed difficulties, probably a database issue.

EDIT: oh wait, that sounds like mixing the difficulties is worse than separating them

colossalvoids · June 15

Still there's a merit asking about things they've promised at the very begining of the game - better instancing rulesets, like loading more likely with your guides and friends or in our case here with people of the same interests, instances would be literally the same as currently running 2-3 at a time to evenly distribute players around but one of them now would more likely to have people with a difficulty slider pushed high enough, just not as a strict rule.

Gabriel_H · June 15

katanagirl1 wrote: »

So I would have thought that the overall number of instances should be about the same for everyone in the same difficulty versus mixed difficulties. That tells me either there aren’t a lot of players using the new difficulties, or that with having one difficulty, even with multiple instances, there is some sharing of the resources (enemies, containers, etc.) and with multiple difficulties they have to be duplicated. I guess they are duplicated within the same instance with mixed difficulties, probably a database issue.

EDIT: oh wait, that sounds like mixing the difficulties is worse than separating them

It's not about number of players in an instance on a particular difficulty, it's about the number that could be there.

Instances take time to spin up. So rather than the players having a 5 minute loadscreen, ZOS spin up an instance of every zone in the game. BUT they want those zones to be reasonably populated, rather than just a handful in one zone, and the majority in another.

Lets say ZOS expect there to be 100 players logging on in the hour after the server maintenance. The spin up 4 instances of every zone in the game. So that they can distribute those 100 evenly. Any more logging in then get distributed to the 4 instances. When they get to say 50 players in each zone, they take a view on how many more will be logging on in the next hour. Lets say another 100. They spin up 2 more instances and start the process again for the new 100. All the while managing the previous 100's distribution as they move around the game world balancing population distribution with expected logins and logoffs. Taking down instances and spinning more up as necessary.

Now multiply all of that by four because they don't know what difficulty players will decide at any given moment to select. That is exponentially more difficult to manage, as well as putting pressure on the servers as a whole as the central database is pinged and communicating with more and more instances.

The only way to alleviate that second part would be to have four times the servers, not just the instances. Each with their own central database, each with their own set of instances, but that would also then require a master database for each server to report back to in order to save data. They'd then have to copy characters over when the player switched difficulty, amounting to a loadscreen (like at logon) and removing the seamless transition.

katanagirl1 · June 15

I agree with all that, there is likely some crossover threshold of total possible number of players versus load per instance that determines the size.

I am thinking of it as how much stuff per instance there is with having players of all difficulties in it, the “weight” he was talking about. You would have to keep track of the enemies hp and damage, plus the player’s hp and damage, and that would each be four times the data that would be in an instance that had only one difficulty. That seems like a greater database load to carry. I wonder if they considered it from that perspective.