PC/EU - Time Trials unplayable during peak hours

Kilnerdyne · July 2020

Userid: @Kilnerdyne
Server: PC EU
Location: All 12-player Time Trials

When: From 6pm - 12pm CEST veteran time trials are unplayable on PC EU.
What happens:

All players in the raid receive high latencys
Skill usage is delayed 2-5 seconds or sometimes does not go off at all.
All enemies dance backwards & forwards on the spot and are unresponsive to taunt.
Incoming & outgoing damage and healing is either delayed or does not happen at all.
Mechanics are delayed by up to 5 seconds making certain mechanics impossible to avoid or counter.

Since when has this been happening: Harrowstorm update was the patch that amplified this problem the most but it has actually been a problem in small parts since way before harrowstorm.
What can be done about this: Maintenance & patching department at zenimax need to advertise another performance update & never deliver.
What will be done about this: Nothing, it might even be made worse.

Kilnerdyne · July 2020

Standard Replies

"Check with your internet provider, it's probably a problem with just you not the game in general."
- We're a multi-national raiding guild (Pathfinders). All 12 players in the group are experiencing the same issue almost every evening & we don't live together on the same connection. Infact we're in at least 8 different countries on different providers.

"Maybe think about upgrading your PC, you're probably just on low fps because your PC sucks."
- I can't speak for others in the raid (although I know at least 3 of them have ridiculous setups) but i'm on a Ryzen 9 3900X @ 4.6GHz | 16GB 3200DDR4 | RTX2080 | 1TB NVMe SSD

"There's ongoing performance patches to make the game more stable, just be patient."
- Since One Tamriel patch this has gone from players being hopeful for the future, to cautiously optimistic, to losing faith, to no longer believing anymore, to dismissing the likelihood any such patch is indeed forthcoming or ongoing.

Elsonso · July 2020

Hopefully, some improvement with Update 27

"Trial Performance: Our metrics have shown that Trial performance on the server has not been a consistent experience. We were able to track down the root cause of this–instance launchers running multiple Trials simultaneously would routinely perform worse overall. We've taken a pass at the algorithm which determined how Trial instances are distributed and now factor this into account. This will decrease the chances of our instance launchers being overloaded in high usage situations and impacting Trial performance as a result."

Flaminir · July 2020

Sadly its been a lot longer than Harrowstorm. But yep... all of this ^^^

Kr3do · July 2020

Elsonso wrote: »

Hopefully, some improvement with Update 27

"Trial Performance: Our metrics have shown that Trial performance on the server has not been a consistent experience. We were able to track down the root cause of this–instance launchers running multiple Trials simultaneously would routinely perform worse overall. We've taken a pass at the algorithm which determined how Trial instances are distributed and now factor this into account. This will decrease the chances of our instance launchers being overloaded in high usage situations and impacting Trial performance as a result."

We've been promised improvements countless times before. My hopes aren't very high.

stefan.gustavsonb16_ESO · July 2020

Good work, really, but too slow.

It's troubling that it took them this long to track down the root cause to be a load balancing problem causing a processing bottleneck and/or a network congestion. That would have been my first guess, and it ought to have been theirs as well. So, they have only now gotten past verifying that their first guess was correct, and made some changes to the instance launcher algorithm to prevent multiple trial instances from being spawned on the same node of their server cluster. That's nice, but thoroughly unimpressive considering the time it took to even begin fixing an urgent and severe problem with trials.

I stopped doing veteran trials a good while ago because of this, and I'm not eager to start again until I hear reports of clear and consistent improvement. I hear nothing but grief and gnashing of teeth from the group I was running with before. They say the random lag makes far too many instances absolutely unplayable during prime time, and about half the time they just give up on a run. It's very disappointing to gather in a group of 12 friends for an hour of fun and have it all fizzle out into nothing.

I sincerely hope this works.

visionality · July 2020

As sad as it is, the PC/EU-server is simply not laid for the number of primetime players it is being sold to. There are ofc programming issues, too, but if you compare the performance of PC/US with PC/EU (assuming the programming issues are the same on both servers), you realize that PC/EU is having additional problems all summing up to 'too many players for too little server'. Whether you go for vet trials in PVE or Cyro-PVP, you have the 2-5 second skill delay everywhere, you can see it even in 4-man-dungeons where you die from invisible boss damage that you clearly avoided 2 seconds ago.

What can be done about it, is very simple: more server capacity on PC/EU.

ZOS likes to argue that primetime on EU is more player-intense than on US, due to the smaller time difference in EU countries. A look on any world clock will tell you how embarrassingly poor that excuse is. Furthermore, even if there would be differences in the login behaviour of players on PC/EU to PC/US (and I dont want to claim there are none), it is not taking a lot of brains to foresee it after five years of selling a MMO and to adapt accordingly.

stefan.gustavsonb16_ESO · July 2020

visionality wrote: »

...
ZOS likes to argue that primetime on EU is more player-intense than on US, due to the smaller time difference in EU countries. A look on any world clock will tell you how embarrassingly poor that excuse is.
...

To be fair, the US population is spread out across four time zones, with large population centers in all of them, while most of the EU players are in a single time zone, with only the UK/Ireland and Russia to either side. However, given that the peak hours are from 6PM to 12PM, it shouldn't make a huge difference. I'm pretty sure it's one of the contributing reasons, but they may indeed be exaggerating the impact from this.

igniz93 · July 2020

I also have an issue where my character will randomly do heavy attacks instead of spells during trials, skills like endless hail not firing off, etc.

stefan.gustavsonb16_ESO · July 2020

When the lag monster hits, it seems anything can happen. Just yesterday I had my ultimate (Shooting Star) glitch against a vet dungeon boss: the ultimate key gave off a "clank" sound and I saw the message "the target is immune". Better than having the ultimate fizzle and lose the points, which also happens a lot, but seriously? "The target is immune"? To Shooting Star as a whole? And that prevents an ability with a strong AoE component from even firing against a target? I have never seen anything like that before, and I see no reason for it. Ten seconds later, the same ability fired just fine against the same target.

Sylosi · July 2020

stefan.gustavsonb16_ESO wrote: »

To be fair, the US population is spread out across four time zones, with large population centers in all of them, while most of the EU players are in a single time zone, with only the UK/Ireland and Russia to either side.

Incorrect.

Portugal is also in the same time zone as the UK, there are also a bunch of countries at +2 like Greece, Finland, Ukraine, Turkey, etc.

stefan.gustavsonb16_ESO · July 2020

Sylosi wrote: »

stefan.gustavsonb16_ESO wrote: »

To be fair, the US population is spread out across four time zones, with large population centers in all of them, while most of the EU players are in a single time zone, with only the UK/Ireland and Russia to either side.

Incorrect.

Portugal is also in the same time zone as the UK, there are also a bunch of countries at +2 like Greece, Finland, Ukraine, Turkey, etc.

Yes, I included only the large population centers for clarity. I should have stated that, sorry. The number of players in Portugal is small compared to UK/Ireland, and even Portugal/UK/Ireland combined is still not a huge portion of the EU population. Greece+Finland+Ukraine+Turkey etc combined is a somewhat larger chunk, agreed, but most players in that zone are most likely from Russia. The point I was trying to make is that a very large portion of the players on the EU server are in the CET zone, while the NA players are more evenly spread out across four zones, so it's not a completely invalid argument from ZOS.

gatekeeper13 · July 2020

I consider myself lucky if I can use a potion or activate a synergy in trials by hitting the X button less than 5 times.

Not even LA apply. And I realize that because my Rele stacks vanish.

FatherDelve · July 2020

Now imagine Cyrodiil at this time, where you're not supposed to follow simple mechanics, and moving out of a red circle wont save your day.

ShawnLaRock · July 2020

Ran CR a few times yesterday unable to use any abilities, swap bars, or break-free - but was still able to access map & inventory menu. Had to completely dashboard to fix. It was also eery to be in the middle of a boss fight with no in-game sound.

S.

Celephantsylvius_Bornasfinmo · July 2020

It's all broken now, anyway.
Dragon's keep shouting in VSS and you literally have to open the crown store to stop it....oO

Glaiceana · July 2020

Even now after 5 years I still hope things will be improved for the PC/EU server... My patience is almost infinite.

paulsgruff · August 2020

Agree 100% with OP. I don't usually get to play during peak hours, but did Cloudrest today (Saturday) at 2pm UK time and it was shocking. Portal completely broken (Examples: 3 fake spears already spawned in upon entering the portal, when the group sent down the real 3, there was 6 on the ground inside the portal. Explosion when you jump back down from the top was so desynced that I went up, waited, jumped down to match what was on screen, picked up a core and was running with it before the explosion happened, killing dds and tank. Character was also completely disappearing at random). Mechanics were also totally desynced in main room (Main boss completely vanishing for up to 15 seconds when teleporting, people getting kicked to login screen at random points on every attempt, synergies showing as normal but unusable, character performing actions but doing no animations at all, character doing animations but skills not actually firing, bar swaps completely breaking, jumps from 125-900 ping when usually experiencing 65-80 maximum in the same content in the early hours of the morning)
#Joke

Gythral · August 2020

Peak hour being all those that the server is not off (for maintinance) it seems

TineaCruris · August 2020

PvP was standard weekend unplayable today too.

BazOfWar · August 2020

This isn't restricted to trials either because it's also happening in Dungeons too at least for me it is.

Getting one shot by bosses on veteran mode because animations are not matching when the damage is put on you, so you think your blocking but your either late or too early.

poe · August 2020

This time its way worse than Harrowstorm. The thing I just CANT fathom is HOW they keep messing this EVERY patch and why there isnt any communication on it, lik WHAT THE HELL IS WRONG WITH THE GAME FFS? And dont you dare saying: "were working on it...", "we are aware....", "we have perfomance patches...." hahaha you are not even a joke anymore ZoS but a farce. You should be ashamed.

BazOfWar · August 2020

poe wrote: »

This time its way worse than Harrowstorm. The thing I just CANT fathom is HOW they keep messing this EVERY patch and why there isnt any communication on it, lik WHAT THE HELL IS WRONG WITH THE GAME FFS? And dont you dare saying: "were working on it...", "we are aware....", "we have perfomance patches...." hahaha you are not even a joke anymore ZoS but a farce. You should be ashamed.

Seen this before is so many mmo's, They need to spend money to sort it out and that is something they are NOT prepared to do because they want to spend as little as possible with maximum profit.

They are like a cowboy plasterer, rather than take a bad wall down they will just add layer upon layer of plaster to cover the cracks.

stefan.gustavsonb16_ESO · August 2020

From the recent comments added to this thread, I take it that the problem persists. I just wonder: did it become less common after they tried to make the server processes for trials spawn more evenly across their hardware to balance the server load? It would be good to hear some statistics from ZOS on this after the recent update. A thread like this will naturally mostly consist of sharing of horror stories, and the problem might be much less severe now. I'm not saying it is, not even having any great hopes for it, just wondering.

Getting 12 people from my old vet trial progress group together to try, only to find that things are still bad, would probably make quite a few of them give up on the game for good. I'm hesitating, and I could really use some good news here.

Czekoludek · August 2020

stefan.gustavsonb16_ESO wrote: »

Sylosi wrote: »

stefan.gustavsonb16_ESO wrote: »

To be fair, the US population is spread out across four time zones, with large population centers in all of them, while most of the EU players are in a single time zone, with only the UK/Ireland and Russia to either side.

Incorrect.

Portugal is also in the same time zone as the UK, there are also a bunch of countries at +2 like Greece, Finland, Ukraine, Turkey, etc.

Yes, I included only the large population centers for clarity. I should have stated that, sorry. The number of players in Portugal is small compared to UK/Ireland, and even Portugal/UK/Ireland combined is still not a huge portion of the EU population. Greece+Finland+Ukraine+Turkey etc combined is a somewhat larger chunk, agreed, but most players in that zone are most likely from Russia. The point I was trying to make is that a very large portion of the players on the EU server are in the CET zone, while the NA players are more evenly spread out across four zones, so it's not a completely invalid argument from ZOS.

Actually in EU you have ppl split through 3 different time zones. You can't say that moscow time zone is not populated with players when zeni launch russian language support because of numbers of players from russia etc. So we have a lot of ppl i'm 3 different time zones yet PC EU is trash when comparing to PC NA with 4 time zones. They should finally spend some money on servers as this joke which they called performance is no longer funny

paulsgruff · August 2020

Also...

Maintenance for the week of August 31
• PC/Mac: No maintenance – August 31

Because it's not needed, clearly. Nothing to see here. Everything is okay

Elsonso · August 2020

BazOfWar wrote: »

poe wrote: »

This time its way worse than Harrowstorm. The thing I just CANT fathom is HOW they keep messing this EVERY patch and why there isnt any communication on it, lik WHAT THE HELL IS WRONG WITH THE GAME FFS? And dont you dare saying: "were working on it...", "we are aware....", "we have perfomance patches...." hahaha you are not even a joke anymore ZoS but a farce. You should be ashamed.

Seen this before is so many mmo's, They need to spend money to sort it out and that is something they are NOT prepared to do because they want to spend as little as possible with maximum profit.

They are like a cowboy plasterer, rather than take a bad wall down they will just add layer upon layer of plaster to cover the cracks.

Actually, one of the problems that ZOS has is that we are all sitting here watching them work. When they mess up, we all know about it and we have to watch while they fix it.

If BGS screws up and needs to tear down the "wall" and rebuild it while making a game like Fallout 4, we never know about it. In cases where it is unfixed on release, it is often fixed in the Unofficial Patch.

If ZOS screws up, it is frequently in the next update and they have to fix it later. In many cases, they actually do fix it later. If they don't, then no one will.

ZOS could double the size of their team and double the time it takes them to make content so that they could spend months in QA fixing things, instead of weeks, and that might solve that part of the problem.

I won't get into the other part of the problem.

stefan.gustavsonb16_ESO · August 2020

Czekoludek wrote: »

You can't say that moscow time zone is not populated with players

I never said any such thing. I do not defend the poor performance of the EU server, nor do I excuse ZOS for ignoring the problem for ages. I just wanted to point out that their argument that the NA population is more evenly spread out across several time zones is not completely without merit. Nowhere did I ever say this makes the horrible server performance okay in any way.