ZOS, Massive Spike in Ping/LAG in Recent Days - What Gives?

Destai

ZOS_Kevin wrote: »

LadyGP wrote: »

@ZOS_Kevin Still nothing from you on the priority of this. Has this been elevated to anything beyond reaching back out to the team? Has leadership been made aware of this issue? Is this issue even on the top 5 priority lists right now?

We have and continue to talk to leadership about this issue. They are aware and we have been providing them with updates with the thoughts, feelings, theories and reports in this thread. We are working on getting more helpful information to share. But no update at this time.

How long does it take? It's a 5+ month old issue.

We're not asking for a solution, we're asking for some concrete details that it's being worked on and how big of an issue it is from your perspective. How is that information not already available?

Can't you just look at JIRA or the ticketing system and tell us the status, priority, severity, and volume of relevant tickets/stories? Is that request being understood?

And quite frankly, given that you do not respond to these suggestions for better communication, I'm not confident they're being understood or seen. I'd really appreciate if you can respond to the suggestions like those above (both mine and others) for interim status updates. There is a difference between telling us the final solution vs telling us what's being worked on. Not responding makes the situation worse.

In any case, I am glad leadership is aware of the issue. I just hope it’s communicated in a way that directly answers our questions.

ZOS_Kevin wrote: »

Just_Attivi wrote: »

lol... ZOS closed a thread actually trying to get information on how many players are affected, that was kind of in an organized manner. Classic. Expected really.

The thread in question was a good thread, but some of the criteria was asking for some information that could be used to find out personal data from folks. So we now have to talk to some folks about what needs to be done before restoring the thread. I have already notified the OP of that thread via DM to note that.

Can you or someone organize a survey then? That would go so much further than the current approach.

dk_dunkirk

twev wrote: »

dk_dunkirk wrote: »

vsrs_au wrote: »

dk_dunkirk wrote: »

Tandor wrote: »

Aurielle wrote: »

Justosay wrote: »

Aurielle wrote: »

Justosay wrote: »

Played the entire last month on the PVE EU server and had no lag or crashes at all.
The only ping increase from 80 to 130 and very rarely to near 300 was when Windows downloaded a new patch in background.
Thank you, "ZOS engineers".

Good for you. Slow clap.

Nothing is downloading in the background when I play the game. Other online games I play function perfectly with low pings and no spikes. ESO is the only game I play that experiences connectivity issues.

But I don't have any "connection issues". Are we playing different games?
Or is this "the only game" acting differently for you and me, and so you blame the game (it's weird to blame yourself)?

If it was a problem with our computers or our internet service providers, the MANY among us having problems in ESO would be having problems in other games as well. We aren't, though. What about this is difficult for you to understand? Just consider yourself fortunate that you're able to play ESO without issues.

That doesn't mean it can't be a networking issue, as all players will be routed differently to the servers and those who struggle with ESO but are fine with other games will be routing differently to ESO than to their other games. It's clearly not a user issue, nor is it an ISP issue as such but some players have found in the past that by approaching their ISP they have been able to overcome any networking issues by having the ISP re-route them. So far as routing is concerned, there is a commonly held belief that this has a lot to do with those routes which pass through Akamai.

Meanwhile we all suffer from the lack of any confirmation from @ZOS_MattFiror and others as to how extensive these issues are according to their server logs and tickets etc, and what they consider the likely cause(s) to be as well as what they are actually doing in their investigations. We need a more substantial update than we've been given to date.

There have been at least a few comments in this MASSIVE thread (forgive me if I've missed some) that describe people sitting in the same house, and one experiencing problems while the other doesn't. Or, at least, one person experiencing the problem in a much more pronounced way. It's hard to blame ISP's or routing or localized caching when the problem varies within the same endpoint.

Network routes can change even when the 2 end points are the same: the internet is designed to be robust, and the route between 2 points can change based on various factors. If one of those routes just happens to use a router that is problematic (e.g. Akamai) and the other doesn't, then they will have different performance.

The "internet" is much less "robust" than you think. Entire ASN's go down and get misconfigured, and people are just stuck. There's usually very little redundancy between you and some service you're trying to get to, unless it's some massive thing like Google or Facebook. While I don't know anything about ESO's architecture -- and who does, definitively? -- I seriously doubt something that only serves, say, 20K average daily players has multiple, live, redundant data centers and active connections. And even if they did, and 2 computers wound up in different facilities, they wouldn't be playing together.

If 2 clients in the same house are on the same "megaserver," and can see each other in game, they're taking the same route.

I defy anyone to show me differing traceroutes to ESO servers from 2 different computers in the same house with characters that can see each other in game, where one is experiencing problems, and the other isn't. Not going to happen. The situation is completely far-fetched, and you're giving ZOS a benefit of the doubt when it's not merited. And even if this were to happen, it would be still on ZOS to fix their caching and proxying configuration, and not just to shrug their shoulders and blame "the network."

I can be playing two accounts on the same computer at the same time, with both characters 'seeing' each other in the same location:
1.) It's not uncommon that character A can be face-to-face trading with character B, While Character B sees Character A standing 30 feet away, looking the other way.

2.) It's also not uncommon that both characters can be standing on the same dead NPC, Character A can loot it while Character B gets the 'out of reach' message.

3.) I've had instances when both characters can be interfacing with each other when one will lose connection to the server and be logged out.

Those three types of common occurrences indicate to me that there's most likely a certain de -synchronization in the engine that isn't even apparent to most people, because most people don't have occasion to have two accounts on one machine matching the actions/situations cited above.

Edit to add: Just for clarity - I occasionally play two accounts on line at the same time on one machine, with each account on a separate monitor, usually so I can craft researchable items on one and trade to the other. Sometimes Character A will be in group with Character B to do things like allow Character B to port to a wayshrine.

You seem to be in a perfect position for a test. I’m going to do some research.

silky_soft

I don't know what's the deal today. But the client says I have 213ms ping, which is 40 is less then normal. But I'm significantly worse off.

I can barley hit anything in pvp and if I'm hit, nothing happens then all the damage at once.

Also in pve I'll instant kill a normal overland monster and it will just hang there with some health. I'm unable to do anything and it will just drop insta dead and I'm back to playing.

LadyGP

ZOS_Kevin wrote: »

LadyGP wrote: »

@ZOS_Kevin Still nothing from you on the priority of this. Has this been elevated to anything beyond reaching back out to the team? Has leadership been made aware of this issue? Is this issue even on the top 5 priority lists right now?

We have and continue to talk to leadership about this issue. They are aware and we have been providing them with updates with the thoughts, feelings, theories and reports in this thread. We are working on getting more helpful information to share. But no update at this time.

Thank you, it's reassuring to hear that leadership is 1) aware and 2) being kept up to date about this.

These are the kind of updates that I think would be helpful. Knowing what is happening (updating leadership but more importantly working on getting something helpful to share) is useful information that I think people can appreciate. Yeah, we don't have a true update on a resolution to the issue but here is where we are at, what is currently happening, and what we are trying to do for future plans - is a nice update.

KaironBlackbard

cptscotty wrote: »

Just_Attivi wrote: »

lol... ZOS closed a thread actually trying to get information on how many players are affected, that was kind of in an organized manner. Classic. Expected really.

HAHAHAHAHA

At least the community got a response

Man. I hop back on a few hours later (aka right now) and find my thread got closed.
I was trying to
A: Get a real headcount, without having 5000 posts from the same person/people creating hoards of pages, as has happened here with reply and reply on reply and reply on reply on reply etc.
B: Get some extra specs that can be used by repair people to solve problems, without the really personal direct location/address nor real names.
C: info is just what people have been putting into this thread, but only randomly: location, system specs, network (aka ISP) specs, and list of issues.
D) Does NOT include residential address or real name, nor IP address. Several people have put their IP address into this thread here, and I feel it could put them in jeopardy, I just wanted a safer way to do it that narrows the field to look at while NOT pinpointing the exact user location.
In being generic, especially if many have the same area/state, or same ISP in said state, I feel it would totally help them narrow it down, potentially determining which ISPs or Servers are bad, and figure out either a fix or a reroute to get it working again.


Two dislikes about the BBCode:
B ) becomes
D : becomes
It messes with my listing (as you can see my D has a different thing than the rest. I normally prefer ")" but because of I did the ":" instead, and had to edit because D made )

KaironBlackbard

Destai wrote: »

ZOS_Kevin wrote: »

Just_Attivi wrote: »

lol... ZOS closed a thread actually trying to get information on how many players are affected, that was kind of in an organized manner. Classic. Expected really.

The thread in question was a good thread, but some of the criteria was asking for some information that could be used to find out personal data from folks. So we now have to talk to some folks about what needs to be done before restoring the thread. I have already notified the OP of that thread via DM to note that.

Can you or someone organize a survey then? That would go so much further than the current approach.

A survey would be nice. That's essentially what I tried to do but because the posts can be read publicly there's that legal issue of potential bad actors using it against people.
Although I thought mine was safer than what people have put in this thread so far.
Truly only asking state and issues and Gamertag, with the optional ISP and hardware. No real names, no IP addresses as has been supplied numerous times in this thread, preferably not MAC addresses (perhaps I should have mentioned that, specifying don't give your MAC address in system (hardware) specs for your safety.)

Is that one of the potential issues @ZOS_Kevin? That I didn't specify limitations?
I did specify to not go further located than state.
Perhaps I should have also mentioned to not give mac address under system specs.

LadyGP

KaironBlackbard wrote: »

Destai wrote: »

ZOS_Kevin wrote: »

Just_Attivi wrote: »

lol... ZOS closed a thread actually trying to get information on how many players are affected, that was kind of in an organized manner. Classic. Expected really.

The thread in question was a good thread, but some of the criteria was asking for some information that could be used to find out personal data from folks. So we now have to talk to some folks about what needs to be done before restoring the thread. I have already notified the OP of that thread via DM to note that.

Can you or someone organize a survey then? That would go so much further than the current approach.

A survey would be nice. That's essentially what I tried to do but because the posts can be read publicly there's that legal issue of potential bad actors using it against people.
Although I thought mine was safer than what people have put in this thread so far.
Truly only asking state and issues and Gamertag, with the optional ISP and hardware. No real names, no IP addresses as has been supplied numerous times in this thread, preferably not MAC addresses (perhaps I should have mentioned that, specifying don't give your MAC address in system (hardware) specs for your safety.)

Is that one of the potential issues @ZOS_Kevin? That I didn't specify limitations?
I did specify to not go further located than state.
Perhaps I should have also mentioned to not give mac address under system specs.

I wish they would do a survey.. I actually made a thread about that very subject.

https://forums.elderscrollsonline.com/en/discussion/667117/zos-do-a-survey-for-performance#latest

But like I said in that thread they are going to come back and say they just did a survey and aren't going to do another one that close back to back.

MedicInTheWild

I have went on strike, after 10 years of playing I have now not logged in in 3 weeks. Watching the player base leave and money lost is the only reason they seem to be interacting now. Performance is not a new problem, hell PVP has been riddled with poor performance since the beginning and yet they add more stuff to increase the problem. Why add new sets when you have 30+ sets that nobody uses anymore, why is the hammer still in pvp when all it does is cause chaos of performance, why nerf/destroy sets that pressure ball groups to encourage ball style play when it lags the server to the lowest point in hell..

The community have already give some awesome ideas to try and help performance, hell Myself I asked why we can not have a campaign to test changes in outside of PTS which is logical as it does not change current campaigns and some cant download pts due to disk space. Make changes and promote play by rewards, should not be that hard crown store gets new stuff all the time just put some of the staff making content to give away.

Performance should be a top priority, why are we focusing on new content when the old doesn't work? Fix the current state of the game then add new content. I myself cant even look in the crown store when the game is broken, why put anymore into a game I have sunk thousands in when I cant even use the mount I buy when im stuck in combat..

XOXO

dk_dunkirk

silky_soft wrote: »

I don't know what's the deal today. But the client says I have 213ms ping, which is 40 is less then normal. But I'm significantly worse off.

I can barley hit anything in pvp and if I'm hit, nothing happens then all the damage at once.

Also in pve I'll instant kill a normal overland monster and it will just hang there with some health. I'm unable to do anything and it will just drop insta dead and I'm back to playing.

Agreed. I’m trying to level morphs in Craglorn tonight, and a quarter to a half the time, my skills are lagging, and I just have to wait for it to clear the queue. For a game that requires 50ms of timing to weave properly, this is flat busted.

Nharimlur_Finor

I was actually expecting something like this:

https://edition.cnn.com/2024/10/05/politics/chinese-hackers-us-telecoms/index.html

to be the problem.

This issue has been in the public domain for close to two weeks, so clearly it is still something else.

Just_Attivi

ZOS_Kevin wrote: »

Just_Attivi wrote: »

lol... ZOS closed a thread actually trying to get information on how many players are affected, that was kind of in an organized manner. Classic. Expected really.

The thread in question was a good thread, but some of the criteria was asking for some information that could be used to find out personal data from folks. So we now have to talk to some folks about what needs to be done before restoring the thread. I have already notified the OP of that thread via DM to note that.

This is a valid reason, then, though I do wonder why not just edit the post? I've had posts edited for simply restating things said by Devs, I'd imagine editing the post to remove specific listed criteria would have been equally swift (though, obviously every company has different SOPs and maybe certain questions in posts trigger thread closure, but adding details to the reason for closing the thread that reflect this would have yielded fewer flames, rather than just 'hey go try to collect data in this 70 page thread instead' as the original take down message implies.)

blktauna

DinoZavr wrote: »

in my humble understanding, players having no, or very little issues, play at EU servers, which are relatively stable nowadays.
NA servers seem to have issues for quite many players. Freezes and desync for several seconds is hard to miss.

I play regularly on the PCEU servers and have plenty of issues. So do a large number of my various guildmates.

blktauna

silky_soft wrote: »

DinoZavr wrote: »

in my humble understanding, players having no, or very little issues, play at EU servers, which are relatively stable nowadays.
NA servers seem to have issues for quite many players. Freezes and desync for several seconds is hard to miss.

Does EU have to go via Akamai?

As I follow the trace, you have to go through akamai to get to eso servers. The usual use of akamai is to cache frequently used info to lighten server load times. Maybe one of those is misconfigured. When I worked at a large sports company's website we used akamai servers to do a/b testing, intercepting ddos attacks and cdn caching. I'm assuming the same use here. We had issues similar with bad configurations but corrected those fast, so I'm not sure what up here.

Aliniel

I'm on EU and I didn't notice these issues - not at such high frequency anyway.

But if it was affecting me, I'd be pissed as hell after 5 months of empty promises.

Maybe it's time to shoot higher - make a petition, send it to the Microsoft gaming division bosses. Sometimes I wonder what they really know of the business ZOS runs and of the poor reputation ZOS has. After all, it's been a few years since Zenimax Media acquisition. It's time to hold Microsoft accountable as well.

Coo_PnT

ZOS_Kevin wrote: »

Just_Attivi wrote: »

lol... ZOS closed a thread actually trying to get information on how many players are affected, that was kind of in an organized manner. Classic. Expected really.

The thread in question was a good thread, but some of the criteria was asking for some information that could be used to find out personal data from folks. So we now have to talk to some folks about what needs to be done before restoring the thread. I have already notified the OP of that thread via DM to note that.

That's not the main problem. The simple fact is, can this problem be fixed or not? That's what we want to hear from ZoS. What is the situation?

NotNi.ya

I had to quit one of my cores because of performance issues. I just couldn't do it any more. It's kinda sad when you stop working for something you're close to because zos refuse to fix the issue!

I'm big mad!

dcrush

PC/NA just now. Game seemed ok until I joined a BG because it’s a battleground event weekend.

The entire BG I had between 800-999+ ping, teleport pads were taking 4-5 seconds to register, skills weren’t going off but somehow I wasn’t disconnected… until the score was 10 points away from the end of the match and I got disconnected. Meaning it most likely won’t count for rewards and I wasted my time.

This game is becoming more unplayable by the day, my playtime already went down massively but I think I’m going to be done pretty soon. It’s been fun but when a game is more frustrating than fun it’s best to step away.

LadyGP

dcrush wrote: »

PC/NA just now. Game seemed ok until I joined a BG because it’s a battleground event weekend.

The entire BG I had between 800-999+ ping, teleport pads were taking 4-5 seconds to register, skills weren’t going off but somehow I wasn’t disconnected… until the score was 10 points away from the end of the match and I got disconnected. Meaning it most likely won’t count for rewards and I wasted my time.

This game is becoming more unplayable by the day, my playtime already went down massively but I think I’m going to be done pretty soon. It’s been fun but when a game is more frustrating than fun it’s best to step away.

At the point that a game becomes frustrating and not fun... that is when you know to step away.

Servadei

I've noticed that ESO sometimes still runs in memory in background processes even after the game is closed. I don't know how long it's been doing this because I only started monitoring my system closely since ESO hangs and disconnects so much to find issues and saw that.

I don't have this problem with any other programs or games, just ESO. My computer is brand new.

YandereGirlfriend

Servadei wrote: »

I've noticed that ESO sometimes still runs in memory in background processes even after the game is closed. I don't know how long it's been doing this because I only started monitoring my system closely since ESO hangs and disconnects so much to find issues and saw that.

I don't have this problem with any other programs or games, just ESO. My computer is brand new.

My subjective experience has been that the "new client" is substantially worse than the old one. The old one, for me, was completely issue-free, whereas the new client will do as you say: hang, refuse to close, throw obtuse network errors, etc.

Really unclear what was "wrong" with the old client but IMO it wasn't broken and it didn't need fixing.

Sluggy

Servadei wrote: »

I've noticed that ESO sometimes still runs in memory in background processes even after the game is closed. I don't know how long it's been doing this because I only started monitoring my system closely since ESO hangs and disconnects so much to find issues and saw that.

I don't have this problem with any other programs or games, just ESO. My computer is brand new.

Comes and goes with the patches. It was like that for me at the very beginning of the year when I reinstalled the game. It would crash at close every time. For me that stopped happening sometime around June maybe? I've also noticed some really bad memory leaks at times. IC in particular. Every time I go into EP's part of the sewers the framerate starts to drop by the minute and won't recover until I restart the game.

Aurielle

dcrush wrote: »

PC/NA just now. Game seemed ok until I joined a BG because it’s a battleground event weekend.

The entire BG I had between 800-999+ ping, teleport pads were taking 4-5 seconds to register, skills weren’t going off but somehow I wasn’t disconnected… until the score was 10 points away from the end of the match and I got disconnected. Meaning it most likely won’t count for rewards and I wasted my time.

This game is becoming more unplayable by the day, my playtime already went down massively but I think I’m going to be done pretty soon. It’s been fun but when a game is more frustrating than fun it’s best to step away.

Yep, it's not worth it to waste valuable free time on something that's causing more frustration than enjoyment. Plenty other MMO fish in the sea out there. If we continue logging in and struggling with these issues, all it communicates to the bean counters is the fact that, hey, we're still logging in. If we're still logging in, there's still a chance we'll spend money on the crown store...

LalMirchi

Well the game decided to zap my User Settings file just now.

Game crashed and when I opened it again I was back at the starting point as if I never played the game before. UserSettings.txt was set to 0 bytes by some mysterious process.



This has happened before so I occasionally duplicate UserSettings.txt so I have a backup. No way I'm going through the initial setup again and duplicating all my settings manually would be a extremely painful and meaningless time-sink.

How does one even attempt to open a ticket on a problem like this?

LadyGP

LalMirchi wrote: »

Well the game decided to zap my User Settings file just now.

Game crashed and when I opened it again I was back at the starting point as if I never played the game before. UserSettings.txt was set to 0 bytes by some mysterious process.



This has happened before so I occasionally duplicate UserSettings.txt so I have a backup. No way I'm going through the initial setup again and duplicating all my settings manually would be a extremely painful and meaningless time-sink.

How does one even attempt to open a ticket on a problem like this?

This has happened to me as well more than once. There are a few people in here who also mentioned that it happened to them when their game crashed. How any of these issues are happening is beyond me.

Dojohoda

So... since whatever this is isn't resolved, what's going to happen when the new battlegrounds drop? New things attract interest and, currently, one can dc while looking at a menu.

Just_Attivi

Dojohoda wrote: »

So... since whatever this is isn't resolved, what's going to happen when the new battlegrounds drop? New things attract interest and, currently, one can dc while looking at a menu.

Dear Players,

We know there is a lot of excitement surrounding the upcoming battlegrounds update! We are excited too, and can't wait to hear all the epic tales of battle in our newsletter!

Due to recent performance issues, we ask players to do their part to minimize potential problems. When you go to enter a queue for battlegrounds, please instead hit [Esc} and close your eyes. Now imagine the epic battle unfolding, and how amazing you are doing in this imaginary pvp battle! We are so proud of you, you won! Next, visit the crown store and buy something to celebrate! we have a whole new line up of restyled flags and banners for you to pay $40 a piece for! These actions should mitigate many of the problems players are facing, while still resulting in a positive experience for all involved! So keep those thinking caps on and imagine away!

---

Jokes aside, I imagine BGs will be still frames presented in a random AI generated order so that we think we are playing, but in reality, we are just holding the player 2 controller while our older sibling plays.

Nharimlur_Finor

We're all failing to see that, just because MS bought the Zenimax Media company and its assets; ESO, as a discrete entity may not actually be that important to them.

YandereGirlfriend

LalMirchi wrote: »

Well the game decided to zap my User Settings file just now.

Game crashed and when I opened it again I was back at the starting point as if I never played the game before. UserSettings.txt was set to 0 bytes by some mysterious process.



This has happened before so I occasionally duplicate UserSettings.txt so I have a backup. No way I'm going through the initial setup again and duplicating all my settings manually would be a extremely painful and meaningless time-sink.

How does one even attempt to open a ticket on a problem like this?

This has happened to be a bunch of times as well.

Think of it like the game intentionally breaking the 4th Wall a la the Psychomantis fight from OG Metal Gear or Inscryption, LMAO.

Just kidding, this is actually really bizarre and definitely should NOT happen.

Seravi

Never mind. It will just get "snipped". Good luck guys and gals.

silky_soft

9/10 melee heavy attack miss even when im ontop of the target tonight. I can follow it up with any skill and they work. But the heavy attack won't. Which means no damage, not resource gain on offbalance target.