ZOS, Massive Spike in Ping/LAG in Recent Days - What Gives?

Orbital78 · July 2024

I was having issues around 2 or 3 months ago, it was only ESO servers effected. I did a speed test to Dallas, TX servers and my speeds seemed bottlenecked or throttled and it was only at prime time hours unfortunately. I even called my ISP to report it, giving IP's and information on the game servers I was having issues with. After a week or two things seemed to have smoothed out for me. I average 60-80 ms most of the time but do get 100-150 ms spikes on rare occasion (excluding Cyrodiil).

As of late I haven't noticed any connection issues in overland and trials. During my connection issues period things were basically unplayable though and sadly it occurred when I often did group content with guilds. If you live in AU or EU and are trying to use NA servers I can understand the connection issues, all of the AU players I have played with in the past have almost all hated their ISPs and I am unsure anything can really be done about that.

edward_frigidhands · July 2024

Jaraal wrote: »

show previous quotes

SeaUnicorn wrote: »

Photosniper89 wrote: »

manukartofanu wrote: »

Photosniper89 wrote: »

Okay - thats kind of what I thought happened but wasn't sure. Disappointing. I'm assuming the decision was made that it's not financially worth it for them to invest in refactoring the game but....

I'd challenge anyone in a leadership role in the company to factor in technical debt they have because they refuse to refactor the code. This is a conversation that hassssssss to happen (most leaders don't even know what technical debt is so it's never even discussed).

I don't think it's a matter of technical debt. You can refactor endlessly, but when the issue is with the servers, it doesn't solve anything. If the problem were in the code, everyone would be experiencing exact same issues. But we see that only a portion of the players are affected simultaniously. This kind of suggests that the questions should be directed not at the programmers, but at the administrators of the whole system.

They just upgraded the servers last year and we are right back to where we are. It's not the servers.

Now - are the servers configured improperly some how... sure that is possible. But throwing money and hardware at the issue didn't fix it.

you do realize one could cheap out on hardware and purchase not enough or of questionable quality. In addition, there are other components of infrastructure besides the servers. New compute does not solve issues with the network. Routers and switches handle network traffic and therefore have direct impact on the ping. Few faulty routers means some people experience issues because their traffic gets routed through those, while remaining population does not experience issue because their traffic gets routed through functional ones. In addition there are network components that connect servers themselves, one faulty router there means certain areas that the server behind faulty router would experience issues.

Mr Firor himself said that the ongoing multithreading/code rewrite was causing instability. So it's not necessarily just a hardware issue.

https://forums.elderscrollsonline.com/en/discussion/637691/server-multithreading-update-july-2023/p1

He also mentioned that they would let us know when they had any success with the coding upgrade. And as we are still awaiting word on that, it's reasonable to assume that there's still much work to do.

That is from a while back. There is a specific set of issues that began and worsened during May 2024 and have been ongoing since.

Jaraal · July 2024

edward_frigidhands wrote: »

show previous quotes

Jaraal wrote: »

SeaUnicorn wrote: »

Photosniper89 wrote: »

manukartofanu wrote: »

Photosniper89 wrote: »

Okay - thats kind of what I thought happened but wasn't sure. Disappointing. I'm assuming the decision was made that it's not financially worth it for them to invest in refactoring the game but....

I'd challenge anyone in a leadership role in the company to factor in technical debt they have because they refuse to refactor the code. This is a conversation that hassssssss to happen (most leaders don't even know what technical debt is so it's never even discussed).

I don't think it's a matter of technical debt. You can refactor endlessly, but when the issue is with the servers, it doesn't solve anything. If the problem were in the code, everyone would be experiencing exact same issues. But we see that only a portion of the players are affected simultaniously. This kind of suggests that the questions should be directed not at the programmers, but at the administrators of the whole system.

They just upgraded the servers last year and we are right back to where we are. It's not the servers.

Now - are the servers configured improperly some how... sure that is possible. But throwing money and hardware at the issue didn't fix it.

you do realize one could cheap out on hardware and purchase not enough or of questionable quality. In addition, there are other components of infrastructure besides the servers. New compute does not solve issues with the network. Routers and switches handle network traffic and therefore have direct impact on the ping. Few faulty routers means some people experience issues because their traffic gets routed through those, while remaining population does not experience issue because their traffic gets routed through functional ones. In addition there are network components that connect servers themselves, one faulty router there means certain areas that the server behind faulty router would experience issues.

Mr Firor himself said that the ongoing multithreading/code rewrite was causing instability. So it's not necessarily just a hardware issue.

https://forums.elderscrollsonline.com/en/discussion/637691/server-multithreading-update-july-2023/p1

He also mentioned that they would let us know when they had any success with the coding upgrade. And as we are still awaiting word on that, it's reasonable to assume that there's still much work to do.

That is from a while back. There is a specific set of issues that began and worsened during May 2024 and have been ongoing since.

The point is that as they are tweaking and changing things, sometimes things get worse. We expect the changes behind the scenes to make things better, but that's not always the case.

Initial metrics, however, show that these steps in multithreading are not having as much of a performance improvement as desired, and so we had to roll back one of the process changes because it introduced an unacceptable level of instability.

Photosniper89 · July 2024

Initial metrics, however, show that these steps in multithreading are not having as much of a performance improvement as desired, and so we had to roll back one of the process changes because it introduced an unacceptable level of instability.

I'd argue that the current state of PvE/PvP is at an unacceptable level of instability.

drip_fromtheinkwell · July 2024

it's been a minute since it happened again, but did anyone else figure out what to do in a "booted from server" > "unexpected internal error" loop? ,_,

Photosniper89 · July 2024

drip_fromtheinkwell wrote: »

it's been a minute since it happened again, but did anyone else figure out what to do in a "booted from server" > "unexpected internal error" loop? ,_,

The only thing I have been able to figure out is that if you get booted you need to wait about 30 seconds to a minute before trying to log back in or you're going to be put in a 5 min time out before you can log in.

I'm assuming this is some kind of DDoS prevention or something but it's poorly designed if so.

blktauna · July 2024

I have hard crashed to desktop on PC EU 4 times in the last 10 minutes. What the actual?

SeaUnicorn · July 2024

Jaraal wrote: »

show previous quotes

edward_frigidhands wrote: »

Jaraal wrote: »

SeaUnicorn wrote: »

Photosniper89 wrote: »

manukartofanu wrote: »

Photosniper89 wrote: »

Okay - thats kind of what I thought happened but wasn't sure. Disappointing. I'm assuming the decision was made that it's not financially worth it for them to invest in refactoring the game but....

I'd challenge anyone in a leadership role in the company to factor in technical debt they have because they refuse to refactor the code. This is a conversation that hassssssss to happen (most leaders don't even know what technical debt is so it's never even discussed).

I don't think it's a matter of technical debt. You can refactor endlessly, but when the issue is with the servers, it doesn't solve anything. If the problem were in the code, everyone would be experiencing exact same issues. But we see that only a portion of the players are affected simultaniously. This kind of suggests that the questions should be directed not at the programmers, but at the administrators of the whole system.

They just upgraded the servers last year and we are right back to where we are. It's not the servers.

Now - are the servers configured improperly some how... sure that is possible. But throwing money and hardware at the issue didn't fix it.

you do realize one could cheap out on hardware and purchase not enough or of questionable quality. In addition, there are other components of infrastructure besides the servers. New compute does not solve issues with the network. Routers and switches handle network traffic and therefore have direct impact on the ping. Few faulty routers means some people experience issues because their traffic gets routed through those, while remaining population does not experience issue because their traffic gets routed through functional ones. In addition there are network components that connect servers themselves, one faulty router there means certain areas that the server behind faulty router would experience issues.

Mr Firor himself said that the ongoing multithreading/code rewrite was causing instability. So it's not necessarily just a hardware issue.

https://forums.elderscrollsonline.com/en/discussion/637691/server-multithreading-update-july-2023/p1

He also mentioned that they would let us know when they had any success with the coding upgrade. And as we are still awaiting word on that, it's reasonable to assume that there's still much work to do.

That is from a while back. There is a specific set of issues that began and worsened during May 2024 and have been ongoing since.

The point is that as they are tweaking and changing things, sometimes things get worse. We expect the changes behind the scenes to make things better, but that's not always the case.
show previous quotes

Initial metrics, however, show that these steps in multithreading are not having as much of a performance improvement as desired, and so we had to roll back one of the process changes because it introduced an unacceptable level of instability.

Game developers are tweaking the game engine, it has nothing to do with actual hardware, especially network. You don't just "tweak" drivers and software of network components, those come from the manufacturer, maintained by manufacturer, and updates of those are released by manufacturer. ZOS engineers are only responsible for patching those on timely manner. In majority of cases that software is proprietary all together. What your saying is equivalent of "I am tweaking the recipe of my apple pie, and because of it my oven does not work anymore"

Bammlschwamml · July 2024

Well if you tweak your apple pie while the oven is on, you will end up with a pile of goo and burn yourself in the process...

edward_frigidhands · July 2024

Bammlschwamml wrote: »

Well if you tweak your apple pie while the oven is on, you will end up with a pile of goo and burn yourself in the process...

I am so lost with this one...

I hope to see how things look today once I am done with work.

SeaUnicorn · July 2024

Jaraal wrote: »

show previous quotes

SeaUnicorn wrote: »

Jaraal wrote: »

SeaUnicorn wrote: »

Photosniper89 wrote: »

manukartofanu wrote: »

Photosniper89 wrote: »

Okay - thats kind of what I thought happened but wasn't sure. Disappointing. I'm assuming the decision was made that it's not financially worth it for them to invest in refactoring the game but....

I'd challenge anyone in a leadership role in the company to factor in technical debt they have because they refuse to refactor the code. This is a conversation that hassssssss to happen (most leaders don't even know what technical debt is so it's never even discussed).

I don't think it's a matter of technical debt. You can refactor endlessly, but when the issue is with the servers, it doesn't solve anything. If the problem were in the code, everyone would be experiencing exact same issues. But we see that only a portion of the players are affected simultaniously. This kind of suggests that the questions should be directed not at the programmers, but at the administrators of the whole system.

They just upgraded the servers last year and we are right back to where we are. It's not the servers.

Now - are the servers configured improperly some how... sure that is possible. But throwing money and hardware at the issue didn't fix it.

you do realize one could cheap out on hardware and purchase not enough or of questionable quality. In addition, there are other components of infrastructure besides the servers. New compute does not solve issues with the network. Routers and switches handle network traffic and therefore have direct impact on the ping. Few faulty routers means some people experience issues because their traffic gets routed through those, while remaining population does not experience issue because their traffic gets routed through functional ones. In addition there are network components that connect servers themselves, one faulty router there means certain areas that the server behind faulty router would experience issues.

Mr Firor himself said that the ongoing multithreading/code rewrite was causing instability. So it's not necessarily just a hardware issue.

https://forums.elderscrollsonline.com/en/discussion/637691/server-multithreading-update-july-2023/p1

that's a year old post. Multithreading is compute i.e. how game processes calculations. Those would be issues with skills not firing correctly, or not feeling "responsive", mob AI not behaving correctly, general lag in large populated areas. Those issues would cause everyone to experience lag. Issues reported here are network related, some people experience lag and can't stay connected, while others are perfectly fine.

If you are referring to the physical network of the game servers, then yes, they have mentioned having to replace failing network ports:

https://forums.elderscrollsonline.com/en/discussion/600966/update-33-pc-launch-postmortem/p1

It could also be related to the sharding, where players on certain shards are having the dysfunction, while those on other shards are having no issues. That could indicate some sort of hardware or software malfunction where the shards connect. It could also be packet loss or data bottlenecking at Akamai, or whatever other anti-DDoS scrubbing service they are currently using. Recent traceroutes have been showing a variety of IPs, whereas it was exclusively Akamai in the past.

Shards are storage drives that are spread across servers and are connected into single DB via network. Your character data like achieves, outfits, inventory is stored there. Realtime events of you moving, firing skills, applying buffs de-buffs are not persisted in the database, because its unreasonable amount of unnecessary data and databases are not designed to process such amounts of data. Such real-time "streams" of data are processed in memory on compute instances and then summaries of what states of characters and the world those actions resulted in are sent to the database. So the skill not firing and all that does not even involve interaction with the shards. It's processed on the compute instances. If there was a miss-communication between compute and store everyone would experience something like rollbacks, or corruption of data, such as items missing, achieves not applying and so on. I am sure ZOS pays ALOT of attention to that part, as this has direct monetary impact and is probably ranked as high inherent risk item. We only really seen major issues with data corruption may be twice in history of ESO: once was straight up PTS was connected to live, so it was saving onto live DB (just a human error, pretty easy to make too, just copy-paste one wrong config line), and there was this one time when servers rolled back by few minutes so people lost some store purchases. Normally company has log summaries of important stuff like purchases, so there would be tools to address that too. In any case issues with shards would be a major problem and we def don't want to experience those.
Current issues we are experiencing and reporting are limited exposure (i.e. some people but not others), hence no pressure to fix them immediately. It's all just simple product management and issue prioritization at this point. But more voices make those issue raise higher on the priority list. So we should keep reporting them.
What does not help is discounting and dismissing people's experiences. Issues are happening. There is no explanation from dev/engineering team regarding the root cause. There are indication of what those issues might be. Posts of 2023 are not an explanation of the root cause of current issue.

Jaraal · July 2024

SeaUnicorn wrote: »

show previous quotes

Jaraal wrote: »

edward_frigidhands wrote: »

Jaraal wrote: »

SeaUnicorn wrote: »

Photosniper89 wrote: »

manukartofanu wrote: »

Photosniper89 wrote: »

Okay - thats kind of what I thought happened but wasn't sure. Disappointing. I'm assuming the decision was made that it's not financially worth it for them to invest in refactoring the game but....

I'd challenge anyone in a leadership role in the company to factor in technical debt they have because they refuse to refactor the code. This is a conversation that hassssssss to happen (most leaders don't even know what technical debt is so it's never even discussed).

I don't think it's a matter of technical debt. You can refactor endlessly, but when the issue is with the servers, it doesn't solve anything. If the problem were in the code, everyone would be experiencing exact same issues. But we see that only a portion of the players are affected simultaniously. This kind of suggests that the questions should be directed not at the programmers, but at the administrators of the whole system.

They just upgraded the servers last year and we are right back to where we are. It's not the servers.

Now - are the servers configured improperly some how... sure that is possible. But throwing money and hardware at the issue didn't fix it.

you do realize one could cheap out on hardware and purchase not enough or of questionable quality. In addition, there are other components of infrastructure besides the servers. New compute does not solve issues with the network. Routers and switches handle network traffic and therefore have direct impact on the ping. Few faulty routers means some people experience issues because their traffic gets routed through those, while remaining population does not experience issue because their traffic gets routed through functional ones. In addition there are network components that connect servers themselves, one faulty router there means certain areas that the server behind faulty router would experience issues.

Mr Firor himself said that the ongoing multithreading/code rewrite was causing instability. So it's not necessarily just a hardware issue.

https://forums.elderscrollsonline.com/en/discussion/637691/server-multithreading-update-july-2023/p1

He also mentioned that they would let us know when they had any success with the coding upgrade. And as we are still awaiting word on that, it's reasonable to assume that there's still much work to do.

That is from a while back. There is a specific set of issues that began and worsened during May 2024 and have been ongoing since.

The point is that as they are tweaking and changing things, sometimes things get worse. We expect the changes behind the scenes to make things better, but that's not always the case.

Initial metrics, however, show that these steps in multithreading are not having as much of a performance improvement as desired, and so we had to roll back one of the process changes because it introduced an unacceptable level of instability.

Game developers are tweaking the game engine, it has nothing to do with actual hardware, especially network. You don't just "tweak" drivers and software of network components, those come from the manufacturer, maintained by manufacturer, and updates of those are released by manufacturer. ZOS engineers are only responsible for patching those on timely manner. In majority of cases that software is proprietary all together. What your saying is equivalent of "I am tweaking the recipe of my apple pie, and because of it my oven does not work anymore"

Well that's nice that you think the current issues are strictly hardware related. The studio director said that software changes they make can cause instability, and instability is certainly symptomatic of some of the issues people are describing in this thread.

I don't understand how some folks think that just because something was said two years ago makes it false today.

fizzylu · July 2024

edward_frigidhands wrote: »

I hope to see how things look today once I am done with work.

Well, when it comes to the game.... I seriously struggle to not see it as a joke at this point. Thought I'd log on just to keep up the daily rewards and immediately got the unable to connect to server error. Tried entering my password three or four times and just gave up because clearly ESO doesn't want me playing it. And you know what game doesn't randomly boot me or just not let me in at all? Many, but specifically my one of choice lately is RuneScape. So.... I know where I'm going for the evening/night.

silky_soft · July 2024

Both pc got kicked same time 5 mins ago at my place. Got locked out for a while with same messages as above.

Photosniper89 · July 2024

SeaUnicorn wrote: »

show previous quotes

Jaraal wrote: »

SeaUnicorn wrote: »

Jaraal wrote: »

SeaUnicorn wrote: »

Photosniper89 wrote: »

manukartofanu wrote: »

Photosniper89 wrote: »

Okay - thats kind of what I thought happened but wasn't sure. Disappointing. I'm assuming the decision was made that it's not financially worth it for them to invest in refactoring the game but....

I'd challenge anyone in a leadership role in the company to factor in technical debt they have because they refuse to refactor the code. This is a conversation that hassssssss to happen (most leaders don't even know what technical debt is so it's never even discussed).

I don't think it's a matter of technical debt. You can refactor endlessly, but when the issue is with the servers, it doesn't solve anything. If the problem were in the code, everyone would be experiencing exact same issues. But we see that only a portion of the players are affected simultaniously. This kind of suggests that the questions should be directed not at the programmers, but at the administrators of the whole system.

They just upgraded the servers last year and we are right back to where we are. It's not the servers.

Now - are the servers configured improperly some how... sure that is possible. But throwing money and hardware at the issue didn't fix it.

you do realize one could cheap out on hardware and purchase not enough or of questionable quality. In addition, there are other components of infrastructure besides the servers. New compute does not solve issues with the network. Routers and switches handle network traffic and therefore have direct impact on the ping. Few faulty routers means some people experience issues because their traffic gets routed through those, while remaining population does not experience issue because their traffic gets routed through functional ones. In addition there are network components that connect servers themselves, one faulty router there means certain areas that the server behind faulty router would experience issues.

Mr Firor himself said that the ongoing multithreading/code rewrite was causing instability. So it's not necessarily just a hardware issue.

https://forums.elderscrollsonline.com/en/discussion/637691/server-multithreading-update-july-2023/p1

that's a year old post. Multithreading is compute i.e. how game processes calculations. Those would be issues with skills not firing correctly, or not feeling "responsive", mob AI not behaving correctly, general lag in large populated areas. Those issues would cause everyone to experience lag. Issues reported here are network related, some people experience lag and can't stay connected, while others are perfectly fine.

If you are referring to the physical network of the game servers, then yes, they have mentioned having to replace failing network ports:

https://forums.elderscrollsonline.com/en/discussion/600966/update-33-pc-launch-postmortem/p1

It could also be related to the sharding, where players on certain shards are having the dysfunction, while those on other shards are having no issues. That could indicate some sort of hardware or software malfunction where the shards connect. It could also be packet loss or data bottlenecking at Akamai, or whatever other anti-DDoS scrubbing service they are currently using. Recent traceroutes have been showing a variety of IPs, whereas it was exclusively Akamai in the past.

Shards are storage drives that are spread across servers and are connected into single DB via network. Your character data like achieves, outfits, inventory is stored there. Realtime events of you moving, firing skills, applying buffs de-buffs are not persisted in the database, because its unreasonable amount of unnecessary data and databases are not designed to process such amounts of data. Such real-time "streams" of data are processed in memory on compute instances and then summaries of what states of characters and the world those actions resulted in are sent to the database. So the skill not firing and all that does not even involve interaction with the shards. It's processed on the compute instances. If there was a miss-communication between compute and store everyone would experience something like rollbacks, or corruption of data, such as items missing, achieves not applying and so on. I am sure ZOS pays ALOT of attention to that part, as this has direct monetary impact and is probably ranked as high inherent risk item. We only really seen major issues with data corruption may be twice in history of ESO: once was straight up PTS was connected to live, so it was saving onto live DB (just a human error, pretty easy to make too, just copy-paste one wrong config line), and there was this one time when servers rolled back by few minutes so people lost some store purchases. Normally company has log summaries of important stuff like purchases, so there would be tools to address that too. In any case issues with shards would be a major problem and we def don't want to experience those.
Current issues we are experiencing and reporting are limited exposure (i.e. some people but not others), hence no pressure to fix them immediately. It's all just simple product management and issue prioritization at this point. But more voices make those issue raise higher on the priority list. So we should keep reporting them.
What does not help is discounting and dismissing people's experiences. Issues are happening. There is no explanation from dev/engineering team regarding the root cause. There are indication of what those issues might be. Posts of 2023 are not an explanation of the root cause of current issue.

Great summary. I agree with the last part of what you said in particular. I wish they would bring out the PvP Leads, bring out Rich, bring out Gina, bring out all of them and just have an honest conversation.

I've mentioned this a few times on here, social media, Twitch Streams (Zos put me in an immediate time out for mentioning PvP) but look at how 343i handled the launch of Halo Infinite. That game launched and was an absolute disaster - people were losing their lids. So what did they do? They got a few key people in front of cameras and they started talking about how they were going to address it. They did monthly 1ish hour (pre recorded) videos where they talked about how they were capturing data, explained how they did that, explained how they were prioritizing issues, why xyz decisions were made, and they talked about a path fwd.

Each month they came out and tried stuff - sometimes things didnt turn out how they wanted and they communicated that with people during those streams. It was a long process, multi year, to get the game to where it is now but they communicated for a while each month how they were working on fixing it. Once they got the majority of the issues fixed they stopped with the videos.

THIS is what I wish zos would do. They used to do stuff similar by posting on the forums but we have all seen how those threads are promised to be updated but then never are.

I really wish ZOS would officially address the issues that have been plaguing PvP for 7 years now. These server issues are honestly only being talked about this much because it's now hitting PvE. @ZOS_GinaBruno @ZOS_Kevin

You don't have to respond to this but if you see this message I'm literally begging you please just take a look at how 343i handled their launch of Halo Infinite and the videos they put out. Their willingness to be frank and have difficult conversations is what I truly respect about them (I'm a HUGE Halo fanboi and was extremely disappointed by MCC and Halo Infinite but respect how they handled it).

Kainarel · July 2024

Got two server connection errors in a row today - for the first time in two months or so after I'd had those when kicked out of the game and unable to return (back then I had to repair to return, and now I wouldn't be able to until launcher is fixed). =(( Had to wait a minute before logging in to grab a daily.

Last night had more lag spikes than usual with red ping numbers, random ones as well as right after teleporting.

Nothing changed despite the maintenance. This is frustrating beyond any measure.

edward_frigidhands · July 2024

I played between 5 AM and 10 30 AM EST today but didn't have the time to enter cyrodill.

Performance outside cyrodill was what I was used to before these recent issues began. I was at 80 to 100 ms without going into cyrodill.

Typically I am 90-120 outside cyrodill and it gets worse when inside cyrodill.

I am wondering if the server restart helped or it was just that I was playing at a low population hour and didn't have the time to enter cyrodill.

I am still connecting from the East Coast and my latency held at 80-100 ms constantly both inside and outside cyrodill without lag spikes before the issues that began during May 2024.

Xarc · July 2024

Launcher is the problem.

Playing directly from eso64.exe = normal ping.

Just tried it and works fine.

@ZOS_GinaBruno

Kainarel · July 2024

Xarc wrote: »

Launcher is the problem.

Playing directly from eso64.exe = normal ping.

Just tried it and works fine.

@ZOS_GinaBruno

Launcher and Steam are indeed a problem that causes very long loading screens and overall lag.

BUT I always play directly from the exe file, using launcher only for patching, and even directly from the file there are issues described in this topic because it's all server-related.

This topic was started precisely when those issues started -- before that time I had no issues, no lag spikes, no red numbers and my ping was 80-100. Now my normal ping is mostly in 120s. It gets more or less normal only when playing in non-populated hours (night, morning) and areas. It hasn't changed yet. Plus, many players are having frequent error 303 issues then can't login back at once and have to wait.

harvey07 · July 2024

today i am not showing any high latency but at vendors and the bank/auction.....there is a 2-3 second delay on selling and listing items.

Jaraal · July 2024

edward_frigidhands wrote: »

I am wondering if the server restart helped or it was just that I was playing at a low population hour and didn't have the time to enter cyrodill.

Performance is consistently better after server reboots, and deteriorates over time. That usually indicates unaddressed memory leaks.

ZOS_Kevin · July 2024

Hi all. After chatting with the team, we're looking to gather more information from players who have been experiencing new and/or worse latency/lag spikes/rubber-banding issues in ESO since the beginning of May 2024 (when the emergent issues in this thread were initially reported.) To help us with our ongoing investigation, could you please answer the following questions?

What server/platform do you play on?
What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)
Around what time of day have you experienced the new and/or worse issues reported in this thread?
Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)
Have you experienced these issues outside of combat situations (in houses, for example)
Have you noticed these issues are worse during any specific times of day? If so, what times?
Have you noticed any changes/improvements as of today?

Wraithseer_ESO · July 2024

What server/platform do you play on?

PC EU

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)

Higher ping and all associated desyncs and disconnects sarting the evening of June 20

Around what time of day have you experienced the new and/or worse issues reported in this thread?

Constantly, but over two weeks average ping went from 450ms to 250ms (used to be 100ms)

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)

Everywhere, see my post in this thread with traceroutes. Desyncs and disconnects aremore common in Cyrodiil and Imperial City.

Have you experienced these issues outside of combat situations (in houses, for example)

Yes

Have you noticed these issues are worse during any specific times of day? If so, what times?

No

Have you noticed any changes/improvements as of today?

No it's still around 250ms with spikes (used to be around 100ms before June 20 and is ~100ms if I connect via VPN)

thepandalore · July 2024

In addition to the latency spikes, players are also encountering problems with block actually blocking and abilities actually firing, despite seeing the shield animation and ability icons depressing on the ability bar. If it helps, a known workaround for the problem with abilities actually firing is to program your gaming mouse to double-input each cast with zero delay.

wrote:

What server/platform do you play on?

PCNA

wrote:

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)

Rubberbanding, block not blocking, player abilities are failing to fire, new enemy casts are starting after enemy deaths

wrote:

Around what time of day have you experienced the new and/or worse issues reported in this thread?

Occurs at all times of day.

wrote:

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)

lucent citadel, blackrose prison, the cauldron, graven deep, banished cells, fungal grotto, sanity's edge, rockgrove, sunspire, southern elsweyr, northern elsweyr, craglorn, apocrypha, the rift, eastmarch

wrote:

Have you experienced these issues outside of combat situations (in houses, for example)

yes, multiple inputs are sometimes required to open storage, interact with NPCs in houses, as well as open chests after combat has ended. the worst of which is at the end of Lucent Citadel. Character will cancel crouch animation 10+ times in a row before actually opening chest

wrote:

Have you noticed these issues are worse during any specific times of day? If so, what times?

issues don't seem to be worse at any given time of day

wrote:

Have you noticed any changes/improvements as of today?

no

Anyammis · July 2024

What server/platform do you play on?

PC NA

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)

Around June/chatper launch the load from login to character select started to take an extremely long time. Once in game, load screens went back to normal.

As of the Monday patch, however, I am getting extreme lag and stuttering now in game. It also seems like ESO is memory leaking again.

Around what time of day have you experienced the new and/or worse issues reported in this thread?

My sleep schedule is erratic and I've been on at various times, time of day does not seem to affect anything related to the issues on my end.

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)

Initial login lag is literally the first login to characters select to in game world. Hopping characters afterwards, hopping zones, etc are fine as long as I don't fully log back out.

For the second issue, I have been to almost every zone that has collectable surveys and Lucent Citadel because right now I'm only really doing the new trial+crafting dailies. The lag seems to be everywhere on my end.

Have you experienced these issues outside of combat situations (in houses, for example)

Doesn't seem better or worse in or outside of houses, in or outside of trials, etc. Just a persistent lag.

Have you noticed these issues are worse during any specific times of day? If so, what times?

Nothing noticeably better or worse time of day wise.

Have you noticed any changes/improvements as of today?

The borderline unplayable lag started after Monday's maintenance for me. The character login is not better either.

Photosniper89 · July 2024

What server/platform do you play on?
- PC/NA

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)
- Massive skill delay, inability to interact with banks/nodes/doors w. 3-5 second delay, rubber banding, skill delays/bar swap delays of ~5 seconds

Around what time of day have you experienced the new and/or worse issues reported in this thread?
- During the day issues are less frequent and delays are smaller 1-2 seconds but during "prime time" issues are 3x to 4x worse

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)
- starter zones, craglorn, Cyrodiil (grayhost), and in my house just decorating

Have you experienced these issues outside of combat situations (in houses, for example)
- Yes, I've experienced issues just sitting inside my house or just porting into a new zone

Have you noticed these issues are worse during any specific times of day? If so, what times?
- Two part yes answer
- - Yes, as the day gets later issues gets worse
- - After maintenance (server reboot) the issues are not as bad but later in the week (especially weekends) it progressively gets worse and worse (almost like a slow memory leak)

Have you noticed any changes/improvements as of today?
- Hard to say because it's always slightly improved the first day of a servers being restarted

VouxeTheMinotaur · July 2024

ZOS_Kevin wrote: »

Hi all. After chatting with the team, we're looking to gather more information from players who have been experiencing new and/or worse latency/lag spikes/rubber-banding issues in ESO since the beginning of May 2024 (when the emergent issues in this thread were initially reported.) To help us with our ongoing investigation, could you please answer the following questions?
What server/platform do you play on?

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)

Around what time of day have you experienced the new and/or worse issues reported in this thread?

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)

Have you experienced these issues outside of combat situations (in houses, for example)

Have you noticed these issues are worse during any specific times of day? If so, what times?

Have you noticed any changes/improvements as of today?

1. NA / PS5

2. Rubber banding / freeze frame, warping, frame rate drop, skill delay around 2-5 seconds or just do not fire at all. bar swap delay of 2-5 seconds, heavy attacks not connecting to enemies at range or close range despite red targeting glow around enemy and crosshair on enemy, combine this with skill delay is frustrating.

3. seemingly any time of day, but mostly during 8pm (EST) to 2 am (EST). Earlier hours of the day are usually *less* severe with these issues but still persistent enough to cause an Error DC at times.

4. Extremely severe in dungeon, trial, delves, dolmens / incursions / vents, arenas including solo arenas, and public dungeons, scrying questline, interacting with NPC's / items / loot nodes (like sacks or crates) in heavily player populated towns. Severe in open zones like Mournhold of Deshaan, Alinor of Summerset, Craglorn's trial gathering spot. It seems to be where combat is heavy and when lot of players are gathered in one area, though I have experienced very minor stuttering while running alone on the mount through zones, my own housing and in other players houses. Dungeons and Trials remain to be the biggest issue for me.

5. Yes, specifically when moving or placing furnishings. Placing furnishings also have a delay of around 2-5 seconds. There is also a pop-up notification that appears at the top right of the screen that says "your previous action has not been saved" or something like that, it comes up often. Accessing the Purchase tab in home decorating also has a delay when viewing furnishings to buy. Opening doors, whether they were apart of the home or a furnishing door / gate also has a delay along with interactable items / furnishings / NPC's like the assistants and crafting tables, up to a point that the "X" button needs to be tapped repeatedly to use said items.

6. Seemingly worse when there are more players active. My availability to play is always during nighttime of EST, sometimes mornings leading into the afternoon. Nighttime is worse than early morning / afternoon. An exact timeframe I would say between the hours of 6 pm and forward (I usually log off around midnight, but it still goes on afterward according to guildmates of the same platform / server, they are located in Australia / Western Canada / Western USA) this is when the issues gradually or suddenly become more apparent, most times it hits like a truck, coming out of nowhere after a couple of hours of "steady but manageable" issues.

7. I have yet to play today or yesterday, but prior to this maintenance, PS5/NA (at least for me personally) seemed to be a BIT better. The rubber banding / lag and skill delay were still there, but it did not spike or happen as often as it was in the prior weeks. I would say whatever the changes, if any were made, to the PS5/NA servers are a good start and seemed to be heading in the right direction as far as a solution or fix, since again, the issue does not happen at the same frequency. However, when it does happen, it hits just as hard as when it was frequent, enough for the same Error to show on the main menu screen. I do not know if there were any changes made, or if it's just because of player population slowing going down since many decided to switch to other games until the issue is fixed.

I will edit this comment later today or tomorrow beneath this sentence after playing the game tonight to give feedback on how the game feels / runs after maintenance. Fingers crossed that its better!!

noblecron · July 2024

Been noticing a lot more lag and fps drops as of late as well. Never had it happen untill the last couple of updates.

Some friends have been having it a lot as well and some can't even log in through steam

Kainarel · July 2024

ZOS_Kevin wrote: »

Hi all. After chatting with the team, we're looking to gather more information from players who have been experiencing new and/or worse latency/lag spikes/rubber-banding issues in ESO since the beginning of May 2024 (when the emergent issues in this thread were initially reported.) To help us with our ongoing investigation, could you please answer the following questions?
What server/platform do you play on?

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)

Around what time of day have you experienced the new and/or worse issues reported in this thread?

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)

Have you experienced these issues outside of combat situations (in houses, for example)

Have you noticed these issues are worse during any specific times of day? If so, what times?

Have you noticed any changes/improvements as of today?

* PC EU
* After the May festival was over - specifically after ZOS fixed the PTS issue for people who were locked out of the game for days - then problems started to show. I had none before that time. There are unexpected lag spikes that could happen in any zone at any moment, just randomly. Ping would go into red numbers for a few seconds, then come back to norm. Latency numbers became higher. With my norm at 80-100 the latency turned into 120-130s which would prevail. More normal numbers show when less players are around in late night hours or early morning. Sometimes lag would show in powers delay or stutter but the numbers would remain normal. There were disconnects happening during Battleground matches or during traveling between wayshrines. When logging in you would be placed in a queue.
* The issues happen at any time of day or night. It's random - in my experience.
* I have been kicked from Battleground matches after everything froze except for me before kicking me. I have been disconnected during traveling. I have experienced lag spikes with red numbers just doing nothing in empty zones while fishing or just collecting resources. Sommerset, Malabal Tor, Vvardenfell, a lot of high latency in Wayrest, even when almost no people are around at night or early morning. Something there just simply puts a load on my cpu and gpu. FPS have been dropping dramatically to 13-20 inside the castle in Solitude and in the city in Galen. Had freezes in dungeons as well.
* Yes, it happens outside the combat, inside the house, outside etc. Randomly.
* Certainly some issues are more vivid during the prime time playing whenever most players are around.
* Ping numbers are better in less populated zones, but in cities and around many other players issues are still present.

efster · July 2024

What server/platform do you play on?
PC/NA

What is the nature of the issues you started experiencing or noticed got worse after the beginning of May 2024? (ex. did characters rubberband/warp, did abilities get delayed/fail to fire, etc.)

Getting booted from server, characters rubber-band, abilities get delayed and fail to fire, takes a long time for interact prompts to fire. Trying to take things out of/put things in the bank takes forever.

Around what time of day have you experienced the new and/or worse issues reported in this thread?
At all times of day.

Where in-game have you experienced these issues? Please be as specific as possible (name/area of overland zone, delve, dungeon, Trial, etc.)
Infinite Archive, various DLC dungeons, my house (Grand Topal Hideaway), Cyrodiil (including alliance base), various delves connected to the Gold Road questline, Mirrormoor incursions, overland zones (Grahtwood, West Weald, Reaper's March, Auridon), trespassing in NPCs' houses in Vastyr, Galen.

Have you experienced these issues outside of combat situations (in houses, for example)
Yes, frequently. Particularly, the game loves to boot me from the server if I try to port directly to a dungeon while in my house.

Have you noticed these issues are worse during any specific times of day? If so, what times?
No. They're equally likely to happen, though for me they are not especially frequent -- there's a period of lag once or twice a play session and the rest of the time it's playable. However, my normal ping overall has been higher lately. I live in Japan, so "normal" means 200-250 ping, but I've been seeing 300-400 a lot more than I'd like, recently. This makes me very concerned for my ability to do anything during Mayhem later this month.

Have you noticed any changes/improvements as of today?
It's hard to say since it is not a constant problem, but I did experience a huge lag spike in the Archive at around 11AM Eastern.