Maintenance for the week of December 23:
• NA megaservers for maintenance – December 23, 4:00AM EST (9:00 UTC) - 9:00AM EST (14:00 UTC)
• EU megaservers for maintenance – December 23, 9:00 UTC (4:00AM EST) - 14:00 UTC (9:00AM EST)

ZOS, Massive Spike in Ping/LAG in Recent Days - What Gives?

  • sshogrin
    sshogrin
    ✭✭✭
    silky_soft wrote: »
    First try into new life gateway. Disconnect.

    Edit 1:
    Did some of the event, teleport back to eastmarch, disconnect.
    1a:
    Disconnect second try.

    Over in EU it didn't matter which account I was using (I have 3 full ESO accounts), I couldn't use the New Life Festival portal near the Impressario without timing out and getting kicked from game back to the Login screen. I ended up just using the Wayshrine and that worked just fine.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    Doing a quick search on reddit it looks like the timing of this thread lines up with peoples experiences there too.
    slr4im71a7ap.png

    Tons of comments in this thread... basically the same as here.
    https://www.reddit.com/r/elderscrollsonline/comments/1dqi480/am_i_the_only_whos_getting_massive_lag_spikes_im/

    @ZOS_Kevin really hope the team can narrow in around this time frame and see what changed.
    Will the real LadyGP please stand up.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    I'm gonna be that person. I put a ton of responses in here, Richs feedback, other dev comments, took information from Reddit and told ChatGPT to analyze all of the available information. I'm sure someone at ZoS has already done this and their team knows more about this issue than ChatGPT but none the less I'm going to put it here in the 0.5% chance it sparks something to help solve the problem.
    The issue appears to be tied to a recent client or server-side update that introduced a new network handling regression. This regression is causing the client to block while waiting for server responses, leading to frequent "stalls" (high ping, freezing) and then sudden bursts of updates once the server data arrives.

    Detailed Analysis:
    Based on the additional user report and eliminating the Bandits UI add-on as a factor, we need to refocus on what could be causing network stalls that affect all players, regardless of their hardware, ISP, or installed add-ons.

    Key points from the user’s comments:

    1. Wait Chain Analysis Shows Network I/O Blocking:
    The user notes that Windows’ built-in wait chain analysis shows the game’s threads waiting on network I/O. This suggests that the game client’s thread(s) responsible for networking are blocked, waiting for a response from the server. The client may be making synchronous network calls or is otherwise unable to proceed with normal rendering and logic updates until it receives needed data packets. As a result, the entire game "feels" frozen because the main loop or critical update thread is stalled.

    2. Cross-Environment Consistency:
    The user reinstalled Windows, refreshed drivers, and still experiences these issues exclusively with this game. Other AAA titles run smoothly on the same system, with no spikes in latency or frame drops. This eliminates local hardware, drivers, and ISP as the cause, reinforcing that the problem is external.

    3. Coincidence with the Most Recent Client Update:
    The issues began following a recent client patch or update. Such timing strongly suggests a code change—possibly a new feature or optimization gone wrong—that introduced a more synchronous or less fault-tolerant network path.

    4. Global, Not Add-On Dependent:
    Initial suspicions pointed towards add-ons like Bandits UI exploiting map pings and causing a network message flood. However, reports from users without any add-ons and across various hardware configurations indicate that the problem persists. This points to a more fundamental network or server-side code issue.

    5. Similar Symptoms, Multiple Users:
    Players report skill delays, high ping, and periods of "no movement" followed by rapid catch-up. This pattern indicates that the server is not delivering updates steadily. Instead, the client starves for data, then receives a sudden burst of packets once the server "catches up." The root cause could be an overloaded server process, a network queue backlogging, or a newly introduced logic path that locks the server’s response temporarily.

    Likely Root Cause:
    The most probable cause is a server or client-side network handling regression introduced by the recent update. On the server side, there could be a message queue buildup or a bottleneck in message processing, causing delayed responses. On the client side, there could be a newly introduced synchronous wait for server data—perhaps a previously asynchronous call is now blocking the main thread. When the server finally sends the data, the client rushes to process everything at once, causing the "speed-up" effect.

    Conclusion:
    Even if add-ons were initially suspected, current evidence points more clearly toward a systemic issue introduced by recent code changes. The client is being forced into synchronous waits for server responses, and when combined with server-side delays, this results in the freezing and catch-up behavior observed. The root cause is most likely a network processing or code regression on the server or client side (or both) introduced during the recent patch, rather than user hardware, ISPs, or add-ons.
    Will the real LadyGP please stand up.
  • Quackery
    Quackery
    ✭✭✭✭✭
    For the love of God, the performance is so bad on PC NA-server that I can't even login to do ONE QUEST and get my event tickets! New Life event POOF!
  • Sakiri
    Sakiri
    ✭✭✭✭✭
    ✭✭
    sshogrin wrote: »
    Sakiri wrote: »
    sshogrin wrote: »
    @KaironBlackbard

    BanditsUI has removed their map pings for now and will be investigating using LibMapPing like all addons should be using which shouldn't cause an overload of the MapPing function.

    What map pings dies it even use? Like what functionality. Because mines never pinged my map that I'm aware of.

    the addons that use data sharing for things like dps and ultimates use a function within LUA called MapPing.
    There is a Library called LibMapPing that the developers of addons should be using so all those data shares and calls are handled by that Library. That Library would actually use just one call per data type for all addons you have installed that are wanting to share or use that data so you don't have 2 or more addons trying to make the call or share for the same thing.
    This is something that was referred to in an earlier comment as a "back channel" use.
    To equate this to graphics, in an RGB image, the user can only see those 3 channels R, G, B, if you add an alpha channel for something like masking, that alpha channel would be like a back channel, something going on behind the scenes. I hope I'm explaining that well enough.

    Ah okay.

    I know wow uses lua for it's stuff but all the libraries are NOT separate addons which is why modding the stuff here was just...weird.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    Quackery wrote: »
    For the love of God, the performance is so bad on PC NA-server that I can't even login to do ONE QUEST and get my event tickets! New Life event POOF!

    Yeah I agree... it's been really rough for me tonight too. I just ran a wireshark and trying to analyze everything now. Maybe I can find something that jumps out.
    Will the real LadyGP please stand up.
  • sshogrin
    sshogrin
    ✭✭✭
    @Sakiri
    "Ah okay.

    I know wow uses lua for it's stuff but all the libraries are NOT separate addons which is why modding the stuff here was just...weird."

    The use of a Library consolidates those functions to streamline the addons. Having multiple instances of the same library just makes the LUA side more bloated and can cause issues because you have multiple of the same addon running.

    Technically a Library really isn't an "addon" perse, but it's a Library of functions for the LUA language to lookup those functions with. Yes, in laymen's terms it's and addon, but it's function is "different" than an addon. The Library itself doesn't work as an addon, but more like knowledge for addons to use...it's like telling the addons that 255R, 255G, and 255B= "white" and to 'print' "white" to screen, and that all other addons that will tell the game to say "white" on screen would use that library and not have multiple instances of the same information or function running.
  • Rkindaleft
    Rkindaleft
    ✭✭✭✭✭
    PlayStation NA.

    Performance has been really inconsistent since the servers went down a week or so ago. Tonight was the worst I've seen in a while.

    Below is a clip of our Unstoppable group running approximately 1-2 hours ago from posting. The first half of the clip is an example of how bad the rubberbanding was (freezing for several seconds and speeding up) and the second part of the clip was literally 4 players getting kicked at the exact same time. I also disconnected tonight but I didn't save the clip. I will state these players are from different continents, so the fact they all got kicked at the same time has nothing to do with their own connections.

    Edit - it keeps giving me an error saying the file size is too large so I'm going to post the link to the YouTube video below.
    https://youtu.be/Md7ZrI7o_nU

    Edited by Rkindaleft on December 20, 2024 7:28AM
    https://youtube.com/@rkindaleft PlayStation NA. I upload parses and trial POVs sometimes.
    6/9 Trial Trifecta achievements.
    Tick Tock Tormentor | Immortal Redeemer | Gryphon Heart | Godslayer | Dawnbringer | Planesbreaker

    Scores:
    VMOL 172,828 (PSNA Server Record)
    VHOF 226,036
    VAS 116,298
    VCR 132,542
    VSS 246,143
    VKA 242,910
    VRG 294,543
  • M0ntie
    M0ntie
    ✭✭✭✭✭
    PC NA performance is the worst it has been in ages. (I usually have decent performance.) The laaagggg of visuals is just horrible. My vpn is saying about the same ping it usually does, so I'm putting this down to ZoS' servers not coping with the New Life Festival. My in game ping is consistently high. The lag is giving me a headache.

    FFS ZoS you know there will be more people on for an event.
  • TX12001rwb17_ESO
    TX12001rwb17_ESO
    ✭✭✭✭✭
    ✭✭✭✭
    I just brought 21,000 crowns and what happens next? my game crashes and I cannot log back in.
  • TX12001rwb17_ESO
    TX12001rwb17_ESO
    ✭✭✭✭✭
    ✭✭✭✭
    I have had enough; I cannot even stay logged in long enough to buy the new house.

    I am getting kicked within less then 20 seconds, is it too much to ask to beable to play the game I paid for?
    Edited by TX12001rwb17_ESO on December 20, 2024 10:38AM
  • silky_soft
    silky_soft
    ✭✭✭✭✭
    couple of bg, lag to the point my game was stalling. some players on the other team definately not feeling any lag as they could take on 6 people no worries.

    what happened in this recent patch?
    Here $15, goat mount please. Not gambling or paying 45 : lol :
    20% base speed for high ping players.
    Streak moves you faster then speed cap.
    They should of made 4v4v4v4 instead of 8v8.
  • Aurielle
    Aurielle
    ✭✭✭✭✭
    ✭✭✭✭✭
    I have had enough; I cannot even stay logged in long enough to buy the new house.

    I am getting kicked within less then 20 seconds, is it too much to ask to beable to play the game I paid for?

    If this were me, I would personally be asking for a refund.
    silky_soft wrote: »
    couple of bg, lag to the point my game was stalling. some players on the other team definately not feeling any lag as they could take on 6 people no worries.

    what happened in this recent patch?

    I still want to know what happened with the patch in May. :)
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    I just brought 21,000 crowns and what happens next? my game crashes and I cannot log back in.

    Thank you for your purchase. JKJK I know the frustration.
    Will the real LadyGP please stand up.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    Aurielle wrote: »
    I have had enough; I cannot even stay logged in long enough to buy the new house.

    I am getting kicked within less then 20 seconds, is it too much to ask to beable to play the game I paid for?

    If this were me, I would personally be asking for a refund.
    silky_soft wrote: »
    couple of bg, lag to the point my game was stalling. some players on the other team definately not feeling any lag as they could take on 6 people no worries.

    what happened in this recent patch?

    I still want to know what happened with the patch in May. :)

    This! I know there has been a lot of talk about the recent patch BUT.... we need to be backing up to May-ish. @ZOS_Kevin thats when it all started.
    Will the real LadyGP please stand up.
  • DeadlySerious
    DeadlySerious
    ✭✭✭
    Cyrodiil raid group today was rubber banding all over the place. We never disconnected, but lag was atrocious with massive lag spikes about every 5-10 minutes.

    This problem gets notably worse during events. The more people using the server, the worse the problem is. I don't know if the problem is hardware or software, but most likely it's a combination of the two. Performance was never as good as it was after they upgraded the NA PC servers, so we know that modern hardware for sure can help.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    Yeah, just like I thought. This issue is being wildly underreported.


    86q1218btdq0.png

    Just look at this thread. https://www.reddit.com/r/elderscrollsonline/comments/1hdo3xc/zoss_rich_lambert_has_posted_an_extensive_qa/

    I too know of 3 or 4 cores that have left eso because of the issues in this thread. ESO playerbase is leaving because of the issues in this thread - period. Yes, there are other issues that go along with their decision but "performance" is the #1 thing I hear across all the various discords/reddit/forum posts I read.

    v8ysnm1g9bio.png

    Literally everyone in this thread has the same reaction as this post
    n5livuixzq5s.png
    Edited by LadyGP on December 20, 2024 3:24PM
    Will the real LadyGP please stand up.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    bt19ytb5scg6.png
    7am3c1c5z1v6.png
    df6lh4yj469u.png
    p8bps8m250th.png
    ue2ihiznpr01.png
    5ap8dybm1o4p.png
    1vjoxtpzpaq0.png
    chk431gwtx47.png
    d30lonxsd9qu.png
    k9odpd1rps36.png
    mc8oq2f895n8.png
    q6uyi0hw8rpf.png
    sxff7zd2rbuf.png
    hv0ylcekp9tt.png
    7bd508pwf6t1.png
    4g0zufyaf56a.png
    h125y4w0nbhx.png
    287x2icpvq82.png
    kl2n2p683ndh.png
    m1jobqwrgtzh.png
    kk58uvueme29.png
    rx7bfzo5ohyo.png
    f5ag2dow39c0.png
    c23sihc3299k.png
    v1taimj1c9vu.png
    4qlcvtf8e16a.png
    o2rcj79hohqp.png
    46hmlxibft7d.png
    ikuky6icqstk.png
    q90vxho3kqqt.png
    uetsuztfq5wf.png
    qqvg8noixo6i.png
    pwhncalvolr2.png
    kljgvlcru3qm.png
    0az7od5ev11o.png
    arryu2d3vu9z.png
    p5zogea2leg6.png
    91simqoz1pcr.png
    g0avpfljmf4x.png
    Edited by LadyGP on December 20, 2024 5:40PM
    Will the real LadyGP please stand up.
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    Meant to tag @ZOS_Kevin @ZOS_JessicaFolsom in the above post.

    I know we just got the big update from Rich - can we expect additional updates, hopefully before Jan 1, on how the investigation is going and if anything has been discovered? Doesn't have to be from Rich directly.. but just... something to help give some confidence to people about the issue?
    Will the real LadyGP please stand up.
  • ZOS_Kevin
    ZOS_Kevin
    Community Manager
    LadyGP wrote: »
    Meant to tag @ZOS_Kevin @ZOS_JessicaFolsom in the above post.

    I know we just got the big update from Rich - can we expect additional updates, hopefully before Jan 1, on how the investigation is going and if anything has been discovered? Doesn't have to be from Rich directly.. but just... something to help give some confidence to people about the issue?

    First, thank you for gathering the screenshots above.

    Regarding updates, we are currently working on a survey to better track information from players who experience performance and lag issues. This is in an effort to better track the information from more places (like Steam, Reddit, Discord Servers, etc,) like what you provided in the screenshots above, just in a more organized way and breaks it down the info that our engineers need. The survey when finished, it will send information directly to our engineers working on this and is intended to better organize the kind of information you took screenshots of above. We are hoping to have this situated in the New Year.

    Beyond what we provided above and Rich's last update, we won't have an update until after the New Year. Most of the team is leaving today for the holidays while the rest are gone after Monday, so no updates will be provided until a few weeks after we are back in the New Year and have a little more time to work.
    Community Manager for ZeniMax Online Studio and Elder Scrolls OnlineDev Tracker | Service Alerts | ESO Twitter
    Staff Post
  • Dojohoda
    Dojohoda
    ✭✭✭✭✭
    ✭✭✭
    Yesterday, the entire time (several hours) and even after restarting, the game had slowed responses. Every input was slowed. For example: interaction with inventory appeared to have a cooldown. When selling to an npc vendor, items did not disappear promptly but instead each transaction took a couple of seconds.

    This refers to my time on PCNA and I live in the USA.
    Fan of playing magblade since 2015. (PC NA)
    Might be joking in comments.
    -->(((Cyrodiil)))<--
  • doesurmindglow
    doesurmindglow
    ✭✭✭✭✭
    Thanks @LadyGP , doing god's work out here.

    I suspect forum moderation also needs a second look: many people are afraid to post their complaints here for fear of getting banned, and this is actively creating blindspots to performance issues with the game that developers need player feedback and information to be able to notice and address.

    We're lucky to have people cross-posting the complaints but ZOS can't keep chilling player feedback on the forums the way they've been and expect a good result in terms of their long-term relationship with their paying customers.
    Edited by doesurmindglow on December 20, 2024 7:40PM
    Guildmaster : The Wild Hunt (formerly Aka Baka) : AD PC/NA
  • ZOS_Kevin
    ZOS_Kevin
    Community Manager
    I suspect forum moderation also needs a second look: many people are afraid to post their complaints here for fear of getting banned, and this is actively creating blindspots to performance issues with the game that developers need player feedback and information to be able to notice and address.

    We are also taking a look at this in the New Year. We have already scheduled some meetings to discuss moderation practices and how we can find a better medium there.

    Community Manager for ZeniMax Online Studio and Elder Scrolls OnlineDev Tracker | Service Alerts | ESO Twitter
    Staff Post
  • doesurmindglow
    doesurmindglow
    ✭✭✭✭✭
    ZOS_Kevin wrote: »
    I suspect forum moderation also needs a second look: many people are afraid to post their complaints here for fear of getting banned, and this is actively creating blindspots to performance issues with the game that developers need player feedback and information to be able to notice and address.

    We are also taking a look at this in the New Year. We have already scheduled some meetings to discuss moderation practices and how we can find a better medium there.

    Thanks, Kevin.
    Guildmaster : The Wild Hunt (formerly Aka Baka) : AD PC/NA
  • LadyGP
    LadyGP
    ✭✭✭✭✭
    ZOS_Kevin wrote: »
    LadyGP wrote: »
    Meant to tag @ZOS_Kevin @ZOS_JessicaFolsom in the above post.

    I know we just got the big update from Rich - can we expect additional updates, hopefully before Jan 1, on how the investigation is going and if anything has been discovered? Doesn't have to be from Rich directly.. but just... something to help give some confidence to people about the issue?

    First, thank you for gathering the screenshots above.

    Regarding updates, we are currently working on a survey to better track information from players who experience performance and lag issues. This is in an effort to better track the information from more places (like Steam, Reddit, Discord Servers, etc,) like what you provided in the screenshots above, just in a more organized way and breaks it down the info that our engineers need. The survey when finished, it will send information directly to our engineers working on this and is intended to better organize the kind of information you took screenshots of above. We are hoping to have this situated in the New Year.

    Beyond what we provided above and Rich's last update, we won't have an update until after the New Year. Most of the team is leaving today for the holidays while the rest are gone after Monday, so no updates will be provided until a few weeks after we are back in the New Year and have a little more time to work.

    Totally makes sense - lots of people out of the office for holidays here as well. Sounds good - excited to see this survey. Thanks for all the hard work and on-going coms - definitely see an overall improvement from everyone involved, much appreciated!
    ZOS_Kevin wrote: »
    I suspect forum moderation also needs a second look: many people are afraid to post their complaints here for fear of getting banned, and this is actively creating blindspots to performance issues with the game that developers need player feedback and information to be able to notice and address.

    We are also taking a look at this in the New Year. We have already scheduled some meetings to discuss moderation practices and how we can find a better medium there.

    YES!!! Love this!
    Will the real LadyGP please stand up.
  • zenonuk
    zenonuk
    Cyrodiil raid group today was rubber banding all over the place. We never disconnected, but lag was atrocious with massive lag spikes about every 5-10 minutes.

    This problem gets notably worse during events. The more people using the server, the worse the problem is. I don't know if the problem is hardware or software, but most likely it's a combination of the two. Performance was never as good as it was after they upgraded the NA PC servers, so we know that modern hardware for sure can help.

    This also reflects my experience ... the game plays very happily with me and a companion in IA, it's plays very happily when I'm in a 4-person dungeon ... for hours in fact in both cases ... as soon as I port out, land in a busy town... bang, the lag is back.

    I thought maybe it could be due to gfx settings, (e.g. from high resolution characters and such like), but despite doing everything to (a) rule out my rig, and (b) reducing eso gfx settings... nope, no change. Tonight I managed a trial without issue, a few blips, but nothing major. Ported out, landed next to Breda... yeah, well I decided to log out.

  • cptscotty
    cptscotty
    ✭✭✭
    The lag is getting worst and more widespread in dungeons. Not sure what is going on and what happened this week but something definitely changed to cause this.

    As I was typing this, the lag was so bad in the dungeon it just disconnected me. Lovely.
  • silky_soft
    silky_soft
    ✭✭✭✭✭
    Half my bg just dc at same time. Both teams affected.
    Here $15, goat mount please. Not gambling or paying 45 : lol :
    20% base speed for high ping players.
    Streak moves you faster then speed cap.
    They should of made 4v4v4v4 instead of 8v8.
Sign In or Register to comment.