PC/NA just went down

ArchangelIsraphel · May 9

Ever since these issues began on May 7th, I've been having intermittent problems.

Normally, ESO runs very smoothly for me. I'm even running on a fresh install from May 5th that resolved a few minor issues I was having.

But ever since the 7th, I've been getting ping spikes regardless of what time I play, and strange latency during inventory management or when interacting with NPC's- especially when buying or selling from merchants.

My ping average is around 80-90 MS on PCNA, I'm spiking upwards of 400 during these strange lag spikes. I'm on the East Coast of the US.

Xandreia_ wrote: »

Desiato wrote: »

Xandreia_ wrote: »

the performance has gone from bad to worse, i was just in a trial where everyone in there kept hitting 999ping, its borderline impossible to do anything @ZOS_Kevin

When were you playing and what geographic area do you play from? PC or console? I'm in NE NA on PC and I played from around 7pm to 2am EDT last night. I played in a 3 trials and 2 arenas without any issues at all.

im uk playing on pc na, alot of the people i was just with are na east

I'm NA east and having a very similar experience as your friends are.

ZOS_Kevin · May 9

Jaimeh wrote: »

I got disconnected to the log in screen with the message 'an error occurred, please wait a few minutes' (PC/NA). Could we get an update on whether these new disconects and lag issues are being looked at @ZOS_Kevin?

These are being looked into. And we are relaying your feedback to the teams looking into the disconnect issues.

Desiato · May 9

ZOS_Kevin wrote: »

These are being looked into. And we are relaying your feedback to the teams looking into the disconnect issues.

@ZOS_Kevin

I haven't been playing, but when I read the responses to this thread after arriving home, I pinged the ip the eso support site describes as the na eso server address and received the following results:

Ping statistics for 198.20.200.1:
Packets: Sent = 310, Received = 304, Lost = 6 (1% loss),
Approximate round trip times in milli-seconds:
Minimum = 57ms, Maximum = 95ms, Average = 71ms

I pinged my ISP's gateway at the same time and did not have any dropped packets.

The last hop before the ESO server for me is 184.26.104.144.

Normally I do not experience packet loss when pinging the eso server.

Edit: The first ping results I posted were from today at around 4 PM EDT. After, I started pinging again and just stopped at 4:38 PM EDT and I'm still experiencing packet loss:

Ping statistics for 198.20.200.1:
Packets: Sent = 1624, Received = 1602, Lost = 22 (1% loss),
Approximate round trip times in milli-seconds:
Minimum = 52ms, Maximum = 91ms, Average = 63ms

Though 1% may not seem much, it's enough to cause players problems when doing something like soloing the unmaker, pvping, or vet pve content.

Edit 2:

Between 4:38 PM to 5:35 PM EDT these were my results:

Ping statistics for 198.20.200.1:
Packets: Sent = 3109, Received = 3062, Lost = 47 (1% loss),
Approximate round trip times in milli-seconds:
Minimum = 51ms, Maximum = 97ms, Average = 65ms

I played during this period and there were minor annoyances. I wouldn't want to pvp or do something challenging.

The dropped packets tend to come in bursts. For example:

Reply from 198.20.200.1: bytes=32 time=70ms TTL=247
Request timed out.
Reply from 198.20.200.1: bytes=32 time=65ms TTL=247
Reply from 198.20.200.1: bytes=32 time=63ms TTL=247
Reply from 198.20.200.1: bytes=32 time=69ms TTL=247
Reply from 198.20.200.1: bytes=32 time=81ms TTL=247
Request timed out.
Request timed out.
Reply from 198.20.200.1: bytes=32 time=67ms TTL=247

If I needed to heal myself or move out of an AE when this sequence happened, it would have been a problem.

Jaimeh · May 9

ZOS_Kevin wrote: »

Jaimeh wrote: »

I got disconnected to the log in screen with the message 'an error occurred, please wait a few minutes' (PC/NA). Could we get an update on whether these new disconects and lag issues are being looked at @ZOS_Kevin?

These are being looked into. And we are relaying your feedback to the teams looking into the disconnect issues.

Thank you for the response!

cicisch · May 9

Definitely seems to be starting back up tonight - playing ToT and opening containers for me is laggy and hitting a world boss took about 10 seconds to actually register.

FlopsyPrince · May 9

@ZOS_Kevin An official response on all these issues that seem to be happening recently would be very nice to have from ZOS.

I realize you may not know it all, but this can shake the confidence when the game starts to be wonky in so many areas.

Note that I am not expecting you or @ZOS_GinaBruno to say anything, but official word when things (hopefully) finally stabilize better would be really nice to have.

Jaraal · May 9

Desiato wrote: »

ZOS_Kevin wrote: »

These are being looked into. And we are relaying your feedback to the teams looking into the disconnect issues.

@ZOS_Kevin

I haven't been playing, but when I read the responses to this thread after arriving home, I pinged the ip the eso support site describes as the eso server address and received the following results:

Ping statistics for 198.20.200.1:
Packets: Sent = 310, Received = 304, Lost = 6 (1% loss),
Approximate round trip times in milli-seconds:
Minimum = 57ms, Maximum = 95ms, Average = 71ms

I pinged my ISP's gateway at the same time and did not have any dropped packets.

The last hop before the ESO server for me is 184.26.104.144.

Normally I do not experience packet loss when pinging the eso server.

Edit: The first ping results I posted were from today at around 4 PM EDT. After, I started pinging again and just stopped at 4:38 PM EDT and I'm still experiencing packet loss:

Ping statistics for 198.20.200.1:
Packets: Sent = 1624, Received = 1602, Lost = 22 (1% loss),
Approximate round trip times in milli-seconds:
Minimum = 52ms, Maximum = 91ms, Average = 63ms

Though 1% may not seem much, it's enough to cause players problems when doing something like soloing the unmaker, pvping, or vet pve content.

Edit 2:

Between 4:38 PM to 5:35 PM EDT these were my results:

Ping statistics for 198.20.200.1:
Packets: Sent = 3109, Received = 3062, Lost = 47 (1% loss),
Approximate round trip times in milli-seconds:
Minimum = 51ms, Maximum = 97ms, Average = 65ms

I played during this period and there were minor annoyances. I wouldn't want to pvp or do something challenging.

The dropped packets tend to come in bursts. For example:

Reply from 198.20.200.1: bytes=32 time=70ms TTL=247
Request timed out.
Reply from 198.20.200.1: bytes=32 time=65ms TTL=247
Reply from 198.20.200.1: bytes=32 time=63ms TTL=247
Reply from 198.20.200.1: bytes=32 time=69ms TTL=247
Reply from 198.20.200.1: bytes=32 time=81ms TTL=247
Request timed out.
Request timed out.
Reply from 198.20.200.1: bytes=32 time=67ms TTL=247

If I needed to heal myself or move out of an AE when this sequence happened, it would have been a problem.

Packet loss and high latency can be traced to the anti-DDoS servers all data is routed to before it gets to the ESO servers. This has been well documented with tracerts for years.

https://forums.elderscrollsonline.com/en/discussion/436992/high-latency-at-akamai-com/p1

Desiato · May 10

Jaraal wrote: »

Packet loss and high latency can be traced to the anti-DDoS servers all data is routed to before it gets to the ESO servers. This has been well documented with tracerts for years.

https://forums.elderscrollsonline.com/en/discussion/436992/high-latency-at-akamai-com/p1

I don't believe that thread from 2018 is relevant anymore. Maybe it is for the login server. One will not have that experience with traceroutes to the ESO NA server. At least not at the moment. In any case, what was shown in my post was not the result of an Akamai black hole and neither is a traceroute to the address at the present time.

The IP address I'm pinging was provided by ZOS in a help article for the purpose of identifying network performance problems.

The latency shown in my ping results is not high, and the dropped packets corresponded with actual issues while playing the game.

It's not the first time I've pinged that address. There is usually no packet loss. Furthermore, there has been no dropped packets for the last hour or so.

Edit: and just after posting that my packet loss cleared up, it returned!

Wildberryjack · May 10

On NA I'm still having problems. I'm getting bad lag spikes constantly, up to 999+, every 10-15 seconds. But I'm also having the game totally freeze between 3 to 6 seconds. These freezes happen 4 to 5 times a minute and they also happen when I'm NOT having a lag spike.

I do get lag spikes on EU also, more than I should, however they aren't as bad nor happen as often and the game does not freeze at all. I barely notice those spikes to be honest. Considering I live in the same state as the NA server it seems I should not be getting better results playing on the EU server. I also play other MMOs with servers in Chicago and California and I'm not having any issues with them at all.

There is something wrong with your NA server and I hope you get it sorted before the new expansion drops because this really is unplayable.

Tallon_IV · May 10

I haven't gotten disconnected yet, but randomly when I try to select dialogue while talking to an NPC there will be a huge delay before it registers. My ping doesn't show as spiking though. Also seeing a few players rubber-banding and teleporting.

Sluggy · May 10

I'm getting constant freezing now. Like it's gone from freezing every twenty to thirty minutes to sometimes less than a second. I've probably lost out of a dozen surveys because I freeze as soon as I start one and it's been consumed without my ever finishing it before I had to restart the game.

Somehow I managed to get a trial going and even finish it but everyone was freezing up for five or ten seconds at a time pretty much nonestop. Luckily it was an easy one and I had enough HoTs all overmyself as the tank that I didn't die. But I'd often find the game was an ACTUAL LITERAL SLIDESHOW that felt more like a recap of the fight than the actual thing live. And I'd frequently see different groups of people dead because of the issues. This is literally falling into the territory of unplayable and regardless of where the issue my be I think ZoS needs to address it because it's affecting a lot of people now.

ArchangelIsraphel · May 10

Yeah, it's been getting gradually worse throughout the day- in the morning (around 9:30 am EDT) it was tolerable, but now you really can't play without risking disconnect while in the middle of something. It seems the later it gets, the worse it gets. (11:29 PM EDT)

I've just been going on to do Endeavors and Writs then logging off to go do other things because I don't want to risk data loss with the sudden disconnects and strange way the server has been behaving

Just_Attivi · May 10

Logged in to do endeavors/collect daily reward, as I already anticipate being unable to play at this point (how sad lol). each crafting station was accompanied by 5 seconds or so of lag, players teleporting all over between each station (while my ping read 120ish), interacting with anything had a delay. went to open my crafting writ rewards, ping 999 and crash. Net has been fine on my end, was just in other applications without computer issues. it is ESO.

Be amazing if we had more insight/plans going forward/any idea whats going on besides 'it is being looked into.'' This is a service most are paying for, and it has been consistently unusable or problematic for over a month now (though these past few days have been the icing on the cake). whether it is at fault of ZOS or not, this has been a month+ of problems, from rollbacks every 10 min at the start of april, the 2 weeks of Tarnished nightmare exploits, to half my pvp friends getting banned for 2 weeks for daring to enter the PTS, to the progressively degrading performance not just in cyro but in PVE land now too, to the point that its unplayable.... there has been continuous issues for over a month.... not just these past few days. I've overlooked a lot of smaller inconveniences/problems with this service for years, but whats going on now.... this has been enough that I have cancelled my sub, def not buying gold road. no point in paying when I cant be playing. Love ESO, no game has PVP like it, but this has just been an unacceptable failure of service, and my faith in a successful chapter launch at this point is... nonexistent. I hope to be proven wrong, and hope this mess gets sorted, but I can't justify supporting it any longer.

/rantover

Genfe · May 10

ZOS_GinaBruno wrote: »

Thanks all, just confirming we've seen your reports today and are looking into things.

Crashed 6 times yesterday. 1 resulting in internal error message making me wait in a second 40q. If I restart my computer I don’t get this message and can log back into Cyrodiil. Randoms skips where everyone freezes then snaps back into reality. Honestly it’s almost comical at this point.

Wildberryjack · May 10

You're coming out with new content soon and World of Warcraft is coming out with new content soon. If you can't stabilize your NA server I definitely will be playing WoW instead. It will also dictate whether I renew my sub or let it go. I'm not paying for something I can't hardly play.

Oh yeah, the freezing is getting worse.

Seravi · May 10

PCNA

Having a hard time today doing anything. I have to hit my keys several times before the game responds. Just trying to do daily writs right now and it is harsh.:(

Hanging up a lot just trying to load my guild store. When I first logged on around 7AMCST this morning no issues. Now at 10AMCST almost unplayable.

davidtk · May 10

I've been playing a bit on NA (I play primarily on EU) the ping I had a while ago (several months ago until last weeks) was in the bearable 120-160 range (playing form the central EU). Not great, weaving was problematic at times, but it was playable.
(But I went to NA mainly for the peace of mind and the experience of starting all over again with no achievements and no stuff collected. It was pretty fun except for some odd things))
Now lately? Unplayable, ping 160/240 -> 460+ (In the most important cases [boss fight f.e.] 999+). Everything slowed down, even interactions with NPCs, even stupid daily craft problematic.
(Okay, I know. I can stay on the EU, but lately I've been experiencing similar problems here on the EU)

StihlReign · May 10

Jaraal wrote: »

Packet loss and high latency can be traced to the anti-DDoS servers all data is routed to before it gets to the ESO servers. This has been well documented with tracerts for years.

https://forums.elderscrollsonline.com/en/discussion/436992/high-latency-at-akamai-com/p1

Good ole Akamai anti-ddos service at it again?

Desiato · May 10

StihlReign wrote: »

Good ole Akamai anti-ddos service at it again?

There's no evidence of that.

My connection to the server is currently flawless. No packet loss and my in-game performance is perfect. Of course, after posting this I feel I am likely to experience issues again!

Ping statistics for 198.20.200.1:
Packets: Sent = 1340, Received = 1340, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 53ms, Maximum = 75ms, Average = 59ms

Tandor · May 10

I imagine it would be useful for ZOS if those experiencing these problems - especially outside Cyrodiil - were to post where in the world they're playing from and through which ISP. Whenever something like this affects only some players and not all it is invariably useful to be able to find some common denominators.

Jaraal · May 10

StihlReign wrote: »

Jaraal wrote: »

Packet loss and high latency can be traced to the anti-DDoS servers all data is routed to before it gets to the ESO servers. This has been well documented with tracerts for years.

https://forums.elderscrollsonline.com/en/discussion/436992/high-latency-at-akamai-com/p1

Good ole Akamai anti-ddos service at it again?

Easy to test.

Windows key + R

Type cmd or cmd.exe in the box.

Type: tracert 198.20.200.1 in the command window and press Enter. Akamai filtering confirmed.

You can also run ping 198.20.200.1

TheMajority · May 10

No maintenance this week doesn't bode well

Desiato · May 10

TheMajority wrote: »

No maintenance this week doesn't bode well

It doesn't necessarily bode poorly either. There may be no issues with the game itself to fix. The issues could be entirely network/data center/zos isp related.

All of my issues the past day correlate with packet loss to the ip address ZOS provides for troubleshooting NA game connection issues.

Today, I have experienced absolutely no issues playing the game, and also no packet loss outside of a few dropped packets during a 10 minute window starting at around 3:45 PM EDT.

Seravi · May 10

I am in High Isle right now and it is horrid. I go to chop wood and five seconds later I chop the wood as I am moving to something else. Trying to get through quest menus is bad. Every few seconds it hangs up and I have to wait so I don't choose something incorrect. Same goes for spells, etc Tried to use Venom Skull and it took almost five seconds for every one I threw to animate. I was already attacking with my daggers and still had the skull animations trying to trigger.

PCNA

tomofhyrule · May 10

Out of curiosity, what have people having issues tried?

I know a guildie in my dungeon/raid group was having constant issues with lag, ping skipes, and rubberbanding since the event, but he recently reinstalled the game completely and that seemed to have fixed about 90% of his issues.

MorninStar · May 10

After selecting character, all you get is a load screen atm. After 10 mins I shut the game down. Any update on what the problem is?

Desiato · May 10

Seravi wrote: »

I am in High Isle right now and it is horrid. I go to chop wood and five seconds later I chop the wood as I am moving to something else. Trying to get through quest menus is bad. Every few seconds it hangs up and I have to wait so I don't choose something incorrect. Same goes for spells, etc Tried to use Venom Skull and it took almost five seconds for every one I threw to animate. I was already attacking with my daggers and still had the skull animations trying to trigger. PCNA

MorninStar wrote: »

After selecting character, all you get is a load screen atm. After 10 mins I shut the game down. Any update on what the problem is?

Just so you know, not everyone is experiencing these problems. I haven't had a problem all day. I just logged in very quickly into a zone packed with players doing their thing.

It might be helpful for any ZOS staff reading this thread to mention which region you play from and any troubleshooting you've done.

If I were you, I would contact ZOS support for help troubleshooting. I'm not saying the issue is on your end. It's possible ZOS is experiencing network issues that affect some players but not others. However at this moment the issues described in those quotes aren't affecting everyone.

Wildberryjack · May 10

Desiato wrote: »

However at this moment the issues described in those quotes aren't affecting everyone.

They're affecting too many players scattered all over for it to be on all of our ends. When I'm on I'm constantly seeing people asking in zone and guild chat if anyone else is freezing and rubber-banding. This is too widespread to be anything other than with their server. I also play on EU and am not having these problems, only on NA is this happening.

Seravi · May 10

Desiato wrote: »

Seravi wrote: »

I am in High Isle right now and it is horrid. I go to chop wood and five seconds later I chop the wood as I am moving to something else. Trying to get through quest menus is bad. Every few seconds it hangs up and I have to wait so I don't choose something incorrect. Same goes for spells, etc Tried to use Venom Skull and it took almost five seconds for every one I threw to animate. I was already attacking with my daggers and still had the skull animations trying to trigger. PCNA

MorninStar wrote: »

After selecting character, all you get is a load screen atm. After 10 mins I shut the game down. Any update on what the problem is?

Just so you know, not everyone is experiencing these problems. I haven't had a problem all day. I just logged in very quickly into a zone packed with players doing their thing.

It might be helpful for any ZOS staff reading this thread to mention which region you play from and any troubleshooting you've done.

If I were you, I would contact ZOS support for help troubleshooting. I'm not saying the issue is on your end. It's possible ZOS is experiencing network issues that affect some players but not others. However at this moment the issues described in those quotes aren't affecting everyone.

I've been troubleshooting on my end all day with all of the things support generally suggests. I am quite confident the issue is not on my side but just reporting here as they asked us to do in hopes this gets worked out. Could just be a hiccup in the traffic somewhere, who knows.

Desiato · May 10

Wildberryjack wrote: »

Desiato wrote: »

However at this moment the issues described in those quotes aren't affecting everyone.

They're affecting too many players scattered all over for it to be on all of our ends. When I'm on I'm constantly seeing people asking in zone and guild chat if anyone else is freezing and rubber-banding. This is too widespread to be anything other than with their server. I also play on EU and am not having these problems, only on NA is this happening.

I'm not saying ZOS isn't experiencing issues, but it's definitely not affecting everyone. I just completed a trial and no one was complaining and I definitely had no issues.

But it's also possible in some cases that players who are experiencing correctable issues on their end will read these threads and presume they are affected, which many not always be the case.