Recent Issues with NA Megaserver

Mogwhite

The famous urban legend of the DDOS attack

POps75p

Thogard wrote: »

Thank you for the update. I have a few questions, if you would be so kind:

1. For the last three weeks or so, we've seen issues with certain nodes near the tail end of many users routes to the server. Were these DDOS attacks also causing those issues? I can confirm that traffic on those nodes would spike coinciding with the "lag spikes" in game.
2. It seems that in the last 72 hours the default route to the servers for many users has changed. Is this related to the DDOS attacks?
3. Can you coment on ZOS's relationship with Akamai technologies?
4. It seems that the default / resting ping for many users has increased even when the traffic flow is normal (no DDOS). Is it fair to assume that this is a side effect of the extra "defenses?"
5. Most importantly, where is the best jump in point for those of us using a VPN to improve ping? It seems as if having the VPN drop me off in Dallas is no longer ideal. My ping maps seem to indicate NorCal / Bay Area as the new, ideal jump in spot.

I have a hard time with this myself, with their record of BS, it's really tough to believe anything they say

AlphaAbsol

squinceybones wrote: »

Surely these ddos people just get sent an email saying you have logged in from a different IP address that never comes, and therefore cannot ddos. What more security could you need?

That's not how it works at all. That's logging into the game which DDoSers are not doing.

Mojmir

How convenient, they can ride this excuse out indefinitely.
it ironically happens during major events they want people to login,patch days,etc.
I guess it's just another random (not really) thing they have no control over.lol

AlMcFly

For the life of me, ever since I read about the advent of DDOS attacks years ago, I've STILL never understood why people even do it. The most "believable" reason I've read is that these people perform DDOS and then try to extort money from the company, blackmailing them if they want the DDOS to stop. To be honest though, how many DDOS scumbag worms have successfully extorted money in this way? I've not heard one instance of companies actually giving into cyber terrorist demands like this.

LegacyDM

I call BS. This is the same excuse used for years. Really? Server performance and crap code has nothing to do with the loading screens? Lag? and disconnects? The fact that when there is a massive tri faction battle during prime-time at chalman and the server comes to a crawl is the work of DDOS? LOL!. How convenient. When there is an event which overloads the server the mysterious DDOS attack excuse shows up.

https://www.youtube.com/watch?v=eKgPY1adc0A

Kronuxx

Mogwhite wrote: »

The famous urban legend of the DDOS attack

Exactly. Sorry but you guys have used this excuse quite often when there has been horrible latency. Somehow the DDOS coincides at the same time more people join into one location known as Cyrodiil? I call bs. Rather ZOS, man up for once, admit your servers need updating, that all the commands sent to the server from people PvPing, causes extra server load and stress, and get to updating, instead of making excuses and lies. I guarantee anyone here, that once this "DDOS" resolves, the server lag will still be as *** as ever, if not "slightly" improved.

C3N50R3D

Thogard wrote: »

Thank you for the update. I have a few questions, if you would be so kind:

1. For the last three weeks or so, we've seen issues with certain nodes near the tail end of many users routes to the server. Were these DDOS attacks also causing those issues? I can confirm that traffic on those nodes would spike coinciding with the "lag spikes" in game.
2. It seems that in the last 72 hours the default route to the servers for many users has changed. Is this related to the DDOS attacks?
3. Can you coment on ZOS's relationship with Akamai technologies?
4. It seems that the default / resting ping for many users has increased even when the traffic flow is normal (no DDOS). Is it fair to assume that this is a side effect of the extra "defenses?"
5. Most importantly, where is the best jump in point for those of us using a VPN to improve ping? It seems as if having the VPN drop me off in Dallas is no longer ideal. My ping maps seem to indicate NorCal / Bay Area as the new, ideal jump in spot.

Anyone in the security industry knows what Akamai does and it does cause some lag sending the traffic through but not enough to explain the ridiculous amount of lag on an ongoing basis and the server not responding to some client commands at all in certain cases. If your ping is above a certain threshold or there is a lot of ping flux, then a Gaming VPN can add consistency at the expense of slightly more lag. Drop point should be as close to ZoS servers as possible with a two server solution. @ZOS_GinaBruno mentioned reaching out to internet providers where they are seeing problematic routes between users and their datacenters. It might be a good idea for ZoS to partner with some of the Gaming VPN companies to provide more stable routes and more consistent server response.

Elsonso

Kronuxx wrote: »

Mogwhite wrote: »

The famous urban legend of the DDOS attack

Exactly. Sorry but you guys have used this excuse quite often when there has been horrible latency. Somehow the DDOS coincides at the same time more people join into one location known as Cyrodiil? I call bs. Rather ZOS, man up for once, admit your servers need updating, that all the commands sent to the server from people PvPing, causes extra server load and stress, and get to updating, instead of making excuses and lies. I guarantee anyone here, that once this "DDOS" resolves, the server lag will still be as *** as ever, if not "slightly" improved.

Hmm. No. I believe they actually are dealing with a DDoS attack. I noticed the signs that they were dealing with one on Wednesday, and it was pretty obvious that it was getting worse on Thursday.

Darkmage1337

ZOS_GinaBruno wrote: »

Hi everyone,

We’d like to briefly explain what’s been going on with the North American server stability lately. In short, we’ve been getting hit with many repeated DDoS attacks (Distributed Denial of Service). When this happens, it floods our service and prevents you from being able to log in, stay connected, or even group properly. Sometimes they only affect a handful of players, and other times they can cause a much greater and widespread impact. Either way, it’s a frustrating situation.

While we do have protection in place (which we obviously can’t discuss publicly), it hasn’t been enough. We’ve been working hard to implement additional security measures to make sure you’re able to log in – and stay logged in – whenever you want to play, and aren’t interrupted. We hope you understand why we’re being somewhat vague and not discussing specifics here. We take this issue very seriously, though, and are working hard to ensure everyone has the best experience possible.

Thank you for your continued support and patience.

In light of DDOS attacks, please extend the Midyear Mayhem PvP event by a week because the entire game is completely unplayable for so many players, not even just Cyrodiil, is completely unplayable.
If you are not a part of the unlucky majority; well, then lucky you...

Dracofyre

only been in one Maddness, and going to miss entire week because of not able to stay connected, contantly booted and unloggingable.
barely long enough for few mins and best time long enough for crafting writs before bootings.
not long emough for harvest nodes.

TazESO

FYI, DDOS attacks are not BS. I have no inside knowledge of this particular situation, but I do know that gaming is now one of the most attacked targets. An entire industry has surfaced to combat DDOS and help with stability of web content streaming, online commerce, and of course online gaming. Akamai, other CDN services, etc.

TazESO

Two examples:

https://www.akamai.com/us/en/resources/protect-against-ddos-attacks.jsp


https://www.cloudflare.com/lp/ddos-b/?_bt=200127107716&_bk=+ddos +prevention&_bm=b&_bn=g&gclid=CKmMyZzIrdUCFUdWDQodcoANUg

agingerinohio

Could all the botters I see have something to do with it? Maybe whatever program they're running is somehow appearing as a DDoS? How else would they remain active for days on end? Also, after being "dealt" with pop right back up by I'm assuming the same people. I was talking to my IT friend about it and he said it sounds like they are using multiple IP's to stay active and in business.

Rohamad_Ali

TazESO wrote: »

Two examples:

https://www.akamai.com/us/en/resources/protect-against-ddos-attacks.jsp


https://www.cloudflare.com/lp/ddos-b/?_bt=200127107716&_bk=+ddos +prevention&_bm=b&_bn=g&gclid=CKmMyZzIrdUCFUdWDQodcoANUg

They've been doing this for years and we get to watch the nerd hackers go into real jails all the time . They think they are so clever until they are giving back rubs to Bubba the bank robber in Federal for three years . All over nerd rage on a video game . Don't drop the soap in there .

Wheyntheblacksmith_ESO

Still not working for me. Looks like I am stuck on EU servers from now on. You can at least transfer my wifes and mines stuff over since I will not be on the NA any longer.

Brettoel

@ZOS_GinaBruno

Thanks alot man , will this mean an extension to the event? i hope you guys figure everything out , maybe best to take a break from making content after this coming DLC to focus on what there is and fixing what needs fixing before moving on

Hyade

Thank you for informing your customers, we appreciate it.

Decon98b16_ESO

ZOS_GinaBruno wrote: »

Hi everyone,

We’d like to briefly explain what’s been going on with the North American server stability lately. In short, we’ve been getting hit with many repeated DDoS attacks (Distributed Denial of Service). When this happens, it floods our service and prevents you from being able to log in, stay connected, or even group properly. Sometimes they only affect a handful of players, and other times they can cause a much greater and widespread impact. Either way, it’s a frustrating situation.

While we do have protection in place (which we obviously can’t discuss publicly), it hasn’t been enough. We’ve been working hard to implement additional security measures to make sure you’re able to log in – and stay logged in – whenever you want to play, and aren’t interrupted. We hope you understand why we’re being somewhat vague and not discussing specifics here. We take this issue very seriously, though, and are working hard to ensure everyone has the best experience possible.

Thank you for your continued support and patience.

Thank you for finally saying something on this topic.
As far as saying it is DDoS attacks - bull

Taysa

So any update on this because Cyrodiil is still unplayable for me. The game randomly locks up. And now I can't even log into the game.

Also, any consideration being made toward extending the event? Since the servers are absolute garbage right now?

Swifigames

Sincerest of thanks for the insight and all the hard work the team puts in for this gaming community. I can't say I know the inner workings of such madness, but I've done some reading on DDoS attacks earlier in the year and I'm sure it can be brutal at times. I send my best wishes to you all and hope for everything to work out like a kinked neck in the end. Then we'll all be able to relax.

Wheyntheblacksmith_ESO wrote: »

Still not working for me. Looks like I am stuck on EU servers from now on. You can at least transfer my wifes and mines stuff over since I will not be on the NA any longer.

How is it playing on the EU server(s)? I'm not certain I want to give it a go...but what's it like? Any differences, laggy, etc? And if that's an option why shouldn't I go that route myself in situations like this?
Thanks.

jolinar83ub17_ESO

Thank you for the update was getting frustrated with support saying it was on my end when like nothing has changed AND I can get on the PTS just fine.

Mozsta69

Thanks for the update at least I know why I lost connection a few times, hopefully it will stop.

LadyDestiny

Thanks for the info....

Pele

Thanks for the update.

Turelus

Thanks for the update Gina, many in the community will appreciate knowing what's going on.

Sigh... I'm glad we had at least one page of nice comments and thankful players before the tinfoil rage prograde showed up.

LegacyDM wrote: »

I call BS. This is the same excuse used for years. Really?

Kronuxx wrote: »

Exactly. Sorry but you guys have used this excuse quite often when there has been horrible latency.

Really? Because I am pretty sure ZOS has rarely come out stating they're under attack, in fact why don't we check how many posts Gina has made with DDoS in the text. Let's have a look... clearly she's been over using this excuse.

LegacyDM wrote: »

Server performance and crap code has nothing to do with the loading screens? Lag? and disconnects? The fact that when there is a massive tri faction battle during prime-time at chalman and the server comes to a crawl is the work of DDOS? LOL!. How convenient. When there is an event which overloads the server the mysterious DDOS attack excuse shows up.

Kronuxx wrote: »

Somehow the DDOS coincides at the same time more people join into one location known as Cyrodiil? I call bs. Rather ZOS, man up for once, admit your servers need updating, that all the commands sent to the server from people PvPing, causes extra server load and stress, and get to updating, instead of making excuses and lies. I guarantee anyone here, that once this "DDOS" resolves, the server lag will still be as *** as ever, if not "slightly" improved.

They've not stated that the Cyrodiil issues are solely because of a DDoS attacks at all, they've said the entire server wide issues currently happening are due to them.
ZOS have actually been very open about their efforts to fix code and server demands which cause the Cyrodiil lag, and have made a number of changes over the last years to help alleviate it more.

Wheyntheblacksmith_ESO

Swifigames wrote: »

Sincerest of thanks for the insight and all the hard work the team puts in for this gaming community. I can't say I know the inner workings of such madness, but I've done some reading on DDoS attacks earlier in the year and I'm sure it can be brutal at times. I send my best wishes to you all and hope for everything to work out like a kinked neck in the end. Then we'll all be able to relax.

Wheyntheblacksmith_ESO wrote: »

Still not working for me. Looks like I am stuck on EU servers from now on. You can at least transfer my wifes and mines stuff over since I will not be on the NA any longer.

How is it playing on the EU server(s)? I'm not certain I want to give it a go...but what's it like? Any differences, laggy, etc? And if that's an option why shouldn't I go that route myself in situations like this?
Thanks.

The ping is a little higher, chat is non-existent. Rather peaceful really. IMO I feel the EU servers are either dead or they now a NA player and don't like us. Hence they refuse to talk or the EU servers are A: That dead or B: The time difference is that drastic and everyone is asleep lol or C: it's filled mostly with *** bots they can't farm worth a damn which leaves a slow player like myself to enjoy his RP walking and still can farm to his heart content while enjoying the views and sites. if you decide to jump on, i have a Fem Bosmer named Nueska Osomi. Like me up so you have a fellow NA player to game with. Least talk to ya know.

Dracofyre

i am not sure if they (cheaters) used 3rd party plug-in for booting unwanted players and blocks other players trying to log in while they still playing and earning AP more than anyone.
might be entire guild cheating

i am only guessing any possible why since this is special summer event, and once the event are over, and likely "DDoS" dissappeared.

Mevrain

Wheyntheblacksmith_ESO wrote: »

Still not working for me. Looks like I am stuck on EU servers from now on. You can at least transfer my wifes and mines stuff over since I will not be on the NA any longer.

How is it playing on the EU server(s)? I'm not certain I want to give it a go...but what's it like? Any differences, laggy, etc? And if that's an option why shouldn't I go that route myself in situations like this?
Thanks.[/quote]

It's fine. Nothing transfers from the NA servers (no guilds, no items, no CP, nothing) so you basically get to start fresh with new chars. Which after all this time is kind of refreshing.

GreenhaloX

Some of you guys at the ZOS lounge must have been working some overtimes, then. I'm not sure if it would be politically correct to say.. hey, it's job security. However, sadly, there are just so many jacked up bad people in the world.