Update 41 is now available for testing on the PTS! You can read the latest patch notes here: https://forums.elderscrollsonline.com/en/categories/pts

Hardware Refresh Work Update – March 31, 2023

ZOS_Kevin
ZOS_Kevin
Community Manager
Hi All, we wanted to share with everyone the latest on our ongoing hardware refresh work. The work on PC EU is nearing completion. We are doing the cutover during an extended maintenance that will start in the evening on April 3 and end in the morning on April 4. Tentative times are April 3 at 8:00 PM EDT to April 4 at 10:00 AM EDT (1:00 AM BST to 3:00 PM BST on April 4). Once the maintenance is complete, we will monitor the PC EU server performance. This maintenance does occur during the Jester’s Festival and is why we preemptively added an extra day to the beginning of the event.

For the remaining servers, below are the latest planned timelines. Please note that these timelines could change if we run into any issues with the hardware refresh process or other issues arise with the ESO live service that takes priority.
  • PlayStation EU: This server is next, and that work will be completed later this month (April.) We will let you all know when we’ve locked in a date.
  • Xbox NA and PlayStation NA: This work is scheduled for completion sometime in May. We will provide dates when we have them.
  • Xbox EU: Additional work is needed in our Frankfurt datacenter before we can do the Xbox EU hardware refresh work. We recognize that this server coming last is not in-line with what we originally communicated in December. This is the server experiencing the least amount of performance/stability issues. The hardware refresh work for this server is planned for some time later this year. We’ll provide updates as we have them.
Thank you!
Edited by ZOS_Kevin on October 4, 2023 7:54PM
Community Manager for ZeniMax Online Studio and Elder Scrolls OnlineDev Tracker | Service Alerts | ESO Twitter
Staff Post
  • thejadefalcon
    thejadefalcon
    ✭✭✭✭✭
    "why we preemptively added an extra day to the beginning of the event."

    So you're anticipating people can't play and earn tickets and your solution is to benefit other servers more than PC EU (since everyone gets the extra day, not just PC EU), rather than just... schedule the maintenance some other time?

    Don't get me wrong, yay for smoother Cyrodiil, but seriously, why are you so determined to consistently schedule maintenance at the worst possible times for your players? This isn't even an event thing, it's just even more ludicrous when it is.
  • AdexiTheOneEyed
    Kevin, is there any update on Whitestrake's Mayhem? I'm still holding out hope we'll get the event, at this point, it feels largely unlikely. Last year, ZOS swapped out the Imperial City event to put in their High Isle Event. As a result, the price of Hakeijo in the market is through the roof, it doesn't help that Whitestrake's hasn't happened yet and that has further increased the prices of certain items that we'd normally get in Mayhem event boxes.

    As always, we sincerely hope for changes and fixed to both Overland and PvP zones.
    Former Empress
  • ZOS_Kevin
    ZOS_Kevin
    Community Manager
    Kevin, is there any update on Whitestrake's Mayhem?

    Nothing to announce right now, but we will have an update on Whitestrake's Mayhem soon.

    Community Manager for ZeniMax Online Studio and Elder Scrolls OnlineDev Tracker | Service Alerts | ESO Twitter
    Staff Post
  • Sidewaves89
    Sidewaves89
    ✭✭✭
    "why we preemptively added an extra day to the beginning of the event."

    So you're anticipating people can't play and earn tickets and your solution is to benefit other servers more than PC EU (since everyone gets the extra day, not just PC EU), rather than just... schedule the maintenance some other time?

    Don't get me wrong, yay for smoother Cyrodiil, but seriously, why are you so determined to consistently schedule maintenance at the worst possible times for your players? This isn't even an event thing, it's just even more ludicrous when it is.

    [snip] tickets, you getting new servers. [snip] relax and enjoy.

    [edited for baiting]
    Edited by ZOS_Icy on March 31, 2023 2:33PM
  • Cobble123
    Cobble123
    ✭✭✭
    "why we preemptively added an extra day to the beginning of the event."

    So you're anticipating people can't play and earn tickets and your solution is to benefit other servers more than PC EU (since everyone gets the extra day, not just PC EU), rather than just... schedule the maintenance some other time?

    Don't get me wrong, yay for smoother Cyrodiil, but seriously, why are you so determined to consistently schedule maintenance at the worst possible times for your players? This isn't even an event thing, it's just even more ludicrous when it is.

    Wow. Just wow.
  • Reverb
    Reverb
    ✭✭✭✭✭
    ✭✭✭✭✭
    I’ll be very interested in hearing if PC EU sees the same pvp performance gains that we did on NA for the first week after the hardware upgrades.
    Battle not with monsters, lest ye become a monster, and if you gaze into the abyss, the abyss gazes also into you. ~Friedrich Nietzsche
  • Ingenon
    Ingenon
    ✭✭✭✭✭
    Thanks for providing this schedule update!

    I am looking forward to the PlayStation NA server upgrade, and getting to play using server hardware that is newer than the PS4 that I already traded in for a PS5. >:)
  • Gadamlub14_ESO
    Gadamlub14_ESO
    ✭✭✭✭
    an extra day for the event because you expect many people to miss a day when the event was already specifically designed to not give you enough shards to earn new cosmetics in the first place even with that extra day?

    Well gee, thanks for forcing me to buy even more event tickets if i want anything from this event since this is during the only time i can play.
    Edited by Gadamlub14_ESO on March 31, 2023 2:48PM
  • colossalvoids
    colossalvoids
    ✭✭✭✭✭
    ✭✭✭✭
    Thanks for the update, new hardware would be the best anniversary present to the community if everything would be set up correctly and next patch won't introduce some additional over the top calculations rendering it useless. Fingers crossed!
  • Poss
    Poss
    ✭✭✭✭
    Xbox EU “This is the server experiencing the least amount of performance/stability issues.”

    Lol ok
  • Destai
    Destai
    ✭✭✭✭✭
    ✭✭
    ZOS_Kevin wrote: »
    Hi All, we wanted to share with everyone the latest on our ongoing hardware refresh work. The work on PC EU is nearing completion. We are doing the cutover during an extended maintenance that will start in the evening on April 3 and end in the morning on April 4. Tentative times are April 3 at 8:00 PM EDT to April 4 at 10:00 AM EDT (1:00 AM BST to 3:00 PM BST on April 4). Once the maintenance is complete, we will monitor the PC EU server performance. This maintenance does occur during the Jester’s Festival and is why we preemptively added an extra day to the beginning of the event.

    For the remaining servers, below are the latest planned timelines. Please note that these timelines could change if we run into any issues with the hardware refresh process or other issues arise with the ESO live service that takes priority.
    • PlayStation EU: This server is next, and that work will be completed later this month (April.) We will let you all know when we’ve locked in a date.
    • Xbox NA and PlayStation NA: This work is scheduled for completion sometime in May. We will provide dates when we have them.
    • Xbox EU: Additional work is needed in our Frankfurt datacenter before we can do the Xbox EU hardware refresh work. We recognize that this server coming last is not in-line with what we originally communicated in December. This is the server experiencing the least amount of performance/stability issues. The hardware refresh work for this server is planned for some time later this year. We’ll provide updates as we have them.
    Thank you!

    Thanks for the update, good to finally get one. Can you ask what are the specific performance issues this fixes?
  • React
    React
    ✭✭✭✭✭
    ✭✭✭✭✭
    Destai wrote: »
    ZOS_Kevin wrote: »
    Hi All, we wanted to share with everyone the latest on our ongoing hardware refresh work. The work on PC EU is nearing completion. We are doing the cutover during an extended maintenance that will start in the evening on April 3 and end in the morning on April 4. Tentative times are April 3 at 8:00 PM EDT to April 4 at 10:00 AM EDT (1:00 AM BST to 3:00 PM BST on April 4). Once the maintenance is complete, we will monitor the PC EU server performance. This maintenance does occur during the Jester’s Festival and is why we preemptively added an extra day to the beginning of the event.

    For the remaining servers, below are the latest planned timelines. Please note that these timelines could change if we run into any issues with the hardware refresh process or other issues arise with the ESO live service that takes priority.
    • PlayStation EU: This server is next, and that work will be completed later this month (April.) We will let you all know when we’ve locked in a date.
    • Xbox NA and PlayStation NA: This work is scheduled for completion sometime in May. We will provide dates when we have them.
    • Xbox EU: Additional work is needed in our Frankfurt datacenter before we can do the Xbox EU hardware refresh work. We recognize that this server coming last is not in-line with what we originally communicated in December. This is the server experiencing the least amount of performance/stability issues. The hardware refresh work for this server is planned for some time later this year. We’ll provide updates as we have them.
    Thank you!

    Thanks for the update, good to finally get one. Can you ask what are the specific performance issues this fixes?

    When they replaced the hardware on PC NA, for a short period(3 or 4 months) cyrodiil server performance was perfect. No lag at all in pop locked campaigns with massive sieges happening.

    After about three or four months, the performance suddenly and drastically decreased. Now, almost a year later, the server performance is quite literally back to where it was pre-hardware refresh. Every night when the campaigns pop lock, they become unplayable. It's very odd that the performance increased so drastically and then decreased so drastically. Almost feels like there were additional (necessary) server resources at the start, that were then turned off or reallocated elsewhere in the later part of the year.
    Edited by React on March 31, 2023 3:26PM
    @ReactSlower - PC/NA - 2000+ CP
    React Faster - XB/NA - 1500+ CP
    Content
    Twitch.tv/reactfaster
    Youtube.com/@ReactFaster
  • Zodiarkslayer
    Zodiarkslayer
    ✭✭✭✭✭
    "why we preemptively added an extra day to the beginning of the event."

    So you're anticipating people can't play and earn tickets and your solution is to benefit other servers more than PC EU (since everyone gets the extra day, not just PC EU), rather than just... schedule the maintenance some other time?

    Don't get me wrong, yay for smoother Cyrodiil, but seriously, why are you so determined to consistently schedule maintenance at the worst possible times for your players? This isn't even an event thing, it's just even more ludicrous when it is.

    This is nonsensical.

    @thejadefalcon You are from the UK, right? In the UK the servers go down on 1 a.m. April 4th and are up at 3 p.m. the same day. This should be plenty of opportunity to get all possible tickets.

    The times seem to be deliberately chosen to avoid EU primetime.
    read, think and write.In that order.
  • ApoAlaia
    ApoAlaia
    ✭✭✭✭✭
    ✭✭
    "why we preemptively added an extra day to the beginning of the event."

    So you're anticipating people can't play and earn tickets and your solution is to benefit other servers more than PC EU (since everyone gets the extra day, not just PC EU), rather than just... schedule the maintenance some other time?

    Don't get me wrong, yay for smoother Cyrodiil, but seriously, why are you so determined to consistently schedule maintenance at the worst possible times for your players? This isn't even an event thing, it's just even more ludicrous when it is.

    This is nonsensical.

    @thejadefalcon You are from the UK, right? In the UK the servers go down on 1 a.m. April 4th and are up at 3 p.m. the same day. This should be plenty of opportunity to get all possible tickets.

    The times seem to be deliberately chosen to avoid EU primetime.

    The whole 'thing' with the current event aside (which is already been covered elsewhere so I won't repeat it here) this is what it looks to me.

    They are starting the maintenance at 1AM BST, and they are not doing it during the anniversary event, I might be suffering from Stockholm syndrome but it seems pretty considerate to me?
  • DirtyDeeds765
    DirtyDeeds765
    ✭✭✭✭
    ZOS_Kevin wrote: »
    Hi All, we wanted to share with everyone the latest on our ongoing hardware refresh work. The work on PC EU is nearing completion. We are doing the cutover during an extended maintenance that will start in the evening on April 3 and end in the morning on April 4. Tentative times are April 3 at 8:00 PM EDT to April 4 at 10:00 AM EDT (1:00 AM BST to 3:00 PM BST on April 4). Once the maintenance is complete, we will monitor the PC EU server performance. This maintenance does occur during the Jester’s Festival and is why we preemptively added an extra day to the beginning of the event.

    For the remaining servers, below are the latest planned timelines. Please note that these timelines could change if we run into any issues with the hardware refresh process or other issues arise with the ESO live service that takes priority.
    • PlayStation EU: This server is next, and that work will be completed later this month (April.) We will let you all know when we’ve locked in a date.
    • Xbox NA and PlayStation NA: This work is scheduled for completion sometime in May. We will provide dates when we have them.
    • Xbox EU: Additional work is needed in our Frankfurt datacenter before we can do the Xbox EU hardware refresh work. We recognize that this server coming last is not in-line with what we originally communicated in December. This is the server experiencing the least amount of performance/stability issues. The hardware refresh work for this server is planned for some time later this year. We’ll provide updates as we have them.
    Thank you!

    Great! Happy to see an update for PSNA and glad it is happening soon! I was disappointed we were back to no concrete-as-can-be-date but you guys got us back to one quickly.

    We know you guys don't hear it enough - we do appreciate what you do.

    @DirtyDeeds765
  • Tommy_The_Gun
    Tommy_The_Gun
    ✭✭✭✭✭
    ✭✭✭✭✭
    I think I am gonna leave some quests done, but not completed, so if I will be lucky I will just login & get event ticket ASAP. At least, this is the only countermeasure I can think of.

    It would be ideal if ZOS would communicate internally better so that the team responsible for the in-game events would know in advance when there are some potenial server down-times and plan events in different days.
    Edited by Tommy_The_Gun on March 31, 2023 7:05PM
  • vgabor
    vgabor
    ✭✭✭✭✭
    You guys just moved the weekly timers reset to Tuesday, so how about you stop doing maintenance on Tuesdays? There are plenty enough days on a week which not collide with the reset times, why not pick one of those?

    @ZOS_Kevin
  • ZOS_Kevin
    ZOS_Kevin
    Community Manager
    It would be ideal if ZOS would communicate internally better so that the team responsible for the in-game events would know in advance when there are some potenial server down-times and plan events in different days.

    Wanted to chime in here and provide a little bit of context. The in-game events team is aware and part of the process of the hardware refresh. In fact, many teams need to be on the same page for a maintenance like this to happen. One thing to remember is, there are many teams working together to pull this off. We understand sometimes that isn't the best timing for events and how that is frustrating. As a reminder, that is why we built in the extra day for Jester's, to try and balance that out. However, we also have to take into account the teams involved and where those teams are. The teams have to be available to do the work. And pushing out one of these dates would inevitably impact when other servers get their refresh. The team considers multiple angles when choosing a time for these and work to create the least friction possible, when possible. Hopefully this provides some context for timings, as events are not the only factor the team has to consider when planning out the maintenance here.

    While we know it is frustrating, we to appreciate everyone's patience and understanding for these maintenance windows.
    Community Manager for ZeniMax Online Studio and Elder Scrolls OnlineDev Tracker | Service Alerts | ESO Twitter
    Staff Post
  • React
    React
    ✭✭✭✭✭
    ✭✭✭✭✭
    ZOS_Kevin wrote: »
    It would be ideal if ZOS would communicate internally better so that the team responsible for the in-game events would know in advance when there are some potenial server down-times and plan events in different days.

    Wanted to chime in here and provide a little bit of context. The in-game events team is aware and part of the process of the hardware refresh. In fact, many teams need to be on the same page for a maintenance like this to happen. One thing to remember is, there are many teams working together to pull this off. We understand sometimes that isn't the best timing for events and how that is frustrating. As a reminder, that is why we built in the extra day for Jester's, to try and balance that out. However, we also have to take into account the teams involved and where those teams are. The teams have to be available to do the work. And pushing out one of these dates would inevitably impact when other servers get their refresh. The team considers multiple angles when choosing a time for these and work to create the least friction possible, when possible. Hopefully this provides some context for timings, as events are not the only factor the team has to consider when planning out the maintenance here.

    While we know it is frustrating, we to appreciate everyone's patience and understanding for these maintenance windows.

    I know it likely isn't at the top of the priority list, but are the developers aware of the timeline with PC NA's server performance in the past year? Are there any plans to address this?

    When we got the fresh hardware, cyrodiil ran flawlessly. I don't say that lightly - at the end of week one post server replacement, I had one particular fight where I was in a keep with over 100 players on the player counter, multiple "ball groups" fighting with one another, over 30 siege deployed across the 3 factions, in a pop locked gray host during the prime hours. There was no ability delay or lag at all. This was completely unheard of - the game never had run this well in it's entire life. Everybody I know that PVPs was ecstatic, and I had numerous friends returning to the game.

    Fast forward 3-4 months post server replacement, and suddenly the performance drastically decreases. GrayHost begins to become "unplayable" again each night once it fills, and any time there is a large conflict (particularly between "ball groups"), the entire campaign experienced skill delay. It seemingly just happened out of nowhere.

    Now, one year post hardware replacement, both CP cyrodiil campaigns become extremely laggy as soon as they fill. There is significant skill delay every single night, and it is very close to the level of unbearable performance we had right before the hardware refresh.

    I'm no server engineer. But I do have friends that work in that field, and what they've suggested to me is that it seems like when the servers were first replaced, there were more resources dedicated to cyrodiil allowing for the flawless performance we didn't think was possible. They have implied that the sudden and drastic decrease 3-4 months later would indicate that some of those server resources were reallocated.
    Edited by React on March 31, 2023 10:20PM
    @ReactSlower - PC/NA - 2000+ CP
    React Faster - XB/NA - 1500+ CP
    Content
    Twitch.tv/reactfaster
    Youtube.com/@ReactFaster
  • pecheckler
    pecheckler
    ✭✭✭✭✭
    Is this going to make cyrodiil not constantly lag like crazy on ps5? It's quite frustrating having abilities not trigger nearly all the time.
    End the tedious inventory management game.
  • ZOS_Kevin
    ZOS_Kevin
    Community Manager
    @React The team is monitoring PC NA performance, but that is all we have to share at this time. However, we will make sure to share (and have been sharing) feedback like this as well.
    Community Manager for ZeniMax Online Studio and Elder Scrolls OnlineDev Tracker | Service Alerts | ESO Twitter
    Staff Post
  • React
    React
    ✭✭✭✭✭
    ✭✭✭✭✭
    ZOS_Kevin wrote: »
    @React The team is monitoring PC NA performance, but that is all we have to share at this time. However, we will make sure to share (and have been sharing) feedback like this as well.

    Thanks for the reply.
    @ReactSlower - PC/NA - 2000+ CP
    React Faster - XB/NA - 1500+ CP
    Content
    Twitch.tv/reactfaster
    Youtube.com/@ReactFaster
  • ArchMikem
    ArchMikem
    ✭✭✭✭✭
    ✭✭✭✭✭
    "why we preemptively added an extra day to the beginning of the event."

    So you're anticipating people can't play and earn tickets and your solution is to benefit other servers more than PC EU (since everyone gets the extra day, not just PC EU), rather than just... schedule the maintenance some other time?

    Don't get me wrong, yay for smoother Cyrodiil, but seriously, why are you so determined to consistently schedule maintenance at the worst possible times for your players? This isn't even an event thing, it's just even more ludicrous when it is.

    Need remind you that the planet has different time zones. Maintenance times will ALWAYS inconvenience someone, regardless of when it's scheduled.
    CP1,900+ Master Explorer - AvA One Star General - Console Peasant - The Clan
    Quest Objective: OMG Go Talk To That Kitty!
  • Destai
    Destai
    ✭✭✭✭✭
    ✭✭
    ZOS_Kevin wrote: »
    It would be ideal if ZOS would communicate internally better so that the team responsible for the in-game events would know in advance when there are some potenial server down-times and plan events in different days.

    Wanted to chime in here and provide a little bit of context. The in-game events team is aware and part of the process of the hardware refresh. In fact, many teams need to be on the same page for a maintenance like this to happen. One thing to remember is, there are many teams working together to pull this off. We understand sometimes that isn't the best timing for events and how that is frustrating. As a reminder, that is why we built in the extra day for Jester's, to try and balance that out. However, we also have to take into account the teams involved and where those teams are. The teams have to be available to do the work. And pushing out one of these dates would inevitably impact when other servers get their refresh. The team considers multiple angles when choosing a time for these and work to create the least friction possible, when possible. Hopefully this provides some context for timings, as events are not the only factor the team has to consider when planning out the maintenance here.

    While we know it is frustrating, we to appreciate everyone's patience and understanding for these maintenance windows.

    Fantastic reply, Kevin. Context like this is really appreciated! Thanks man.
  • vsrs_au
    vsrs_au
    ✭✭✭✭✭
    ✭✭
    Destai wrote: »
    ZOS_Kevin wrote: »
    It would be ideal if ZOS would communicate internally better so that the team responsible for the in-game events would know in advance when there are some potenial server down-times and plan events in different days.

    Wanted to chime in here and provide a little bit of context. The in-game events team is aware and part of the process of the hardware refresh. In fact, many teams need to be on the same page for a maintenance like this to happen. One thing to remember is, there are many teams working together to pull this off. We understand sometimes that isn't the best timing for events and how that is frustrating. As a reminder, that is why we built in the extra day for Jester's, to try and balance that out. However, we also have to take into account the teams involved and where those teams are. The teams have to be available to do the work. And pushing out one of these dates would inevitably impact when other servers get their refresh. The team considers multiple angles when choosing a time for these and work to create the least friction possible, when possible. Hopefully this provides some context for timings, as events are not the only factor the team has to consider when planning out the maintenance here.

    While we know it is frustrating, we to appreciate everyone's patience and understanding for these maintenance windows.

    Fantastic reply, Kevin. Context like this is really appreciated! Thanks man.
    I wholeheartedly endorse that comment, and @ZOS_Kevin you get an 'awesome' from me for every post in this thread. They're much appreciated.
    PC(Steam) / EU / play from Melbourne, Australia / avg ping 390
  • karthrag_inak
    karthrag_inak
    ✭✭✭✭✭
    ✭✭
    Khajiit is certainly impressed with the performance he's been having this evening. Lagged out to launch screen 3 times.
    PC-NA : 19 Khajiit and 1 Fishy-cat with fluffy delusions
  • LadyLethalla
    LadyLethalla
    ✭✭✭✭✭
    @ZOS_Kevin Please please please no maintenance on April 5th... it's my birthday...!
    x-TallyCat-x // PC EU DC - For the Covenant! // ESO Platinum trophy - 16th May 2017Proud member of the Aetherium Alliance.Melbourne Australia - the land of Potato Internet.WTB ESO OCEANIC SERVER
  • Sarannah
    Sarannah
    ✭✭✭✭✭
    ✭✭✭
    Awesome!!! And thanks for adding an extra day of jester's festival in advance, for those who might miss out on a day of tickets.

    Would love it if we could get an update on the gameplay implications of the server hardware refresh/upgrade. Like will this allow for more character slots per server, more inventory space, more bank storage space, higher furniture limits in houses, etc? Does this refresh/upgrade have any other gameplay implications?
    The only thing we hear about this is that the hardware refresh/upgrade should make the game/Cyrodiil run smoother, but any other possible implications have never been officially addressed.
  • Northwold
    Northwold
    ✭✭✭✭✭
    vgabor wrote: »
    You guys just moved the weekly timers reset to Tuesday, so how about you stop doing maintenance on Tuesdays? There are plenty enough days on a week which not collide with the reset times, why not pick one of those?

    @ZOS_Kevin

    I'm not understanding why this is significant? You might equally complain if it was Monday that that leaves you a day less to meet the reset timers, or if it was Wednesday that that means you've lost a day from the timers. There is no eighth day to do maintenance on that will suit everyone's personal niggles.
  • Lebkuchen
    Lebkuchen
    ✭✭✭✭✭
    @React I hope one day you will get at least one real answer to one of your questions.
Sign In or Register to comment.