7 years gone in a moment

Alurria

Well @Yormula I sincerely am hoping for a good resolution to this problem for you. I wouldn't wish this on any other gamer. Thinking of all the time you put into your character. Personally I am attached to my character after all these years playing and I know how I would feel. I will continue to hope that any goodwill that Zos offers you also spreads throughout the player base and make those like myself who view this thread feel better about how Zos customer service treats the player base. The win win is if you are truly helped to your satisfaction and the rest of us know it. I have been in management customer service for over 35 years so.. I do know this a little kindness, understanding and empathy goes a long ways to building a stellar reputation. GL @Yormula I'm rooting for you

JoeCapricorn

SilverBride wrote: »

Yormula wrote: »

SilverBride wrote: »

... The story just doesn't add up.

Thank you for the insistent and continuous vote of confidence. As you may have noticed, other threads with issues based purely on disciplinary actions got that fact mentioned very quickly (and closed).

I am not trying to discredit you, but think there are too many unknowns to know exactly what is going on. Everyone is so quick to jump on the "Let's bash ZoS" bandwagon but there were still unanswered questions.

I'd like to add that ZoS did contact you on 6/14/21 as you've shown, and aren't just ignoring this issue as so many have accused them of.

Any kind of discussion of specific disciplinary actions would get a thread locked, even if the OP didn't mention that it was a disciplinary action, so it is safe to assume that is not the case here.

I think what instead is going on is something of a perfect storm of a bunch of issues. 9 months ago the world was very much in the midst of a raging pandemic, ZOS was working from home. This change in environment is partly to explain why all updates since Greymoor have been less smooth. Then you have an issue where a data-set is corrupted, which is not an easy fix especially for standard support. This might be something that requires a programmer's expertise.

If that is the case, it isn't so simple as asking a Data Systems Programmer to look at the corrupted file and <hAcKeR tYpE> on the keyboard in five minutes to fix it. They might also be working on future updates, such as Waking Flame and Deadlands, as well as 2022's year of content, potential new systems, etc., so peeling off from all of that work to help 1 customer is a fairly big ask.

With that said, this thread has gotten quite the attention, and it would be in the community's best interest if Yormula is helped to a satisfactory resolution. The solution might not be simple, but if one can be found it should be implemented.

I would say if there was a cold storage back-up or older version of Yormula's character somewhere, that it could be used to rebuild his save file. If I had something go this terribly wrong, I'd rather start over from last November instead of 2014 (although, let's be honest, I'd much rather nothing go terribly wrong so I'm going to go ahead and knock on wood until my knuckles are red).

GenjiraX

SilverBride wrote: »

there were still unanswered questions.

Though the rest of your post was demonstrably false, you got this bit correct. Those questions were and are:

How was this allowed to happen?

Why has Yormula’s account and character data not been restored from a valid backup?

What measures have ZOS put in place to ensure this will not and cannot happen to any other players?

If ZOS can’t protect our data, they’re taking us for fools.

Over to you, ZOS...

ApostateHobo

Good lord still no resolution to this? I wish I could say I'm surprised, but I'm sadly not. I hope they eventually get this fixed for you because this seriously sucks.

Galadan

What kind of database are not editable, what kind of IT system has no logs and backups, and dev support cannot restore missing data for one single customer with this logs?

I understand it's not a trivial task and could took a weeks of work, but data management is one of f fundamentals of every large IT infrastructure. If we lost a client data in our company, the whole IT department would be handcuffed to a computers till find a solution or manually restore it.

I don't beleive such strong and large team as ZOS could not do anything about this. I rather suppose it's a question of their priorities and motivation, and it's a sad thing.

ZOS, guys, please, don't leave your loyal customer, investigate such incident and show that you care.

@Yormula i truly hope that this situation will be solved for you, man. As many here, ill be wathing this thread and wait.

ApoAlaia

Personally I would not mind if the ESO+ sub would go up in order to cover the financial burden of having readily available database backups and the protocols in place to perform a safe restore in case something goes awry.

Not just for ESO+ subscribers mind you, for players in general.

Of course this is not without its flaws; some may argue that is an exploitable avenue, others that shouldering the financial burden is unfair, however the bottom line is that running without a recovery solution as we can see with the case at hand poses a significant risk.

I know that if I were in OP's shoes no amount of dismissive 'is just a game, get a life' or 'you should not have gotten invested then' hand waving would diminish my mental anguish and frustration in any way.

FlamingMeat

ApoAlaia wrote: »

Personally I would not mind if the ESO+ sub would go up in order to cover the financial burden of having readily available database backups and the protocols in place to perform a safe restore in case something goes awry.

This should already be a thing. It is even in games running on engines as old as OSRS. Backups and logs reaching back years (see the compensation to the mod Jed fiasco, tracking hacked accounts from over a year prior to the scandal and compensating them). POE has really good backups/logs and has been implemented for me even recently for a glitch that occurred while I wasn't playing 2 years ago that wiped/updated some legacy items.

What I'm getting at is there are no reasons I can think of that this shouldn't already be a thing. It takes a miniscule amount of data to store.

Pauwer

Hope this gets resolved for you

Guess what a heart attack it is when the game crashes so bad on console that there is a "data save file corrupted" message and you have to choose to overwrite it and then you load back into game and it plays the very first intro video to game and then takes you to first character creation screen? That is always such fun.

LordArconSeptim

Dude it's not fair if you get unlocked thread with hundreds of reply that you think you can get unban or reset whatever it is...there are so many who lost their accounts because of rival guilds mass reports and similar things of hacks and simulation...

Suna_Ye_Sunnabe

Why has there been no word on this from zos yet?!

Odovacar

Still hoping something can be resolved, soon. #fingerscrossed

MasterSpatula

Just got caught up on the thread, and I remain:

1. Depressingly unsurprised this happened.
2. Impressed by OPs level of calm and rationality throughout the thread.
3. Astonished by ZOS's unwillingness to just come into this thread and either assure us that this can't happen to us or at least reassure us that they're working on making sure it can't happen to us.

SilverBride

ZoS contacted the OP on 6/14, and he posted a screenshot of the message. They also are not allowed to discuss another player's specific issues publicly.

spartaxoxo

SilverBride wrote: »

ZoS contacted the OP on 6/14, and he posted a screenshot of the message. They also are not allowed to discuss another player's specific issues publicly.

They are allowed. This is not a disciplinary action nor is it about personally identifiable information such as his real name.

GenjiraX

SilverBride wrote: »

ZoS contacted the OP on 6/14, and he posted a screenshot of the message. They also are not allowed to discuss another player's specific issues publicly.

Yes, everyone saw that. Nevertheless -

MasterSpatula wrote: »

Astonished by ZOS's unwillingness to just come into this thread and either assure us that this can't happen to us or at least reassure us that they're working on making sure it can't happen to us.

What’s your point, @SilverBride ? You’re responding to a statement nobody made.

SilverBride

My point is that they have contacted the OP. They are not just ignoring him. And ZoS cannot reassure that nothing will ever happen to anyone ever again because unforseen things sometimes just happen.

That being said, they can put out a better effort to make amends.

MasterSpatula

SilverBride wrote: »

And ZoS cannot reassure that nothing will ever happen to anyone ever again because unforseen things sometimes just happen.

That's a misrepresentation of what's being asked for. This situation reoccurring is no longer an "unforeseen event."

This situation only occurred because of inadequate or failed backup procedures. If it's inadequacy, they can implement more robust backup procedures. If not, they can investigate what went wrong in this circumstance and correct the failure. Either way, they can let us know.

None of these are too much to ask for, In fact, they shouldn't have to be asked for. The silence feels disrespectful.

Chips_Ahoy

I do not understand the need to divert attention from the main issue with absurd conspiracies, the only thing that will be achieved is that they close the thread.

I hope they find a solution.

luckily my experience with technical support is amazing.

Freeflyer212

Chips_Ahoy wrote: »

I do not understand the need to divert attention from the main issue with absurd conspiracies, the only thing that will be achieved is that they close the thread.

I hope they find a solution.

luckily my experience with technical support is amazing.

Is it me or is it a conspiracy my brother? (joke from an old tv show)

Alinhbo_Tyaka

SilverBride wrote: »

And ZoS cannot reassure that nothing will ever happen to anyone ever again because unforseen things sometimes just happen.

Based on what can be seen in this thread yes ZOS can reassure that this would not happen again. The issue appears to be ZOS attempting to correct the OPs problem and instead breaking the character beyond repair. There are multiple references to the broken part being a save file. If it is truly a file then standard procedures in any basically competent IT shop would be to back up the file and also make a copy. Any modifications would be made to the copy which when complete and verified would be used to replace the broken original. If things went to pieces the backup could be restored leaving everything in its original state. If the save file is really an entry/entries or row/rows in a database best practices would be to make a copy of the tables in a test database, make the changes and verify them. If they were ok then a save point in the live database and the update made. In the event something went wrong updating the live database a rollback could readily be performed because of the previous save point. The key is standard procedures always allow the data being updated to be restored to its original state in the event something goes wrong. Based on the information we have available it is the lack of backup and ability to restore the character to its original state that are the issue not that the attempt to fix the problem broke things further.

Rukia541

This has to be the biggest gaming yikes of all time, RIP bro. I wonder if this would be fixed if they had just made achievements account wide, as it seems some timey wimey crap with the character specifically or they just don't back up data properly. I know in WoW you can straight up delete a character and get it back again, doesn't matter how long afterwards.

starkerealm

Galadan wrote: »

What kind of database are not editable...

A surprising number, actually.

At least, not, user editable. This is especially true when you consider the perspective. The end user can't edit their database entry, obviously, but even the Customer Support team has limited access to the database, and limited opportunity to manipulate it, and this entire incident should illustrate why you restrict database access.

A customer support rep did get in there and modify the database, and the end result was that they destroyed the character.

Galadan wrote: »

...what kind of IT system has no logs and backups, and dev support cannot restore missing data for one single customer with this logs?

Who keeps comprehensive logs going back over 9 months on every record update? Keeping in mind that there are millions of records, and each active record receives thousands to tens of thousands of updates each day?

Where do you store these logs?

Do you even bother to log everything? (And, the answer to this one, in this case, is, "no.")

If the user's account was corrupted entirely, it's likely they could restore the entire account from a backup (or at least clone a new one from a backup), but it would mean wiping away the last 9 months of progress on the account. Something few people would be willing to accept.

And, no, it's not really that weird that the character records can't be easily edited separately from the account record. You do see stuff like this in databases, where it's not as simple as simply plugging in a new sub-record and you're good to go. Especially given that there have been issues in the past with players seeing another player's characters on their account during login. I don't have any real insight to the database architecture, but it wouldn't surprise me if you can't simply revise a character, without having to tweak a lot of other fields along the way.

Galadan wrote: »

I understand it's not a trivial task and could took a weeks of work, but data management is one of f fundamentals of every large IT infrastructure. If we lost a client data in our company, the whole IT department would be handcuffed to a computers till find a solution or manually restore it.

I remember while working for FedEx, I pulled a package off a truck. The thing weighed nearly 200lbs. The belt manager was annoyed. I was annoyed. Ground doesn't move anything heavier than 150. This thing should have been sent freight. I suggested, "maybe we should just call freight, and have them send someone over to pick it up." My superior's response was, "but, who's going to pay for that?"

Your company has a valuable contract.

@Yormula is paying 15 bucks a month (at most.) Your average coder at ZOS is making roughly 120k a year. So, if you took a programmer from the database team, told them to reconstruct this account, and it took them two weeks, that's a cost of $2500. Now, it's possible Yormula has spent more than that on ESO, but it's extremely likely that, even over seven years, they have not. Meaning reconstruction would be more expensive than @Yormula is worth as a customer. (No offense, man.)

If you're looking at a multi-million dollar contract, tying up a team for a couple weeks is still worth it, because you're throwing 20-30k at saving a contract worth 100 times that. But, that's not relevant here, because one individual customer isn't worth that much.

Galadan wrote: »

I don't beleive such strong and large team as ZOS could not do anything about this. I rather suppose it's a question of their priorities and motivation, and it's a sad thing.

ZOS, guys, please, don't leave your loyal customer, investigate such incident and show that you care.

@Yormula i truly hope that this situation will be solved for you, man. As many here, ill be wathing this thread and wait.

amm7sb14_ESO

Rukia541 wrote: »

This has to be the biggest gaming yikes of all time, RIP bro. I wonder if this would be fixed if they had just made achievements account wide, as it seems some timey wimey crap with the character specifically or they just don't back up data properly. I know in WoW you can straight up delete a character and get it back again, doesn't matter how long afterwards.

They can here too. When I came back in December 2019, I deleted what was my original launch character, because I wanted to start over (I was only level 25). A couple days later, I changed my mind and decided I wanted to keep my original character, so I petitioned ZOS, and they gave him back to me. I didn't have any gear upon getting him back, but they were able to restore my character.

dem0n1k

@Yormula is paying 15 bucks a month (at most.) Your average coder at ZOS is making roughly 120k a year. So, if you took a programmer from the database team, told them to reconstruct this account, and it took them two weeks, that's a cost of $2500. Now, it's possible Yormula has spent more than that on ESO, but it's extremely likely that, even over seven years, they have not. Meaning reconstruction would be more expensive than @Yormula is worth as a customer. (No offense, man.)

I see what you are saying & it's true that any company must do a cost analysis for any job to see if it's worth doing. However, at this stage, it's not just Yormula's character that is at stake. It is now a pretty serious PR issue if the player base loses confidence in the company's ability to have data integrity. Perhaps that customer confidence is worth justifying the work required... I hope so for all player's sakes.

GenjiraX

dem0n1k wrote: »

@Yormula is paying 15 bucks a month (at most.) Your average coder at ZOS is making roughly 120k a year. So, if you took a programmer from the database team, told them to reconstruct this account, and it took them two weeks, that's a cost of $2500. Now, it's possible Yormula has spent more than that on ESO, but it's extremely likely that, even over seven years, they have not. Meaning reconstruction would be more expensive than @Yormula is worth as a customer. (No offense, man.)

I see what you are saying & it's true that any company must do a cost analysis for any job to see if it's worth doing. However, at this stage, it's not just Yormula's character that is at stake. It is now a pretty serious PR issue if the player base loses confidence in the company's ability to have data integrity. Perhaps that customer confidence is worth justifying the work required... I hope so for all player's sakes.

The impact to ZOS’s (not to mention Microsoft’s) reputation if they continue to fail to address the wider situation here, on the ESO forum, where they have control, and it breaks out to their social media accounts could be significant.

They must have considered this, which makes their continued silence the more bizarre.

starkerealm

amm7sb14_ESO wrote: »

Rukia541 wrote: »

This has to be the biggest gaming yikes of all time, RIP bro. I wonder if this would be fixed if they had just made achievements account wide, as it seems some timey wimey crap with the character specifically or they just don't back up data properly. I know in WoW you can straight up delete a character and get it back again, doesn't matter how long afterwards.

They can here too. When I came back in December 2019, I deleted what was my original launch character, because I wanted to start over (I was only level 25). A couple days later, I changed my mind and decided I wanted to keep my original character, so I petitioned ZOS, and they gave him back to me. I didn't have any gear upon getting him back, but they were able to restore my character.

And, that's a potential option, ask customer service to roll that character back more than 9 months, to before the original error occurred. There is a sticky problem here where we don't know what the options @Yormula has been offered. We're also not sure what options are really available.

So, this is speculation, but we can reasonably guess that the character's record was corrupted when customer support deleted the achievement. From personal experience with databases, a corrupted record can spread to other parts of the database. It is possible, though we don't know for sure, that deleting the character could cause that corruption to "migrate," to the next character record. Meaning, you'd have a (slow) cascade, as you deleted and restored characters. Given some elements of the corruption affected the account as a whole, it's possible that the account data itself is partially corrupted, meaning the entire account would need to be salvaged.

We don't know how frequent database backups are (especially older ones) as it's now been 10 months, and we don't know how having everyone working from home may have contributed to this.

Like, I get that it's a frustrating situation, but, we don't know what solutions are viable and which aren't. Again, if @Yormula has the option, a rollback may be the best they can hope for. If that is, in fact, an option.

starkerealm

dem0n1k wrote: »

@Yormula is paying 15 bucks a month (at most.) Your average coder at ZOS is making roughly 120k a year. So, if you took a programmer from the database team, told them to reconstruct this account, and it took them two weeks, that's a cost of $2500. Now, it's possible Yormula has spent more than that on ESO, but it's extremely likely that, even over seven years, they have not. Meaning reconstruction would be more expensive than @Yormula is worth as a customer. (No offense, man.)

I see what you are saying & it's true that any company must do a cost analysis for any job to see if it's worth doing. However, at this stage, it's not just Yormula's character that is at stake. It is now a pretty serious PR issue if the player base loses confidence in the company's ability to have data integrity. Perhaps that customer confidence is worth justifying the work required... I hope so for all player's sakes.

It's not a serious PR issue.

I'm sorry, like, I get what you're thinking, where you're coming from, and you're not wrong to have that approach, the problem is, nobody will remember.

Outriders had an issue where players were losing their inventory (and in some cases) their characters. That was a serious PR issue because it affected a significant number of players.

You may already know about this because it was covered in games media, and by influencers. It was widesperad enough that people could see this happening, and there were many affected accounts.

Yormula is one person. And, I say this with respect for them, they're pretty a pretty minor figure in the community. It's not a "major," PR issue. You aren't reading about Yormula's character being corrupted on IGN, Polygon, or Kotaku, you're seeing it here, in the forums.

This pales in comparison to the storage chest bug from when those were first implemented, that was more widespread, and still didn't get attention.

So, unfortunately, this is not really a PR issue. It's not news, and as a result, no PR needed.

It also appears to be a one-off event. We're not seeing reports of this popping up elsewhere. So, while I hope Yormula can get something worked out, I don't think there's much we can do, and just riling ourselves up on their behalf isn't particularly constructive.

GenjiraX

starkerealm wrote: »

dem0n1k wrote: »

@Yormula is paying 15 bucks a month (at most.) Your average coder at ZOS is making roughly 120k a year. So, if you took a programmer from the database team, told them to reconstruct this account, and it took them two weeks, that's a cost of $2500. Now, it's possible Yormula has spent more than that on ESO, but it's extremely likely that, even over seven years, they have not. Meaning reconstruction would be more expensive than @Yormula is worth as a customer. (No offense, man.)

I see what you are saying & it's true that any company must do a cost analysis for any job to see if it's worth doing. However, at this stage, it's not just Yormula's character that is at stake. It is now a pretty serious PR issue if the player base loses confidence in the company's ability to have data integrity. Perhaps that customer confidence is worth justifying the work required... I hope so for all player's sakes.

Yormula is one person....

It also appears to be a one-off event. We're not seeing reports of this popping up elsewhere. So, while I hope Yormula can get something worked out, I don't think there's much we can do, and just riling ourselves up on their behalf isn't particularly constructive.

Soulshine wrote: »

This is not as rare as it sounds, which is really concerning. Your are the third person now I have seen with an issue like this, reported in a year's time.

The other two are former guild-mates of former guilds. One lost his original NB character in pretty much the manner OP described. They never recouped his char and just gave him gold and a new char. The second person had his original DK char just mysteriously vanish from server, no explanation ever as to why. Also told here's some gold and move on. The first person is still playing afaik, the second one is no longer playing ESO having left the game fairly upset over the whole situation. Says he's not coming back.

It does give one pause for sure, and when you are someone with a very old account especially, since we do have a lot invested at this point. Those of us here since beta have achievements and items that simply are not possible to recover if we lose our chars from server. Really awful that there is so little in place to address this kind of issue.

“This is not as rare as it sounds...”

If it it can happen to one, it can happen to any of us. That’s what some of us are riled up about.

RevJJ

Yesterday, my game crashed three times in a row when I tried to log into a specific character. Tried other characters, no problem. But this one, my main, made the game crash three times in a row.

Because of this thread, my first thought was “if I need to raise a ticket for this there is a chance they completely mess up my character. I’d rather quit the game than go through that whole process”.

The fact that this can happen sucks, but the way ZOS is dealing with it is the real issue. I love this game but if something like this happened to me I would be out and it seems more people feel that way. Do better ZOS, if you want to keep your customers.

starkerealm

GenjiraX wrote: »

“This is not as rare as it sounds...”

If it it can happen to one, it can happen to any of us. That’s what some of us are riled up about.

That's actually the problem, it is as rare as it sounds. Bonus points in that we don't know, for a fact, that it's the same issue.

You're correct that it could, hypothetically, happen to anyone, but it's rare enough that you don't hear about this frequently. We've got one first hand example, but after that, it's "a friend of my friend's former roommate's dog sitter said..."

We've got three characters that are locked to prevent login. Right? Well, do we? No, we don't. We know that one of Soulshine's examples was a character just flat out vanishing. Which could be as simple as the ex-guildie's mother yanking them away from the keyboard, and then their younger brother deleting said character. It wouldn't be the first time that something like that happened.

We've got one that was locked, "pretty much the manner OP described," so, does that mean they had an achievement fail to fire, contacted customer support, and had their character bjorked, leading to customer support locking the character?

If that's true, then it supports the idea that someone in Customer Support was removing achievements when they really shouldn't have, and, in the process, breaking characters.

Which, kinda suggests, if you have a problem with an achievement breaking... don't contact customer support during last year's pandemic.

This is the problem, it's not widespread. In this thread, we've had, maybe, three examples, and one of those is definitely something else entirely.