Items worth 26m+ missing - ESO support refused to clear things up

Amottica

Looks like this is getting resolved, as in the person affected will be able to find out what seems to have happened and why Zenimax did not rectify the situation. Well, after they make few more posts.

I do not know if this is the case, but I will still suggest players make sure they use a good password for their game and that it is different than what is used for the connected email. A lot of people use very lame passwords and often use the same password for everything. That makes it easy for crooks to take it all and mess up people's lives.

nightstrike

Zinpix wrote: »

Thanks for bringing this to the forums Banshee!

I just got access to the forums and will try to contact Kevin.

The only thing which kept me in the game so far is the friends Ive made in game, my guild(s) and progress group.
I am quite tired, disappointed and disillusioned at the moment, but the support here helped me feel a bit better.

//Zinpix

Just FYI, support can see the logs for every inventory change, and they can tell you exactly what happened. However, your initial 5 or 6 responses will be either automatically generated, or someone saying "I can't do that". You have to keep replying to them until you get assigned to a support person who feels like helping. It sometimes helps to just keep replying with "please escalate this" or some such thing.

I've submitted a lot of tickets over the years. Things almost always get resolved, but it takes an alarming amount of work, perseverance, hundreds of emails begging for someone to actually look into your issue, and frankly sometimes calling and leaving a ton of voicemails. Eventually, you'll find someone nice, though. And when you do, that person will instantly solve the problem, because it's absolutely trivial for them to 1) look at the logs, 2) explain what happened, and 3) restore items if it's warranted. Sadly, most just don't want to do those things, so they lie and say it's impossible.

EDIT: I forgot to add, when they close your ticket without fixing it, it does usually help to fill out the survey and complain that they closed it without fixing it. That tends to alert a higher level person (sometimes) who looks into it and fixes it.

Pixiepumpkin

ZOS_Kevin wrote: »

Hi BansheeVT. Thanks for reaching out on behalf of the player. As mentioned in our guidelines and also noted in this thread, we do not discuss tickets and actions outside of the player involved.

If the player involved would like to reach out to me via DM, I can let them know what Customer Service noted, take what issues they may have and how they should proceed.

why not, playres have a right to know when things are not working especially if u dont help the player

SeaGtGruff

nightstrike wrote: »

Zinpix wrote: »

Thanks for bringing this to the forums Banshee!

I just got access to the forums and will try to contact Kevin.

The only thing which kept me in the game so far is the friends Ive made in game, my guild(s) and progress group.
I am quite tired, disappointed and disillusioned at the moment, but the support here helped me feel a bit better.

//Zinpix

Just FYI, support can see the logs for every inventory change, and they can tell you exactly what happened. However, your initial 5 or 6 responses will be either automatically generated, or someone saying "I can't do that". You have to keep replying to them until you get assigned to a support person who feels like helping. It sometimes helps to just keep replying with "please escalate this" or some such thing.

I've submitted a lot of tickets over the years. Things almost always get resolved, but it takes an alarming amount of work, perseverance, hundreds of emails begging for someone to actually look into your issue, and frankly sometimes calling and leaving a ton of voicemails. Eventually, you'll find someone nice, though. And when you do, that person will instantly solve the problem, because it's absolutely trivial for them to 1) look at the logs, 2) explain what happened, and 3) restore items if it's warranted. Sadly, most just don't want to do those things, so they lie and say it's impossible.

EDIT: I forgot to add, when they close your ticket without fixing it, it does usually help to fill out the survey and complain that they closed it without fixing it. That tends to alert a higher level person (sometimes) who looks into it and fixes it.

If you've ever worked in any kind of customer support in an organization, you might know that an organization will typically have rules and procedures in place for different levels of employees. You shouldn't assume that every single support rep is even authorized to do whatever it is you think they should be doing.

LadyLethalla

Years ago I lost a bunch of purple mats and was refused support. ZOS' TOS state they've no obligation to help... I understand however, that one would be 'rather miffed' (to put it mildly) at such a loss as OP's friend.

nightstrike

SeaGtGruff wrote: »

nightstrike wrote: »

Eventually, you'll find someone nice, though. And when you do, that person will instantly solve the problem, because it's absolutely trivial for them to 1) look at the logs, 2) explain what happened, and 3) restore items if it's warranted. Sadly, most just don't want to do those things, so they lie and say it's impossible.

If you've ever worked in any kind of customer support in an organization, you might know that an organization will typically have rules and procedures in place for different levels of employees. You shouldn't assume that every single support rep is even authorized to do whatever it is you think they should be doing.

I should assume that support reps will say "I have to escalate this" instead of "that's impossible". They never do. It's not just an issue of training people to know what fixes need a supervisor to approve. The first tier people are either taught or otherwise encouraged to reject as much help as possible.

Please remember that some of us, myself included, have spent thousands of dollars on this game. For that amount, I personally expect better support. If I paid ten bucks one time 5 years ago, then fine, scrape the bottom of the barrel. But I pay a lot more, and I therefore expect a lot more.

And yes, I have plenty of experience in this area. Different companies have different approaches with different end user quality.

ZOS_Kevin

@Zinpix, no worries. I just DM'ed you.

ZOS_Kevin

Pixiepumpkin wrote: »

why not, playres have a right to know when things are not working especially if u dont help the player

Yes, of course players have the right to know if something is not working. However, right now with the information that we have, there is nothing to suggest that this is an issue outside of the specific player. This is one reason why we do not discuss tickets or actions outside of those involved.

We would rather be certain when letting players know there is a wider issue instead of causing unnecessary panic for players if this is an isolated issue. If we identify an issue that impacts the wider playerbase, we will certainly reach out and make sure players know.

Chips_Ahoy

ZOS_Kevin wrote: »

If we identify an issue that impacts the wider playerbase, we will certainly reach out and make sure players know.

"Sorry, it isn't a crown item, we can't restore it.".

If this is true, it is an issue that impacts the wider playerbase.

Onigar

Read all the comments here in this post makes and me think if something disappeared from my char bags, craft bag, bank or house storage I really would not know. Having played for a long time I have plenty of stock of pretty much everything so I just use without looking to see what's there.

Its not like there is anything physical here, all the items are electronic and part of one or more databases.

Databases require regular maintenance and like any other software can have problems.

When something untoward is noticed by a player ZOS support really need a good idea of the date & time so they can target the logs.

Perhaps an inventory transaction tracker addon or something like this would help which just logs to an external text file char login+out, before and after transaction stock levels as well being able to do a consistency check separately for char bags, craft bag, bank or house storage.

Until there is evidence of a larger problem it is probably hard for ZOS to justify Dev time to properly investigate when there is already so much on their work list.

nightstrike

That's not a bad idea. There have been plenty of inventory bugs so far that have resulted in lost items, so there is a precedent for wanting extra verification.

Pinesy

Anyone else remember when storage boxes for homes came out? They were immediately buggy, and I lost _so much_ gear. I was so flabbergasted that I didn't even try to get any of it back.

Xebov

Onigar wrote: »

Read all the comments here in this post makes and me think if something disappeared from my char bags, craft bag, bank or house storage I really would not know. Having played for a long time I have plenty of stock of pretty much everything so I just use without looking to see what's there.

Its not like there is anything physical here, all the items are electronic and part of one or more databases.

Databases require regular maintenance and like any other software can have problems.

When something untoward is noticed by a player ZOS support really need a good idea of the date & time so they can target the logs.

Perhaps an inventory transaction tracker addon or something like this would help which just logs to an external text file char login+out, before and after transaction stock levels as well being able to do a consistency check separately for char bags, craft bag, bank or house storage.

Until there is evidence of a larger problem it is probably hard for ZOS to justify Dev time to properly investigate when there is already so much on their work list.

I would not be suprised if Addons account for some of the item losses. They can be very powerful and some (like Rule Based Inventory) require alot of configuration and if using them wrong they could easily destroy items. Given how databases are working having random bugs for loosing entries only on specific players seems to be unlikely.

trailofundead

I've had exactly the same problem, though different materials. All of the mats I use for writs disappeared bewteen the morning and afternoon. That's all my rubedite ore, ancestor silk, sanded ruby ash, rubedite leather and scraps, and platinum ounces. All set to zero when I know I had plenty. Maybe other mats too but impossible to check.

It had happened twice before but just with platinum ounces, so I was keeping a close eye on them - I had about 2200 before they disappeared.

I wondered whether it's an exploit through an addon, it was the first time in a while I had sold directly from my craft bag though I don't know whether the previous losses also timed with that.

I submitted a ticket but I never expect a response even when it's not the holidays. I will keep reminding support though, especially if it's happening to other people. I'll also be keeping a very close eye on the amounts of my valuable and commonly used mats.

Thanks for posting, and to the guildie who flagged this post to me.

trailofundead

Well I thought I'd submitted a ticket - I sumbitted one in game and then a second via the website but neither appear in t'my support'. No wonder nothing gets done when even submitting a ticket doesn't work.

nightstrike

Xebov wrote: »

Given how databases are working having random bugs for loosing entries only on specific players seems to be unlikely.

It's less that it's specific players and more perhaps a specific sequence of actions that only specific players are triggering that isn't part of QA testing. This may be timing related, or perhaps a difficult race condition to trigger.

For instance, perhaps you have to be clicking a particular button at the same time as an asynchronous server side background task runs, and the two actions are not protected with a mutex. Something like that. It's easier than you think to create hard to find and special case bugs. I troubleshoot race conditions like this all day long.

wolfie1.0.

nightstrike wrote: »

Xebov wrote: »

Given how databases are working having random bugs for loosing entries only on specific players seems to be unlikely.

It's less that it's specific players and more perhaps a specific sequence of actions that only specific players are triggering that isn't part of QA testing. This may be timing related, or perhaps a difficult race condition to trigger.

For instance, perhaps you have to be clicking a particular button at the same time as an asynchronous server side background task runs, and the two actions are not protected with a mutex. Something like that. It's easier than you think to create hard to find and special case bugs. I troubleshoot race conditions like this all day long.

I have definitely seen some crazy bugs that have occurred at work that only trigger if you are doing x>y>z while also doing a>b>c. They can be crazy isolated, specific and rare.

BalticBlues

The core of the problem is: WE PLAYERS CANNOT DO ANYTHING.

At home, we can do a BACKUP of our personal data.
So if a hard drive dies, we can restore our backup on a new drive.
In ESO, we players cannot do a backup of our personal data.
So we players have to rely on ZOS for a backup of the server data.

If things go sideways, especially after YEARS of investing in ESO:
1. Players need to be sure that ZOS can restore a backup.
2. Players need to be given a reason if ZOS cannot restore a backup.

So what in this case is preventing ZOS from giving the player
at least THE REASON why they cannot restore his data from backup?

I can think of several reasons, like

• somehow there was a server failure and no backup
• somehow ZOS does now want to restore the backup

IMHO at least the customer deserves to know THE HONEST REASON behind it all.

Perhaps the LOG shows that things were not stolen but destroyed? But why should he do this? Perhaps there was an unhappy lover who started his PS/CONSOLE up to delete all things? We do not know - but life can be nasty. Anyway, if something like this will ever happen to us, we can only pray that ZOS is able and willing to help.

AlnilamE

BalticBlues wrote: »

The core of the problem is: WE PLAYERS CANNOT DO ANYTHING.

At home, we can do a BACKUP of our personal data.
So if a hard drive dies, we can restore our backup on a new drive.
In ESO, we players cannot do a backup of our personal data.
So we players have to rely on ZOS for a backup of the server data.

If things go sideways, especially after YEARS of investing in ESO:
1. Players need to be sure that ZOS can restore a backup.
2. Players need to be given a reason if ZOS cannot restore a backup.

So what in this case is preventing ZOS from giving the player
at least THE REASON why they cannot restore his data from backup?

I can think of several reasons, like

• somehow there was a server failure and no backup
• somehow ZOS does now want to restore the backup

IMHO at least the customer deserves to know THE HONEST REASON behind it all.

Perhaps the LOG shows that things were not stolen but destroyed? But why should he do this? Perhaps there was an unhappy lover who started his PS/CONSOLE up to delete all things? We do not know - but life can be nasty. Anyway, if something like this will ever happen to us, we can only pray that ZOS is able and willing to help.

You do realize that restoring the game database from backup means rolling the game back for all players, right? In this case by a week or so, which is when the problem occurred?

A more sensible option is for customer service to investigate what happened, and if appropriate mail the player the items, rather than causing a major disruption to the entire playerbase via a backup restore.

BalticBlues

AlnilamE wrote: »

You do realize that restoring the game database from backup means rolling the game back for all players, right?

In which century are you living? This 202x, not 199x. 30 years difference means,
there is no need to restore an entire database to restore a particular data record

Elsonso

BalticBlues wrote: »

AlnilamE wrote: »

You do realize that restoring the game database from backup means rolling the game back for all players, right?

In which century are you living? This 202x, not 199x. 30 years difference means,
there is no need to restore an entire database to restore a particular data record

That assumes that the database you want to restore from (a) has the data that is needed, and (b) is online and accessible via query. We don't know a lot about their database environment, but we do know that the databases are big. Mind-bogglingly big. Heck, just the amount of mail that they _deleted_ earlier this year was mind-bogglingly big, and that was just the one database. I would not assume that backups are standing by waiting for someone to query them.

nightstrike

BalticBlues wrote: »

AlnilamE wrote: »

You do realize that restoring the game database from backup means rolling the game back for all players, right?

In which century are you living? This 202x, not 199x. 30 years difference means,
there is no need to restore an entire database to restore a particular data record

You're not entirely wrong, but database manipulation isn't really the first thing to try. They can just mail you almost any item in the game.

The sequence I go through when a glitch destroys an item is to:

1) file a ticket in game
2) reply back to the first 5 or 10 responses with "please try again", "read what I wrote", "please escalate this", "please check the log", etc.
3) when the ticket gets closed (it always does), fill out the survey and request it be reopened.
4) eventually, I'll get someone to actually look at a log and send it to me, showing the glitch. We then have a talking point to discuss. They then almost always mail me the missing item(s).

It's not an easy process. It takes weeks, sometimes a month. But it does work. Most recently, I lost some purple BG chests due to a bug with duplicate leads. Ultimately, they couldn't replace the lost style pages inside the bugged chests, but they instead just gave me a handful of new chests and luck got me half of the missing pages back. We still had non-DM BG at the time, so I just called it square at that point.

(Note! BG chests are still bugged, so don't hoard them and do your leads immediately!)

This thread is veering off topic (and I admit that I'm contributing to that), so it's likely to get closed soon. I would just kindly ask ZOS, before that happens, to please take this thread as an example of how support could do better. Take it as a training opportunity maybe to treat customer complaints of lost items with perhaps more thoroughness. Given my own experience, I can guess how this user's experience could be better, and there's likely room for improvement across the board. Please try to learn from that.

IronWooshu

Maybe the items he bought were dupped or stolen from another account and when returned to the original player was taken from him. I'm on console so I don't have to really worry about that stuff but I remember the Deadric Throne dupe that happened awhile back, maybe someone found something like that and ZOS caught on and made things right.

Supreme_Atromancer

I lost my Facemask and Encasement of Anguish in this way. So disappointing cause I treasured those items and held on to them for literal years. Then they were just gone.

I'm sure this sort of stuff happens all the time on a smaller scale. It may just be that often you don't even notice the things that go.

BalticBlues

Elsonso wrote: »

We don't know a lot about their database environment, but we do know that the databases are big. Mind-bogglingly big.

That is correct. However, ZOS now even has implemented a mechanism to swap unused accounts out of the live database to backup media (to lower the server burden) and/or restore an old account from backup media into the live database once an old user returns and logs in again. With such mechanisms and a solid backup strategy, they can restore any account from any given date.

hafgood

No they can't. To restore a character from an earlier date they have to have a saved version on that date.

Your assumption is that they have unlimited copies of the database going back to when it started. Do you have any idea how much space that would take up?

In addition you are also forgetting that the database will have been updated many times in that period, a year one database would not drop into the current patch database due to all the changes that have occurred since.

So to claim they can restore back to any period time shows a severe lack of understanding of how databases work. They can probably restore anything post CP2 from any save points they have but it's unlikely they will have daily saves. So, in the case in point, if we assume it happened on the 15th of the month, if the last save with the relevant data on was the 1st of the month the player will potentially lose 2 weeks of game play. Obviously the more recent the issue the more likely they are to have more saves but these things take time to find the relevant save and then to drop it back in

nightstrike

BalticBlues wrote: »

Elsonso wrote: »

We don't know a lot about their database environment, but we do know that the databases are big. Mind-bogglingly big.

That is correct. However, ZOS now even has implemented a mechanism to swap unused accounts out of the live database to backup media (to lower the server burden) and/or restore an old account from backup media into the live database once an old user returns and logs in again. With such mechanisms and a solid backup strategy, they can restore any account from any given date.

Ok, I know I said I was contributing to this being off topic, but.....

That's not what sharded storage is or how it works. You can think of it much more like keeping used accounts in a cache and unused accounts out of the cache. That's completely different than keeping used accounts in "a live database" and unused accounts "on backup media".

daim

I dont really care how their database works, its not my job. I just dont want to lose stuff. Im pretty sure im in a majority here.

Hopefully this will be looked into, in Detail! Its highly worrying.

ADarklore

daim wrote: »

I dont really care how their database works, its not my job. I just dont want to lose stuff. Im pretty sure im in a majority here.

Hopefully this will be looked into, in Detail! Its highly worrying.

I'm kind of curious as to WHY this is highly worrying when so far it's only been ONE person here with this problem at this time? If there were more than one, I'd be concerned... but 'highly worrying' over one person's incident... yeah, I'm not worried at all.

SeaGtGruff

.

ADarklore wrote: »

daim wrote: »

I dont really care how their database works, its not my job. I just dont want to lose stuff. Im pretty sure im in a majority here.

Hopefully this will be looked into, in Detail! Its highly worrying.

I'm kind of curious as to WHY this is highly worrying when so far it's only been ONE person here with this problem at this time? If there were more than one, I'd be concerned... but 'highly worrying' over one person's incident... yeah, I'm not worried at all.

Me, neither.

Maybe it's just me, but I think this whole thread smells a bit suspicious, starting with the accusatory wording of the title.