How will the AI that monitors our in-game chats effect RP?

katanagirl1

So the profanity filter and the reporting system aren’t enough I guess. Sounds like a zero tolerance policy that even if everyone is okay with those words you still can’t say them.

Congratulations ZOS, you have already effectively killed zone chat for all reasons other than gold sellers, guild recruiters, and selling in zone chat. Seriously, no one is saying anything anymore. Those are the things everyone really hates, too. Zone chat could be entertaining and it could sometimes be horrible. I can’t really defend it other than that it made the game feel more alive. Now it’s like there is no one else on.

It’s not just zone chat that is the problem now, from what I have read here. Even private chat can get you a ban.

Lately there have been a lot of decisions made that have upset the playerbase. I hope this is not the final nail in the coffin. I personally don’t have to worry about typing anything offensive myself but I’m an outlier. This is a multiplayer game and I do have to rely on others to get some things done. If everyone leaves that will be problematic.

Destai

Elsonso wrote: »

TaSheen wrote: »

"offshore resources reading chats"?

Everyone is "offshore" to someone.

I am not sure where this "offshore" thing is coming from, except from the player above. Does anyone know what they are talking about?

Not really. In HR and staffing, offshore specifically means countries like India and PH.

I manage offshore resources quite frequently, and it’s a common occurrence in global IT to leverage offshore resources due to the lower costs. It’s a common outsourcing practice.

My firm staffs content moderation and the resources largely come from offshore, but there are a few local call centers that contract out work.

TaSheen

Destai wrote: »

Elsonso wrote: »

TaSheen wrote: »

"offshore resources reading chats"?

Everyone is "offshore" to someone.

I am not sure where this "offshore" thing is coming from, except from the player above. Does anyone know what they are talking about?

Not really. In HR and staffing, offshore specifically means countries like India and PH.

I manage offshore resources quite frequently, and it’s a common occurrence in global IT to leverage offshore resources due to the lower costs. It’s a common outsourcing practice.

My firm staffs content moderation and the resources largely come from offshore, but there are a few local call centers that contract out work.

Yes. I understand that. I was specifically questioning the almost antagonistic use in pointing to CS in ESO.

PrayingSeraph

Im late to the party here so forgive my ignorance, but are people getting their entire account banned for apparent chat violations? Unless its something too egregious, cant they just mute your ability to chat in game and not ban you from playing? Thats what I always assumed would happen.

endorphinsplox

ZOS_Kevin wrote: »

Hi All,

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.

• We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.
• The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.
• To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.
• As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

So basically, ZOS is calling this a "hiccup", refusing to explain why they implemented this system with no warning, claiming the disciplinary actions are performed by a real person after manually reviewing flagged content, and that a real, living, breathing human being, employed by a major developer, genuinely believed that someone jokingly referring to a furnishing as looking like a certain bodily fluid was reasonable to categorize as a "serious crime, hate speech, or extreme case of harm"?

Not only that, but you aren't planning on stopping the NSA style monitoring of what we once believed were private chats, won't give us a list of words we can not say, and won't acknowledge that the casualties of this extreme error in judgement you call a "hiccup" outweigh any potential benefit it could have had, given that many cheaters, bullies, and scammers are still present throughout the game, completely unaffected by this change?

Yeah I waited for a response, and this confirms that ZOS is just going down a road I can not follow.

Dominion_Nightblade

Apparently, Call of Duty is doing it as well.

DenverRalphy

katanagirl1 wrote: »

So the profanity filter and the reporting system aren’t enough I guess. Sounds like a zero tolerance policy that even if everyone is okay with those words you still can’t say them.

Congratulations ZOS, you have already effectively killed zone chat for all reasons other than gold sellers, guild recruiters, and selling in zone chat. Seriously, no one is saying anything anymore. Those are the things everyone really hates, too. Zone chat could be entertaining and it could sometimes be horrible. I can’t really defend it other than that it made the game feel more alive. Now it’s like there is no one else on.

It’s not just zone chat that is the problem now, from what I have read here. Even private chat can get you a ban.

Lately there have been a lot of decisions made that have upset the playerbase. I hope this is not the final nail in the coffin. I personally don’t have to worry about typing anything offensive myself but I’m an outlier. This is a multiplayer game and I do have to rely on others to get some things done. If everyone leaves that will be problematic.

ZoS hasn't changed anything. It's the same system that's been in place since year one.

AngryPenguin

DenverRalphy wrote: »

katanagirl1 wrote: »

So the profanity filter and the reporting system aren’t enough I guess. Sounds like a zero tolerance policy that even if everyone is okay with those words you still can’t say them.

Congratulations ZOS, you have already effectively killed zone chat for all reasons other than gold sellers, guild recruiters, and selling in zone chat. Seriously, no one is saying anything anymore. Those are the things everyone really hates, too. Zone chat could be entertaining and it could sometimes be horrible. I can’t really defend it other than that it made the game feel more alive. Now it’s like there is no one else on.

It’s not just zone chat that is the problem now, from what I have read here. Even private chat can get you a ban.

Lately there have been a lot of decisions made that have upset the playerbase. I hope this is not the final nail in the coffin. I personally don’t have to worry about typing anything offensive myself but I’m an outlier. This is a multiplayer game and I do have to rely on others to get some things done. If everyone leaves that will be problematic.

ZoS hasn't changed anything. It's the same system that's been in place since year one.

If you are not a ZOS employee working directly on this issue then you can't be any more sure about what's going on than the rest of us.

spartaxoxo

DenverRalphy wrote: »

katanagirl1 wrote: »

So the profanity filter and the reporting system aren’t enough I guess. Sounds like a zero tolerance policy that even if everyone is okay with those words you still can’t say them.

Congratulations ZOS, you have already effectively killed zone chat for all reasons other than gold sellers, guild recruiters, and selling in zone chat. Seriously, no one is saying anything anymore. Those are the things everyone really hates, too. Zone chat could be entertaining and it could sometimes be horrible. I can’t really defend it other than that it made the game feel more alive. Now it’s like there is no one else on.

It’s not just zone chat that is the problem now, from what I have read here. Even private chat can get you a ban.

Lately there have been a lot of decisions made that have upset the playerbase. I hope this is not the final nail in the coffin. I personally don’t have to worry about typing anything offensive myself but I’m an outlier. This is a multiplayer game and I do have to rely on others to get some things done. If everyone leaves that will be problematic.

ZoS hasn't changed anything. It's the same system that's been in place since year one.

Kevin stated they are working with new tools that they are still training on. Here's that part of the quote, snipped for brevity.

ZOS_Kevin wrote: »

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups.

AngryPenguin

HatchetHaro wrote: »

ZOS_Kevin wrote: »

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

If I might make a suggestion, since the automated checks will always flag these "potentially harmful words or phrases", it means your customer support representatives will have to check through a large number of messages, and therefore also increasing the likelihood of misinterpretation of chat messages and therefore sending out undeserved warnings, suspensions, or bans.

Instead of having automated checks flood the system with false positives, perhaps it'd be better to have the players themselves determine whether a message was harmful. For example, if a player receives a potentially harmful message, they then can determine for themselves whether or not it was harmful to them, and then have the ability to flag those phrases themselves for customer support to review and take action on. I'm thinking they can right click the player's name in the chat for that option. We can call it the "Report Player" button!

This is exactly right on.

We already have a profanity filter and a report player option. We certainly didn't need the addition of a hyperactive and highly inaccurate AI system getting involved. Who ever heard of a robot getting offended anyway? Why would a robot care if someone insulted it? Would calling a robot a useless tin can make it cry?

It's undeniably obvious that this new system of monitoring and banning is not working correctly, so please disable it. If it were just causing the player base a "hiccup" very few would be complaining about it. This is a much more serious issue than Kevin's statement acknowledges. Not to mention just flat out disrespectful of the player base.

Additionally, this isn't just a problem for RP'rs. This is a problem for anyone who ever uses in game chat for anything. So pushing all the discussion of this issue into an RP thread is not appropriate either. It's just making the people reporting this issue less noticeable and harder to be heard.

Syldras

AngryPenguin wrote: »

We already have a profanity filter and a report player option. We certainly didn't need the addition of a hyperactive and highly inaccurate AI system getting involved. Who ever heard of a robot getting offended anyway? Why would a robot care if someone insulted it? Would calling a robot a useless tin can make it cry?

Considering it has no human sensivities, a bot would actually even be better than a human reviewer who could be influenced by personal opinions, upbringing, culture and other such factors - wouldn't the problem be that it would also not understand context, fiction vs reality, jokes, etc.

Anyway, ZOS says it's only auto-flagging, but then the cases get sent to a human reviewer. The question is why that process leads to so many wrong bans. If it's outsourced: Are the people making these decisions reliable? Do they understand the language well, including idioms and colloquialisms? Are they provided enough of the chat to be able to see context (one line only won't help)? Are they flooded with cases so much that they don't have enough time to review them properly?

AngryPenguin

Syldras wrote: »

AngryPenguin wrote: »

We already have a profanity filter and a report player option. We certainly didn't need the addition of a hyperactive and highly inaccurate AI system getting involved. Who ever heard of a robot getting offended anyway? Why would a robot care if someone insulted it? Would calling a robot a useless tin can make it cry?

Considering it has no human sensivities, a bot would actually even be better than a human reviewer who could be influenced by personal opinions, upbringing, culture and other such factors - wouldn't the problem be that it would also not understand context, fiction vs reality, jokes, etc.

Anyway, ZOS says it's only auto-flagging, but then the cases get sent to a human reviewer. The question is why that process leads to so many wrong bans. If it's outsourced: Are the people making these decisions reliable? Do they understand the language well, including idioms and colloquialisms? Are they provided enough of the chat to be able to see context (one line only won't help)? Are they flooded with cases so much that they don't have enough time to review them properly?

For six years ZOS told us they were going to fix performance in PvP/Cyrodiil.

Syldras

We can still work with he hypothesis based on what info they give us. And then the question is: Why is the human decision-making so faulty?

Let's say if it's outsourced and the reviewers get paid per solved case, I'd not be surprised for many mistakes, because it leads to the reviewers hurrying through the process to get as many cases closed in the shortest time possible.

In the end, we can only speculate. I just hope that ZOS is aware of the current problem now and will act accordingly.

Arunei

brylars wrote: »

I think the issue is done.

We asked the queation. We got our official answer.

Nothing will be done because ZOS is denying the accusations and has said it will not change anything.

Since this thread will do nothing more to enact change @ZOS_Kevin please close the thread.

How about we not call for closing a thread where discussions are still being had and questions asked? You have no way of knowing whether we're going to get more answers or not. So please don't try to shut down the conversation of a very serious matter.

endorphinsplox wrote: »

ZOS_Kevin wrote: »

Hi All,

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.

• We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.
• The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.
• To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.
• As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

So basically, ZOS is calling this a "hiccup", refusing to explain why they implemented this system with no warning, claiming the disciplinary actions are performed by a real person after manually reviewing flagged content, and that a real, living, breathing human being, employed by a major developer, genuinely believed that someone jokingly referring to a furnishing as looking like a certain bodily fluid was reasonable to categorize as a "serious crime, hate speech, or extreme case of harm"?

Not only that, but you aren't planning on stopping the NSA style monitoring of what we once believed were private chats, won't give us a list of words we can not say, and won't acknowledge that the casualties of this extreme error in judgement you call a "hiccup" outweigh any potential benefit it could have had, given that many cheaters, bullies, and scammers are still present throughout the game, completely unaffected by this change?

Yeah I waited for a response, and this confirms that ZOS is just going down a road I can not follow.

All of this. We need answers. What we got was a vague admission of guilt and ""assured"" that actual real people made these choices. How are we supposed to trust that they won't keep making bad choices or letting personal opinion on what they're reading make choices?

1) Why were we not informed of this change? This is a MASSIVE issue, as chat is one of the biggest damn features in an MMO. How would anyone think this wasn't a major addition to the game that needed to be announced?

2) What REAL assurances can we be given that these actions won't be made willy-nilly in the future like they are now? Will agents be held accountable if they take actions against accounts that did not warrant said actions? And no, don't tell us that what happened was the result of new tools and training (talking to the devs here, not Kevin, just a heads-up). Like I said, there shouldn't need to be any sort of advanced training on how to read context.

3) As others have asked, myself included, what terms/phrase/words/etc will get us in trouble? You can't implement a rule and then not explain fully how to avoid breaking it. Will this list take into account the fact that a word might be a slur or crude in one language but is a normal innocent word in another?

4) Will the fact that the filter in-game is ridiculously sensitive? It finds ""naughty"" words by ignoring spaces or considering certain numbers as letters (such as 5 being used in place of an S or 0 for an O or 7 for T)?

I know I have more questions but I can't think of them right now because I'm super tired.

Ingel_Riday

Wanted to chime in again, real quick.

@ZOS_Kevin : If the goal was to kill zone chat, you did it. At least from my experience over the past two days, Glenumbra-Barrens is dead. Every zone feels dead. Nobody wants to risk having their account suspended or banned, so people are saying nothing outside of an occasional guild spam or "WTS item X for exorbitant sum" message.

Frankly, I find this depressing. One of the things that has kept me subscribed since 2017 is the community. The incessant, increasingly galling FOMO events / daily log-ins / what-have-you play a role, sure, but I actually like ESO people. If you all ever got too crass for me, I could always block you... but I didn't have to do that very often. Outside of the awful anniversary event RNG staff nightmare, none of you ever drove me into a feeding frenzy of peeved whining (got a strike on my forum account for that one, too. Some of you got me riled up right and proper, haha. Pure angry-idiot mode).

But hey, can't risk people being offended by private chats they can't see, or leave ZOS / Microsoft at risk of finding themselves running afoul of restrictive FCC laws because they didn't toe the Draconian line and sew our mouths shut. So here we are.

I always wondered what this game would be like if they made an offline mode without any players around. Would it hold up, or would it need npcs running around like goobers to compensate? Spoiler alert: it doesn't hold up. I love doing things solo, but I love reading your general chat comments while doing so. Without those, it's genuinely missing something.

Years from now, if this game ever gets re-balanced and released as a giant single player experience, I hope they work in fake general chat to complete the experience. Some AI-driven absurdity to fill the void.

Going to do the smart thing and follow the crowd. Ditch general chat and only look at it if I need to recruit someone for a world boss. I'll miss you guys.

Edit addition: you know the grim thing about it all? In this age of technology so advanced that it'd be magic to our ancestors, I'm feeling increasingly isolated. Nobody hangs out together in real life anymore, and every digital space is becoming on Orwellian nightmare. Over half my comments on Youtube get hidden, with the system hiding them being left purposefully vague and obtuse lest I learn a way to bypass it. I don't dare chat with people on PlayStation or Xbox, and now I won't dare chat with any of you here (in true fashion, ZOS hasn't published any of the parameters. Purposefully vague and obtuse). "Digital social spaces" are increasingly anything but nowadays.

Just sad. God help the powers that be when they lose the reins on these systems. You think that you'll enjoy this level of Big Brother censorship when it's turned on you? We'll see, because it'll happen eventually. *shrug*

Eh, I'm rambling. Shouldn't get discouraged. I'll just find a new community to hang out with. New people to chat with. It's not the end of the world. Maybe I'll finally join a meet-up group of Spanish speakers and make some new real-life buds. Then I can chat with them freely without any moral crusaders clamping our mouths shut in the name of progress.

Syldras

Not to excuse any of the debacle, but on PC EU I've not noticed any big difference. Especially in the starter zones the chat is cluttered as usual, including the usual Vulkhel Guard wayshrine stuff (who came across it knows what I mean).

I'm still curious whether the people who were banned were all on the US server? And were these chats all in English?