How will the AI that monitors our in-game chats effect RP?

Idelise · September 19

Idea for a solution:

Moving to speaking brainrot language fluently. Confuse the bots.
https://anythingtranslate.com/translators/brainrot-translator/

Syldras · September 19

I think since it's a more complicated topic, they might have to confer with their experts (legal, programmers,... - everyone who is involved in is somehow) to make a correct statement. That can take some time.

By now, I'd expect them to tell us that now there's an automoderator (I'd rather suspect keyword list than a true AI) and that we have agreed on the TOS, so it's correct that they enforce it. And that in case of faulty decisions, we can message them to review the case.

Beyond that, I'd really like to see them acknowledging the problems we have named in this thread, e.g. a bot not recognizing context and cultural and language differences.

SuspensionDispersingAutomaton · September 19

Idelise wrote: »

Idea for a solution:

Moving to speaking brainrot language fluently. Confuse the bots.
https://anythingtranslate.com/translators/brainrot-translator/

It do be like that, fam, y'all better griddy into dat. Bots be edging fr.

Syldras · September 19

Elsonso wrote: »

A lot of people don't maintain a puritan demeanor at all times.

And some use insults that colorful that the bot doesn't recognize them

Ugrak · September 19

Idelise wrote: »

Idea for a solution:

Moving to speaking brainrot language fluently. Confuse the bots.
https://anythingtranslate.com/translators/brainrot-translator/

Bots will be updated. Go directly to number stations and one time pads I guess.

Synapsis123 · September 19

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

SilverBride · September 19

Synapsis123 wrote: »

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

Unless the gibberish puts together letters that spell offensive words.

Elsonso · September 19

Synapsis123 wrote: »

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

Bet you get banned for that. Spamming, probably.

tomofhyrule · September 19

Here's the thing - I'm sure this is not coming from ZOS. This is something that's likely coming down from higher than them. It would have been nice for them to give a warning that "Starting with U43, we will be cracking down on violations of the Code of Conduct, including in closed chat messages," or start by giving warnings instead of going straight to bans, or something of the sort. That would at least let people know that this moderation was coming.

Still, trusting an automod (and for the looks of it, it's an automod, not an AI) to be judge, jury, and executioner has numerous times been shown to lead to problems, and not for the people getting actioned.

Remember when Baldur's Gate 3 was released on consoles, and then XBox players found themselves getting banned because 'they had saved images displaying nudity,' when it turned out that they were literally screenshots from the game? And then it took ages to unban them and the developers had to fight with Microsoft to be like "hey, so our game has nudity, as it says in the rating, which you knew about. Banning people for simply taking a screenshot of the game is a bit ridiculous."

I'm also sure that this won't change the way most people play. People are likely to be less willing to use the in-game chat, but most players use Discord or something of the sort anyway to talk with their friends and still carry on their conversations. The toxic people are still gonna be toxic, but it may at least make all those nice players stop trying to talk to their dungeon randoms and explain mechanics when needed.

I_killed_Vivec · September 19

Tandor wrote: »

I_killed_Vivec wrote: »

The geese will be flying east tonight

Are you a secret service agent passing on a coded message to your handler?

Indeed. It was one of the conspiracy theories, that private chat would be used for nefarious communications... and AI could detect this and rat us out to the feds

SuspensionDispersingAutomaton · September 19

Synapsis123 wrote: »

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

Elsonso wrote: »

Synapsis123 wrote: »

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

Bet you get banned for that. Spamming, probably.

Spam is also bannable, even in whispers to friends, according to this 2021 article:

https://web.archive.org/web/20210410122352/https://gamestoday.info/pc/the-elder-scrolls/update-on-permanent-ban-finally-some-justice/

The CS bot will just call you a bot yourself and then you will need to respond at least a billion of times before an actual human reads your appeal.
I don't know the person who wrote the article, but my friend was warned for spamming his empty guild chat with no members in order to test connectivity while lagging, so such cases are also real.

Syldras · September 19

I_killed_Vivec wrote: »

Indeed. It was one of the conspiracy theories, that private chat would be used for nefarious communications... and AI could detect this and rat us out to the feds

It's not a conspiracy theory, it's actually what companies who monitor their chats or private messages always claim as a reasoning ("If we don't monitor, some evil terrorists criminals unwashed scoundrels will use the chat to talk about plans to kill unlive (insert name of random person here)!" - Of course no person with more than 3 braincells would write about such plans openly.

WynnGwynn · September 19

a bot calling real people bots is the kind of thing I love to see. /s

Pelanora · September 19

Synapsis123 wrote: »

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

Encryption?

Pelanora · September 19

Syldras wrote: »

I think since it's a more complicated topic, they might have to confer with their experts (legal, programmers,... - everyone who is involved in is somehow) to make a correct statement. That can take some time.

By now, I'd expect them to tell us that now there's an automoderator (I'd rather suspect keyword list than a true AI) and that we have agreed on the TOS, so it's correct that they enforce it. And that in case of faulty decisions, we can message them to review the case.

Beyond that, I'd really like to see them acknowledging the problems we have named in this thread, e.g. a bot not recognizing context and cultural and language differences.

You could be invoking the tos with them and see what happens. Given you're in the EU and have the right to be free of automatic decision making.

Desiato · September 19

Many years ago when I was still a young man, my best friend and I used to greet each other exclusively with insults until we were in our mid-30s.

I guess if we were playing eso together back then, we might have gotten banned.

Growing up, we were both taught "sticks and stones may break your bones but words will never hurt you." We reinforced that by insulting each other, as part of the process of toughening up, an essential part of entering adulthood.

It's obviously important to protect customers from harassment and other forms of abuse, but words shared between consulting adults should be off-limits for both ZOS staff and automated processes. Proactive enforcement has no place in group and private chats.

People who question whether ESO is financially successful should look at this policy as proof that it is because a struggling business would never consider alienating paying customers in this way.

sarahthes · September 19

Elsonso wrote: »

Synapsis123 wrote: »

Someone make an addon that changes what you type into chat into gibberish and only people who have the addon can see what you actually said. That way we can get around the chat bot blocking normal activity.

Bet you get banned for that. Spamming, probably.

There is an automated social ban for spamming that it is possible to trigger. Saw it in action about 4 years ago.

ZOS_Kevin · September 19

Hi All,

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.

We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.
The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.
To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.
As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

Sleepsin · September 19

ZOS_Kevin wrote: »

Hi All,

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.
We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.

The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.

To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.

As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

How far down the line is the 3rd party contractor that is reading our private chat?

SuspensionDispersingAutomaton · September 19

ZOS_Kevin wrote: »

No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

The humans hired were no more useful than automatic bots, as they banned me for consensual chat among close friends and sent a generic copypasted reply I saw they sent to others affected, with absolutely no consideration for my explanation and reasoning that it was all consensual.

I don't have faith that if I return and spend even more time on this game, I will not just be autobanned again for juicy words in private because another agent will be just as careless and prudish as the one I had.
And next time it will most likely have less public attention and be swept under the rug, like many other cases in the past when people I knew kept contacting support trying to explain their situation, but were ignored by the CS.

Best of luck moving forward with your new systems, I guess.
I will play things that don't make me fear being banned whenever I chat in private.
"Private" I mean, as the support "kindly" explained to me.

Tommy_The_Gun · September 19

The serious issue I am seeing here is that in game chat is also an actual communication channel for many players and it is not always "game related" stuff that is being discussed but also RL stuff. If some one for example asks "how is your day going ?" I guess it is all fine but... how to put it. I guess an example will be best. Lets say some one will mention a military conflict that happens somewhere on the world. And they will mention something harmless, like "I wish this conflict ended" etc. The AI will for sure detect such person & maybe even actually ban for no good reason (just in case) or at least flag such person for human moderation.

So um... consider this a feedback I guess.

EnerG · September 19

ZOS_Kevin wrote: »

Hi All,

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.
We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.

The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.

To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.

As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

This also doesn't answer the question of what kind of program is scrubbing chat to begin with, and how it's affecting latency in the game. Please give a more clear answer about WHAT is sending "concerning" chats to support, and how intensive the system being used is.

Ugrak · September 19

Desiato wrote: »

It's obviously important to protect customers from harassment and other forms of abuse, but words shared between consulting adults should be off-limits for both ZOS staff and automated processes. Proactive enforcement has no place in group and private chats.

They could maybe instead just hook something up directly to the profanity filter setting, so that players with the filter on simply do not receive communication from players with the filter off.

Turn off seeing other potentially offensive actions too, such as the crouch animation, mudballs, etc. ESO safe-mode.

And/or instead of eavesdropping and punishing, the automod could be set up so that all communication is filtered through it in real time, so that anything that triggers it is simply rejected with a warning.

Desiato wrote: »

People who question whether ESO is financially successful should look at this policy as proof that it is because a struggling business would never consider alienating paying customers in this way.

The caveat however, especially with MMORPG accounts, is that customers pay with some expectation that they will have access to the product, and that the product will deliver on it's expected value for some time.

If well founded fear of losing such accounts to automoderation of harmless chatter leads to mute communities, then this fundamentally undermines the MMORPG experience. Why keep investing time and money into that?

"North Korea Online" is the type of gaming experience apt to elicit an exodus; making future financial success far less certain.

Juomuuri · September 19

Thank you for the follow-up Kevin! Very relieved to hear there's no such thing as auto-bans - hopefully this means we can all chat and RP without fear again.

Destai · September 19

ZOS_Kevin wrote: »

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

Thanks for coming back to this and responding to our concerns, appreciate the engagement.

ZOS_Kevin wrote: »

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

I really think you should share their guide for what "potentially harmful terms and phrases" actually is. Without knowing what's allowed or not, we're all going to be just waiting for the hammer to drop anytime we make an adult joke. Like if I joke about the lag in game, am I risking a ban for bashing? IMO, the only thing these guys should be looking for is the gold spammers that are in every zone.

ZOS_Kevin wrote: »

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups.

A few hiccups? Dude, come on. Do you know how stressful it is for someone to potentially get banned from an account that they've spent countless hours and dollars on because you guys rolled out some new tools unannounced?

Oh, and I was looking for an announcement of this feature on your website. I wasn't finding it. Why wasn't there advance notice of this coming out? Was it being released soon so the larger gaming community knows this practice exists?

ZOS_Kevin wrote: »

But we want to stress a few core points.
We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.

The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.

To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.

As a company we also abide by the Digital Service Act law and all similar laws.

Are they reading our private, unreported messages - yes or no?

ZOS_Kevin wrote: »

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

Good call. Just for our information, are these offshore resources reading chats that they may not have the cultural context to understand? Hopefully people are being compensated for what was the result of training people on new technology.

ZOS_Kevin wrote: »

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

I'm sorry, but this doesn't alleviate any concern. If anything, it makes me more concerned, especially seeing how moderation's played out here and on streams. A lot of us are guilds that are 18+ and make mature jokes. The last thing anyone needs to worry about is a joke getting them banned when it was among consensual parties in private.

One last question - is there or will there be similar monitoring on in-game voice chat?

Inari Telvanni · September 19

ZOS_Kevin wrote: »

Hi All,

We want to follow up on this thread regarding moderation tools and how this intersects with the role-play community. First, thank you for your feedback and raising your concerns about some recent actions we took due to identified chat-based Terms of Service violations. Since you all raised these concerns, we wanted to provide a bit more insight and context to the tools and process.

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.
We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.

The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.

To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.

As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

We hope this helps to alleviate any concern around our in-game chat moderation and your role-play experiences. We understand the importance of having safe spaces for a variety of role-play communities and want to continue to foster that in ESO.

Thanks for the reponse, Kevin. Do you know when the "iterating on some processes" will be finished?

TaSheen · September 19

"offshore resources reading chats"?

Dragonnord · September 19

From ZOS_Kevin:

"No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on."

Thank you Kevin. Because several people were blaming ZOS and AI of automatic banning.

As I said, people becoming paranoid with AI.

I hope @StaticWave and @Heren are relieved now.

Erissime · September 19

What in the whole fetching Oblivion is this all about? If there is a censorship working around - can we get a specific list of bannable words and terms and be off with it already? Chat will be monitored anyway - and rpers should know how to waltz past words and terms without them having to be downright vulgar.

So can we just have those terms and continue to live our lives without so much hassle? Specific and in depth - words you know have a potential to trigger your tools , so that people know exactly what to avoid and clear the air.

Thank you.

Syldras · September 19

I mean, in a way I can understand that some keywords are autoflagged, even in private chats. That is when it comes to the topic of severe crimes. Imagine the scandal if indeed something serious happens and the media make a scandal out of "The perpetrators communicated through the ESO chat unseen" - ZOS certainly wants to avoid such a reputation loss. But the flagging of simple swear words... oh well.

Ugrak wrote: »

And/or instead of eavesdropping and punishing, the automod could be set up so that all communication is filtered through it in real time, so that anything that triggers it is simply rejected with a warning.

If the filter remains as sensitive as it is, this is no solution either. It will lead to what I've seen in the comment section of some news sites: No matter how you rewrite it, you'll always get a "profanity detected" message, until you give up posting altogether.

TaSheen wrote: »

"offshore resources reading chats"?

Sounds like the Amazon callcenters operating mainly from India and Pakistan. The employees are usually students and while they are usually very willing to help, language often is a problem.