Is it true you can get banned by AI in zone chat?

Vonnegut2506 · May 18

Just a language curiosity, but when did we start replacing "suspended" which used to mean a temporary forced timeout from the game with "banned" which used to mean you were booted out for good? I read a lot of these threads with people claiming to be banned when they mean a temporary forced timeout. It isn't just this game either, it is the same thing over on the WoW forums.

SpiritofESO · May 18

My feeling is that language, our language, is important and we should always be careful about our words. Even though many people feel they are anonymous while playing an online game, the truth actually is that you display your "character" every time you speak, or, in this case, type in chat.

I have no idea if you can get banned by AI in this game, but it would not surprise me.

And please remember the Golden Rule: those who have the gold, make the rules. Therefore, ZOS makes the rules.

Jaimeh · May 18

Yes, people should generally mind what they say in chat, but context and nuance exist, and AI does not take them into consideration. All it does is create a culture of fear to the point that people in order not to deal with the headache of an accidental issue opt not to talk in chat at all. In an MMO game...

Vonnegut2506 · May 18

Jaimeh wrote: »

Yes, people should generally mind what they say in chat, but context and nuance exist, and AI does not take them into consideration. All it does is create a culture of fear to the point that people in order not to deal with the headache of an accidental issue opt not to talk in chat at all. In an MMO game...

To be fair, I usually shut off zone chat due to stupidity long before AI moderation was even a glimmer on the horizon. I will have to turn it back on to see if things are better or not now that it exists. I would be willing to bet Auridon is still filled with racist chat most of the time as it used to be.

LordDragonSlayer · May 19

The short answer is yes.

I am not sure why they are looking at private messages unless the recipient complains but their AI system looks all messages from my understanding and based on the words being used, they can ban.

In fact, their AI can ban for many reasons and usually it will say something like cheating or exploitation or language or harassment, etc.. Currently, there is an issue with newer players it seems who are getting banned for no reason or at least they are claiming there is no reason.

I have reported people who harass because they will enter a trial group, pull a boss, get everyone killed, and then leave but I was told that is normal game play. I am like really? But if you teabag a person in pvp, that is considered harassment and is a cause for being banned. So harassment already has different meanings and is therefore, not well defined.

The problem with the cheating/exploitation type of banning is that you have to know it is cheating and/or an exploitation in order not to use it. They and others will claim it is clear in the TOA/TOS etc but lets face it, those are written by lawyers and they are never clear. Yes, some are an obvious exploitation such as paying for something with gold, then doing something so the gold never actually leaves your account. The TOA/TOS will say something like exploitation is a cause for banning your account but never actually really define what exploitation is other than to say that it is something outside the normal game play. See above as an example.

Which leads to what exactly is normal game play? I am not the one writing the code and I am not the one that is the designer so how am I supposed to know that this is not normal game play. If they catch someone exploiting something, they should be told that is an exploitation and any further use of this exploitation will result in their account being banned.

FeedbackOnly · May 19

Mayrael wrote: »

Unfortunately, just respecting the rules isn’t enough anymore, because even jokes can get us into trouble these days. In my opinion, this is already an infringement on free speech that borders on totalitarianism. Nowadays, anything and anyone can be offended, and we’re reaching the point of absurdity. It really saddens me that this is seeping into games. This is one of the reasons I’ve stepped back from social interactions in ESO, and I get the impression that game chats in general are no longer vibrant hubs of fascinating conversations. Instead, they’ve become lifeless bulletin boards. The moment you bring up any topic, even one related to the game, someone will inevitably take offense at the way it’s said, the form, or the content.

Yeah that is what I was wondering about. Thank you

Destai · May 19

Xarc wrote: »

Destai wrote: »

I personally avoid chat in this game entirely. It’s not worth the stress to wonder if some inane mistake is going to cost me my account. There’s too much conflicting information on how AI moderation was implemented. Would rather gold spammers dealt with.

It's a mmo, not a solo game...

are you at least using vocal?

I use either in game voice or psn chat. I have a pretty established crew, so not really much need for text chat.

JoeCapricorn · May 19

Not too long ago I noticed someone make an egregious typo when trying to use the word "bigger". The unfortunate adjacency on standard QWERTY keyboards meant a different term was instead posted to guild chat.

As far as I know, they did not get in trouble. I have no doubt the AI saw that incident and alerted a human, but as long as it does not do the enforcing itself automatically, that sort of use is fine. And can it even be called AI if all it is doing is flagging a chat when particular words are used? I should also add that in subsequent messages, they corrected their error and it's clear from natural language that it was an error. AI is still unable to perfectly discern natural language, and is thus quite error prone (I know someone who made the same typo on Facebook and got actioned before they could edit it, but that was years ago before they added in a feature that sometimes works where it gives you time to do the edit)

That being said, I have no plans on testing this. Academic researchers that use budgeted funds to generate new accounts might be able to though (and probably not focus on one game or even identify which games were used).

moderatelyfatman · May 19

JoeCapricorn wrote: »

Not too long ago I noticed someone make an egregious typo when trying to use the word "bigger". The unfortunate adjacency on standard QWERTY keyboards meant a different term was instead posted to guild chat.

As far as I know, they did not get in trouble. I have no doubt the AI saw that incident and alerted a human, but as long as it does not do the enforcing itself automatically, that sort of use is fine. And can it even be called AI if all it is doing is flagging a chat when particular words are used? I should also add that in subsequent messages, they corrected their error and it's clear from natural language that it was an error. AI is still unable to perfectly discern natural language, and is thus quite error prone (I know someone who made the same typo on Facebook and got actioned before they could edit it, but that was years ago before they added in a feature that sometimes works where it gives you time to do the edit)

That being said, I have no plans on testing this. Academic researchers that use budgeted funds to generate new accounts might be able to though (and probably not focus on one game or even identify which games were used).

Yeah, a group of Twitch streamers got in trouble for naming this mythological creature during stream because it sounded like a very bad word.
Given that so many words and names in foreign languages can get you banned, I'm not sure how ZOS can fairly police this?

tincanman · May 19

Thought it might be interesting, if ironic, to ask a machine ( yep, that one ) how it evaluates the behaviour of the zos central scroooooootiniser (cf: Joe's Garage, Frank Zappa).

In the elder scrolls online game is AI used to ban players without checking by humans?
In the elder scrolls online game does the use of AI in the process of banning player accounts result in false positives? What would the likely estimated rate of these false positives be?
Does the use of AI in pre-emptive moderation where no player has reported an issue in the elder scrolls online game constitute a infringement on rights such as freedom of speech? Does the blanket application of such AI tools to all text shared in game indicate an invasion of privacy or a progressive move towards totalitarianism in that different and distinct cultures are judged by the same or similar metrics?

In the second question it picked up on my spelling error but, fortunately, I wasn't baned banned so was able to proceed to question 3.

The very last line of the 3rd response may have been introduced by a luddite. Or a cyberdyne systems model 101.

Zodiarkslayer · May 19

The right to free speech does not cover hate speech.

I find it befuddling when people rather want to argue about AI, when the reason it is needed in the first place is the decline of human decency, decorum and respect for one another.

cuddles_with_wroble · May 19

Zodiarkslayer wrote: »

The right to free speech does not cover hate speech.

I find it befuddling when people rather want to argue about AI, when the reason it is needed in the first place is the decline of human decency, decorum and respect for one another.

I don’t think any rational person is against banning people for hate speech, the issue is that the ai cannot distinguish between tones and emotions yet, the AI can ban you for making a joke to your friend in whispers…

The other issue is that if you do get banned, zos customer support is a horrid experience and you will likely never get unbanned even if the ai made a mistake

Hotdog_23 · May 19

The sad part for me is this is a MMO after all, and social is part of it by nature. People are not robots, and all our interactions with others cannot just be simple and direct. Yes, there is appropriate and inappropriate language, and no one should have to endure negative hateful language. ZOS is obligated to do something about creating a safe environment within the game. I see it as a delicate balance and a big gray area.

What worries me is with all the talk of banning weather, justified or not. They are pushing you, me, and others to be less social in the game for fear of “AI” ban, which is taking part of the charm out of ESO by encouraging fewer social interactions with others.

I believe no one should get an auto "AI" ban, and allowing a system that can and does auto ban someone without human intervention is just being lazy on ZOS part and really all games that use this same kind of feature. Any ban should be approved and looked at by the actual person. Let the system flag it, but it must be reviewed by an actual human being before the ban goes in place. Companies are trying to be cheap by using AI to moderate rather than actual people. They make tons of money on ESO; a small team of moderators would cost, but it is the cost of doing business responsibly. By letting such practices as auto ban in place they are slowing taking the fun out of the game and killing it a little bit.

Stay safe

Pixiepumpkin · May 19

Xarc wrote: »

Pixiepumpkin wrote: »

I was banned.

I sent in a ticket showing that I had not done anything wrong.

The ban was reversed, I was awarded crowns.

wow.
interesting thing

I think that when you do a lot of PvP and for a while, you gain experience and end up knowing/being known by certain people with whom a certain rivalry can arise and sweet words can be exchanged. Sometimes it goes too far; that's part of human relationships too.
My IgnoreList has been full for a few years now, and sometimes it's hard to hold back, especially when someone provokes you, but hey, that's how it is.

I was not really doing PVP at the time and had no "enemies" or rivals or anything. This happened in zone chat, I am mostly silent in zone chat, but this one time I spoke up expresing the actual definition of a word and how it is used in dailly communication.

frogthroat · May 19

Pretty sure this is not AI bannable offence, but it surely is profanity filtered. If you speak other language than English, in my case Finnish, and use the Finnish word "and" followed by any word starting with the letter p, that will be censored unless the filter is off. It confused me why it would censor the word "and". But that word followed by the letter p happens to be a slur in English.

So in Finnish we have to keep the profanity filter off even if we wanted it on because it censors one of the most common words we have. I am sure the AI is not banning us for it, at least not yet, because then all Finns would be banned. So the day all Finns vanish from this game, it's because AI decided to start banning people based on guild chat talks.

sleepy_worm · May 19

Zodiarkslayer wrote: »

The right to free speech does not cover hate speech.

I find it befuddling when people rather want to argue about AI, when the reason it is needed in the first place is the decline of human decency, decorum and respect for one another.

Reproductive fluid is not hate speech.

Elsonso · May 19

JoeCapricorn wrote: »

Not too long ago I noticed someone make an egregious typo when trying to use the word "bigger". The unfortunate adjacency on standard QWERTY keyboards meant a different term was instead posted to guild chat.

As far as I know, they did not get in trouble. I have no doubt the AI saw that incident and alerted a human, but as long as it does not do the enforcing itself automatically, that sort of use is fine.

I have to think that if that had been reviewed by a Human that the person would have been suspended or banned. It is reasonable to assume that the player got lucky.

Destai wrote: »

Xarc wrote: »

Destai wrote: »

I personally avoid chat in this game entirely. It’s not worth the stress to wonder if some inane mistake is going to cost me my account. There’s too much conflicting information on how AI moderation was implemented. Would rather gold spammers dealt with.

It's a mmo, not a solo game...

are you at least using vocal?

I use either in game voice or psn chat. I have a pretty established crew, so not really much need for text chat.

It is only a matter of time, months, not years, before voice chat is correctly scanned in real time. I am sure scanning is happening already, but real-time accuracy will get to the point where it will be the basis for reliable enforcement. This will start with people who speak the language without a heavy accent, but eventually, understanding will expand to include accented language.

I stay away from text and voice chat that is linked to the game and platform as much as possible. A quick "tyfg" at the end of a dungeon is about the extent of it.

cuddles_with_wroble wrote: »

I don’t think any rational person is against banning people for hate speech

This is a slippery slope. As such, I am conditional on the subject. Basically... who defines what is "hate speech" and are they worthy of being assigned to that role?

ZOS, XBox, and Microsoft can define it how they please, per game, but such definitions may not apply outside of their purview.

I see this as core to the problem. The boundaries of "hate speech" are rather fuzzy. This is just the social and political nature of such lists. They tend to be influenced by a particular perspective. The definition is subjective, not objective.

ZOS does not define the bounds of their "hate speech" definition. This makes things vague, but it does allow them to change it as the need arises.

xylena_lazarow · May 19

Zodiarkslayer wrote: »

The right to free speech does not cover hate speech.

As mentioned earlier ITT the bots can't tell between hate speech and someone making a typo on Crystal Frag. To make matters worse, the malicious trolls have learned they can simply misspell their hate speech and the bots won't notice. People are just leaving chat in an MMO game because they can be banned for something taken badly out of context, while a bunch of malicious jerks continue to exploit flaws in the system without consequence. Just like IRL...

robwolf666 · May 19

FeedbackOnly wrote: »

Getting back to ESO and want to make sure it's safe. Don't want to invest time in another game ruined by AI so would appreciate warning.

No idea, I never use zone chat these days, haven't for years. So unless AI can somehow read minds, I'm safe.

Azphira · May 19

Maybe one day there will be technology where people can say what they want and people that don't want to hear it can put them on a list where they never hear them again.

scrappy1342 · May 20

Elsonso wrote: »

It is only a matter of time, months, not years, before voice chat is correctly scanned in real time. I am sure scanning is happening already, but real-time accuracy will get to the point where it will be the basis for reliable enforcement. This will start with people who speak the language without a heavy accent, but eventually, understanding will expand to include accented language.

if they aren't already there, they are very close. we have a meta quest 3 and when i'm around loud obnoxious children, my system is always asking me if i want to block these ppl. pretty sure i saw warnings about language because the games/ai will pick up on it and can ban you.

yNiriel · May 20

3 of my elderly friends got banned
One got his account back just to got banned 4 days later, he doesn't speak in zone chat, doesn't cheat in any ways, the only thing he does is questing and nodes...
7 years spent on a game just to got banned for no reason
I'm literally scared to play, I don't pick up nodes from the ground or talk to anyone since this started
These were things I used to do often but now I've stopped because I'm afraid of being banned.
This was my and my friends' routine and they are getting banned one by one

yNiriel · May 20

Renato90085 wrote: »

I never see this happen in my language
Is this only happen in english?

If your language is Portuguese, yes it can happen
My friend who is also called Renato was banned for using a bad words in porguese in group chat

Frayton · May 20

Pixiepumpkin wrote: »

HOWEVER....ZOS said I STILL have a mark on my account (I think if you get three, you get perma banned).

This scared me so much I spent 15 dollars on a name change to change a character who's name MIGHT be taken out of context that I had used for 6 years and never reported (it was insanely mild).

It's pretty sad when people are not only afraid of ZOS but also give them money to avoid their wrath. I think there might even be a term for that.

Renato90085 · May 20

yNiriel wrote: »

Renato90085 wrote: »

I never see this happen in my language
Is this only happen in english?

If your language is Portuguese, yes it can happen
My friend who is also called Renato was banned for using a bad words in porguese in group chat

nope,I am from asia,and yesterday i test ,i can use my language say most bad word

zaria · May 20

moderatelyfatman wrote: »

There have been numerous cases of people getting banned in private chats too.

A famous case involved someone in a trial group chat using a crude term to joking describe some glowing spheres that appear in vSO hard mode and got banned even though it was in a group chat and no one complained.

In another case in Cyrodill some typed 'Flag' to instruct players to go to the flag but accidentally hit the semicolon instead of the letter l and got autobanned.

That is the problem with F:ag? Some sort of weird code?

moderatelyfatman wrote: »

There have been numerous cases of people getting banned in private chats too.

A famous case involved someone in a trial group chat using a crude term to joking describe some glowing spheres that appear in vSO hard mode and got banned even though it was in a group chat and no one complained.

In another case in Cyrodill some typed 'Flag' to instruct players to go to the flag but accidentally hit the semicolon instead of the letter l and got autobanned.

What is the issue with 'Flag' where l is replaced by : ?
To me the resulting word sounds like nonsense, will not type it here as you did not so it sounds dangerous somehow.

Syldras · May 20

At least the auto-censor (not sure whether it's auto-reported, too) seems to ignore all kinds of punctuation marks as well as spaces. So it even censors part of the message if 2 words (even with a comma or exclamation mark between them) somehow form a "bad" word together. Most often seen when people write in chat to whisper them.

Pixiepumpkin · May 20

Servadei wrote: »

Pixiepumpkin wrote: »

HOWEVER....ZOS said I STILL have a mark on my account (I think if you get three, you get perma banned).

This scared me so much I spent 15 dollars on a name change to change a character who's name MIGHT be taken out of context that I had used for 6 years and never reported (it was insanely mild).

It's pretty sad when people are not only afraid of ZOS but also give them money to avoid their wrath. I think there might even be a term for that.

I can't disagree and the fear is real.

Soarora · May 20

zaria wrote: »

moderatelyfatman wrote: »

There have been numerous cases of people getting banned in private chats too.

A famous case involved someone in a trial group chat using a crude term to joking describe some glowing spheres that appear in vSO hard mode and got banned even though it was in a group chat and no one complained.

In another case in Cyrodill some typed 'Flag' to instruct players to go to the flag but accidentally hit the semicolon instead of the letter l and got autobanned.

That is the problem with F:ag? Some sort of weird code?

moderatelyfatman wrote: »

There have been numerous cases of people getting banned in private chats too.

A famous case involved someone in a trial group chat using a crude term to joking describe some glowing spheres that appear in vSO hard mode and got banned even though it was in a group chat and no one complained.

In another case in Cyrodill some typed 'Flag' to instruct players to go to the flag but accidentally hit the semicolon instead of the letter l and got autobanned.

What is the issue with 'Flag' where l is replaced by : ?
To me the resulting word sounds like nonsense, will not type it here as you did not so it sounds dangerous somehow.

To British people, the word refers to cigarettes. To Americans, it’s a slur against gay people. But also at least some gay people have reclaimed the word.
See, these nuances are not going to be picked up by AI. If it’s in dms and, for example, someone I know is gay says that word in reference to themselves then they’re well within their rights to…

Syldras · May 20

Soarora wrote: »

To British people, the word refers to cigarettes. To Americans, it’s a slur against gay people. But also at least some gay people have reclaimed the word.

In theory, everyone can insult themselves how ever they want, it's no one else's business. I see doing it publicly critically, though, for 2 reasons: Everyone could throw around the crassest slurs for "fun" and then just claim it's okay because they belong to that group, which in some cases can't be proven or disproven easily. Bystanders who are confronted with these words as insults regularly might not find hearing them to be empowering at all.

Private conversation, of course, is something different, but then the only thing that matters is consent anyway, no matter what topics are discussed, themes roleplayed, or swear words used.

Anyway, I generally don't think there should be some auto-report function in general. If something comes across as insulting or inappropriate, people can just report it. I'd rather see a focus on reacting on these reports in a reasonable way.