How will the AI that monitors our in-game chats effect RP?

Ugrak

"ESO AI" is likely to return a lot of results about AI for analysis of data from the Earth System Observatory.

Might be a few articles referring back to the forum and reddit discussions but I doubt there is much more info out there as this is quite new.

Hopefully when an official answer does get posted it'll just turn out to be some kind of unintentional behavior across the systems and people responsible for processing this.

The_Boggart

Be careful about nursery rhymes as well
I got a ban by a bot once for mentioning 3 little d***y birds, these bots never read context

WuffyCerulei

“AI” in that way usually just means code or something that automatically monitors for certain words, but it’s a catch-term cuz it’s all the rage for companies rn. It’s not the same as the actual AI that’s used in gaming for things like NPCs, bosses, etc. I mean there should be some monitoring though, considering some people are WAY too comfortable dropping real slurs and whatnot with no repercussions. In-game or lore-based terms shouldn’t be “banned” though. The Elder Scrolls series has always been nitty-gritty with certain topics, and no one should be punished for following that. Though of course there are lines even RPers should not cross.

pklemming

Calling this AI is also a misnomer. It is not a LLM, just a bot.,

jle30303

The_Boggart wrote: »

Be careful about nursery rhymes as well
I got a ban by a bot once for mentioning 3 little d***y birds, these bots never read context

Now imagine if you were quoting the fairy story or pantomime about a certain former Mayor of London of medieval times, Mr Richard "***" Whittington and his famous cat...

jle30303

oops, somehow posted twice. And guess what, it censored his nickname, which was (and indeed still is) a common abbreviation of Richard.

[EDIT] And weirdly enough, when I removed the asterisks and re-edited it, it wasn't censored. Goodness only knows why this happens, or did not happen.

[EDIT AGAIN] And now returning to the thread, it's censored the nickname again!

Which only goes to show: Automation and AI does not understand context, and should NEVER be used to censor things. Not in chat, and not in private.

Arunei

Kappachi wrote: »

AltmerGF wrote: »

Can anecdotally say I was using some less than wholesome language with my friends while playing this weekend and haven't gotten banned, so hopefully we're just looking at some sort of overzealous moderation glitch

even amongst private chats you agreed to a TOS when you started the game, if you're throwing out the real world equivalents of n'wah/s'wit/etc then you're bound to get banned. keep it civil in-game even amongst friends.

Absolute NOT. In what world does it make sense where consenting individuals, who are okay with crude or lewd languages and speaking in PRIVATE CHANNELS, are getting banned for banter and joking with each other? It doesn't affect you or anyone else if a friend of mine playfully calls me a word that rhymes with itch because of an RP twist for example, and I'd much prefer if what is MY business stays MY business.

Look at the nature of the franchise's universe for crying out loud. I've already gone over this point and so have others. It's rated M for mature language and themes IN THE GAME UNIVERSE ITSELF. If the game doesn't have to remain civil why should we? The game is allowed to say ""naughty"" words and reference dark themes at will but us players have to use PG talk even amongst private chats?

And again, consider that there are words in English that are innocent to us but are offensive in some way in other languages, and vice-versa. People are getting banned for using LORE-FRIENDLY TERMS. How in the world is that justifiable? In what world does it make sense the game can say a word but players CAN'T?

There. Is. No. World. Where a bot or AI that CANNOT read context should be giving warnings, suspensions, or bans. No one needs to "remain civil" in private chats with their friends that are all totally fine with bantering that includes name-calling and colorful language. People are getting hit for ""naughty words"" in private instances like MA for heck's sake. What, do we need to be civil to the mobs so we don't hurt their feelings before we kill them?

Syldras

Found only one magazine article (that was also made into a Youtube video), to be honest. If there are more: Why not linking them all here? It would certainly show ZOS how urgent the situation feels for players.

(Although we can discuss about the quality of journalism when articles already claim "AI moderation" as a fact when there's nothing but a suspicion yet. But that's a different topic.)

Elsonso wrote: »

Personally, I still think this is "business as usual" for ZOS. They have always screened what people say, in every venue that they are present in, and they use pattern matching to pull out naughty words. They have banned or suspended people in the past for it. These actions have been long been rumored to have an automated component to them. The only thing new here is that people are blaming "AI" for this, and I am not certain that actually represents much of a change, if it is even relevant in this context.

Can't make it part of the AI scare without claiming it was AI

The_Boggart wrote: »

Be careful about nursery rhymes as well
I got a ban by a bot once for mentioning 3 little d***y birds, these bots never read context

There once was a man from Balmora...
But I won't tell you anything about him, just in case rhymes are evil in general

wilykcat

Hot take: I'm fine with the ai. The rule is simple. Don't post anything inappropriate in the chat = no penalty.

On the otherhand, this is why human moderators are needed. To review what the ai does and to make sure those messages in the chat are an actual offense or not.

CoolBlast3

wilykcat wrote: »

Hot take: I'm fine with the ai. The rule is simple. Don't post anything inappropriate in the chat = no penalty.

On the otherhand, this is why human moderators are needed. To review what the ai does and to make sure those messages in the chat are an actual offense or not.

Oh come on. So RPers can't RP anything beyond "My Little Pony" without getting flagged? I understand being weary of explicit mentions of like Molag's sphere in public hubs or w/e, but most RPers RP in housing anyways these days. So you're telling me we deserve to have a bot flag us (and getting seemingly banned/account striked) for RP where everyone consents to be there?

This is a PG18/M Rated game. If people are so scared of naughty words they should go play Wizard101

opallithia

Still nothing from ZOS, huh? https://massivelyop.com/2024/09/16/roleplaying-elder-scrolls-online-players-raise-red-flags-over-automated-chat-moderation/

Come on guys.

Syldras

wilykcat wrote: »

Hot take: I'm fine with the ai. The rule is simple. Don't post anything inappropriate in the chat = no penalty.

Where would you draw the line when it comes to "inappropriate"? In my opinion, players should be able to discuss the things happening in game with each other, including the content of quests. And some quests are gruesome.

SilverBride

@ZOS_Kevin Is there any word yet? This is very disturbing and we really need to know what is going on. Thanks.

Onomog

SilverBride wrote: »

@ZOS_Kevin Is there any word yet? This is very disturbing and we really need to know what is going on. Thanks.

I'm not optimistic that, *if they do say anything*, it will be either informative or reassuring.

I'm eager to be proven wrong, though.

Tazzy

As a supporter, I would be cautious about making a statement now.
Meanwhile this is more of a matter for the legal department than for support.

thorwyn

Hot take: I'm fine with the ai. The rule is simple. Don't post anything inappropriate in the chat = no penalty.

Hot take: what you are personally fine with is irrelevant.
The majority of adult people don't need a super nanny watching their personal communications, ESPECIALLY not when it is a private, consensual conversation between a limited number of people. I'm paying ZOS to play a game, not to monitor my language in 1v1 chats with my friends. How I am talking to my friends privately is none of someone's business.
As we have seen, the "rules" are not that simple.

manukartofanu

Arunei wrote: »

Kappachi wrote: »

AltmerGF wrote: »

Can anecdotally say I was using some less than wholesome language with my friends while playing this weekend and haven't gotten banned, so hopefully we're just looking at some sort of overzealous moderation glitch

even amongst private chats you agreed to a TOS when you started the game, if you're throwing out the real world equivalents of n'wah/s'wit/etc then you're bound to get banned. keep it civil in-game even amongst friends.

Absolute NOT. In what world does it make sense where consenting individuals, who are okay with crude or lewd languages and speaking in PRIVATE CHANNELS, are getting banned for banter and joking with each other? It doesn't affect you or anyone else if a friend of mine playfully calls me a word that rhymes with itch because of an RP twist for example, and I'd much prefer if what is MY business stays MY business.

Look at the nature of the franchise's universe for crying out loud. I've already gone over this point and so have others. It's rated M for mature language and themes IN THE GAME UNIVERSE ITSELF. If the game doesn't have to remain civil why should we? The game is allowed to say ""naughty"" words and reference dark themes at will but us players have to use PG talk even amongst private chats?

And again, consider that there are words in English that are innocent to us but are offensive in some way in other languages, and vice-versa. People are getting banned for using LORE-FRIENDLY TERMS. How in the world is that justifiable? In what world does it make sense the game can say a word but players CAN'T?

There. Is. No. World. Where a bot or AI that CANNOT read context should be giving warnings, suspensions, or bans. No one needs to "remain civil" in private chats with their friends that are all totally fine with bantering that includes name-calling and colorful language. People are getting hit for ""naughty words"" in private instances like MA for heck's sake. What, do we need to be civil to the mobs so we don't hurt their feelings before we kill them?

It would be fair if private chats in the game were not accessible by default. But since they are accessible, it's obviously unclear to the bot who is a friend and who isn't. In a system where absolutely anyone can send you a private message saying anything at all, monitoring private chats is quite justified. It probably just needs to be implemented differently. Although the alternative is apparently just banning based on any report without investigating, and that's not great either.

thorwyn

It would be fair if private chats in the game were not accessible by default. But since they are accessible, it's obviously unclear to the bot who is a friend and who isn't. In a system where absolutely anyone can send you a private message saying anything at all, monitoring private chats is quite justified.

If anyone is offended by a private message, they can report the offense and let ZOS decide based on the chatlogs AFTERWARDS. It is NOT a justification to tolerate an intrusion of privacy.

jaws343

thorwyn wrote: »

It would be fair if private chats in the game were not accessible by default. But since they are accessible, it's obviously unclear to the bot who is a friend and who isn't. In a system where absolutely anyone can send you a private message saying anything at all, monitoring private chats is quite justified.

If anyone is offended by a private message, they can report the offense and let ZOS decide based on the chatlogs AFTERWARDS. It is NOT a justification to tolerate an intrusion of privacy.

You are playing a game and chatting in someone else's platform, literally the only privacy you are guaranteed in private chat is privacy from other players. Zos shouldn't, and doesn't have to, tolerate their game chat system being used for communication they do not condone, even if it is done under the guise of role play or under a "shield" of acceptance by the parties involved in the conversation.

thorwyn

So, would you say the same if it wasn't ZOS but your ISP that's monitoring your private chats? Or your phone company? After all, you are just using their plattform and they don't have to tolerate their chat system being used for communication they do not condone.

Edit:
At the end of the day: if ZOS thinks that it is more important to survey private chats of their players in search of potential "nono" words than to actually sell a game... fine. I'm not forcing them to take my money.

Syldras

manukartofanu wrote: »

In a system where absolutely anyone can send you a private message saying anything at all, monitoring private chats is quite justified.

What happened to self-responsibility? Why would someone else have to shield me from things that a hypothetical someone might find insulting or crass or whatever without giving me the possibility to decide for my own whether I feel disturbed by it or not? A report button is all that is needed.

jaws343 wrote: »

Zos shouldn't, and doesn't have to, tolerate their game chat system being used for communication they do not condone, even if it is done under the guise of role play or under a "shield" of acceptance by the parties involved in the conversation.

There's some truth in this... In the end, we're using ZOS' plattform and ZOS can decide what kinds of content they want to accept. So if they don't want, let's say, certain roleplay topics, even if played consensually, they could restrict that. Fine.

But the the autofiltering like it currently is still doesn't work. It's too faulty. It has too many false positives, especially when it comes to completely normal words in other languages that are marked as "bad" by the word filter because they look like a slur in English. Something as faulty as that should not be able to autoban people. It's unavoidable that a real person checks and decides.

xclassgaming

Still not seeing any proof this is happening, im starting to think this is just a lie spread by someone who got banned and isn't telling the whole or entire truth.

ArchangelIsraphel

It's a rather dystopian feeling to step away from your favorite game to deal with IRL things for a little while, only to return to find something like this has happened.

I feel like I've stepped back in time to when I was 17 years old in high school, and had to watch every little thing I said within earshot of an adult because I could get a weekend detention for having a civil discussion about adult themes. And by adult themes, I do not mean anything of a romantic nature (I'll pick my euphemisms with caution- after all, I wouldn't want to use proper biological terms, lest they be regarded as inappropriate =_=) I mean discussions regarding historical events or plot lines in stories that had difficult themes. Even though both myself and my friends were consenting to the discussion, we were frequently told off for discussing -history-. History! A consensual discussion!

And that's an important factor which is being overlooked by the company. Consent. This quote from SuspensionDispersingAutomaton says it all:

On September 14, I received a 72 hour suspension for something I know for a fact no one reported and was said by me to my fully consenting friend in a private party.
When attempted to explain that and appeal the ban, the customer support declined and kept the ban on my account, sending me a generic reply, telling me that every private chats in this game are subject to automatic moderation and don't require reports, and that next ban could be permanent. To say this is highly disturbing is to say nothing.

Private chats don't require reports. This perhaps one of the most important points in this entire discussion, and the most disturbing.

So, a "victim" is not required for it to be assumed that wrongdoing has taken place? No one has reported what was said- no one was hurt, no one was made to feel uncomfortable, no one even wanted a report to take place. And yet accounts are being actioned against to protect them from something they desired no protection from, because they consented to the conversation that was taking place?

No one is accusing anyone of wrongdoing- and yet we have a bot deciding a violation of the TOS has taken place simply because of what was said, rather than because of the context in which it was said. When no one within the conversation even desires the protection the bot supposedly provides.

This essentially robs players of personal agency. We are adults, perfectly capable of consenting to conversations which contain difficult themes, the very themes which are explored within the story told in the game itself. How are we to discuss the plot, or even ask other users about the content of a quest, if we must fear being banned for doing so? Not only role play has been effected by this, but discussion as a whole. Banter between friends, and entire facets of different relationships that could develop with the game (between characters, or real people) are truncated by the looming presence of automated moderation.

I have probably repeated several points that have already been made in this thread. But I would respectfully like to ask the moderation team to take our criticism into consideration- we are not children, or teenagers. We need to be able to explore the themes within the game, both good and bad, and converse naturally between one another in private in order for the game to function. For socialization between players to take place at all, we must feel comfortable and secure. By using such heavy handed, automatic moderation, you are interfering with our ability to form friendships naturally, to express ourselves, and to interact with the game world at all.

It isn't for you to decide how two consenting adults should speak to one another in private. If neither is uncomfortable, if neither wants or desires intervention, then none should take place.

Elsonso

opallithia wrote: »

Still nothing from ZOS, huh? https://massivelyop.com/2024/09/16/roleplaying-elder-scrolls-online-players-raise-red-flags-over-automated-chat-moderation/

Come on guys.

I am not certain how urgent a response will be, given that so many years have gone by with little notice.

spartaxoxo

Even if they decide it's needed to monitor chats for illegal activity, it does not make sense for the banning to be automated or for that to be misused to decide that two adult players obviously joking around with one another is ban worthy. Such a tool should be extremely narrowly applied.

Sepultura_13

xclassgaming wrote: »

Still not seeing any proof this is happening, im starting to think this is just a lie spread by someone who got banned and isn't telling the whole or entire truth.

I agree. Someone has successfully managed to troll the forums with a wild conspiracy theory, it seems...

valenwood_vegan

xclassgaming wrote: »

Still not seeing any proof this is happening, im starting to think this is just a lie spread by someone who got banned and isn't telling the whole or entire truth.

Agree. I had a little free time this morning and searched through various reddit threads about this, as people may understandably be reluctant to post screenshots on the forums... after an hour, was unable to find anything concrete except a link to a now deleted screenshot that was allegedly an email from zos rejecting a user's appeal over being banned.

To be clear, *if* zos is actually doing what's been alleged, I think it's a terrible idea and needs to be reconsidered. But it's hard to get outraged about a rumor. Hoping that official statement from ZoS is still coming and will help clear things up.

xclassgaming

valenwood_vegan wrote: »

xclassgaming wrote: »

Still not seeing any proof this is happening, im starting to think this is just a lie spread by someone who got banned and isn't telling the whole or entire truth.

Agree. I had a little free time this morning and searched through various reddit threads about this, as people may understandably be reluctant to post screenshots on the forums... after an hour, was unable to find anything concrete except a link to a now deleted screenshot that was allegedly an email from zos rejecting a user's appeal over being banned.

To be clear, *if* zos is actually doing what's been alleged, I think it's a terrible idea and needs to be reconsidered. But it's hard to get outraged about a rumor. Hoping that official statement from ZoS is still coming and will help clear things up.

Oh yeah i 100% agree with you. If it IS true, ZOS needs to stop this asap and the people who were banned rping (not the ones who were probably shouting slurs in zone chat or harassing people; they should stay banned) should be unbanned asap.

But yeah, i have looked about on the forums; reddit. Nothing actually pointing to this being true, so until i see hard proof i'm not buying it; i think someone made this up to excuse their bad actions most likely and it's turned into a frenzy; which is why ZOS should reply and clear it up. Again; if this is true then like i said needs to be rectified asap, but for now; i don't buy it.

Syldras

xclassgaming wrote: »

Oh yeah i 100% agree with you. If it IS true, ZOS needs to stop this asap and the people who were banned rping (not the ones who were probably shouting slurs in zone chat or harassing people; they should stay banned) should be unbanned asap.

Everyone, as long as it was consensual, no matter if rp or other types of conversation.

Ugrak

xclassgaming wrote: »

Nothing actually pointing to this being true, so until i see hard proof i'm not buying it; i think someone made this up to excuse their bad actions most likely and it's turned into a frenzy; which is why ZOS should reply and clear it up. Again; if this is true then like i said needs to be rectified asap, but for now; i don't buy it.

Yeah I've so far reserved my judgement. Still subscribed and playing as normal. I do enjoy this game, imperfect as it is.

The truth is in a kind of quantum superposition state until something concrete collapses it into a definable point. Though the fact that it is believable is worrisome on it's own. That may of course be more a sign of the times overall, but deserved or not, ZOS does have a reputation for aggressive moderation, lending it plausibility.

If there is no truth to it, it should be easy for ZOS to lay it to rest.

And if it does turn out to be a real thing that has occurred, there's still the question whether it is a bug or issue with a third-party. So yeah, Schrödinger's scandal for now.