How will the AI that monitors our in-game chats effect RP?

brylars

This can be answered by the mods and closed if necessary by them after they answer.

In a time when gaming studios use AI more and more to mod chat, there is a risk that during RP an AI will not be able to discern between normal conversation and a storyline being acted out. Tamriel has many dark figures, cultures, etc. Naturally there are some RPers who create characters that embrace the associated dark themes and language. As a community we welcome help in making our public and private conversations more wholesome and inclusive.

That being said, I personally have a concern how this will affect RPers in their dark stories, those that parallel what already exists in the stories of ESO and other TES games. Also, if it is used in PMs, how will this affect private conversations between two consenting adults or married people?

I know that AI use is a hot topic and it is not my intention to stir up trouble, but this is troubling for that portion of the players that favor this platform for RP and appreciate all of the items created to improve the immersion.

Thank you for your help.

PeacefulAnarchy

Are you just assuming there's an AI that monitor chat, or is this stated somewhere?

brylars

I am just asking how AI will affect RP since some RP has dark themes. Topic and descriptions will come up that might offend people in normal real life conversations:

Themes like:
How Dunmer really treat their slaves
One race's view of another race
Vigilante religious fervor or anti-religious rhetoric
Political intrigue
Plotting murder
Conversation between lovers


The list can go on and on.

alternatelder

brylars wrote: »

I am just asking how AI will affect RP since some RP has dark themes. Topic and descriptions will come up that might offend people in normal real life conversations:

Themes like:
How Dunmer really treat their slaves
One race's view of another race
Vigilante religious fervor or anti-religious rhetoric
Political intrigue
Plotting murder
Conversation between lovers


The list can go on and on.

You didn't answer the question. Where are you getting this info that AI monitors chat?

DigiAngel

Please....ZOS can't even fix the servers....and your asking about tacking on AI? Currently there's no evidence AI is even present in this game. If it actually WAS added, I'm sure it would be terrible as most AI is.

Nitewalker314

There have been reports made this morning in various RP communities on Discord with a screenshotted warning/ban notification, allegedly after colorful comments were made in whispers or solo arena instances, regarding furnishings/NPCs, unseen by anyone else. Leading to the belief that, if no one else has seen the messages, no one could report them (the warning also says that ZOS doesn't require a report), so who caught and flagged the comments? Possibly AI.

There is no official announcement anywhere about an AI bot that I've been able to find.

That being said, it would be terrible for such a thing to exist and just flag people left and right for roleplay versus *actual* harmful, toxic, inappropriate, etc. language.

noblecron

Actually the AI was added. Several friends already got bot messaged warnings despite using lore friendly terms

valenwood_vegan

Is this just a theoretical question? Has there been some announcement that AI is currently, or will imminently be, used to monitor our in-game chats?

Anyway, I doubt this is something that the forum mods would be able to answer in any kind of in-depth way... it sounds like a complex topic that might be best answered by an AI developer... particularly if only theoretical in nature.

Of course *if* such a thing has actually been added to ESO, it would certainly be helpful to get an official statement about it. (If there's evidence of this, it might be helpful to share it here so others on the forums can better understand what's being discussed).

drip_fromtheinkwell

alternatelder wrote: »

brylars wrote: »

I am just asking how AI will affect RP since some RP has dark themes. Topic and descriptions will come up that might offend people in normal real life conversations:

Themes like:
How Dunmer really treat their slaves
One race's view of another race
Vigilante religious fervor or anti-religious rhetoric
Political intrigue
Plotting murder
Conversation between lovers


The list can go on and on.

You didn't answer the question. Where are you getting this info that AI monitors chat?

"AI" doesn't necessarily mean chatGPT, or other generative AI platforms. What they probably mean is, there is now a bot monitoring all communication in the game, private or not, reported or not— and that should concern everyone, but particularly people who participate in RP. TES has a lot of dark themes, including lore around Molag Bal and vampires, and werewolves, to the Dark Brotherhood (crazy that they JUST released DB crates, as a side note). I don't personally participate in Dark RP or the like, but others should be able to. It IS a part of the lore, after all.

tyran404_ESO

yes I've had guild members come out today with them getting warnings and suspensions on their accounts. The biggest suspect one was for said individual making a lewd joke to a consenting friend in pm whom didn't report it. The response they got from CS was to the effect "We monitor all chat channels for inappropriate language and toxicity at all times and if you want to keep using our service, please refrain from doing it again." I would post the CS screenshots but I think that's not allowed on the forums here? Correct me if i'm wrong there.

This has already begun hitting people engaging in Role-play. So i would appreciate any word on this to pass along to my guild that's currently panicking.

Tandor

There are some unacceptable words and phrases that are not permitted under the TOS, and while I respect role-playing and role-players I would be very much against players being able to wriggle out of effective moderation by claiming "but I was just role-playing". It should be perfectly possible for role-players to address mature themes in role-play without breaching the TOS restrictions that everyone else has to follow.

Inari Telvanni

This is a major concern and it almost feels like it has the potential to be a draconian suppression of the themes, however dark, that make up the franchise and the players' free exploration of them through an otherwise lacking online medium. This should be reassessed and fine-tuned.

Nitewalker314

Tandor wrote: »

There are some unacceptable words and phrases that are not permitted under the TOS, and while I respect role-playing and role-players I would be very much against players being able to wriggle out of effective moderation by claiming "but I was just role-playing". It should be perfectly possible for role-players to address mature themes in role-play without breaching the TOS restrictions that everyone else has to follow.

While it is true that we should all follow the Code of Conduct (which also supports roleplayers, yay!), an automatic flagging system lacks the human element to discern context and validity of whatever was flagged.

Folks here are worried about being wrongly flagged, not legitimately. Nor are they supporting the use of "it was just roleplay" as a way to skirt breaking the rules.

Stafford197

tyran404_ESO wrote: »

yes I've had guild members come out today with them getting warnings and suspensions on their accounts. The biggest suspect one was for said individual making a lewd joke to a consenting friend in pm whom didn't report it. The response they got from CS was to the effect "We monitor all chat channels for inappropriate language and toxicity at all times and if you want to keep using our service, please refrain from doing it again." I would post the CS screenshots but I think that's not allowed on the forums here? Correct me if i'm wrong there.

This has already begun hitting people engaging in Role-play. So i would appreciate any word on this to pass along to my guild that's currently panicking.

That’s pretty weird if they’re monitoring private Whispers or even group chats between friends for explicit keywords.

And if the current restrictions are loose, then who’s to say these restrictions will not be tightened in the future? The entire idea of moderating non-reported text (outside of bot message spam) is bad imo.

brylars

Tandor wrote: »

There are some unacceptable words and phrases that are not permitted under the TOS, and while I respect role-playing and role-players I would be very much against players being able to wriggle out of effective moderation by claiming "but I was just role-playing". It should be perfectly possible for role-players to address mature themes in role-play without breaching the TOS restrictions that everyone else has to follow.

The TOS? They know RP. Some used to be part of it. They know the themes they themselves have created and when they are recreated by players, we expect not to be banned for it. So that is the question. Will we be banned for RPing a Mephala worshipper, Dibella worshipper, a Morag Tong agent, Dark Brotherhood assassin, or a noble wanting to remove a ruling party, etc.

This is a simple question needing a simple answer then the topic can go away.

Jsmalls

This. Is. WILD.

The game is rated M17+. If I want to joke around with my friends in-game then let me.

Man the world is becoming ridiculous.

tyran404_ESO

The problem lies in two things. This just doesn't effect RP. Below are two instances I've already had brought forward.

One Instance was a Private unreported PM that compared the far grave bubbles to something else. If neither consenting party in this pm was hurt why the disciplinary action? It is also an M rated game.

Another instance: Player tells a maelstrom arena boss to **** off in spacial chat, no one is around cause its maelstrom arena. They got flagged by the same automod. We wouldn't want the npcs feelings to get hurt i guess?

Secondly if it is a Bot doing? A Bot cannot understand or comprehend context. Which especially effects roleplayers.

Lets do an example of a roleplay scene, it is a combat between heroes and cultists: "Player A watches their friends suffer grievous wounds, they charge the menacing cultists and yell. "**** you!" as they charge with their sword out to end the cultists."

The bot could flag it and issue a automod warning if the example 2 is evidence.

Tandor

Nitewalker314 wrote: »

Tandor wrote: »

There are some unacceptable words and phrases that are not permitted under the TOS, and while I respect role-playing and role-players I would be very much against players being able to wriggle out of effective moderation by claiming "but I was just role-playing". It should be perfectly possible for role-players to address mature themes in role-play without breaching the TOS restrictions that everyone else has to follow.

While it is true that we should all follow the Code of Conduct (which also supports roleplayers, yay!), an automatic flagging system lacks the human element to discern context and validity of whatever was flagged.

Folks here are worried about being wrongly flagged, not legitimately. Nor are they supporting the use of "it was just roleplay" as a way to skirt breaking the rules.

While I've no doubt that monitoring is already, and always has been, automated initially I haven't seen any indication that ZOS intend to drop the any and all human interaction in resulting disciplinary action - including in the appeal process.

My concern about players claiming "it was just roleplay" relates more to those who are not role-playing but claim that as an excuse for breaching the TOS in order to get out of disciplinary action because it's ok to breach the TOS so long as they're role-playing.

None of which is to say that I don't have any concerns about the increased use of AI in our lives, not least having just today had to use a "live chat bot" to ask whether a particular product was was produced in a particular way, only to get the response that unfortunately the company doesn't give free samples. Utterly useless, whereas my follow-up email received a full and detailed staff reply an hour or so later giving me a very helpful answer to the question.

Sallymen

Not sure when or how they implemented this AI moderation but No automated system nor your average moderator for that matter will be able to detect jokes, context, or simple misunderstandings. Which is why it is a bad idea to implement it in the first place.

Example: Two friends joke around and insult each other in whisper - AI moderation sees that the friends say offensive words in tell - AI moderation hits one or both friends of a warning. Because AI moderation is not able to detect that these two friends are joking or just making fun of each other for the hell of it.

Syldras

Tandor wrote: »

There are some unacceptable words and phrases that are not permitted under the TOS, and while I respect role-playing and role-players I would be very much against players being able to wriggle out of effective moderation by claiming "but I was just role-playing". It should be perfectly possible for role-players to address mature themes in role-play without breaching the TOS restrictions that everyone else has to follow.

I think it's clear from the context whether something is roleplay or not. And no, I don't consider walking up to a random stranger and insulting them with a TES slur roleplay. But if two or more people have a longer conversation acting as their characters, it's usually very distinguishable to a real, out of character conversation. At least for human mods it would be.

If AI is/was used, it will lead to problems. It will definitively not understand the difference between a real conversation and roleplay. The current automatic word filter is ridiculously faulty already (always has been since I started playing this game 8 years ago) and very often filters totally harmless words because it also "recognizes" them within other words and doesn't even take punctuation or spaces into consideration. It's basically a sequence of letters that the filter reacts on, nothing more. And if it's an insult with 3 or 4 letters only (there are such), this sequence of letters will show up very often in all kinds of other words and sentences. Not even taking into consideration other languages that might have closely looking absolutely harmless words that the filter also sees as insults or slurs. Try writing the word for "black" in Spanish, the word for "less" in German... If you write the German word for "smoked turkey ham", you even have 2 insults in one word

TheBardAtTheInn

It wasnt roleplay, but a friend of mine got a 3-day ban for whispering me a single mean name. No context to be seen in previous chats, nothing else in the message because we were in voice chat. She wasn't being actually malicious towards me, just bantering, but she still caught a ban for it without being reported by the only person that would've seen it (me).

For clarity, she said the term that the Shrek character "Lord Farquaad"s name is a play on.

jaws343

drip_fromtheinkwell wrote: »

alternatelder wrote: »

brylars wrote: »

I am just asking how AI will affect RP since some RP has dark themes. Topic and descriptions will come up that might offend people in normal real life conversations:

Themes like:
How Dunmer really treat their slaves
One race's view of another race
Vigilante religious fervor or anti-religious rhetoric
Political intrigue
Plotting murder
Conversation between lovers


The list can go on and on.

You didn't answer the question. Where are you getting this info that AI monitors chat?

"AI" doesn't necessarily mean chatGPT, or other generative AI platforms. What they probably mean is, there is now a bot monitoring all communication in the game, private or not, reported or not— and that should concern everyone, but particularly people who participate in RP. TES has a lot of dark themes, including lore around Molag Bal and vampires, and werewolves, to the Dark Brotherhood (crazy that they JUST released DB crates, as a side note). I don't personally participate in Dark RP or the like, but others should be able to. It IS a part of the lore, after all.

Monitoring keywords does not equal AI or bots... Setting up a monitor on all in-game coms to flag specific words used in chat literally requires zero AI. It's no different than a basic word sensor on forums or in character creation. A simple query against a database of flagged words that returns a simple result.

Inari Telvanni

TOS says,

" Content Moderation

To the extent that ZeniMax performs any content moderation of UGC to ensure its compatibility with these Terms of Service (including the Code of Conduct or any relevant EULA), such content moderation may be carried out via human review as well as through the use of AI-powered proactive and reactive moderation methods including without limitation, software that uses algorithmic decision making.

ZeniMax's proactive content moderation includes without limitation using tools to block and filter UGC that is illegal and/or incompatible with these Terms of Service.

Reactive content moderation methods include without limitation user reporting features which allow You to inform ZeniMax of the behavior or Content of other users that You have encountered which you believe is illegal and/or incompatible with these Terms of Service (including the Code of Conduct) and any such behaviour or Content can be reported to ZeniMax by contacting ZeniMax Customer Support at help.bethesda.net or help.elderscrollsonline.com. If You are in-Game, You can report an issue using an in-Game help feature where applicable. Where ZeniMax is required to do so by virtue of the Statutory Obligations (as defined in Section 1), ZeniMax shall advise You of remedial steps action taken against another user as a result of Your report including details of what steps ZeniMax has taken to investigate your report, if ZeniMax has removed Content that You have reported or if any other restrictions have been applied to the Content or the other user. "

This is an enormous red flag.

Kashya_Vulano

jaws343 wrote: »

Monitoring keywords does not equal AI or bots... Setting up a monitor on all in-game coms to flag specific words used in chat literally requires zero AI. It's no different than a basic word sensor on forums or in character creation. A simple query against a database of flagged words that returns a simple result.

Here's a snippet from the TOS. It doesn't have to mean AI, you're right, but this is also in the terms of service.

jaws343

Inari Telvanni wrote: »

TOS says,

" Content Moderation

To the extent that ZeniMax performs any content moderation of UGC to ensure its compatibility with these Terms of Service (including the Code of Conduct or any relevant EULA), such content moderation may be carried out via human review as well as through the use of AI-powered proactive and reactive moderation methods including without limitation, software that uses algorithmic decision making.

ZeniMax's proactive content moderation includes without limitation using tools to block and filter UGC that is illegal and/or incompatible with these Terms of Service.

Reactive content moderation methods include without limitation user reporting features which allow You to inform ZeniMax of the behavior or Content of other users that You have encountered which you believe is illegal and/or incompatible with these Terms of Service (including the Code of Conduct) and any such behaviour or Content can be reported to ZeniMax by contacting ZeniMax Customer Support at help.bethesda.net or help.elderscrollsonline.com. If You are in-Game, You can report an issue using an in-Game help feature where applicable. Where ZeniMax is required to do so by virtue of the Statutory Obligations (as defined in Section 1), ZeniMax shall advise You of remedial steps action taken against another user as a result of Your report including details of what steps ZeniMax has taken to investigate your report, if ZeniMax has removed Content that You have reported or if any other restrictions have been applied to the Content or the other user. "

This is an enormous red flag.

No it is not..

Zos, as a company that can be held liable for conversations that might happen through the use of their platform, have a responsibility to make sure that they can monitor illegal things being done in their game. Even if those things are done in private chats. If you are using their tools to do illegal things, they are legally put in a spot to intervene or face severe legal repurcussions if they do not.

gamma71

If AI monitored cryodil every one would be banned the first 15 minutes it took effect.😁

Inari Telvanni

jaws343 wrote: »

Inari Telvanni wrote: »

TOS says,

" Content Moderation

To the extent that ZeniMax performs any content moderation of UGC to ensure its compatibility with these Terms of Service (including the Code of Conduct or any relevant EULA), such content moderation may be carried out via human review as well as through the use of AI-powered proactive and reactive moderation methods including without limitation, software that uses algorithmic decision making.

ZeniMax's proactive content moderation includes without limitation using tools to block and filter UGC that is illegal and/or incompatible with these Terms of Service.

Reactive content moderation methods include without limitation user reporting features which allow You to inform ZeniMax of the behavior or Content of other users that You have encountered which you believe is illegal and/or incompatible with these Terms of Service (including the Code of Conduct) and any such behaviour or Content can be reported to ZeniMax by contacting ZeniMax Customer Support at help.bethesda.net or help.elderscrollsonline.com. If You are in-Game, You can report an issue using an in-Game help feature where applicable. Where ZeniMax is required to do so by virtue of the Statutory Obligations (as defined in Section 1), ZeniMax shall advise You of remedial steps action taken against another user as a result of Your report including details of what steps ZeniMax has taken to investigate your report, if ZeniMax has removed Content that You have reported or if any other restrictions have been applied to the Content or the other user. "

This is an enormous red flag.

No it is not..

Zos, as a company that can be held liable for conversations that might happen through the use of their platform, have a responsibility to make sure that they can monitor illegal things being done in their game. Even if those things are done in private chats. If you are using their tools to do illegal things, they are legally put in a spot to intervene or face severe legal repurcussions if they do not.

If I'm getting banned for being crass in whispers then, yes, that is a red flag. Nothing illegal about swearing at an enemy NPC or making a lewd remark about Fargrave water globules.

SkaraMinoc

There are some cases where chat logs might need to be broadly scanned for law enforcement purposes. Video game chat isn't a private, encrypted chat where people can send any message content they want. There are other tools for that. If something is found by AI, the game company's Customer Service team would do a manual review.

Syldras

The question is what exactly the TOS say: Is writing certain things forbidden, or is pestering other people?

I mean, it might make no sense logically (if we make the assumption that rules exist not to harm others), but if the TOS say you're forbidden to write certain types of things, I guess you're forbidden to write them ( = to make input and bring them on the server), even if you only whisper to unbothered friends or talk to yourself in a instance where you're alone.

Do I find that useful? No. Not at all. For me, it's about harm done or not done. Also: Context matters. And word filters and AI are still very faulty.

I'm just stating how a the wording in the TOS would make a difference.

Arunei

It's been ages since I've even looked at this forum, let alone posted in it, but I think this is a good topic to break my posting fast on.

@ZOS_GinaBruno Please for the love of everything tell us there are plans to change this. People should NOT be getting autobans from a MACHINE that CANNOT TELL CONTEXT. I thought the devs were RP friendly? =/ People getting banned for use of lore-friendly terms? People getting banned for talking in a chat where no one else is? People getting banned for doing adult-themed RP in a Mature rated game?

Ain't no way.

ragnarok6644b14_ESO

The biggest thing an AI/algorithm cannot filter for is consent of the 'receivee'. The biggest thing for the RP community is 'OOC consent' - that is to say 'out of character agreement to participate/exist within the subject matter at hand'. This means that regardless of whether or not a character consents (say, a paladin busting in on a Mephala cult conducting some ritual), the player behind the paladin (on the keyboard) is not offended by the imagery that the scene includes.

A machine cannot determine whether the player behind the keyboard consents to the activity/messaging/descriptions before them on the screen.

Protecting people from things they themselves consent (or even *wish*) to bear witness to is where, in my opinion, moderation crosses the line. This isn't a case where one person objects out-of-character and the others 'force' or 'coerce' them to go along (any indication of non-consent out of character crosses the line and almost every RPer I've ever met would stop the RP at once in that case).