Is AI Automatic Moderation Killing the RP Scene?

Ruthless · December 21

Let us know your thoughts?

TaSheen · December 21

Don't know. I don't RP.

[Clarification: I RP my girls in my own mind; I do not interact with others in game.]

Destai · December 21

Yup

It certainly makes me nervous to use in game chat.

Avran_Sylt · December 21

Yup

I don’t RP.

But if AI automatically moderates chat without any user reports, then it also cannot tell the difference between consensual conversation between two parties, and non-consensual conversation.

As such, all parties subject to any inappropriate language are victims, and all parties using such language are aggressors, even if such a scenario was consensual.

Basically, it lacks context. And I assume RP is all about that (Having assigned roles and all that).

I guess this would hurt more lore-based RP (racism abound), heavy topics being discussed in-game (not necessarily even RP), and ERP (for obvious reasons).

I’ve only heard about this Auto-AI chat moderation getting people banned from snippets here and there on this forum and Reddit, so I cannot say how much of an issue it really is (or how often players just use something like discord).

But at a fundamental level it does cause chat to become a minefield even within a trusted group.

Servadei · December 21

Yup

It's not just the rp community but players in general are scared to use chat.

Madarc · December 21

Yup

I have little contact with RP groups at the moment, but the little I do get to see has already changed, or only takes place to a very limited extent ingame..

ArchangelIsraphel · December 21

Yup

My guild has taken all RP outside of the game entirely to avoid the AI system misinterpreting a post. The themes of our stories involve war, political upheaval, and betrayal between the alliances- which of course leads to (entirely fabricated) arguments between characters and strong words being thrown back and fourth. As we do not want any of this to be seen as real when it is not, we have totally withdrawn all chat activity from inside of the game.

We used private housing, guild chat, and party chat and did not publicly roleplay. However, seeing as we no longer feel safe roleplaying within the game, we also no longer recruit and our events have become rather limited.

Sakiri · December 21

Yup

If it's not in a public area it shouldn't be moderated. My house? Safe space. I can dictate if people came come in. Groups, voluntary. Don't like the chat, leave. Instances? Same thing. Whispers? We're perfectly capable of reporting them.

They can do this but not stop invite bots.

manukartofanu · December 21

Nope

Where did you even get the idea that there is some sort of AI moderation here?

Varana · December 21

Yup

Not here, in game.

Avran_Sylt · December 21

Yup

manukartofanu wrote: »

Where did you even get the idea that there is some sort of AI moderation here?

I’m tempted to go into a solo dungeon instance and see if I can’t trigger auto moderation myself (with innocent misspellings as if I were typing fast) myself, to see if it’s actually a thing or not.

If I get flagged for text in a place where literally no one around sees the text, that’s heavily indicative of overreach.

Marto · December 21

Nope

I don't think so. And it doesn't seem like most of the RP community thinks so either.

I was there when it happened. Literally in the same Discord channel as the user whose case started this controversy when they received a warning on their account (not even a ban). If I recall correctly, it took less than 48h for the warning to be removed, and over the next couple days the other affected users were unbanned.

Many of us were certainly scared, but as more and more information came out it became clear this was nothing more than a string of false positives caused by overly strict moderation guidelines. Pretty much exactly what Rich Lambert confirmed.

It's not the first time that happens, either. I've been in the RP community since 2015, and I remember around 2019 moderation got a bit more strict, leading to some friends getting temporary bans when RPing mature themes. But that was short lived, after a few months things calmed down.

ZOS moderation guidelines are stricter than they should be. I agree with that. And by all means, bring attention to it. But don't act as if this is ZOS purposely trying to ruin your day. False positives suck. They really do. I get it. But they happen because the devs want to catch people who are genuinely toxic.

Right now, there doesn't seem to be many, if any cases. I've not seen anyone in big RP communities getting wrongfully banned for a couple months now. I don't think we have anything to worry about.

Avran_Sylt · December 21

Yup

Marto wrote: »

I don't think so. And it doesn't seem like most of the RP community thinks so either.

I was there when it happened. Literally in the same Discord channel as the user whose case started this controversy when they received a warning on their account (not even a ban). If I recall correctly, it took less than 48h for the warning to be removed, and over the next couple days the other affected users were unbanned.

Many of us were certainly scared, but as more and more information came out it became clear this was nothing more than a string of false positives caused by overly strict moderation guidelines. Pretty much exactly what Rich Lambert confirmed.

It's not the first time that happens, either. I've been in the RP community since 2015, and I remember around 2019 moderation got a bit more strict, leading to some friends getting temporary bans when RPing mature themes. But that was short lived, after a few months things calmed down.

ZOS moderation guidelines are stricter than they should be. I agree with that. And by all means, bring attention to it. But don't act as if this is ZOS purposely trying to ruin your day. False positives suck. They really do. I get it. But they happen because the devs want to catch people who are genuinely toxic.

Right now, there doesn't seem to be many, if any cases. I've not seen anyone in big RP communities getting wrongfully banned for a couple months now. I don't think we have anything to worry about.

Oh interesting, wasn’t sure this was something that was addressed by mods already, I’ve only heard hearsay. There a thread with the responses?

NotaDaedraWorshipper · December 21

Yup

manukartofanu wrote: »

Where did you even get the idea that there is some sort of AI moderation here?

There's a lot of evidence with people being banned for no reason. Most recent was some kids uncle or such on reddit, who gifted ESO to his niece and nephew and both were banned for cheating while one was in the tutorial and the other in character creator.

spartaxoxo · December 21

NotaDaedraWorshipper wrote: »

manukartofanu wrote: »

Where did you even get the idea that there is some sort of AI moderation here?

There's a lot of evidence with people being banned for no reason.

There's definitely something happening that seems to be having a very negative impact on new players. No idea what. But I've seen mention of it on Steam too.

Ugrak · December 22

NotaDaedraWorshipper wrote: »

manukartofanu wrote: »

Where did you even get the idea that there is some sort of AI moderation here?

There's a lot of evidence with people being banned for no reason. Most recent was some kids uncle or such on reddit, who gifted ESO to his niece and nephew and both were banned for cheating while one was in the tutorial and the other in character creator.

That's unlikely to be AI chat moderation.

Rather, there have been cases where the anti-cheat software of other games have been in conflict with that of ESO; triggering early bans such as that.

The evidence for AI chat moderation is the people sanctioned for private chats where no player reported anything. According to ZOS, it is not fully automated, but uses AI to monitor and flag concerning chat for a human to review. Presumably outsourced to some large customer service ticket handling outfit, due to the manpower required to process the output of an MMO in a timely manner.

Not much better than full automation through AI, as understanding of context may be limited at best. Not to mention the potential that if the subcontractor management is pushing workers to close more tickets faster or some such KPI-chasing, they may just default to quickly going along with whatever the automated flagging says.

Arunei · December 22

They've said it's not AI that filters through the chat, but rather a bot that looks through offline copies of chat (it's supposedly not scanning live chats) and if it flags something it then forwards that to an actual person for review.

I still don't think it's necessary to the degree they've made it, where people were (maybe still are) getting warned or banned for saying things in private instances. Someone in one of my RP guilds got a 72 ban I think it was for saying a few choice words about the whole thing in one of his own player houses, with no one else around. It was something akin to "we don't need to be censored" but with some curse words thrown in. Nothing offensive or insulting towards anyone, just ranting to the void, and he got actioned for it.

I'm hearing less about it now, maybe that's because they're finally getting on CS to ONLY action accounts that are actually doing something bad like harassing others or discussing doing illegal things irl, rather than taking action just because they feel offended over something or because they're having a bad day.

Sleepsin · December 22

Everytime I play I see gold seller spam directing people to their website. It's the same message written the same way everytime. You would think the AI moderation would be able to detect that.

NotaDaedraWorshipper · December 22

Yup

Ugrak wrote: »

NotaDaedraWorshipper wrote: »

manukartofanu wrote: »

Where did you even get the idea that there is some sort of AI moderation here?

There's a lot of evidence with people being banned for no reason. Most recent was some kids uncle or such on reddit, who gifted ESO to his niece and nephew and both were banned for cheating while one was in the tutorial and the other in character creator.

That's unlikely to be AI chat moderation.

Rather, there have been cases where the anti-cheat software of other games have been in conflict with that of ESO; triggering early bans such as that.

The evidence for AI chat moderation is the people sanctioned for private chats where no player reported anything. According to ZOS, it is not fully automated, but uses AI to monitor and flag concerning chat for a human to review. Presumably outsourced to some large customer service ticket handling outfit, due to the manpower required to process the output of an MMO in a timely manner.

Not much better than full automation through AI, as understanding of context may be limited at best. Not to mention the potential that if the subcontractor management is pushing workers to close more tickets faster or some such KPI-chasing, they may just default to quickly going along with whatever the automated flagging says.

The weird chat moderation is definitely the biggest evidence, but as mentioned there are multiple. It's a lot of weird moderation going on in the game recently that should not happen if it was a person that actually properly looked at it.

The chat being looked through, no matter if online or offline shouldn't be a thing, because context is even difficult for a human being to interpret in text, so a computer is even less likely. If it's not reported, it shouldn't be looked at.

AvalonRanger · December 22

Yup

AI can't understand non English word. Usually it picked up wrong word which should not be prohibited.

AnduinTryggva · December 22

ArchangelIsraphel wrote: »

My guild has taken all RP outside of the game entirely to avoid the AI system misinterpreting a post. The themes of our stories involve war, political upheaval, and betrayal between the alliances- which of course leads to (entirely fabricated) arguments between characters and strong words being thrown back and fourth. As we do not want any of this to be seen as real when it is not, we have totally withdrawn all chat activity from inside of the game.

We used private housing, guild chat, and party chat and did not publicly roleplay. However, seeing as we no longer feel safe roleplaying within the game, we also no longer recruit and our events have become rather limited.

To be honest I think if they want AI to take the role of refereeing instead of humans which I can understand from an economic point of view they should limit the unsolicited AI refereeing to occasions where clearly defined infringements like use of inappropriate wording that are racist, homophobic or hate speech occur.

It should not interfere in an unsolicited way where people insult each other in a "normal" manner (I don't consider insulting normal unless as a part of rp of course - just to be clear). The AI should in these cases inly start refereeing upon player report.

DemonicGoat · December 22

Arunei wrote: »

They've said *snip*

Theyve said a lot of things.

Marto · December 22

Nope

Avran_Sylt wrote: »

Oh interesting, wasn’t sure this was something that was addressed by mods already, I’ve only heard hearsay. There a thread with the responses?

Here is the most recent response, from game director Rich Lambert. He confirmed it has no impact on performance, that the issue was caused by human moderators being too strct, and that the automated system has been in use for a few years now (meaning everyone claiming there's a "new AI moderation system" is simply lying)

ZOS_RichLambert wrote: »

Can someone confirm the role of the AI chat monitoring here?
We’ve extensively investigated if this could have an impact on game performance. The way the tool works, it sifts through chat logs that are exported from the game. So, it isn’t integrated into ESO, and never actually touches the game or the chat servers.
Regarding the tool itself, it helps us identify at risk chat and areas of concern faster. But our agents make the determination on whether an account is suspended or banned. We do have some auto kick rules for chat spam, but anything automated is temporary. Our agents review every automatic action against account history before taking any permanent account action.
We have also been using the tool for a couple years and nothing has changed in how we log chat, so the timing doesn’t line up. The situation a couple months ago with increased actioning related to chat logs was more a case of changes to our processes and training up new personnel than with the tool itself. Ultimately, though, yes we did look at if the tool was having any noticeable impact to game performance and it’s not.

Here is the original response from Community Manager Kevin about the controversy, a couple days after it started.

ZOS_Kevin wrote: »

As with any online game, our goal is to make sure you all can have fun while making sure bad actors do not have the ability to cause harm. To achieve this, our customer service team uses tools to check for potentially harmful terms and phrases. No action is taken at that point. A human then evaluates the full context of the terms or phrases to ensure nothing harmful or illegal is occurring. A human is always in control of the final call of an action and not an AI system.

That being said, we have been iterating on some processes recently and are still learning and training on the best way to use these tools, so there will be some occasional hiccups. But we want to stress a few core points.
We are by no means trying to disrupt or limit your role-play experiences or general discourse with friends and guildmates. You should have confidence that your private role-play experiences and conversations are yours and we are not looking to action anyone engaging in consensual conversations with fellow players.

The tools used are intended to be preventative, and alert us to serious crimes, hate speech, and extreme cases of harm.

To reiterate, no system is auto-banning players. If an action does occur, it’s because one of our CS agents identified something concerning enough to action on. That can always be appealed through our support ticketing system. And in an instance where you challenge the appeal process, please feel free to flag here on the forum and we can work with you to get to the bottom of the situation.

As a company we also abide by the Digital Service Act law and all similar laws.

To wrap this up, for those who were actioned, we have reversed most of the small number of temporary suspensions and bans. If you believe you were impacted and the action was not reversed, please issue an appeal and share your ticket number. We will pass it along to our customer service to investigate.

And if you were to search all those reddit threads from users claiming they were wrongfully banned, pretty much all of them have Kevin or Jessica asking for ticket numbers. So it's clear ZOS is investigating what exactly is causing the uptick in false positives.

Sakiri · December 23

Yup

I still feel hate speech is subjective. Dunmer, for example, are incredibly racist and xenophobic by nature and refer to argonians as farm equipment, as in, slavery.

You can't RP that without catching flak. You can't do ERP with your spouse either(I know folks into that but always with consenting partners) because it'll catch it.

Whats offensive in one culture isn't in another - many words offensive in English are not in Sweden, for example. Ignoring cultural differences can lead to problems and a lot of times it just slips.

And on top of it, offense is taken, not given. I'm autistic, I offend folks without trying, and people tend to be offended by just about anything somehow these days.

It *needs* to get rid of bots.

Arunei · December 23

DemonicGoat wrote: »

Arunei wrote: »

They've said *snip*

Theyve said a lot of things.

Yes, and...? Plenty of stuff they've said has been honest and the truth, so there's no real reason to doubt that they're telling the truth about the bot and how it functions. Believing ZOS is always lying and being contrary would take way more energy than taking their word on stuff until proven wrong.

LikiLoki · December 23

Yup

I already wrote in another topic that was immediately closed and I'll repeat it here.: A couple of weeks ago, on one of the social networks, one player complained that he was blocked from the game for 3 days for insulting another player in a personal chat with a friend. They did this to each other in whispers. Isn't it outrageous that there is a penalty for private conversations that no one should hear?

tsaescishoeshiner · December 23

Has anyone been banned or actioned for it recently? Based on the comment above, it sounds like it was happening over a short time frame. Is anyone doing heated or spicy roleplay the past few weeks and having no issue with players getting banned for fictional scenarios?

Desiato · December 23

I used to RP in a more traditional sense as a child in the 80s and a young person in the 90s with things like AD&D, MUSHes and MOOs, but not for decades. So I have some perspective.

I would never consider it in ESO based on the kind of misunderstandings that are possible with ESO CSRs. Just look at the threads about forum bans here.

The idea that a private conversation could so easily be exposed to another person without either party requesting it is a total dealbreaker.

In that vein, I'll never even consider discussing anything serious at all in ESO. I don't get to know other players the way I used to unless it's in discord.

This should be seen as a problem to ZOS because it limits the kinds of connections and trust players can build with each other -- which is a big part of social gaming.

Sakiri · December 23

Yup

Desiato wrote: »

I used to RP in a more traditional sense as a child in the 80s and a young person in the 90s with things like AD&D, MUSHes and MOOs, but not for decades. So I have some perspective.

I would never consider it in ESO based on the kind of misunderstandings that are possible with ESO CSRs. Just look at the threads about forum bans here.

The idea that a private conversation could so easily be exposed to another person without either party requesting it is a total dealbreaker.

In that vein, I'll never even consider discussing anything serious at all in ESO. I don't get to know other players the way I used to unless it's in discord.

This should be seen as a problem to ZOS because it limits the kinds of connections and trust players can build with each other -- which is a big part of social gaming.

I kinda agree. Private discussion should remain private. What if I'm discussing something super messed up in my life with my bestie that I've not even told my spouse yet? I don't really want zos reading that but due to the nature of "super messed up" it could be flagged.

bmnoble · December 23

Yup

I don't RP, what I have noticed however is both guild and zone chat have been a lot quieter these days, still see the looking for groups or guild recruiting messages and the occasional question when someone needs help other than that, rarely ever seem to see any jokes, chatting in chat the last few months.

AvalonRanger · December 23

Yup

Those automatic blocker should be disable now.

1, The language knowledge of AI is too much poor, especially AI can't understand
non English word. Non English chat became chaos now.

2, AI can't understand conversation situation properly.

The Elder Scrolls Online

Is AI Automatic Moderation Killing the RP Scene?

Is AI Automatic Moderation Killing the RP Scene? 84 votes