Suggestions To Assist With Moderating Toxic Behavior

endorphinsplox

So after the recent disaster with the new automated chat moderation system, resulting in rampant bans and suspensions, and the insultingly unsatisfactory response from the dev team followed by the thread topic being shut down with open questions still being asked, I decided to suggest some alternative moderation tools to help with limiting toxicity while also not spying on what are supposed to be private conversations, or persecuting innocent people for victimless crimes.

The first tool I'd like to mention is a profanity filter. Yes, we already have one, but I wanted to remind everyone that it exists.

The second tool I'd like to mention is an "ignore player" feature. Yes, this one is also something we already have but I have an idea for improving it: remove the 100 blocked players limit. There is no reason I can think of for limiting the amount of players that a person can prevent interaction with. Its wild to me that we have more Style Material items in game than we can have friends or blocked players, which is of course why the only somewhat believable explanation of database limitations doesn't make sense either. Another potential improvement is to remove blocked players from the user's view. This would prevent things like mudball trolling, emote spam, ability spam, etc. Obviously, this part wouldn't be applicable to PvP zones, but it should be fine everywhere else.

On top of that, players should not be able to direct message or otherwise interact with people they have blocked. This is used as a tool for trolling because a bully can take control and make conversations one-sided until the target blocks them as well. Of course, a person could still set themselves to offline mode to create one-sided conversations, but that could also be changed so that direct messaging someone while in offline mode could automatically open up return communication from the targeted player. That would deter would-be trolls rather quickly.

The next tool that could use some help is the "Report Player" function. This should always be the go-to for taking action against players who bully others in non-public channels, but as it stands it doesn't work very well. In all likelihood, the reason why we don't see extreme language such as slurs in public channels, is that many players would report the behavior at once, which add up quickly and get noticed unlike individual reports which can fly under the radar. Now, I understand that not all reports are made in good faith and careful evaluation needs to be performed in order to determine if a report is valid to avoid giving toxic players the power to remove innocent people, and when a person gets reported by a lot of people, its more likely they were genuinely in the wrong, and its therefore easier to take action on them. The problem is, there is no consequence for false reporting. In fact, we don't even get told what the resulting action of a report actually was, in order to "protect their privacy" even though that is clearly not something that ZOS is interested in doing, given the surveillance state they've created.

Other companies will actively respond to player reports that have resulted in disciplinary action which encourages accurate reporting, and will also sometimes respond to false reports by saying no violation was found. These are common practices in more successful multiplayer games that should be adopted here.

I think its also important to note that suspensions and bans are a pretty extreme reaction to naughty words. What I would suggest here is simply muting repeat offenders for set durations in-game so they can still play but do receive some level of punishment so they know their actions aren't appropriate. Population numbers wouldn't take as much of a hit, and morale would drop much less quickly overall.

In closing, I do want to mention that targeting specific words as being unacceptable in any context is not an effective strategy. While I think active prevention of the use of racial slurs or other extremely language in public channels is a good idea, monitoring private messages for a wide array of adult oriented words that aren't even considered offensive only spreads fear and kills morale. On top of that, language evolves quickly, especially under the pressure of censorship. Codes can be utilized to replace words that otherwise would be caught by a filter like this game has. The answer to insulting speech is not restriction, it is more speech. No one is helped by being shielded from potentially offensive language, they are only weakened by the malnutrition of hiding from the sun.

wilykcat

Wow. 😲 Very interesting post.

I do agree with chat restrictions on players who do use swear words and other forms of inappropriate language. The player isn't fully suspended, he/she won't beable to use the chat but can still play the game.

I disagree with the removal of the profanity filter. It blocks more than just common swear words, it also blocks hateful slurs. Not everyone here wants to see and experience a game or a forum full of profanity (profanity(swear words) makes me feel uncomfortable). If Zenimax doesn't want profanity in the game and/or on the forums then they have the right to block it.

I also agree with the updating the report function and the giving notice that the report has got sent. And the ability to ignore/block more players.

Another suggestion I would add is the ability for accounts to heal after a moderation action happens. Similar to what league of legends does.

karthrag_inak

endorphinsplox wrote: »

... resulting in rampant bans and suspensions...

Khajiit has seen no evidence of this. But then again, khajiit always prefaces any chat discussions with "This statement is not true" just to prime the AI pump, as it were.

Syldras

wilykcat wrote: »

I do agree with chat restrictions on players who do use swear words and other forms of inappropriate language. The player isn't fully suspended, he/she won't beable to use the chat but can still play the game.

In general chat or in private chats?

wilykcat wrote: »

I disagree with the removal of the profanity filter. It blocks more than just common swear words, it also blocks hateful slurs. Not everyone here wants to see and experience a game or a forum full of profanity (I’m an adult and profanity(swear words) makes me feel uncomfortable).

It's optional right now and that's the way it should remain, especially considering that it leads to many false positives, especially in other languages than English. Communication is hardly possible if every third word is blurred out - and my friend usually doesn't swear.

Sleepsin

You forgot to mention another way to assist with moderation. [snip]

[Edit for minor bait]

TaSheen

Moderation should be limited to truly egregious texts or behaviors. Minor swear words should never be moderated. Seriously, damn and hell don't need moderating, and in other countries there are yes, words that are "egregious" in the good old Victorian prudist US, but there's a limit.

Or there SHOULD be.

SeaGtGruff

karthrag_inak wrote: »

endorphinsplox wrote: »

... resulting in rampant bans and suspensions...

Khajiit has seen no evidence of this.

I was about to post something similar. I wonder how many players were actually banned-- and one player who used Twitch to publicly protest their ban was back to streaming the game after a few days, so it would also be interesting to know how many players were permanently banned versus receiving a temporary suspension.

But in addition to those numbers, I'm curious what percentage were actually due to AI supervision.

In any case, the best way to avoid a ban or suspension is to moderate our own behavior. That might not help against false or mistaken reports made by players who are trying to "get even" or who wrongly think someone is cheating, but it should help protect us from being written up by the "AI chat police" for using naughty words.

spartaxoxo

While I agree that AI moderating private chat is a concern, I don't think they need to drastically change things up. There were also so few bans that people questioned their very existence. One unfair ban was too many but it's not accurate to say it was rampant.

They should simply limit the use of AI moderating to only extreme examples for example, criminal conduct. Leave the offensive language to actual player reports.

I do agree with your PM changes though.

Thee_Cheshire_Cat

spartaxoxo wrote: »

While I agree that AI moderating private chat is a concern, I don't think they need to drastically change things up. There were also so few bans that people questioned their very existence. One unfair ban was too many but it's not accurate to say it was rampant.

They should simply limit the use of AI moderating to only extreme examples for example, criminal conduct. Leave the offensive language to actual player reports.

I do agree with your PM changes though.

And stay out of what is said in player housing.

Syldras

Thee_Cheshire_Cat wrote: »

And stay out of what is said in player housing.

That might be more of a technical problem because it's very probably that the automatism doesn't "see" your location, so it can't distinguish whether you're using general chat while you're alone in a house or the Archive or while you're surrounded by 100 other players in the middle of Wayrest.

endorphinsplox

Syldras wrote: »

That might be more of a technical problem because it's very probably that the automatism doesn't "see" your location, so it can't distinguish whether you're using general chat while you're alone in a house or the Archive or while you're surrounded by 100 other players in the middle of Wayrest.

This is probably true, however its more concerning in that case that the developers of an MMO that's been out for 10 years didn't have the basic understanding that location, timing, and context are all important elements of language.

Bradyfjord

The AI monitoring shows how difficult (and possibly unethical) it could be to proactively get involved. It is easy to misunderstand human interactions.

This is why it is so important for us to use tools like ignore lists. And when necessary, report the action, including date, time, and other context so the investigation can go smoothly.

Syldras

endorphinsplox wrote: »

This is probably true, however its more concerning in that case that the developers of an MMO that's been out for 10 years didn't have the basic understanding that location, timing, and context are all important elements of language.

The strangest thing to me is that there is an automatic scanning of messages for completely mundane things like insults. I could see them doing it for actual criminal content, that would be a case that I might find justified. But other than that...?!

Makes me wonder whether they're afraid that some people might write unfriedly or immoral looking things in chat, even consensually or in roleplay, and it somehow surfaces and media make a scandal out of it. Like "Look! People in ESO can use slurs in chat! And some even describe torture scenarios (let's just ignore that it's consensual roleplay and the person is playing an evil vampire or necromancer)!".

endorphinsplox

Syldras wrote: »

The strangest thing to me is that there is an automatic scanning of messages for completely mundane things like insults. I could see them doing it for actual criminal content, that would be a case that I might find justified. But other than that...?!

Makes me wonder whether they're afraid that some people might write unfriedly or immoral looking things in chat, even consensually or in roleplay, and it somehow surfaces and media make a scandal out of it. Like "Look! People in ESO can use slurs in chat! And some even describe torture scenarios (let's just ignore that it's consensual roleplay and the person is playing an evil vampire or necromancer)!".

And at the end of the day, I could definitely still be incredibly insulting and abusive without using flaggable words because its all about context and intent, 2 things that are impossible for an automated system to understand and determine, especially because humans already struggle with it. Half the time someone gets offended by something I've said these days its in a context where I'm not trying to be insulting or anything, I just said something that someone has a personal history with I could never have controlled for.

darvaria

The problem is that some guilds/groups are mass reporting players that want to see removed.

Make it simple if you send a tell to a player, then no matter what their status is, you can reply. This player opened a line of conversation.

Change the crouch emote to just a dark cloud. NO crouching, or let AI pick up someone using multiple crouches. BEST solution is to change it.

katanagirl1

darvaria wrote: »

The problem is that some guilds/groups are mass reporting players that want to see removed.

Make it simple if you send a tell to a player, then no matter what their status is, you can reply. This player opened a line of conversation.

Change the crouch emote to just a dark cloud. NO crouching, or let AI pick up someone using multiple crouches. BEST solution is to change it.

Crouching is a valid form of gameplay for thieving activities. There is no crouch emote. Banning crouching or making a cloud around thieves who are crouching is not acceptable.

Erickson9610

I think ZOS should keep using their tools to stop bad behavior. "Private" messages in-game were never private.

Varana

Using swear words is not bad behaviour unless specifically directed at another player (or group of them).

I_killed_Vivec

Syldras wrote: »

endorphinsplox wrote: »

This is probably true, however its more concerning in that case that the developers of an MMO that's been out for 10 years didn't have the basic understanding that location, timing, and context are all important elements of language.

The strangest thing to me is that there is an automatic scanning of messages for completely mundane things like insults. I could see them doing it for actual criminal content, that would be a case that I might find justified. But other than that...?!

Makes me wonder whether they're afraid that some people might write unfriedly or immoral looking things in chat, even consensually or in roleplay, and it somehow surfaces and media make a scandal out of it. Like "Look! People in ESO can use slurs in chat! And some even describe torture scenarios (let's just ignore that it's consensual roleplay and the person is playing an evil vampire or necromancer)!".

I'm inclined to agree, but just a moment's thought about why they would do it and you get the inevitable "Think of the children!"...

While you might believe that you are indulging in consensual torture roleplay (whatever floats your boat, but definitely not for me), to someone else it might look like grooming a minor. Yes, we know that minors shouldn't be playing; yes, we know that they do. Just one incident of that getting in the news and all hell breaks loose.

Obviously that's bad business for ZoS.

Remember, the terms and conditions aren't there to protect us, they are there to protect the company.

Elsonso

Syldras wrote: »

Thee_Cheshire_Cat wrote: »

And stay out of what is said in player housing.

That might be more of a technical problem because it's very probably that the automatism doesn't "see" your location, so it can't distinguish whether you're using general chat while you're alone in a house or the Archive or while you're surrounded by 100 other players in the middle of Wayrest.

Yeah, that is possible. A lot of software and processes are designed for the needs of that day, and ZOS may not have cared where the player was standing because they were just looking to see if Player A said bad thing B to confirm a player report.

That said, I would seriously SMH if they tried to deploy a global chat monitoring system without first realizing that they didn't have the data collection for it to work properly.

Syldras wrote: »

Makes me wonder whether they're afraid that some people might write unfriedly or immoral looking things in chat, even consensually or in roleplay, and it somehow surfaces and media make a scandal out of it. Like "Look! People in ESO can use slurs in chat! And some even describe torture scenarios (let's just ignore that it's consensual roleplay and the person is playing an evil vampire or necromancer)!".

I can see this, but on a much larger scale. Bad actors can use private chats to exchange information completely unrelated to the game or gaming situation they are in.

If ZOS ever takes ESO free to play, where anyone can just create an account and log in, I could see this as being a more significant problem than it is today.

Sleepsin

I would be more sympathetic towards ZOS and their moderation if I didn't see gold sellers spamming the chat channel on a daily basis.

LadyGP

karthrag_inak wrote: »

endorphinsplox wrote: »

... resulting in rampant bans and suspensions...

Khajiit has seen no evidence of this. But then again, khajiit always prefaces any chat discussions with "This statement is not true" just to prime the AI pump, as it were.

Someone in my guild got 72 hour ban for talking about peoples DPS at the end of a trial and ways they could work on improving it.

Be apart of any guild/discord and you'll see there are a crap ton of bans going out then being reverted. The annoying part is the person who is banned has to provide context AFTER they have been banned. It seems like the AI is only getting a tiny section of the context thus making it look very bad.

I'm sure there are many instances of the AI working as intended but I've seen a lot of bad instances recently.

xylena_lazarow

Shut down offline tells and most of this game's toxicity problems immediately disappear.

LadyGP

xylena_lazarow wrote: »

Shut down offline tells and most of this game's toxicity problems immediately disappear.

Most underrated comment in this thread.

Syldras

I_killed_Vivec wrote: »

While you might believe that you are indulging in consensual torture roleplay (whatever floats your boat, but definitely not for me), to someone else it might look like grooming a minor.

I don't know how it looks if minors are groomed, but I somehow doubt it usually looks like roleplay where evil necromancer x throws Breton hero y into his dungeon (haven't played such scenarios in game chat yet, but I can imagine how it would look like). Maybe an onlooker might interpret the whole thing sexualized, but that's in the head of that person. Where to draw the line? Is a bandit kidnapping a noble and tying him up inappropriate, too?

Then of course, some people really ERP and if it turns out one participant is underage... "Think of the children!" indeed.

Elsonso wrote: »

I can see this, but on a much larger scale. Bad actors can use private chats to exchange information completely unrelated to the game or gaming situation they are in.
If ZOS ever takes ESO free to play, where anyone can just create an account and log in, I could see this as being a more significant problem than it is today.

Strictly seen, to avoid this, they would have to close chat altogether (or have a human check every single line) because messages could be coded or written in a way to avoid the typical trigger words for the filter. And I'd assume that most people who have something criminal in mind would do exactly that. I still can see how ZOS would filter for words that hint on criminal intent, as a legal protection, so they could claim that they at least did everything possible (or maybe there are even legislations where they are forced to do that now?). It's still ridiculous if the filter is set off by simple swearwords, the word for an illegitimate son like Martin Septim, or the German word for "less" (that happens to include a few letters that look like the name of a West African country).

LPapirius

wilykcat wrote: »

Wow. 😲 Very interesting post.

I do agree with chat restrictions on players who do use swear words and other forms of inappropriate language. The player isn't fully suspended, he/she won't beable to use the chat but can still play the game.

I disagree with the removal of the profanity filter. It blocks more than just common swear words, it also blocks hateful slurs. Not everyone here wants to see and experience a game or a forum full of profanity (profanity(swear words) makes me feel uncomfortable). If Zenimax doesn't want profanity in the game and/or on the forums then they have the right to block it.

I also agree with the updating the report function and the giving notice that the report has got sent. And the ability to ignore/block more players.

Another suggestion I would add is the ability for accounts to heal after a moderation action happens. Similar to what league of legends does.

There is already a profanity filter that is turned on by default. Problem solved. If people can't tolerate swear words, they can leave the filter on and never see a dirty word.

LPapirius

Varana wrote: »

Using swear words is not bad behaviour unless specifically directed at another player (or group of them).

If nobody reports the colorful language as harassment, no violation of the ToS took place. Period.

endorphinsplox

Erickson9610 wrote: »

I think ZOS should keep using their tools to stop bad behavior. "Private" messages in-game were never private.

You only talk this way until something you don't find problematic gets you a 3 day suspension during an event.

Elsonso

LadyGP wrote: »

karthrag_inak wrote: »

endorphinsplox wrote: »

... resulting in rampant bans and suspensions...

Khajiit has seen no evidence of this. But then again, khajiit always prefaces any chat discussions with "This statement is not true" just to prime the AI pump, as it were.

Someone in my guild got 72 hour ban for talking about peoples DPS at the end of a trial and ways they could work on improving it.

I would suggest that this is not why they were suspended. That may have been what they were doing, but it was not why they were suspended.

shadyjane62

I just stopped talking to anyone period. I do not say anything in chat in pvp ever from this day forward. I no longer help anyone lest it be misconstrued.

Is this sufficient?

Stafford197

karthrag_inak wrote: »

endorphinsplox wrote: »

... resulting in rampant bans and suspensions...

Khajiit has seen no evidence of this. But then again, khajiit always prefaces any chat discussions with "This statement is not true" just to prime the AI pump, as it were.

I still see some people say this. What evidence are you looking for?

It is against Forum rules to post actual evidence, which is why people have instead posted their stories. ZOS_Kevin also officially acknowledged the whole situation, including both their new automated system and the numerous incorrect bans.

Is the official statement not good enough evidence? I’m just curious what you are looking for given that you are speaking on a platform where posting the evidence you want is prohibited lol.