How will the AI that monitors our in-game chats effect RP?

SeaGtGruff

Stafford197 wrote: »

ZOS said they would reply here this week,

Technically, no they didn't.

ZOS_Kevin wrote: »

Since it’s pretty late in the day on a Friday, we probably won’t have any feedback until earlier next week.

"Probably" does not mean "definitely."

Basically, Kevin said that, it being late Friday, the soonest they'd be able to give any feedback about people's concerns would be sometime this week.

Furthermore, Kevin isn't in control of whoever would most likely be making any decisions about what said feedback should be, so it isn't fair to Kevin for anyone here to try to hold him to a presumed hard deadline of this week based on when he said they would "probably" have some feedback to give us.

Also, given that ZOS is now owned by Microsoft, ZOS might conceivably need to get feedback from Microsoft before they can give any feedback to us, depending on who is ultimately in control of this particular issue.

Pelanora

Arunei wrote: »

ESO_player123 wrote: »

Arunei wrote: »

@manukartofanu @jaws343
Are you guys being purposefully obtuse? You're saying that we should all be at risk of warning, suspension, or bans because of a computer program that can't tell context deciding private conversation between adults needs auto-moderating?

I'm sure, then, you'd be totally fine if your phone provider was monitoring every single private call you had and threatening to call the police on you if you ever said a bad word or something lewd?

I'm truly struggling to understand why y'all think it's cool for people to have accounts actioned for the HORRIBLE OFFENSE of using colorful language among friends in private instances and settings. So like...what if someone opens up about trauma they've dealt with and say they were S/A'd but use the word grape without the g, and the bot that can't tell the context that word was used in decides it must be a threat or an insult?

What about the issue that's been brought up multiple times with words in different languages being able to have innocent meanings in their native tongue but being offensive in another?

The report feature exists for a reason. No actions should be taken against accounts by a bot. Ever. And nothing anyone can say will change that.

I think your phone provider example is not a good one here. While subscribing to a phone service you do not sign any agreements about what you are allowed to say over the phone. People in this thread posted parts of the ToS that describe the topics/language that is not going to be tolerated. And we all signed it.

In that case I'd like to bring up something someone else did above me. Why wait going on 11 years to start handing out actions against accounts then? I will admit I've had my share of spicy RP. I've had my share of gruesome RP. No characters I've ever played use explicit language, or if they do, a lot of the time it's Old English like arse (which is used in-game) or something like "sod" in place of the f-word.

If this has always been in the ToS why have I never once, in like 9-ish years of RP, ever been hit by this bot/AI/whatever? Why is it only now being an issue? There are a few potential reasons.

This is all happening and it's a recent addition, thus it only really recently seeming to be such a huge issue.

This is all happening and it's because that computer program is glitching badly somehow. Maybe it existed this whole time and properly flagged things for human review, maybe it was recently added, but either way it's bugging out.

It's not actually happening and people are just stirring the pot trying to create drama where none exists. Though I feel if this were the case, the devs should have been able to quickly be like "Oh nah this isn't actually happening people are just out here spreading worry and chaos all's good" by now.

The fact that we don't have any sort of comment saying it's not an issue is very concerning. It likely means the devs are having to check things, maybe check other things depending on what they found, and potentially check even more branching stuff. Which means something either did go wrong or this was a deliberate decision and now it's a matter of them trying to work on a damage control response.

And I think we all know how long THAT can take.

Also I know we don't sign user agreements for phone service, and honestly I didn't phrase it right. I should have said they monitor all your private calls with an automatic system that can potentially flag you for use of "bad language". It was meant to highlight how ridiculous it is for anyone to think actions made by machines that can't consider context is justifiable. Do those people not realize they could be hit by this same system as anyone else, especially considering the whole accidentally triggering the filter thing? Nearly all of us should realize it really is too sensitive and nothing that trips that easily should be responsible for actioning accounts.

Nobody can break the law to enforce a rule. Whatever rules they have in the tos, they can't just do whatever they like to enforce them. So the usage of data needs to be within the terms of the tos and of the various privacy regimes they operate under.

Pelanora

SilverBride wrote: »

ESO_player123 wrote: »

People in this thread posted parts of the ToS that describe the topics/language that is not going to be tolerated. And we all signed it.

This is true. However this does not mean that monitoring our private conversations is a good idea, and it is certainly not good customer service.

We need some communication about this.

Yea there's a big difference between saying 'this is the behaviour i expect of you' and 'here is the bot that will watch every single thing you do in case you don't behave'.

You know, like between Christmas, and, installing that pyscho elf to watch all year long.

ESO_player123

SilverBride wrote: »

ESO_player123 wrote: »

People in this thread posted parts of the ToS that describe the topics/language that is not going to be tolerated. And we all signed it.

This is true. However this does not mean that monitoring our private conversations is a good idea, and it is certainly not good customer service.

We need some communication about this.

@SilverBride
Using only the part of my reply that you kept in the quote changes the meaning of my reply. This part was meant to emphasize the difference between subscribing to the services of a phone provider and getting access to the chat in ESO (lack of a signed agreement about what can be communicated in the former).

I'm not trying to take sides here but waiting for an official replay to see what's what. Personally, I treat the content of any chat in any game like something that can be stored/recorded. So, I would never assume that that information is private.

Tazzy

SeaGtGruff wrote: »

Stafford197 wrote: »

ZOS said they would reply here this week,

Technically, no they didn't.

ZOS_Kevin wrote: »

Since it’s pretty late in the day on a Friday, we probably won’t have any feedback until earlier next week.

"Probably" does not mean "definitely."

Basically, Kevin said that, it being late Friday, the soonest they'd be able to give any feedback about people's concerns would be sometime this week.

Furthermore, Kevin isn't in control of whoever would most likely be making any decisions about what said feedback should be, so it isn't fair to Kevin for anyone here to try to hold him to a presumed hard deadline of this week based on when he said they would "probably" have some feedback to give us.

Also, given that ZOS is now owned by Microsoft, ZOS might conceivably need to get feedback from Microsoft before they can give any feedback to us, depending on who is ultimately in control of this particular issue.

Totally agree here. In large companies, customer support has to get approval from superiors on such escalating issues about how exactly to respond and what exactly to respond. It doesn't speed up the process if users are constantly demanding answers. (I'm not saying that it shouldn't be discussed further; it's actually good for keeping the thread current.)

LesserCircle

The annoying part for me is that these horrible things never get big on youtube or social media like they do when it happens with other games, we can complain all we want in the forums but it won't reach anyone outside of our ESO players and even then the forums are a minority too.

Elsonso

SeaGtGruff wrote: »

Also, given that ZOS is now owned by Microsoft, ZOS might conceivably need to get feedback from Microsoft before they can give any feedback to us, depending on who is ultimately in control of this particular issue.

Even if the policy comes from XBox or Microsoft, someone at ZOS knows what is going on and can explain what is happening. It isn't like someone at Microsoft is sneaking in and updating the naughty word filter without anyone at ZOS knowing about it.

This particular area, namely the people who deal with ToS, is well off stage and we rarely hear anything about them or what they are doing. I really don't expect any answers, even if Kevin does come back with more to say.

Idelise

LesserCircle wrote: »

The annoying part for me is that these horrible things never get big on youtube or social media like they do when it happens with other games, we can complain all we want in the forums but it won't reach anyone outside of our ESO players and even then the forums are a minority too.

I am pretty sure I saw an article about it somewhere... https://massivelyop.com/2024/09/16/roleplaying-elder-scrolls-online-players-raise-red-flags-over-automated-chat-moderation/

And for now, until the issue is resolved, I unsubbed. Ain gonna support such decisions with my money.

Tandor

sarahthes wrote: »

Tandor wrote: »

CrazyKitty wrote: »

Tandor wrote: »

Pelanora wrote: »

Kappachi wrote: »

AltmerGF wrote: »

Can anecdotally say I was using some less than wholesome language with my friends while playing this weekend and haven't gotten banned, so hopefully we're just looking at some sort of overzealous moderation glitch

even amongst private chats you agreed to a TOS when you started the game, if you're throwing out the real world equivalents of n'wah/s'wit/etc then you're bound to get banned. keep it civil in-game even amongst friends.

That's ridiculous. Zos isn't some moral police over private citizens consenting private chat.

If you want a private chat, don't log into someone else's software in order to have it. When you sign up to the TOS then they apply to the whole game., not just the bits you think they should apply to.

Pardon me, but I thought ESO was rated "M" for mature. No?

Indeed, but all that refers to is mature content in the game, it doesn't give players the freedom to ignore the TOS.

It's not the TOS. It's the Code of Conduct that governs behaviour.

As I understand it, the Community Code of Conduct governs behaviour on the forums, and the Rules of Conduct are part of the Terms of Service which govern behaviour in the game.

Elsonso

Tandor wrote: »

sarahthes wrote: »

Tandor wrote: »

CrazyKitty wrote: »

Tandor wrote: »

Pelanora wrote: »

Kappachi wrote: »

AltmerGF wrote: »

Can anecdotally say I was using some less than wholesome language with my friends while playing this weekend and haven't gotten banned, so hopefully we're just looking at some sort of overzealous moderation glitch

even amongst private chats you agreed to a TOS when you started the game, if you're throwing out the real world equivalents of n'wah/s'wit/etc then you're bound to get banned. keep it civil in-game even amongst friends.

That's ridiculous. Zos isn't some moral police over private citizens consenting private chat.

If you want a private chat, don't log into someone else's software in order to have it. When you sign up to the TOS then they apply to the whole game., not just the bits you think they should apply to.

Pardon me, but I thought ESO was rated "M" for mature. No?

Indeed, but all that refers to is mature content in the game, it doesn't give players the freedom to ignore the TOS.

It's not the TOS. It's the Code of Conduct that governs behaviour.

As I understand it, the Community Code of Conduct governs behaviour on the forums, and the Rules of Conduct are part of the Terms of Service which govern behaviour in the game.

The ToS, CoC, and Privacy Policy govern both.

Arunei

Carcamongus wrote: »

If it's against the rules to say "vulgar" words, like that one starting with an F, then how come the profanity filter can be turned off? Perhaps we need some clarifications on more than just this rumored bot.

Oh, and by "that one starting with an F", I obviously meant "fadoodle".

This is actually a very good point no one else has made yet. What's the point of a filter if we aren't allowed to use ""naughty words"" to begin with?

Pelanora wrote: »

Arunei wrote: »

ESO_player123 wrote: »

Arunei wrote: »

@manukartofanu @jaws343
Are you guys being purposefully obtuse? You're saying that we should all be at risk of warning, suspension, or bans because of a computer program that can't tell context deciding private conversation between adults needs auto-moderating?

I'm sure, then, you'd be totally fine if your phone provider was monitoring every single private call you had and threatening to call the police on you if you ever said a bad word or something lewd?

I'm truly struggling to understand why y'all think it's cool for people to have accounts actioned for the HORRIBLE OFFENSE of using colorful language among friends in private instances and settings. So like...what if someone opens up about trauma they've dealt with and say they were S/A'd but use the word grape without the g, and the bot that can't tell the context that word was used in decides it must be a threat or an insult?

What about the issue that's been brought up multiple times with words in different languages being able to have innocent meanings in their native tongue but being offensive in another?

The report feature exists for a reason. No actions should be taken against accounts by a bot. Ever. And nothing anyone can say will change that.

I think your phone provider example is not a good one here. While subscribing to a phone service you do not sign any agreements about what you are allowed to say over the phone. People in this thread posted parts of the ToS that describe the topics/language that is not going to be tolerated. And we all signed it.

In that case I'd like to bring up something someone else did above me. Why wait going on 11 years to start handing out actions against accounts then? I will admit I've had my share of spicy RP. I've had my share of gruesome RP. No characters I've ever played use explicit language, or if they do, a lot of the time it's Old English like arse (which is used in-game) or something like "sod" in place of the f-word.

If this has always been in the ToS why have I never once, in like 9-ish years of RP, ever been hit by this bot/AI/whatever? Why is it only now being an issue? There are a few potential reasons.

This is all happening and it's a recent addition, thus it only really recently seeming to be such a huge issue.

This is all happening and it's because that computer program is glitching badly somehow. Maybe it existed this whole time and properly flagged things for human review, maybe it was recently added, but either way it's bugging out.

It's not actually happening and people are just stirring the pot trying to create drama where none exists. Though I feel if this were the case, the devs should have been able to quickly be like "Oh nah this isn't actually happening people are just out here spreading worry and chaos all's good" by now.

The fact that we don't have any sort of comment saying it's not an issue is very concerning. It likely means the devs are having to check things, maybe check other things depending on what they found, and potentially check even more branching stuff. Which means something either did go wrong or this was a deliberate decision and now it's a matter of them trying to work on a damage control response.

And I think we all know how long THAT can take.

Also I know we don't sign user agreements for phone service, and honestly I didn't phrase it right. I should have said they monitor all your private calls with an automatic system that can potentially flag you for use of "bad language". It was meant to highlight how ridiculous it is for anyone to think actions made by machines that can't consider context is justifiable. Do those people not realize they could be hit by this same system as anyone else, especially considering the whole accidentally triggering the filter thing? Nearly all of us should realize it really is too sensitive and nothing that trips that easily should be responsible for actioning accounts.

Nobody can break the law to enforce a rule. Whatever rules they have in the tos, they can't just do whatever they like to enforce them. So the usage of data needs to be within the terms of the tos and of the various privacy regimes they operate under.

Oh yeah I'm aware, I even brought up that point several pages back. A company claiming they have the right to do something in their ToS doesn't magically make illegal things legal. A company can't claim they'll come assault you for doing X thing because assault is illegal and law takes precedence over non-legal rules, such as a ToS.

sarahthes

Arunei wrote: »

Carcamongus wrote: »

If it's against the rules to say "vulgar" words, like that one starting with an F, then how come the profanity filter can be turned off? Perhaps we need some clarifications on more than just this rumored bot.

Oh, and by "that one starting with an F", I obviously meant "fadoodle".

This is actually a very good point no one else has made yet. What's the point of a filter if we aren't allowed to use ""naughty words"" to begin with?

Pelanora wrote: »

Arunei wrote: »

ESO_player123 wrote: »

Arunei wrote: »

@manukartofanu @jaws343
Are you guys being purposefully obtuse? You're saying that we should all be at risk of warning, suspension, or bans because of a computer program that can't tell context deciding private conversation between adults needs auto-moderating?

I'm sure, then, you'd be totally fine if your phone provider was monitoring every single private call you had and threatening to call the police on you if you ever said a bad word or something lewd?

I'm truly struggling to understand why y'all think it's cool for people to have accounts actioned for the HORRIBLE OFFENSE of using colorful language among friends in private instances and settings. So like...what if someone opens up about trauma they've dealt with and say they were S/A'd but use the word grape without the g, and the bot that can't tell the context that word was used in decides it must be a threat or an insult?

What about the issue that's been brought up multiple times with words in different languages being able to have innocent meanings in their native tongue but being offensive in another?

The report feature exists for a reason. No actions should be taken against accounts by a bot. Ever. And nothing anyone can say will change that.

I think your phone provider example is not a good one here. While subscribing to a phone service you do not sign any agreements about what you are allowed to say over the phone. People in this thread posted parts of the ToS that describe the topics/language that is not going to be tolerated. And we all signed it.

In that case I'd like to bring up something someone else did above me. Why wait going on 11 years to start handing out actions against accounts then? I will admit I've had my share of spicy RP. I've had my share of gruesome RP. No characters I've ever played use explicit language, or if they do, a lot of the time it's Old English like arse (which is used in-game) or something like "sod" in place of the f-word.

If this has always been in the ToS why have I never once, in like 9-ish years of RP, ever been hit by this bot/AI/whatever? Why is it only now being an issue? There are a few potential reasons.

This is all happening and it's a recent addition, thus it only really recently seeming to be such a huge issue.

This is all happening and it's because that computer program is glitching badly somehow. Maybe it existed this whole time and properly flagged things for human review, maybe it was recently added, but either way it's bugging out.

It's not actually happening and people are just stirring the pot trying to create drama where none exists. Though I feel if this were the case, the devs should have been able to quickly be like "Oh nah this isn't actually happening people are just out here spreading worry and chaos all's good" by now.

The fact that we don't have any sort of comment saying it's not an issue is very concerning. It likely means the devs are having to check things, maybe check other things depending on what they found, and potentially check even more branching stuff. Which means something either did go wrong or this was a deliberate decision and now it's a matter of them trying to work on a damage control response.

And I think we all know how long THAT can take.

Also I know we don't sign user agreements for phone service, and honestly I didn't phrase it right. I should have said they monitor all your private calls with an automatic system that can potentially flag you for use of "bad language". It was meant to highlight how ridiculous it is for anyone to think actions made by machines that can't consider context is justifiable. Do those people not realize they could be hit by this same system as anyone else, especially considering the whole accidentally triggering the filter thing? Nearly all of us should realize it really is too sensitive and nothing that trips that easily should be responsible for actioning accounts.

Nobody can break the law to enforce a rule. Whatever rules they have in the tos, they can't just do whatever they like to enforce them. So the usage of data needs to be within the terms of the tos and of the various privacy regimes they operate under.

Oh yeah I'm aware, I even brought up that point several pages back. A company claiming they have the right to do something in their ToS doesn't magically make illegal things legal. A company can't claim they'll come assault you for doing X thing because assault is illegal and law takes precedence over non-legal rules, such as a ToS.

That said, the likelihood that what they're doing is illegal is miniscule, given how many lawyers versed in the relevant international laws are likely to have written it.

Defending a TOS against claims that it's illegal is far more expensive than paying a lawyer to write a legal one.

Bammlschwamml

LesserCircle wrote: »

The annoying part for me is that these horrible things never get big on youtube or social media like they do when it happens with other games, we can complain all we want in the forums but it won't reach anyone outside of our ESO players and even then the forums are a minority too.

I think most people are sick and tired of thinking about the end of the world. They just wait for it all to be over...

Syldras

sarahthes wrote: »

That said, the likelihood that what they're doing is illegal is miniscule, given how many lawyers versed in the relevant international laws are likely to have written it.
Defending a TOS against claims that it's illegal is far more expensive than paying a lawyer to write a legal one.

It happens every now and then.
https://www.cnbc.com/2022/12/19/epic-games-to-pay-520-million-in-fines-to-ftc.html
https://www.gamesindustry.biz/epic-11m-fined-for-breaking-eu-consumer-law-in-netherlands
https://www.gamesindustry.biz/valve-fined-usd3-million-after-misleading-steam-refund-policy-found-in-breach-of-australian-law
https://www.accc.gov.au/media-release/sony-to-pay-35-million-penalty-for-misrepresenting-playstation-gamers-rights

Jaraal

The profanity filter has been in place for 10+ years, so technically an "AI" bot has been censoring us for years. So the idea of private chats being off limits to such coding is quite assumptive. The only thing new about this is that ZOS has suddenly decided to be more aggressive in banning violations of the Code of Conduct.

baratron

I don't have a problem with private conversations being monitored and/or logged by an AI. As a Known Female Player, this offers me a small amount of protection against harassment, because if someone says something obnoxious to me, there will be a record.

What I have a problem with is the idea that people are receiving automated bans for saying things to their friends.

There is a big difference between a complete stranger calling me the word for a female dog, versus someone who has been on my Friends list since 2014!

Any automated monitoring should take the relationship between individuals in the conversation into account. A simple check for Friends list would help tremendously.

Then again, if this is what's been causing the extremely unstable ping since the patch on Monday, I question its helpfulness to us as players. I can cope with high ping as long as it's constant. 300 ms to 600 ms to 167 ms to 900 ms to 999+ ms to 200 ms within 90 seconds is extremely hard to adjust to.

Four_Fingers

Just a quick note, all phone providers have a TOS as well which nobody reads when they sign their contract or pay their first bill. The TOS usually contains info that your phone may be monitored for purposes of law enforcement, usually taking a court order. Not saying that is relevant to this subject but does exist.

manukartofanu

Arunei wrote: »

@manukartofanu @jaws343
Are you guys being purposefully obtuse? You're saying that we should all be at risk of warning, suspension, or bans because of a computer program that can't tell context deciding private conversation between adults needs auto-moderating?

I'm sure, then, you'd be totally fine if your phone provider was monitoring every single private call you had and threatening to call the police on you if you ever said a bad word or something lewd?

I'm truly struggling to understand why y'all think it's cool for people to have accounts actioned for the HORRIBLE OFFENSE of using colorful language among friends in private instances and settings. So like...what if someone opens up about trauma they've dealt with and say they were S/A'd but use the word grape without the g, and the bot that can't tell the context that word was used in decides it must be a threat or an insult?

What about the issue that's been brought up multiple times with words in different languages being able to have innocent meanings in their native tongue but being offensive in another?

The report feature exists for a reason. No actions should be taken against accounts by a bot. Ever. And nothing anyone can say will change that.

So, are you also being purposefully obtuse? I’ve clearly expressed my position, so please read it and understand it before pouring out your emotional response.

Syldras

Four_Fingers wrote: »

Just a quick note, all phone providers have a TOS as well which nobody reads when they sign their contract or pay their first bill. The TOS usually contains info that your phone may be monitored for purposes of law enforcement, usually taking a court order. Not saying that is relevant to this subject but does exist.

Not sure how it's in the US, but in my country there has to be a court order, it has to be a very severe crime, and there has to be "imminent danger". Those are the preconditions. Otherwise: no chance (not saying the secret service might not doing it anyway, but this is how the laws are).

karthrag_inak

Khajiit wagers the AI is ignorant of Ta'agra

Jer iss vara zira gzalzi an jer dps vaba aki'i !!

Kenibi jer zedu shabar aki'i trun!!

boi_anachronism_

No to all of this. This is a mature title. ZOS need i remind you to go look at the lore of your own story?! Really? Ladies, gents, and those inbetwix i give you: the lessons of vivec and remind you how vivic, IN CANNON, discovered the secrets of chim. Look it up.

Get a grip zos. Ai is not near developed enough to distinguish these things. Leave it to the players to make reports. This is absurd. I am not an rper but i for one will be using other avenues for in game communication like discord or psn from now on unless its x'ing for a group or selling an item in zone chat. I am not about to get banned because i made an off color joke to a friend or something equally stupid.

Syldras

karthrag_inak wrote: »

Khajiit wagers the AI is ignorant of Ta'agra

I bet it's ignorant of Finnish as well.

Wait until you accidentially use a word in Ta'agra that looks like a horrible slur in French or Spanish (the word filter does also work for these languages)...

manukartofanu

Syldras wrote: »

I think no one here wants a toxic community.

To be honest, after reading this thread, I’m not so sure that absolutely everyone here wants a non-toxic community.

Syldras wrote: »

That said, I have not been affected by an autoban yet, maybe it's because of my writing style, maybe because I'm in EU, I don't know, but I have seen the autocensor that blurrs certain words out. From what I've seen so far, it's so sensitive, it's like it was made for literal children! A censor filter that filters so strictly that people couldn't even talk about the content of a quest without the risk of a ban is a failure, plain and simple.

I haven’t received any auto-bans either, although I often run into the filter. This especially happens with certain combinations of letters and numbers when I’m writing about prices in the game. I have no idea what they mean, but sometimes several messages in a row get filtered while I try to figure out how to phrase it so that the filter allows the correct number to be shown. If people are really getting banned because of the auto-filter, then that’s a problem, but it doesn’t seem to be the case here, since the complaints are about AI, and the auto-filter isn’t AI.

Syldras wrote: »

And yes, of course I understand that real mods cost money and and automatic moderation would be cheaper and easier (although I'd say that decent moderation is a part of running a game/service online that should certainly be considered a part of the normal game maintenance, and I'd think that ZOS, as a big company, would take that into consideration in their calculations beforehand). But what is it worth if it's not functioning correctly and angers the playerbase?

I’m working on training LLMs for specific work-related tasks. When it comes to content moderation, this is exactly where LLMs can perform much faster and better than any moderation team. LLMs are essentially designed to work with text. The only question is how they are set up. If what people here understand as AI is actually an LLM, then we should be glad that it's being fine-tuned. Those who deserve to be banned will be banned, and those who know how to communicate properly, even in roleplay, have nothing to worry about once the model is tuned correctly. The issue of context isn’t even relevant anymore. LLMs are now capable of recognizing any context and explaining to you in plain human language what was said and how. What is appropriate or inappropriate is a matter of how the model is set up, not about context.

Syldras wrote: »

I'm on PC and all I have to do is right-click on a person's name in the chat, then a menu pops up, including "whisper", "group invite", "spamlist", "report", etc.

Indeed, I checked, and there is such a button. It’s possible that if you block the person first, the button disappears. I don’t remember exactly what the issue was now, I just remember that I wasn’t able to report someone during a card game. I remember the first part: to report, you have to surrender, and I didn’t want to do that, so I blocked the person instead. Why it became difficult to report after that, I don’t remember.

Syldras wrote: »

You usually get a message that your report was received, but they won't tell you about the outcome. At least that's the way it had been a few years ago (I don't report often, only if people throw around racist, homophobic or other slurs in general chat - because I indeed don't want this toxicity in this game).

I see. In other projects, if a report leads to a ban, there is an automatic notification. I think that’s the right approach, as it lets you know whether it’s even worth bothering with reporting and whether such tickets are actually being reviewed.

ssewallb14_ESO

I'm curious as to whether or not other Microsoft owned games are taking a similar approach to monitoring and moderation.

tomofhyrule

There’s also a major difference between “this call may be monitored or recorded” and “if you engage in what an unthinking being considers wrongthink, you will be permanently removed.”

While some incidences should be immediately and harshly actioned (e.g. using game chat to plan a real-life crime), many of the incidents people are talking about are much more innocuous and require at maximum a warning instead of a ban, if not just a total “this isn’t important enough to pursue” dismissal.

After all, many of us on the forums have gotten posts snipped because something was read in one way and not meant as such. Can you imagine how much communication would be stifled if every minor <snip> resulted in a perma-ban?

Consider: even if the Code of Conduct declares (in an overly broad way, admittedly) that those actions are against the CoC, do we consider…

• Actively advocating violence against people in real life
• Relentless harassment of another player
• Telling a friend you don’t like to run Cloudrest because “Z’maja is a [snip] fight”

… to be equally bad? Do these deserve equal punishment? Because if a bot or AI is unilaterally banning people based on just a very open definition of naughty words, that’s essentially what you’re saying.

Even in a wider scope, there’s a reason the Draconian constitution of 620 BCE is not used in most societies around the world today. If every crime deserves the death penalty, then where do you go? Most modern societies temper that with anti-cruel-and-unusual-punishment provisos in their constitutions.

So yes, all communications are recorded and nobody should be surprised if they’re stored in some random vault. But we should also not assume that friendly jibing between consenting friends, where none of the people who see that chat are offended and want to report it, needs to result in immediate and automatic bans.

wolfie1.0.

Keep in mind that even though the even though ZOS is US based, the 1st ammendment does not apply. Zos can limit what you say and how they monitor it.

That said, I would hope that all AI reports get manual reviews before action is taken. Ai's and algorithmic programs have proven to not only lack nuance and take in context but proven to be overwhelmingly aggressive in how they work. Even the in game profanity filter is overwhelmingly aggressive in filtering out words. As a rule I don't use swear words or curse. But even with that in mind it's hard to have a conversation with the filter on, making it pretty much a pointless endeavor.

DenverRalphy

Is there any recorded example of bans resulting from faulty AI chat monitoring actually happening in ESO?

I've seen claims. I've seen links to articles, but those articles reference this forum as a source, which rules them out. I've seen one article that references the same claims on reddit, but those reddit claims also cite this forum for its claims, and/or make claims without foundation as well.

I am genuinely interested in seeing any concrete example.

JavaRen

I'm still searching for any evidence of anything actually happening.

tomofhyrule

manukartofanu wrote: »

If what people here understand as AI is actually an LLM, then we should be glad that it's being fine-tuned. Those who deserve to be banned will be banned, and those who know how to communicate properly, even in roleplay, have nothing to worry about once the model is tuned correctly. The issue of context isn’t even relevant anymore. LLMs are now capable of recognizing any context and explaining to you in plain human language what was said and how. What is appropriate or inappropriate is a matter of how the model is set up, not about context.

If I may ask one question regarding this statement:

If we assume that there is an LLM AI being trained to handle automodding of chats, at what point is it trained enough and appropriate to roll it out? What is the rate of false to true results that we should aim for?

You’re saying that eventually it will be able to understand what constitutes roleplay and what constitutes irl hate speech. But if it’s catching a bunch of false positives while it’s learning, then is that actually going to be a problem for the game to ban these people unilaterally and then spend the resources to have support manually unban them (and deal with a set of disgruntled players), or would it be better to lessen the punishment while the model is still learning so as not to inconvenience those players (and keep them interacting with the game)?

For example, I know in the US, we will usually have a period of a few weeks or months if a law is newly enforced (e.g. if a permanent red light camera is added to a road) where people are given warnings before the fines are applied. Similarly, it should not be the norm to suddenly enforce the CoC and ban people automatically without warning on the actions of an as-yet-untrained software.

MudcrabAttack

LLMs aren’t smart enough to decide when to ban paying customers, and probably won’t be any time soon. Especially when those paying customers spread the word to other paying customers

There’s still no word back from the ZOS team since Friday?

If this is something as simple as XYZ player was once reported by an actual human being, then placed in a sub-category of naughty list where the AI scrutinizes everything by that player, it might bother me less. I’ve seen evidence of that in the past, and if that’s the case here they could at least come out and say something to that effect.