How will the AI that monitors our in-game chats effect RP?

Syldras · September 15

SilverBride wrote: »

When we sign up for any private service we agree to their Terms of Service. If their Terms of Service says they have access to all our activity while using it then we have given them permission and have no legal recourse.

Law is above TOS. German telecommunication law definitively forbids it. EU laws? I think it's included in either the European Electronic Communications Code or the ePrivacy directive.

Pelanora · September 15

SilverBride wrote: »

Syldras wrote: »

SilverBride wrote: »

Privacy laws protect the consumer's private information, such as name, address, payment information. This doesn't fall into that category.

Depends on legislation. In EU, chats and private messages are protected just as phone calls and letters are. I don't know about the US, of course.

When we sign up for any private service we agree to their Terms of Service. If their Terms of Service says they have access to all our activity while using it then we have given them permission and have no legal recourse.

That doesn't mean that how they choose to use this permission is in the best interest of the company or their consumers. If they are indeed monitoring our private conversations they need to rethink this choice.

Did you read the tos i posted. You are incorrect.

SilverBride wrote: »

Sleepsin wrote: »

This isn't a free speech issue, this is a privacy issue. Privacy laws apply to companies even on platforms the company owns.

Privacy laws protect the consumer's private information, such as name, address, payment information. This doesn't fall into that category.

Not that I'm defending them monitoring our private conversations. I find it unnecessary and invasive and am disturbed by this.

As chat is processed as identified information not anonymously, It is personal information under privacy laws.

Pelanora · September 15

I_killed_Vivec wrote: »

Elsonso wrote: »

However, all good AI policies should include a "no action before human review" because AIs are not actually 'intelligent'. They aren't 'artificial' either, if you think about it.

Not only that but the results of a human review can be fed back into the machine learning so that it can hone its skills and stop making obvious Scunthorpe errors.

But that's an expensive process, requiring human involvement. Remember that the parent company (Microsoft) is pouring literally billions into AI precisely so that human involvement can be dispensed with, so they might not be inclined to fund the human oversight.

In fact I'm sure they are working on the basis of "ban first (AI), review later (human)".

Banning through automatic decision making will be against their Terms of Service for a significant chunk of their customers.

spartaxoxo · September 15

Pelanora wrote: »

SilverBride wrote: »

Syldras wrote: »

SilverBride wrote: »

Privacy laws protect the consumer's private information, such as name, address, payment information. This doesn't fall into that category.

Depends on legislation. In EU, chats and private messages are protected just as phone calls and letters are. I don't know about the US, of course.

When we sign up for any private service we agree to their Terms of Service. If their Terms of Service says they have access to all our activity while using it then we have given them permission and have no legal recourse.

That doesn't mean that how they choose to use this permission is in the best interest of the company or their consumers. If they are indeed monitoring our private conversations they need to rethink this choice.

Did you read the tos i posted. You are incorrect.

The part of the TOS you quoted does not apply to the US.

SilverBride · September 15

Pelanora wrote: »

As chat is processed as identified information not anonymously, It is personal information under privacy laws.

Show me where it says this.

Even if chat is identified information in that ZoS knows who owns the account, monitoring our chat does not violate the privacy laws. Sharing this identifying information, or selling it to third parties would be a violation. But this isn't.

I_killed_Vivec · September 15

Syldras wrote: »

I_killed_Vivec wrote: »

Are you upskirting Vivvy?

The better question is: Who by Mephala's right teat designs a statue that lets you look directly at the depicted person's God's crotch when walking by? It's basically right in front of your face if you stand next to it (somewhere in Necrom City, btw).

Edit: Yes, I know there are many depictions of nude Greek and Roman gods (and even a few emperors), but then it's usually, well, nude. Why add a loincloth and then choose a pose where it's completely useless?!

I don't know, from some angles...

I tried to make an Alma character

coz I killed her too!

and was disappointed that I couldn't achieve the same "you can't see anything but I'm not wearing undies" look carried off so well by the false god floozy.

It seems strange to me that you can go and look at her but you can't create a character that looks like her.

I also have Sotha Silly characters ... I would have killed him too, but Mad Alma got to him first

opallithia · September 15

Many people have expressed the concerns I share already, but I want to add my voice to the list of those who think this is sets an unsettling and dangerous precedent.

My hope is that the point of this AI was to flag things that would then be reviewed by a real person. I'm guessing that maybe something went wrong and the AI is now handing out bans/warnings/suspensions when it was only meant to flag things.

However, if that is not the case and ZOS is in fact using AI to screen messages and hand out bans based on an arbitrary filter, this makes me incredibly nervous for many reasons.

I've been roleplaying in this game pretty consistently since 2016 with a few breaks here and there. I don't keep coming back for the game itself, I come back specifically for the roleplay, which is where I made some incredible friends over the years. In my opinion, there's no better MMO out there for RP right now. That will change, though, if this moderation is allowed to continue.

One of the greatest things about The Elder Scrolls is our freedom to create our own stories within it. If we are being "pre-screened" or "pre-moderated," we will no longer have the freedom to enjoy our stories the way we want to enjoy them. Understand, people will leave because of this. Roleplayers tend to spend a lot of money on the game, so money will leave the game as well.

And what for? What's the point of this kind of moderation? The game is already Mature/18+ rated, and it already has a profanity filter on by default. I've heard rumors that it's to prevent harassment from players who are appearing offline, but that's literally what the ignore/report features are for. The game already *has* all the tools needed for moderation. If people are still complaining about harassment, it's because there aren't enough staff handling reports and banning people who are doing the harassing, and this type of job is not something that can be done by technology. This is work that needs to be done by a real person.

If ZOS is doing this to cut costs, it's punishing everyone, not to mention a loyal part of its fanbase (roleplayers), so that they can not pay workers. That should make everyone nervous.

This doesn't even begin to cover the extreme level of privacy invasion that is monitoring our private messages. I know it's technically allowed under the ToS, but that doesn't mean it's not disturbing. Unless something is reported, why are private messages between friends being moderated? I understand monitoring things for illegal activity, but for swear words? Really? You have to understand the way this makes the company look, and it's not good. A game where people can't talk freely with one another is not a game that can thrive.

I really hope this is just an error and not a deliberate choice, because if it is, shame on you, ZOS. Everyone needs to speak up about this and make their voices heard.

Sepultura_13 · September 15

spartaxoxo wrote: »

In the US, people under the age of 18 are able to play and buy M rated games. Any big game made by an American company with no restrictions on which servers they can join can and will have minors present.

(Bolded by myself for emphasis on the relevant and important information)

@spartaxoxo - thank you for clarifying. You stated the facts much more succinctly than I did.

Warhawke_80 · September 15

Since I believe in the Dead internet theory none of it really matters... everyone I interact with is some form of bot, so no actual people will read what I type...

/looks around...
What?

Syldras · September 15

So minors can buy this game and play it in the US. Okay. But does that mean the ZOS would legally have to use special measures to protect them (that, as a side-effect also affect adult players) even if they are not the intended audience and warnings are given that the games have adult topics?

The whole situation somehow sounds like showing a movie intended for adults (as in: a story including strong language, violence, nudity, drug use,...) in the cinema and then forbidding the adult visitors to talk about it, because children might be present and could be disturbed by the talk (but, for whatever reason, not by the movie itself).

Pelanora · September 15

spartaxoxo wrote: »

Pelanora wrote: »

SilverBride wrote: »

Syldras wrote: »

SilverBride wrote: »

Privacy laws protect the consumer's private information, such as name, address, payment information. This doesn't fall into that category.

Depends on legislation. In EU, chats and private messages are protected just as phone calls and letters are. I don't know about the US, of course.

When we sign up for any private service we agree to their Terms of Service. If their Terms of Service says they have access to all our activity while using it then we have given them permission and have no legal recourse.

That doesn't mean that how they choose to use this permission is in the best interest of the company or their consumers. If they are indeed monitoring our private conversations they need to rethink this choice.

Did you read the tos i posted. You are incorrect.

The part of the TOS you quoted does not apply to the US.

No company like zos can afford to run 2 different automated systems because some users may be under eu rules while some are not and somehow the systems need to know this before they process the data.

This is the interesting impact of the EU laws on digital systems. This is why US presidents fly to Europe to discuss EU data processing laws.

This is why Google makes products it sells comply with EU rules, then rolls them out globally to everyone.

Pelanora · September 15

SilverBride wrote: »

Pelanora wrote: »

As chat is processed as identified information not anonymously, It is personal information under privacy laws.

Show me where it says this.

Even if chat is identified information in that ZoS knows who owns the account, monitoring our chat does not violate the privacy laws. Sharing this identifying information, or selling it to third parties would be a violation. But this isn't.

Use of personal information needs to be under the tos including the privacy tos. That includes all information that is personally identifiable. Ie chat not made anonymous before processing.

If the use is outside the tos, it's in breach.

Making automatic decisions based on processing personal information with significant effect on the players, is against the tos.

Question is whether a perma ban is significant. I'd say it is. Perhaps the lawyers had more in mind under 'significant'

Monitoring the chat isn't the issue. I'd hope they monitoring it for 'wow this mechanism sucks' 'man I hate this WB' 'damn this Cyrodiil'.......It's acting on the monitoring that's at issue. So it's the use of the data and monitoring, to act against the player. If that's not within the tos, then there's a problem.

SilverBride · September 15

Pelanora wrote: »

SilverBride wrote: »

Pelanora wrote: »

As chat is processed as identified information not anonymously, It is personal information under privacy laws.

Show me where it says this.

Even if chat is identified information in that ZoS knows who owns the account, monitoring our chat does not violate the privacy laws. Sharing this identifying information, or selling it to third parties would be a violation. But this isn't.

Use of personal information needs to be under the tos including the privacy tos. That includes all information that is personally identifiable. Ie chat not made anonymous before processing.

They are not violating any privacy laws unless they share or sell the consumer's personal information with third parties. Them just seeing it themselves doesn't violate anything.

Pelanora · September 15

SilverBride wrote: »

Pelanora wrote: »

SilverBride wrote: »

Pelanora wrote: »

As chat is processed as identified information not anonymously, It is personal information under privacy laws.

Show me where it says this.

Even if chat is identified information in that ZoS knows who owns the account, monitoring our chat does not violate the privacy laws. Sharing this identifying information, or selling it to third parties would be a violation. But this isn't.

Use of personal information needs to be under the tos including the privacy tos. That includes all information that is personally identifiable. Ie chat not made anonymous before processing.

They are not violating any privacy laws unless they share or sell the consumer's personal information with third parties. Them just seeing it themselves doesn't violate anything.

They can only use data as per the tos. Zos can provide it to third parties as per their tos. I'm not sure what you mean by 'seeing". The use of data is what is defined in a tos and under various privacy regimes what matters is the use is notified when data is obtained, consent is obtained, not what the use is. (On the whole and edge cases excluded). The actual use is the tos, the law sits over that.

If it's in the tos, if an organisation provided notice when it got the data, it can use data accordingly. If not, it can't. Just because an organisation holds data, doesn't mean they can do anything they want with it.

JavaRen · September 15

Maybe they are using AI instead of human review -because- of privacy concerns. This way no person is ever aware of the contents of your communiations.

Disclaimer: I'm not a lawyer or any kind of proffessional in this regard.

spartaxoxo · September 15

As a reminder, this usage is in the TOS.

Content Moderation

To the extent that ZeniMax performs any content moderation of UGC to ensure its compatibility with these Terms of Service (including the Code of Conduct or any relevant EULA), such content moderation may be carried out via human review as well as through the use of AI-powered proactive and reactive moderation methods including without limitation, software that uses algorithmic decision making.

ZeniMax's proactive content moderation includes without limitation using tools to block and filter UGC that is illegal and/or incompatible with these Terms of Service.

Reactive content moderation methods include without limitation user reporting features which allow You to inform ZeniMax of the behavior or Content of other users that You have encountered which you believe is illegal and/or incompatible with these Terms of Service (including the Code of Conduct) and any such behaviour or Content can be reported to ZeniMax by contacting ZeniMax Customer Support at help.bethesda.net or help.elderscrollsonline.com. If You are in-Game, You can report an issue using an in-Game help feature where applicable. Where ZeniMax is required to do so by virtue of the Statutory Obligations (as defined in Section 1), ZeniMax shall advise You of remedial steps action taken against another user as a result of Your report including details of what steps ZeniMax has taken to investigate your report, if ZeniMax has removed Content that You have reported or if any other restrictions have been applied to the Content or the other user.

Syldras · September 15

JavaRen wrote: »

Maybe they are using AI instead of human review -because- of privacy concerns. This way no person is ever aware of the contents of your communiations.

And this was fine under two circumstances:
- applying it only for topics that indicate that a real and serious crime could be planned
- double-checking of the "positives" by a real human ZOS employee

Doesn't explain something as silly as banning people for swearing.

Elsonso · September 16

Generally, I assume that all Terms of Service and Software License Agreements are written to allow the Company to do whatever they want, within the limit of whatever jurisdiction they adhere to, while restricting what the Consumer is allowed to do.

Drinks_from_Ponds · September 16

Where are good places to find RP at? I only know of Evermore inn.

Alp · September 16

Drinks_from_Ponds wrote: »

Where are good places to find RP at? I only know of Evermore inn.

Probably depends on your platform and server.

blktauna · September 16

I'm sure some actual lawyers will be interested. Jurisdiction and what can actually be enforced in a contract are very nitpicky things

Drinks_from_Ponds · September 16

Alp wrote: »

Drinks_from_Ponds wrote: »

Where are good places to find RP at? I only know of Evermore inn.

Probably depends on your platform and server.

I am on PC-NA

SilverBride · September 16

Not everything that is covered by the ToS is a good idea to put into practice. I find it very disturbing that adults playing a mature game are being monitored in their private chats looking for things they could be moderated for.

Rkindaleft · September 16

A bit late to the party - but this is genuinely awful.

I was on holiday for a while and I came back to people today in my discords telling us to watch what we say in DMs because you're running a risk of getting banned by what seems to be unchecked chat moderation.

Whose idea was that? In this case the AI simply cannot determine the frame of reference or the circumstances of what a message’s real meaning is or why it’s written. The AI is unable to distinguish between a message that is genuinely offensive or a message that is written in jest, for friendly banter between mates, for RP or any other innocuous reason.

Unrelated but I absolutely adore this game, I was subbed for like 7 years straight until Gold Road, ended my sub 3 months ago because it felt to me like it was just one unpopular change after another, and this is just another unpopular decision in a list that seems to get increasingly longer.

katanagirl1 · September 16

All this monitoring and the gold sellers are still spamming in every zone chat I visit.

MJallday · September 16

PeacefulAnarchy wrote: »

Are you just assuming there's an AI that monitor chat, or is this stated somewhere?

If there is, it must be asleep. Or brain dead, due to the amount of toxicity I see daily.

dk_dunkirk · September 16

jaws343 wrote: »

Inari Telvanni wrote: »

TOS says,

" Content Moderation

To the extent that ZeniMax performs any content moderation of UGC to ensure its compatibility with these Terms of Service (including the Code of Conduct or any relevant EULA), such content moderation may be carried out via human review as well as through the use of AI-powered proactive and reactive moderation methods including without limitation, software that uses algorithmic decision making.

ZeniMax's proactive content moderation includes without limitation using tools to block and filter UGC that is illegal and/or incompatible with these Terms of Service.

Reactive content moderation methods include without limitation user reporting features which allow You to inform ZeniMax of the behavior or Content of other users that You have encountered which you believe is illegal and/or incompatible with these Terms of Service (including the Code of Conduct) and any such behaviour or Content can be reported to ZeniMax by contacting ZeniMax Customer Support at help.bethesda.net or help.elderscrollsonline.com. If You are in-Game, You can report an issue using an in-Game help feature where applicable. Where ZeniMax is required to do so by virtue of the Statutory Obligations (as defined in Section 1), ZeniMax shall advise You of remedial steps action taken against another user as a result of Your report including details of what steps ZeniMax has taken to investigate your report, if ZeniMax has removed Content that You have reported or if any other restrictions have been applied to the Content or the other user. "

This is an enormous red flag.

No it is not..

Zos, as a company that can be held liable for conversations that might happen through the use of their platform, have a responsibility to make sure that they can monitor illegal things being done in their game. Even if those things are done in private chats. If you are using their tools to do illegal things, they are legally put in a spot to intervene or face severe legal repurcussions if they do not.

No on the no.

As of right now, the hotly-debated "Section 230" absolves "platforms" of legal culpability that might arise from content created by their users. It's the only reason that Facebook ever existed to begin with. They have an army of people policing content to conform to their TOS, but even they would go out of business if they had to ensure that literally everything on their service was legal. So, no, under Section 230, ZOS can NOT be held liable for what people say in chat on their platform. That being said, there's talk of repealing 230, and maybe ZOS just wants to get ahead of the game here, but that's a different conversation.

Jaraal · September 16

tyran404_ESO wrote: »

I would post the CS screenshots but I think that's not allowed on the forums here? Correct me if i'm wrong there.

There are plenty of instances of screenshotted customer service interactions that have been posted here in the forums. The key is redacting any potential identifiers, such as character names, guild names, etcetera. Screencaps that don't render the participants anonymously are subject to the "naming and shaming" section of the TOS.

Stafford197 · September 16

opallithia wrote: »

Many people have expressed the concerns I share already, but I want to add my voice to the list of those who think this is sets an unsettling and dangerous precedent.

My hope is that the point of this AI was to flag things that would then be reviewed by a real person. I'm guessing that maybe something went wrong and the AI is now handing out bans/warnings/suspensions when it was only meant to flag things.

However, if that is not the case and ZOS is in fact using AI to screen messages and hand out bans based on an arbitrary filter, this makes me incredibly nervous for many reasons.

I've been roleplaying in this game pretty consistently since 2016 with a few breaks here and there. I don't keep coming back for the game itself, I come back specifically for the roleplay, which is where I made some incredible friends over the years. In my opinion, there's no better MMO out there for RP right now. That will change, though, if this moderation is allowed to continue.

One of the greatest things about The Elder Scrolls is our freedom to create our own stories within it. If we are being "pre-screened" or "pre-moderated," we will no longer have the freedom to enjoy our stories the way we want to enjoy them. Understand, people will leave because of this. Roleplayers tend to spend a lot of money on the game, so money will leave the game as well.

And what for? What's the point of this kind of moderation? The game is already Mature/18+ rated, and it already has a profanity filter on by default. I've heard rumors that it's to prevent harassment from players who are appearing offline, but that's literally what the ignore/report features are for. The game already *has* all the tools needed for moderation. If people are still complaining about harassment, it's because there aren't enough staff handling reports and banning people who are doing the harassing, and this type of job is not something that can be done by technology. This is work that needs to be done by a real person.

If ZOS is doing this to cut costs, it's punishing everyone, not to mention a loyal part of its fanbase (roleplayers), so that they can not pay workers. That should make everyone nervous.

This doesn't even begin to cover the extreme level of privacy invasion that is monitoring our private messages. I know it's technically allowed under the ToS, but that doesn't mean it's not disturbing. Unless something is reported, why are private messages between friends being moderated? I understand monitoring things for illegal activity, but for swear words? Really? You have to understand the way this makes the company look, and it's not good. A game where people can't talk freely with one another is not a game that can thrive.

I really hope this is just an error and not a deliberate choice, because if it is, shame on you, ZOS. Everyone needs to speak up about this and make their voices heard.

Went ahead and informed my guilds about what’s going on. ZOS said they would reply here this week, so based on what they say it could definitely be worth having larger content creators cover this topic to get the word out!

ZOS losing the trust of consumers is not a good marketing strategy for their upcoming AAA title.

alternatelder · September 16

Drinks_from_Ponds wrote: »

katanagirl1 wrote: »

All this monitoring and the gold sellers are still spamming in every zone chat I visit.

[snip]

Pretty sure it's a perma ban for buying gold from gold sellers.

[edited to remove quote]