33

A couple of days ago, Stack Exchange Inc., the company running these sites, announced a new policy for handling AI content. This policy goes against our own, local policy on how to deal with AI content, where the U&L community made it very clear that such content was not welcome: How do we want to handle AI-generated answers?

I am writing this post to make sure that the U&L community is aware of this new network-wide policy and to provide a place where we can discuss it and maybe decide how to handle it. The full text of the new policy can be read below, but I urge you to visit (and vote on) the original on Meta Stack Exchange as well:

We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high. Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. Finally, internal evidence strongly suggests that the overapplication of suspensions for AI-generated content may be turning away a large number of legitimate contributors to the site.

In order to help mitigate the issue, we've asked moderators to apply a very strict standard of evidence to determining whether a post is AI-authored when deciding to suspend a user. This standard of evidence excludes the use of moderators' best guesses based on users' writing styles and behavioral indicators, because we could not validate that these indicators are actually successfully identifying AI-generated posts when they are written. This standard would exclude most suspensions issued to date.

We've also identified that current GPT detectors have an unacceptably high false positive rate for content on our network and should not be regarded as reliable indicators of GPT authorship. While these aren't the sole tools that moderators rely upon to identify AI-generated content, some of the heuristics used have been developed with their assistance.

We've reminded moderators that suspensions (and typically mod messages as well) are for real, verifiable malfeasance only, and should not be enacted on the basis of hunches, guesses, intuition, or unverified heuristics. Therefore, we are not confident that either GPT detectors or best-guess heuristics can be used to definitively identify suspicious content for the purposes of suspension.

As always, moderators who identify that a user has a problematic pattern of low-quality posts should continue to act on such users as they otherwise would. Indicators moderators currently use to determine that a post was authored with the help of AI can in some cases form a reliable set of indicators that the content quality may be poor, and moderators should feel free to review posts as such. If someone is repeatedly contributing low-quality content, we already have policies in place to help handle it, including a suspension reason that can, in those cases, be used.

11
  • 15
    I should also point out that the private policy shared with moderators (yes, they told us one thing and you another and we're not allowed to share) is much more restrictive than what can be seen here. What you have is a description of the policy, not the explanation of the "strict standard" we've been asked to apply.
    – terdon Mod
    Jun 2, 2023 at 22:18
  • So you can't suspend users posting AI-generated crap, but deleting that is still fine, right?
    – muru
    Jun 3, 2023 at 3:27
  • 10
    @muru no, in private, staff have told us we can take no "Moderation action", including deletion.
    – terdon Mod
    Jun 3, 2023 at 12:09
  • 3
    Ah, ok. But regular users like me can still vote to delete them, I suppose.
    – muru
    Jun 3, 2023 at 14:36
  • 5
    @muru I sure hope so.
    – terdon Mod
    Jun 3, 2023 at 14:37
  • You all would have gotten a more understanding vision if my requests for transparency on how the ban on generated content was applied and how it was evaluated was met without so much push back. This kind of SE response was the very thing I was trying to avoid. Moderated response, based on data, rather than kneejerk and inflexible responses based on rhetoric are my mojo, you all should try too.
    – Braiam
    Jun 7, 2023 at 13:57
  • 3
    Yes, @Braiam, we are all about inflexible, knee jerk responses. If that's your takeaway after all these years, I don't know what to tell you.
    – terdon Mod
    Jun 7, 2023 at 14:43
  • 1
    It takes only one bad experience, to spoil the entire thing. What do you tell me? Here, look at the data, pour over it and analyze it. Lets make this thing better for all. That's what you should tell me. That's what I would tell you if our positions were reversed. Hell, I would not wait for this comment either. So, try again.
    – Braiam
    Jun 7, 2023 at 15:06
  • BTW, ask physics.se mods to get the same table they got which was what I asked, that would be a starter. physics.meta.stackexchange.com/a/14441/30753
    – Braiam
    Jun 7, 2023 at 15:36
  • 6
    I honestly have no idea what you are talking about. I remember you had posted something about AI a while back, but I don't remember the details. Nor, frankly, is this the place to discuss them. If you have something to say, open a new meta discussion.
    – terdon Mod
    Jun 7, 2023 at 22:22
  • I don't know how do you not know what I was talking about, I literally posted both comments withing 30 minutes one of the other and less than an hour from yours. So, if you do not have idea what I'm talking about, maybe read the first comment which is like 4 comments above yours and don't even need to be unfolded.
    – Braiam
    Jul 6, 2023 at 16:13

2 Answers 2

11

Personally, I consider the previous policy as both appropriate and existentially necessary for the future of the community. I support and encourage any moderators here in joining the general strike.

While I respect the right of each person to choose for themselves, I'm looking for the exit. I just don't see that there is much meaningful room for compromise here on the community side. None of us are interested in spending our time competing with, much less refining, garbage generated output - for others to claim both the credit and the profit. From what I gather that sentiment applies to the mods as well.

I'm still watching for now, but I'm already starting to move over to Codidact and see how I can contribute there. As a result, I now care more about the data dumps, so this is doubly bad timing for SE. In order for me to resume meaningfully contributing here as well, I would need to know my effort will be equally available to all in the dumps going forward.

Basically, because of this AI debacle, I trust SE significantly less than before. Because I trust them less, the dumps are more vital consideration for my participation than ever before. Thus unless they reverse course on both issues, I will probably, finally, fade away from here entirely.

Either way, best of luck to all those working to resolve the situation with SE, and I remain grateful to everyone who has contributed to this wonderful community resource over the years.

0

May I start by thanking you for bringing this to the community's attention. This is a very serious matter that concerns us all. We are all very worried about the extremely detrimental effect that AI content could have in the future.

I personally am worried about the issue generally after I stumbled on a website for a supposed "School" containing thousands of articles which I believe might have been largely generated through AI content for the sake of one or perhaps a few people's CV. This does not bode well for the future of knowledge sites like U&L.

But Keep your powder dry...

May I urge our moderators here to exercise caution and restraint on this topic. Keep your heads cool as possible.

I acknowledge the current "general strike" happening over this matter and I realise our own moderators might have chosen to take part. I would urge you not to take part in future action until, at least, some more has been made clear:

Give SE opportunity to fail

I find this teaching one of the most excruciating to follow as a mentor, but it is invaluable. Until the real impact of this policy can be directly seen combined with the [lack of] tooling SE have chosen to adopt, none of us really know the future.

SE have asked moderators not to take action and there is open discussion that they have set a strict policy that general user base is not privy too. What that does is to put the issue squarely in the hands of SE. SE might well have good reason for keeping even more hidden from moderators to prevent leaks to the public.

It will be really painful to watch them fail at this, but this is really a topic where we MUST let them try [and fail] rather than not even letting them try.

SE are not wrong in the [publicised] justification for this

I think our moderators here are ace! Really some of the best on SE.

SE have highlighted two particularly thorny issues:

  • There is a worryingly high number of false positives
  • There is a real concern about racial bias

Nobody is calling anybody racist! The problem is that people have used writing style as an indicator of ChatGPT content and this is absolutely prone to false positives. ChatGPT was trained from real language after all!

What's really worrying about this is the idea that people who were educated by particular schools or who learned English through a particular educational system are more prone to having their content falsely flagged as AI generated... leading to a really pernicious racial bias that cannot be easily identified or controlled.

The extent of this problem is not yet well understood, but at this time it's of sufficient concern that it could genuinely get big organisations sued.

There are better tools than people for this

SE have not publicised, nor are they likely to publicise, the tools they engage to begin catching AI content. But theoretically technology is likely to be better than people at this.

Again, we have great moderators. But I want our moderators to spend their time moderating user content, not chasing down bots and those who copy-paste from ChatGPT.

There are many features of cheats that become so much easier to spot with technology and with data that is not available even to moderators. E.g. ChatGPT does not commonly spend time writing its posts and forming saved drafts as they go. Click-through time between reading a question and posting sizeable answers is much lower. The relationship between which questions appeared on screen and how many immediately got answered by that user is wholly different. The time spent browsing questions is different.

Strike action hurts us all

As a gentle reminder, those who get hurt by this strike most are the user base. Historically strike action is as likely to anger customers as much as change the minds of management... and that anger, historically, gets directed at those striking.

I really think our mods here on U&L are great. I am genuinely worried that further action could create "friction" between users and moderators.

This isn't to say anyone should put up with poor treatment. But where strikes have such a poor success rate and high collateral cost, I'd advise anyone to exhaust as many possible alternatives as you can find.

And finally

The final solution to this hellish nightmare MUST NOT be asking moderators to clean up a flood using a mop and bucket. I really do empathise with moderators who feel so very let down by SE right now.

Instead of using this strike action to demand back your shiny mod hammer, consider demanding that SE do more to implement better tools themselves that will prevent it even getting to your inbox.


You guys are AWESOME! Hope to see you back here soon.

32
  • 7
    First, thank you for posting a very measured and constructive answer, as is your wont. I think you're wrong, for reasons I will explain, but it's always good to hear a different point of view, and especially when presented earnestly and politely. So thanks! [cont...]
    – terdon Mod
    Jun 6, 2023 at 22:05
  • 8
    Now, there are a couple of misconceptions here: first, technology sucks at detecting AI, at least at the moment. In fact, that is precisely the point: SE assumed we used AI detectors, which we don't 'cause they don't work very well. At best, the AI detector would be one more data point. Next, SE has no reliable way of detecting AI posts, unfortunately. This much they have shared. So no, tech is worse than humans, at the moment, at detecting AI content. [cont...]
    – terdon Mod
    Jun 6, 2023 at 22:05
  • 10
    And, by the way, SE have shared some data with us (finally) and there are some trends there that are worrying, but it isn't at all clear that the data actually support the conclusions they have drawn. For one thing, and this is important, they cannot point to even a single example of an unfair suspension. Don't get me wrong, I'm sure there are some, humans are fallible and we all make mistakes. But SE do not have any such data. Not a single, confirmed false positive suspension or deletion for AI content.
    – terdon Mod
    Jun 6, 2023 at 22:05
  • 12
    The crux of the matter, however, is that after the last time SE behaved really, really badly, they promised to do better. Specifically, they promised that all new policies would be reviewed by representatives of the community in the form of a group of moderators. They promised that policies would go through a period of feedback. They even told us this in writing. And yet, here we are again. [cont...]
    – terdon Mod
    Jun 6, 2023 at 22:06
  • 7
    Once more, the moderators are being accused unfairly and slandered in the press, once more the community is treated like a product and it feels like nothing has changed. Yes, the moderators are convinced the new AI policy is wrong, but perhaps more importantly, we feel we are being lied to again. [cont...]
    – terdon Mod
    Jun 6, 2023 at 22:06
  • 7
    We feel, again, that the company running these sites not only takes the work we (all of us, all those who participate on these sites, not mod only) do for granted and that while it expects us to follow the rules, they are free to break the binding promises they gave us. So the AI issue is the tip of the iceberg. Had the company come in and simply asked us to maybe lay off suspending for a while, we'd have been more than happy to do it. Unfortunately, they came down on us like a ton of bricks, laying down the law and requiring us to go against the community consensus of our own sites.
    – terdon Mod
    Jun 6, 2023 at 22:06
  • 7
    So this is not just about AI. It is about how the company treats the very communities that power its growth. Which is why I, personally, have indeed joined the strike. I have chosen not to post about it here, because I don't want to bring the drama myself, but this has affected me on a very personal level and it was a choice between a strike in the hope of effecting change or just walking away. Not all the U&L mods are striking, and that is absolutely fine. Everyone should do what their own conscience dictates, and I have nothing but respect and affection for my fellow mods.
    – terdon Mod
    Jun 6, 2023 at 22:08
  • 7
    Oh, and for the record, this hasn't been a big deal here. Yes, we have deleted a few posts for being AI generated (and we are absolutely confident we were right, there was a pattern of posting, and not even a complaint), but we have issued a negligible number of suspensions over this. It's SO that has had to deal with AI posts in their thousands.
    – terdon Mod
    Jun 6, 2023 at 22:16
  • 1
    @terdononstrike I think you make some very important points particularly on SE corporate culture. When you have the time and emotional bandwidth, I think it'd be worth getting these into a follow up answer here. We certainly both agree that SE has a painful history of "mistreating" it's community. Trying to give them the benefit of the doubt they must handle a horrifying dichotomy balancing corporate goals against community stuardship. They have in my view however slipped on a number of bannana skins over the past few years which have left us all a bit frustrated. Jun 6, 2023 at 22:18
  • 2
    "Through no fault of moderators' own, we also suspect that there have been biases for or against residents of specific countries as a potential result of the heuristics being applied to these posts. " Sounds like the mods are being called bigots too unlearned as to recognize their own bigotry. Quite unacceptable from a corporate communication.
    – Chindraba
    Jun 6, 2023 at 23:43
  • 2
    @Chindraba no I don't accept that point at all. Actually I think this is a topic that must not be silenced but discussed much more freely. We have become too sensitive about the topic of racism to admit one terrible truth: Racial bias is a natural and very common consequence of normal human pattern matching and simply not knowing which patterns only exist in your own culture. This means everyone will naturally show racial bias and we have to take a lot of personal time and effort individually unlearn our own unhelpful stereotypes. Jun 6, 2023 at 23:50
  • 10
    As for that second-to-last section, if you really think volunteer mods should stay working for a company that doesn't care for them, for free, regardless of the treatment, just for the sake of not annoying users, well, I suppose I can only invite you to seriously reconsider what that sounds like. Besides, wouldn't volunteer moderators and contributors striking en masse only help SE's opportunity to try what it is they appear to want (and to fail)?
    – ilkkachu
    Jun 7, 2023 at 7:42
  • 3
    ... that is I largely view strike action like a hail mary pass trying to avoid quitting all together. My heart in writing this post is "please don't quit". But at the same time I'd defend anyone's decision to do so under the circumstances. Jun 7, 2023 at 9:02
  • 6
    Honestly, I do wonder why so many mods bother with trying to talk sense to/with the company (e.g. "Please follow your own policies on making new policies") and all the other "fuss" instead of just accepting that SE doesn't want to play with them the way they'd hoped, and walking away. (Ok, the fuss might help get support for the cause and eventually get other mods and users to do the same.)
    – ilkkachu
    Jun 7, 2023 at 14:12
  • 2
    @ilkkachu I think someone tried that model, it was called "experts exchange" and it annoyed me so much I went into my own google settings to say don't return links from this site, ever. I wonder sometimes what happened to the website but I'll never know because google won't tell me. 🤭 Jun 7, 2023 at 14:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .