June 5 General Moderation Strike: Recap, Possible roles, consequences and other relevant info

Question

I won't beat the bush here: for cognizance of the community, I would briefly state all the recent events that unfolded over the past week or so. And then (my personal opinion) there would be a message for the community.

What is happening?

The menace of AI-generated content is not a new thing. But how they were and would be handled is the bone of contention.

SE Inc. claimed (albeit without bothering to show the exact figures, methodologies and analyses):

We recently performed a set of analyses on the current approach to AI-generated content moderation. The conclusions of these analyses strongly indicate to us that AI-generated content is not being properly identified across the network, and that the potential for false-positives is very high.

Then they rather blatantly imposed on the moderators new policy to handle suspension emanating from such posts and dictated:

require[ing] an immediate cessation of issuing suspensions for AI-generated content and to stop moderating AI-generated content on that basis alone, affording only one exceptionally rare case in which it was permissible to delete or suspend for AI content.

Why did Mods go to strike?

As has been conspicuous in the actions taken by SE, of late, this time too it was marred by poor interaction with the stakeholders and rather stern standing against any form of constructive feedbacks.

More specifically:

The new policy overrode established community consensus and previous CM support, was not discussed with any community members, was presented misleadingly to moderators and then even more misleadingly in public, and is based on unsubstantiated claims derived from unreviewed and unreviewable data analysis. [...] There has been a lack of communication with moderators and a lack of communication with the community. When communication happened, it was one-sided, with Stack Exchange, Inc. being unwilling to receive critical feedback.

So

As of today, June 5th, 2023, a large number of moderators, curators, contributors, and users from around Stack Overflow and the Stack Exchange network are initiating a general moderation strike. This strike is in protest of recent and upcoming changes to policy and the platform that are being performed by Stack Exchange, Inc.

Moderators' response

Moderators are a vital cog in the machine. They countered the allegation made:

The decision to delete content or suspend users is always taken seriously, and experienced SE moderators are not foolish enough to rely blindly on ... any automated tool, really. Just like we don't automatically suspend everyone who shows up in "suspicious votes" tools, we don't automatically delete every post that automatic GPT detector tools claim is AI-generated, nor suspend every such poster.

To reiterate:

Just one tool saying one answer is AI is not enough to get a suspension. Moderators look for various heuristics, including posting patterns, answer speed, writing style, and others, in order to identify AI-generated content. This can include checking an automated detector, but this is not a primary method used, due to the false positive rate. While any one method can be unreliable, when an answer matches multiple heuristics, moderators can be relatively confident that the post is indeed AI-generated.

As Sycorax - On Strike aptly asserted:

The change in policy is poorly justified and communicated (and SE employees have said contradictory things in different channels in the past week), but the worse part of it is that SE is ordering us, as elected moderators, unilaterally on how to do our volunteer work. We were elected to apply our best judgment to maintain & improve our sites, using donated time and expertise, but SE Inc has decided that they Know Best, Actually (™️)... for secret reasons.

In solidarity with the mods

The recent rule imposition is an antithesis of what SE stands for. Our long goal is to nurture a high-quality repository of no-nonsense q&a. Botched implementation of such half baked rules and diktats would be nothing but an hindrance with respect to the spirit of SE with the pretext of attracting new users (which is also important but not in the name of loosening moderation over GPT generated garbage).

So, I urge the community to sign this open letter - yes, even users without diamonds can do that too - and show solidarity with our moderators. We are proud two of our moderators have signed in the letter (Sycorax - On Strike ♦, Scortchi - Reinstate Monica ♦).

As the letter suggests:

Even if you are not a moderator you can participate in the strike by:

$\bullet$ Not voting on posts

$\bullet$ Not submitting edits

$\bullet$ Not reviewing in the review queues

$\bullet $ Not commenting

$\bullet$ Not flagging posts

without resorting to any form of vandalism or other destructive tactics.

I would like to quote Scortchi - Reinstate Monica:

These rights hardly amount to democracy; but they do amount to the difference between enlightened despotism & mere despotism, between governance through promulgated standing laws & through extemporary arbitrary decrees.

All are welcome to act according to their own will - they can agree without even adhering to the suggestions altogether.

---

Update as of $26$th of June: link.

Update as of $4$th of August: Moderation strike: Results of negotiations. Also this.

Answer 1

I'm primarily participating in the strike because SE has

1. Used the private, moderator-and-staff-only Teams platform to state a new policy for moderating AI-generated content. Mods are instructed not to share things posted to the private Teams platform, so I won't go into detail. The justification for this change in policy makes allusion to an analysis that SE conducted internally, but was not shared with the moderators at the time the policy was announced. (Some elements have been shared since then, but I haven't had time to review it.)
2. Stack Exchange, Inc. seems poised to unilaterally change the moderator agreement, without soliciting our input.

Both announcements make factual claims about how AI content is moderated that I have trouble believing.

My objections are

1. The two statements are not entirely consistent with each other. I can't follow a policy that is inconsistent. Moreover, SE staff have also stated different versions of the policy in mod-only chatrooms, so even among SE staff, there is not a coherent understanding of the policy.

2. It strikes me as deeply troubling to have a secret policy (the one posted to the private Teams) that governs how users interact with the site. I think the site can only work and be fairly moderated of the site is transparent about that process.

3. Diamond moderators and site curators want to be a part of the solution to moderating AI content. We want to find bad content and remove it. Stack Exchange has the stated purpose of creating a high-quality repository of question-and-answer content and this is a goal that we believe in; it's why we joined the site and volunteered our time. Catching us by surprise with a binding change to how we moderate content is not the way to improve how we do that task.

4. Moderators have been asking for years to have better tools to help us detect low-quality content. Stack Exchange Inc. hasn't built them. But now that the network is facing a massive influx of AI-generated content, Stack Exchange has additionally announced that we are forbidden from using GPT detectors as a heuristic to aid in detection of low-quality content. Ok, fine, great, we can accept the loss of a tool that's prone to error, but if Stack Exchange not going to build any tools that do work, how are we supposed to do this job?

All of this is avoidable. Stack Exchange, Inc. has policies and processes for changing how the site works. There are also innumerable ways that they could solicit moderator input and feedback on how to achieve their goals. The sites themselves are home to hundreds, if not thousands, of experts in a variety of fields. Moreover, these people are all actively involved in using and moderating the sites. Surely there is some value to be found in tapping their expertise in curating content.

I recognize that my objections are primarily based in procedural issues, not the policy itself. It might be hard to understand why I feel so strongly about procedure, but imagine that you had volunteered to a particular role and then were subjected to arbitrary revisions to what that role is and the expectations for your performance. Moreover, you do not have the tools you need to do the job efficiently, so you improvise with some (relatively poor) tools... then they take away those tools. I think most people would object to that.

I think that a lot of the problems come down to poor communication. I am consistently baffled by how Stack Exchange staff interact with the volunteers (diamond moderators or otherwise) who do the grunt work of keeping garbage off the sites.

The coordinated action of a group of moderators working together to influence Stack Exchange, Inc. seems like the best way to effect change. This is why I decided to join the strike.

---

Whuber writes

My sense is we had developed a consensus that we should continue moderating posts on their intrinsic merits -- not on whether a chatbot (or for that matter, a computer algebra system or even a calculator) was employed to create them. (Personally--this might not be part of the consensus--I see potential advantages in the judicious use of chatbots to perform research and improve writing style, provided the poster takes responsibility for the accuracy of their work.)

As a philosophy of moderation, I agree: the reason to remove low-quality content is that it is low-quality content. If a responsive, factually correct and well-reasoned answer is submitted, we are all better off, regardless of the tools that produced it.

The problem is that the numbers are against us. AI is an automatic bullshit machine, so automation or persistence among bad actors might flood the site with useless content. The past few decades are replete with examples of cases where humans were overwhelmed with computer-generated bullshit. There was a period not too long ago when spam nearly made email unusable. To this day, various kinds of computer security issues persist due to bad actors using automated tooling to create malware and malicious websites.

This is especially close to home for me because I work in the computer security field. I've seen how this might end: without dramatic improvements to the tooling and automation around how content is curated, Stack Exchange will not be a useful resource because it will be overwhelmed with garbage, AI-generated answers.

So far, Cross Validated has been fortunate that we have not seen the same flood of poor answers that the high-traffic network sites have. But I don't know how long that will last.

---

Regardless of other site moderators participating or not participating in the strike, I hold them in the highest esteem.

Answer 2

Thanks to all the moderators --- your great work is appreciated

I'm not a moderator, and I have only a thin knowledge of this controversy, based on reading some of the posts relating to this strike. My main purpose in this answer is to thank all our moderators for their work on this site and to note that I empathise with the difficulties they are experiencing dealing with an influx of AI-generated answers. This seems to me to be an inherently hard problem to deal with, so I can only hope that our moderators are given appropriate support. I'm a big fan of whuber's recommendation that moderators continue moderating posts on their intrinsic merits (once they are no longer on strike). I have confidence that our moderators can do a good job dealing with AI-generated answers if properly supported.

With that it mind, a general request from the community:

• Do not post AI-generated answers without attribution to the relevant AI platform. If you want to post an answer that was generated using an AI language tool, make it clear in your answer that this is where it came from and cite the source appropriately (e.g., cite the tool you used and the query that generated the text). Let moderators deal with your post with the aid of candid attribution of your sources.

• If you are the unfortunate victim of an incorrect attribution of AI-usage, and your question/answer is closed, deleted, etc., try to remain calm and understand that some false positives are an inevitable result of binary predictions applying to thousands of data points. (As statisticians, this should not come as a shock to us.) If you have been incorrectly flagged as using an AI language tool for an answer, stay calm and talk to the moderators about it. It is understandable to feel somewhat offended at having been flagged as AI if you have written your own work, but temper that feeling by acknowledgement that perfect prediction in all cases is an unrealistic expectation.

Answer 3

Some context

This is news to me, so please don't read anything into what actions I might have taken or did not take. I devote my time to this site, not to SE generally; but I would think that in any real strike the organizers would (at the very least) have attempted to contact all diamond mods across the system.

About the letter

I have followed some of the Meta discussion about AI-generated content and am aware of SE's recent statements about it. However, I have not even read the entire open letter because its governing assumption, "Stack Overflow, Inc. has decreed a near-total prohibition on moderating AI-generated content," does not accord with the facts as I know them. That misconstrues the statements I have read and appears to me like an overblown reaction. Unless additional, contrary facts come to my attention, I don't think I will be signing that letter nor changing my interaction with the site.

Implications for us

As far as how CV moderates AI-generated content, your diamond mods have been discussing this privately for over half a year (since the public release of ChatGPT). My sense is we had developed a consensus that we should continue moderating posts on their intrinsic merits -- not on whether a chatbot (or for that matter, a computer algebra system or even a calculator) was employed to create them. (Personally--this might not be part of the consensus--I see potential advantages in the judicious use of chatbots to perform research and improve writing style, provided the poster takes responsibility for the accuracy of their work.)

What your mods have been doing

We are, however, closely monitoring the site for evidence of abuse of chatbots and we have been quick to close down activity that reflects such abuse, such as posts that spew statistical nonsense or users who appear to be spamming the site with nothing but chatbot output. But this doesn't require any change in our moderation policy at all and, as far as I can tell, is consistent with recent policy statements from SE.