As a lot of us know, AI has become a increasingly bigger problem on Dev.to. Whether it's spam or listicles, we've all likely seen AI generated content in one form or the other. Some of the more active Dev.to moderators, myself included, have discussed this and see it as an important issue, but because a lot of that discussion takes place behind closed doors (in other words, the Dev.to moderator discord), I wanted to compile a short post of thoughts and ideas that brings a bit of transparency to the ongoing conversation about AI generated content.
Complaints about AI generated content
It feels like so many of the posts are clout-chasing LLM-generated crap.
-An anonymous moderator
It's not exactly DEV's fault, the internet is mostly AI-generated content and listicles these days.
-An anonymous moderator
I used to read some good stuff and it’s been non stop LLM pushing and listicle style for a while. I admittedly have stopped marking low quality and have abandoned even attempting to read Dev for the past couple months.
-An anonymous moderator
As for my own thoughts, AI has helped low quality content become much more popular. Low quality content used to be pretty easy to spot, but it's harder to filter through low quality content now because 90% of is AI generated.
So what should we do?
That last part was a little bit nihilistic, so I'd like to bring some ideas about how to deal with AI generated content from our moderators to the table. I'd also like to hear from you guys (the comments, that is), so if you have any ideas please feel free to comment them!
... maybe an automatic quality checker would be nice on DEV, like an API that after posting an article checks for the chances of it being AI-generated ...
-An anonymous moderator
... (how about we) force new accounts to wait a {periodOfTime} before they can post content on Dev. This eliminates a large chunk of the regurgitated AI post and run for clicks.
-An anonymous moderator
Have we considered having a "listicle" type, and then letting folks filter those out?
-An anonymous moderator
I think that there are two stages to this idea: suppress publishing of bad written / poor AI content; encourage people to get back to read good content on DEV; ..." Side note: "this idea" pertains to the idea of an automatic quality checker and the potential side effects of it.
-An anonymous moderator
I'd like to acknowledge one potential issue: false positives with AI detectors. This could certainly be an issue, but the amount of false positives would adamantly be negligible. Additionally, these false positives could simply be appealed, and a human could look over the post to clarify that it isn't AI generated.
To conclude...
What do you think about these ideas? As I previously mentioned, if you have any yourself, please feel free to say something. AI generated content is a community-wide issue, and it should be handled as such.
A side note for the people that I've quoted: If you would like to be credited (i.e. your username instead of "An anonymous moderator"), just let me know. I figured it would be better to air on the safe side and not involve anyone who didn't want to be involved.
Top comments (52)
Since I’ve followed the conversation from the start (includes the Dev.to Discord moderation server). I do wanna add some more detail on the "automatic quality checker".
My thoughts were a "trustScore" let’s say between 0-100. This score is then given depending on certain checks, now whoever uses the API could choose when and what to do, is a disclaimer good enough or completely removing the content, if the score is let’s say 90 or above.
Any thoughts?
I like that. I think platforms like GTPZero already kind of do that, but one could certainly expand on that to include other factors (not sure what factors, but still). You could also consider forcing a post that meets your "TrustScore" requirement to have a little badge at the top that says something along the lines of "Hey! This post is very likely to be AI generated" (unless the author appeals, of course).
Anyways, a lot of this is still "theory", and implementing it and putting it into practice may be a pretty big challenge -- not that I would know where to start.
This way of handling AI-generated content should be quite fair, and if we add another level of validation, it could be the best way to handle the situation, in my opinion. I don't think one way of validating is good enough, as there's a chance of incorrectly flagging content.
In my opinion, if the "trustScore" is above, let's say, 90, then decrease the reach of the post in the feed. If the post has that higher trust score with "x" number of moderators flagging it as AI content using a "Flag to Admins 🚩" button, then completely remove the post after informing the user that the post will be removed from DEV in "x" days and suspend the user for "x" days.
To date, I have come across various completely AI-written blogs that are just a good sight away from being noticed. We definitely cannot depend on a single tool like GPTZero to flag AI-generated content, as I have seen it flagging completely human-written content with slightly "better grammar words" 🤷 as AI-generated.
An article can be "AI-assisted" rather than "AI-generated," so we need to take care of that as well.
I definitely don't seem to find this approach any good as @anmolbaranwal has already said it above, many senior devs just joined might be willing to write here. But seeing this timeout sort of thing is already super disappointing, and they might end up moving away from DEV platform itself.
I think I saw a similar post from @thepracticaldev a few months ago. I would definitely love to know the team's thoughts on this.
I appreciate your mention of considering multiple different factors before labeling a post as AI. It's just a generally good idea.
As for the difference between AI assisted and AI generated articles.... I'm really not sure what to say. There are certainly situations where AI assisted content is completely fine (see @syakirurahman 's comment), but there are also situations where I feel that AI assisted content leads to lazy writing.
Anyways, I know there's more that I could talk about regarding your comment, but I'm tired and it's late. Thank you for the input and ideas!
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
Exactly useless.
And you’re totally right, but this is just the tip of the iceberg.
Here I do actually have some ideas on how to solve that problem. But I like to keep it a secret for now 😅 not that it’s groundbreaking.