I've got a new feature almost ready to go. I'm pretty stoked about this one because I've been wanting it for quite literally years, but it was never possible on Reddit.
Hey, guess what? We're not on Reddit!
But before I continue, I want to temper expectations. This is a prototype of a first revision of an experimental feature. It is not going to look impressive; it is not going to be impressive. There's a lot of work left to do.
The feature is currently live on our perpetually-running dev site. Log in, click any thread, and go look below the Comment Preview. You'll see a quokka in a suit asking you for help. (His name is Quincy.) Click the cute li'l guy and you'll be asked to rate three comments. Do so, and click Submit. Thank you! Your reward is another picture of Quincy and a sense of satisfaction.
So, uh . . . . what?
Okay, lemme explain.
This is the first part of a feature that I'm calling Volunteering. Once in a while, the site is going to prompt you to help out, and if you volunteer, it'll give you a few minutes of work to do. Right now this is going to be "read some comments and say if they're good or not". Later this might include stuff like "compare two comments and tell me if one of them is better", or "read a comment, then try to come up with a catchy headline for it".
These are intentionally small, and they're entirely optional. You can ignore it altogether if you like.
I'm hoping these can end up being the backbone of a new improved moderation system.
Isn't this just voting, but fancy?
You'd think so! But there are critical differences.
First, you do not choose the things to judge. The system chooses the things it wants you to judge. You are not presented with thousands of comments and asked to vote on the ones you think are important, no, you are given (at the moment) three specific comments and information is requested of you.
This means that I don't need to worry about disproportionate votecount on popular comments. Nor do I need to worry about any kind of vote-brigading, or people deciding to downvote everything that a user has posted. The system gets only the feedback it asks for. This is a pull system; the system pulls information from the userbase in exactly the quantities it wants instead of the userbase shoving possibly-unwanted information at the scoring systems.
Second, you can be only as influential as the system lets you. On the dev site you can volunteer as often as you want for testing purposes, but on the live site, you're going to - for now - be limited to once every 20 hours. I'll probably change this a lot, but nevertheless, if the system decides you've contributed enough, it'll thank you kindly and then cut you off. Do you want to spend all day volunteering in order to influence the community deeply? Too bad! Not allowed.
But this goes deeper than it sounds. Part of having the system prompt you is that not all prompts will be the system attempting to get actionable info from you. Some of the prompts will be the system trying to compare your choices against a reference, and the system will then use this comparison to figure out how much to trust your decisions.
That reference, of course, is the mods.
I've previously referred to this as the Megaphone system or the Amplifier system. One of our devs called it a "force multiplier". I think this gets across the core of what I'm aiming for. The goal here is not majority-rules, it's not fully decentralized moderation. It's finding people who generally agree with the mods and then quietly harnessing them to handle the easy moderation cases.
(We have a lot of easy moderation cases.)
There's another important point here. The mods are only human and we make mistakes. My hope is that we can get enough volunteer help to provide significantly more individual decisions than the mods can, and my hope is that the combined efforts of several people who don't quite agree with the mods in all cases is still going to be more reliable than any single mod. In fact, it wouldn't surprise me if there's people out there who are better at judging posts than our mods are! It's just hard to find you; some of you may not even comment, and you're pretty undiscoverable right now, but you will certainly get a chance to volunteer!
Also, this will hopefully improve turnaround time a lot. I'm tired of filtered comments taking hours to get approved! I'm tired of really bad comments sticking around for half a day! There are many people constantly commenting and voting, and if I can get a few minutes of help from people now and then, we can handle those rapidly instead of having to wait for a mod to be around.
Wow! You get all of this, with absolutely no downsides or concerns!
Well, hold on.
The big concern here is that virtually nobody has ever done this before. The closest model I have is Slashdot's metamoderation system. Besides that, I'm flying blind.
I also have to make sure this isn't exploitable. The worst-case scenario is people being able to use this to let specific bad comments through. I really want to avoid that, and I've got ideas on how to avoid it, but it's going to take work on my part to sort out the details.
And there's probably issues that I'm not even thinking of. Again: flying blind. If you think of issues, bring 'em up; if you see issues, definitely bring 'em up.
Oh man! So, all this stuff is going to be running real soon, right?
Nope.
First I need some data to work off. Full disclosure: all the current system does is collect data, then ignore it.
But it is collecting data, and as soon as I've got some data, I'll be working on the next segment.
This is the first step towards having a platform that's actually better-moderated than the current brand of highly-centralized sites. I don't know if it'll work, but I think it will.
Please go test it out on the dev site, report issues, and when it shows up here (probably in a few days) click the button roughly daily and spend a few minutes on it. Your time will not be wasted.
Blocking
Right now this site's block feature works much the same as Reddit's. But I want to change that, because it sucks.
My current proposal is:
-
If you block someone, you will no longer see their comments, receive PMs from them, or be notified if they reply to your comments.
-
This does not stop them from seeing your comments, nor does it stop them from replying to your comments.
-
If they attempt to reply to your comment, it will include the note "This user has blocked you. You are still welcome to reply, but your replies will be held to a stricter standard of civility."
-
This note is accurate and we will do so.
That's the entire proposed feature. Feedback welcome!
User Flair and Usernames
We're going to start cracking down a bit on hyperpartisan or antagonistic user flair. Basically, if we'd hit you with a warning for putting it in a comment, we'll hit you with a warning for putting it in your flair. If anyone has a really good reason for us to not do this, now's the time to mention it!
Same goes for usernames. On this site, you can actually change your display username, and we're just leaving that in place. So we'll tell you to change your name if we have to. Extra for usernames: don't use a misleading or easily-confused username, okay? If it looks like you're masquerading as an existing well-known user, just stop it.
I'm currently assuming that both of these fall under our existing ruleset and don't need new rules applied. If you disagree strongly, let me know.
The Usual Stuff
Give feedback! Tell me how you're doing? Do you have questions? Do you have comments? This is the place for them!
Are you a coder and want to help out? We have a lot of work to do - come join the dev discord.
Jump in the discussion.
No email address required.
Notes -
Quick Volunteer Janitor analysis update!
I've got it spitting out Pretty Accurate Results, to the point where the best way to find bugs is now to look for posts that it thinks we should have modded but didn't, or posts that it thinks we shouldn't have modded but did, and figure out what happened. As a quick cursory glance, the answer in about half the remaining cases is either "we made a mistake" or "ehhh, that could have gone either way", which suggests it's now about as accurate as the mods are. And there's still things I have left to improve! So this is Very Promising overall.
One of the more fascinating results of this was to look at the most accurate volunteers. Out of top ten, nine of them have made less than 100 comments; in fact, half of them have made less than 25 comments, including two of the top three. My tool spit out a giant list of names and I said "who the hell are these people" and I had to go look them up to see if they were actual people. They are! They're just people who don't post a lot. This all suggests that there's a ton of near-lurkers out there who are reading stuff in detail and who have a very good idea of the community norms.
Hello, lurkers! Thank you for being here! I'm not even directing this to the set of you who are volunteering (but extra thanks to you), but to everyone who's reading; part of my goal here is just to be a place for people to see discussions, and I'm glad to know that there are people who are seeing discussions. Y'all are great.
I've got a few more pieces to put in, then I have to figure out how to connect this to the live database in a useful fashion, then I'm going to be initially setting it up as an assistance tool for the mods. If everything pans out, though, it's going to be handling the vast bulk of the moderation work for us in the future (though we're still going to be the ones verifying warnings and bans and writing the actual messages; no fully-automated harsh penalties will be applied.)
As part of this changeover I plan to set up a bit of a more formal warning/ban system so we can link related posts. Right now there's an issue where if someone goes and spams terrible posts over half the community, we tend to attach the ban message to one of them and just ignore the rest, which leads to people thinking that "the rest" did not get moderator attention. With this tool it'll be easy to group those up and just click a little checkbox that says "make a link to connect all these", and the goal is that users will see a note on each questionable post saying "this was bad and deserved moderator attention, but we applied the actual moderator action to this other message, [click here]".
Anyway, y'all are doing great, thank you for the frankly unexpected amount of quokka-clicking you've been doing. This will all make the community better.
This is a really cool feature/stab at crowdsourced modding! Well done!
I find it really fun to use, too.
More options
Context Copy link
The last janitor-duty thing I got gave me only two posts. Have you changed it? (I think two might be a bit worse effect/friction ratio than three.)
At the moment it pops up the window if there's any posts, then gives you up to three. It's possible to get as low as one if you're unlucky. I'm going to end up tweaking this to something like "wait until you have three posts unless one of them is getting kinda old, in which case just give up and give the user what you have available".
(it's actually possible to get "zero" if the last one got approved between you seeing the banner and clicking on it, but it'll just show you an apology and not start the cooldown timer)
I've been getting mostly one post per Quincy banner recently.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Are people who conduct more ratings given higher weight than those who conduct fewer? There are some days when I don't get around to checking The Motte, and I'm wondering if my "score" suffers as a result.
In the current implementation, it takes a reasonable number of ratings for it to start being confident that you're consistent. But I think once you've rated twenty or thirty posts, that effect is essentially gone.
Right now there's no time-based falloff; I'll probably add one at some point, but it's going to be on the order of months, not on the order of days.
I have no plans to turn this into a Daily Quest :)
More options
Context Copy link
More options
Context Copy link
I'm a lurking volunteer. I just had https://www.themotte.org/post/317/culture-war-roundup-for-the-week/54744?context=8#context given to me to rate. I felt a three way conflict.
It is a superb piece of satire, obviously good.
I'm satired-out. There is a lot of satire on the internet. Too much, give me a break. Gut says: puke!
I like the https://www.themotte.org/rules#Disagreement rule, which the comment is breaking. That should be a warning.
I went with "bad". The instruction do say go with your gut.
I think that the disagreement rule is a good rule that we should uphold, partly for the stated reason, partly for my point 2. It might be easier for the volunteers to uphold it if there were a button with a label that was the terse version of "Brilliantly funny sarcasm, but bad, because brilliantly funny sarcasm is fentanyl for discussion."
Yeah, there are absolutely nasty edge cases.
The big reason I've been avoiding adding more buttons is because the decision I'm asking you to make is the same decision the mods have to make. You're right in that that is a good summary! But at some point we need to decide what to do about it, whether we need to respond or not, and saying "there are both pros and cons to this comment" is a completely accurate statement that nevertheless fails to answer the question.
Also, this is all planned to be algorithmically handled, so if a computer program gets that response, well . . . what's it going to do with it?
Even in the tests right now, I'm boiling all the responses down to "bad" and "not-bad". I do plan to extend that in the future to capture some of the nuance people are providing, but that's hard, and I have no idea how I'd deal with something complicated like you're suggesting there.
tl;dr: Yeah, it's a tough situation, thank you for making a decision, that is exactly what I wanted you to do :)
More options
Context Copy link
Sometimes I suspect I've been handed an AAQC reported comment because I can’t see what’s wrong with it. Then I check context and I instantly see why someone thinks it’s bad. Sometimes the inverse happens.
More options
Context Copy link
More options
Context Copy link
Hi to you too and thanks.
More options
Context Copy link
More options
Context Copy link