My Crowdsourced Follow-Up About Crowdsourcing 59
In my last article, I proposed an algorithm that Facebook could use to handle abuse complaints, which would scale to a large number of users while also making it difficult for co-ordinated mobs to get unpopular content removed by filing complaints all at once. I offered a total of $100 to readers sending in the best suggestions for improvements, or alternative algorithms, or fatal flaws in the whole idea that would require starting from scratch. As the suggestions were coming in, Facebook obligingly kept the issue in the news by removing a photo of two men kissing from a user's profile, sending a form letter to the user that they had violated Facebook's prohibition on "nudity, or any kind of graphic or sexually suggestive content". (It would be a cheap shot to say that a photo of a man and a woman kissing probably would not have been removed; in truth, probably just about anything will get removed from Facebook automatically if enough users file complaints against it, which is the problem for unpopular but legal content.)
How would these complaints have been handled under my proposed algorithm? The gist of my idea was that any users could sign up to be voluntary reviewers of "abuse complaints" filed against public content on Facebook. Once Facebook had built up a roster of tens of thousands of reviewers, new abuse complaints would be handled as follows. When a complaint (or some threshold of complaints) is filed against a piece of content, a random group of, say, 100 users could be selected from the entire population of eligible reviewers, and Facebook would send them a request to "vote" on whether that content violated the Terms of Service. If the number of "Yes" votes exceeded some threshold, the content would be removed (or at least, put in a high-priority queue for a Facebook employee to determine if the content really did warrant removal). The main benefit of this algorithm is that would be much harder for co-ordinated mobs to "game the system", because in order to swing the vote, they would have to comprise a significant fraction of the 100 randomly selected reviewers, and to achieve that, the mob members would have to comprise a significant fraction of the entire reviewer population. This would be prohibitively difficult if hundreds of thousands of users signed up as content reviewers.
All of the emails I received -- not just "almost" all of them, but really all of them -- contained some insightful suggestions worth mentioning, although there was some duplication between the ideas. If you didn't see the last article, you might consider it worth while to stop reading before proceeding further, and mull over the description of the algorithm above to see how you would improve it. Then read the suggestions that came in to see how well your ideas matched up with the submissions I received.
The upshot is that nobody found what I believed to be completely fatal flaws, although one reader brought something to my attention that might cause trouble for the algorithm after a few more years. Beyond that, reader suggestions could be divided essentially into two categories. The first category of suggestions related to ensuring that the basic premise would actually work -- that the votes cast by a random sample would be representative of general user opinion, and could not be gamed by a coordinated mob or a very resourceful cabal trying to game the system. The second category of suggestions started by assuming that the voting system would work, and suggested other features that could be added to the algorithm -- or, in one case, an entire alternative algorithm to replace it.
To begin with the attacks and counter-attacks against the basic voting algorithm. Walter Freeman and Haydn Huntley independently suggested monitoring for users who vote in a small minority in a significant portion of vote-offs, and reducing their influence in future votes (by either not inviting them to vote on future juries, or sending them the future invites but then ignoring their votes anyway). The assumption is that if a user is frequently among the 10% who vote "Yes [this is abuse]" when the other 90% of respondents are voting "No [this is not abuse]", or vice versa, then that user is voting randomly, or their point of view is so skewed that their votes could safely be ignored even if they are sincere. I like the idea of eliminating deadweight voters, but this might also incentivize voters to vote the way they think the crowd would vote, instead of voting their true opinions -- for example, if they were called to vote on an anti-Obama page that showed Obama wearing a Hitler mustache. Some people's knee-jerk reaction would be to call the page "racist" or "hate speech" or "a threat of violence", even though comparing Obama to Hitler is not, strictly speaking, any of those things. If I were voting my honest opinion, I would count that page as "not abuse". But if I knew that I were voting along with dozens of other people, and my future voting rights might be revoked if I didn't vote with the majority, I might be tempted to vote "abuse".
Similarly, Walter Freeman and reader "mjrosenbaum" both suggested setting deliberate traps for deadweight users, by creating artificial cases where the answer was pre-determined to be obviously yes or obviously no, calling for votes, and revoking privileges for users who gave the wrong answer. This would eliminate the problem of borderline cases like the one above, where smart users think, "I suspect the majority will give the wrong answer, so I'm just going to go with the crowd, to keep my voting rights." On the other hand, it's more labor for Facebook to create the cases, and any public content authored by them -- especially content that is deliberately crafted to be "questionable" -- would probably have to run a gauntlet of being reviewed by lawyers and PR mavens before being released. My suggestion would be to use these artificial scenarios periodically to make sure that the system is working (i.e. that juries are giving the right answers), but it would be too inefficient to use it to try and weed out problem voters.
In fact, these and several other suggestions fell into a category of ideas that could possibly improve the efficiency of the algorithm by reducing voter shenanigans (where "efficient" means that fewer users have to be invited to each vote-off in order to get statistically valid results), but might not be worth the effort. As long as most of the votes cast by users are sane and sincere, all you have to do is invite enough voters to a vote-off, and the majority will still get the correct answer most of the time, even if you have problem voters in the system. That's the simplest possible algorithm. The more complicated an algorithm you come up with, the more likely that Facebook (or any other site you recommended this to) would just throw up their hands and say, "Sounds too hard", and leave the idea dead in the water. That's why I like the algorithm as lean and tight as possible.
So it's not quite like designing an algorithm for your own use, where you could feel free to introduce more complications as long as you're responsible for keeping track of them. In recommending an algorithm for widespread adoption, the basic form of the algorithm should be as simple as possible. In the case of the voting algorithm some interesting wrinkles may come up if you don't eliminate problem voters, but this is not fatal to the idea as long as it's still true that, given a large enough random sample of voters, the majority will tend to vote the correct answer.
For example, James Renken pointed out that as voters dropped out due to boredom, the remaining users casting votes would tend to be either (1) weirdos who just wanted to view questionable material; and (2) prudes bent on removing as much material from Facebook as possible. But that's OK, as long as those two groups vote sanely enough (or as long as there are enough sane users outside those two groups) that material which does violate the TOS, tends to get more "Yes [this is abuse]" votes than material that doesn't. Then all you have to do is make the jury size large enough to make a statistically significant distinction between those two cases.
Similarly, Joshua Megerman suggested surveying users for their religious, political, and other beliefs when they sign up as volunteer reviewers (they could of course decline the survey). This makes it possible, insofar as people answer truthfully, to make sure that a jury is composed of a group with diverse belief sets. (On the other hand, users could game the system by reporting beliefs that are the opposite of what they truly feel. For example, if you're a leftist, register as a right-winger. Then when an abuse case comes before you, if it's a piece of content more offensive to leftists, then the real leftists on the jury will tend to vote against it -- but as a registered right-winger, you'll be able to cast a vote against it as well, and you'll be displacing a real right-wing voter who probably wouldn't have voted that way, so your vote will be worth more!) Again, it's fine if Facebook wants to do this, but even without collecting this data and simply selecting jurors at random, it should still be true that genuinely abusive pages get more "Yes" votes in a jury vote, than non-abusive pages.
Lastly in the "keep the jurors honest" category, Paul Ellsworth suggested allowing jurors to anonymously review each other -- when a given juror is chosen for the "hot seat" (perhaps randomly, perhaps as a result of a history of skewed voting), other jurors are randomly selected from the voting pool, to review that juror's voting record and decide whether that juror has been voting honestly and judiciously, or not. When I first read this idea, I instinctively thought that because a contaminated jury pool would be reviewing itself, it would not be able to reduce the percentage of problem voters, but a little more thought revealed that this isn't true. Suppose initially your jury pool consists of 80% "honest voters" and 20% "dishonest voters", that honest voters who review the voting record of another voter will always vote correctly whether that person is "honest" or "dishonest", and that dishonest voters will always vote incorrectly. It's still the case that when a voter's record is reviewed by a panel of, say, 20 other voters, virtually 100% of the time the majority will get the right answer. If you strip voting rights from a voter whenever a jury of other voters determines them to be a "dishonest voter", then over time, the percentage of honest voters in the system will creep from 80% to 100%. So again, this might work, and again, it might just be adding unnecessary complexity if the basic algorithm could work without it.
Note that none of these precautions would address the case of a "sleeper" voter -- a voter who joins the system with the sole intention of voting incorrectly on particular types of cases (perhaps planning on voting "yes" to shut down pages made by a particular organization, or pages advocating a particular view on a single issue), while still planning to vote correctly on everything else. By voting honestly in all other cases, they prevent themselves from being flagged by the system for casting too many minority votes, or from being blacklisted by other jurors for having a questionable overall voting record. The only real way I can see to address this problem is to hope that such users are outnumbered by the honest users in the system, and that juries are large enough that the chances of "rogue voters" gaining a majority on any one jury are nearly zero.
Which brings us to the one potentially fatal weakness in the system that I'm aware of: reader George Lawton referred me to a program run by the U.S. government to create armies of fake accounts to infiltrate social media, named, apparently without irony, Earnest Voice:
The project aims to enable military personnel to control multiple 'sock puppets' located at a range of geographically diverse IP addresses, with the aim of spreading pro-US propaganda.
An entity with the resources of the U.S. military could potentially create enough remote-controlled voters to overwhelm the system. I'm not sure if there is a way to deal with a system if the majority of voters are compromised. Presumably by making all decisions appealable to a core group of trusted Facebook employees at the top (although this then creates a bottleneck and limits scalability, especially if filing an appeal is free and all the parties who lose abuse cases are constantly filing appeals to the next level up).
Now. On to the second category of suggestions: Assuming the majority of voters are honest, what other features would be desirable to build into the system?
Walter Freeman, on the subject of filing appeals, suggested putting appealed pages in a special queue where they could be publicly viewed and users could comment on the ongoing appeals process, in addition to reading arguments posted by either side; this also negates the censorship itself due to the to the Streisand effect. I agree, but it's not obvious why this is a desirable feature. This does create perverse incentives, since some users could get extra traffic for their content by creating a page that makes whatever argument you're trying to promote, spiking it with some TOS-violating content, waiting for the page to get shut down, appealing the decision, and enjoying all the extra Streisand attention that it gets while on public display during the "appeal".
Meanwhile, James Renken pointed out that the system would work best for content that was originally public anyway, like a controversial Facebook page or event. If someone filed a complaint regarding a private message that they received, and they wanted a "jury vote" about whether the content of the message constituted abuse, then either the sender or the recipient would have to waive their right to privacy regarding the message before it could be shared with jurors. If the message really was abusive, then in some cases the recipient might waive their privacy rights -- reasoning that they didn't mind sharing the nasty message that someone sent them, in order to get the sender's account penalized. The problem arises if the message also contains sensitive personal facts about the recipient, which they wouldn't want to share with anonymous jurors. The system could allow them to black out any personal information before submitting the message for review, but that creates a recursive problem of abuse within the abuse system -- how do you know that someone didn't alter the content (and thus the offensiveness) of the message through their selective blacking-out? So it's not obvious whether this idea could be applied to non-public content at all.
Reader George Lawton suggested allowing content reviewers to vote on the funniest or weirdest content they had to review, to be posted in a public "Hall of Infamy". I love the thought of this, but I think Facebook's lawyers would be uncomfortable glamorizing anything questionable even if it were ultimately voted to be non-abusive (and certainly if it was voted to be abusive). Besides, this also has the perverse-incentives problem -- tie your message to something that you know will not only get an abuse complaint, but will hopefully end up in the Hall of Weird. (Even without the abuse jury system, there are already plenty of incentives for people to make a political point and hope that it will go viral.)
David Piepgrass suggested that new content reviewers should be allowed to specify certain types of content that they don't want to be asked to review -- nudity, graphic violence, etc. This sounds like a good idea. He adds that users probably shouldn't be able to opt-in only to review certain categories of content (or jurors might sign up only to review nudity, and then who would be left to review the death threats?).
Finally, in the other corner: Jerome Shaver suggested bypassing the jury voting system altogether and working on a heuristic algorithm to determine when abuse reports were being submitted by organized mobs of users, based on the patterns shown by mutual friendships between the users filing the abuse reports. The difficulties in designing such an algorithm, are too complicated to summarize quickly, and could fill an entire separate article. (Convince yourself that it's not an easy problem to solve. You can't just ignore abuse complaints from clusters of users that have many mutual friendships, because it can happen that real tight-knit communities of users might file abuse complaints against a piece of content, where the complaints are actually genuine.) But again, there is the problem that if a proposed solution is too complicated or too nebulous, Facebook has the excuse that they are "weighing several options", that they're "already working on something similar internally", etc. The jury vote system has the advantage that it can be described in just a few sentences, and the general public always knows whether it has been implemented or not -- which means that as long as abuses of the complaint system continue, people can ask, "Why doesn't Facebook try this?"
You'll notice this is just a laundry list of the ideas I received, without any definitive conclusions about which ones are good or bad, but that's all I was going for. The original algorithm, I could argue with the force of mathematical proof that, under certain reasonable assumptions, it would work. There's no such proof or disproof for any of the suggested modifications, so I don't feel as strongly about any of them. But at the top of the article I suggested for readers to stop reading and see how many of these ideas they could come up with on their own. How did you do?
The final honor roll of readers who were each the first, or only, person to submit an original idea: Walter Freeman (bonus points for getting in several good ones), James Renken, Joshua Megerman, Paul Ellsworth, George Lawton, Jerome Shaver, and David Piepgrass. Most of them volunteered to donate their winnings to charity, and agreed to let me donate their share to Vittana, which arranges microloans to college students in developing countries. One preferred a charity of their choosing, and only one actually kept the money. To be clear, for future contests, it's awesome if you want to donate the money to charity, but it's not dickish to keep it. That was the original deal after all.
So, all very clever and interesting suggestions, some of which might inspire readers to keep coming up with their own further variations. I said which ideas I probably would have incorporated and which ones I wouldn't, and I'm sure many of you would tell me that I'm wrong on some of those points. Although from here on out you're doing it for free.
tl;dr (Score:5, Funny)
Re: (Score:3)
Re: (Score:2)
Do I get $25 for parroting the joke?
Re: (Score:2)
I read the first couple paragraphs, and I've actually decided it's worth $100 for me to NOT read the article.
Does this guy even work for Facebook? He doesn't seem to be in any position to actually implement the idea... so what's the point? Brainwanking?
Re: (Score:2)
But hey, if it doesn't spin your prop, don't think about it.
Cognitive penis-waving comparison (Score:2)
so what's the point? Brainwanking?
Encephallic show-off.
Re: (Score:1)
Enough of the bullshit! (Score:1)
Crowdsousing, the Cloud, Web 2.0, Cyberwhatever. Enough of the bullshit buzzwords!
Cyberwhatever (Score:2)
Re: (Score:3)
Mod parent up.
For the tl;dr crowd, to summarize:
It's basically the /. mod system, except mods only vote on posts/messages that are flagged by a user as violating the ToS and the choices are 'OK' or 'Not OK'.
As for who watches the watchers, it's basically the /. meta-mod system.
The "algorithm" is safety in numbers. You get a large enough pool of mods, and hopefully the "normal" people sufficiently outnumber the extreme folks who mod everything as 'Not OK' or who volunteer to mod just to get a peek at naught
Going with the majority? (Score:2)
How is voting the way you feel the majority of people will vote a good thing? While I don't want to out and out say that Democracy doesn't work, there certainly are instances where the majority of people are wrong. The whole crowd-sourcing system doesn't really account for this. Still, it's probably still better than existing systems.
Democracy doesn't work (Score:2)
I'll say it.
Democracy doesn't work. I want my information organised by smart algorithms (Google); not stupid people (facebook).
Re: (Score:2)
Because plutocracies are so much better.
How about we just leave all the content online and let people pick and choose what they want.
That's the [redacted by crowd-sourced censors] joy of the internet
FTFY.
Re: (Score:1)
Re: (Score:2)
How is voting the way you feel the majority of people will vote a good thing?
He's not saying people should strive to vote with the majority. He's worried about folks voting with the expected majority to hide their bias.
Someone who decides to vote against everything is easy to find. Once you get a large enough set of 99-to-1 votes where the same person is the 1, you know you can ignore that person's vote. Whether it's someone who things everything is inappropriate, doesn't understand the ToS, or whatever, the author's feeling is once you've identified whether something is inapprop
Re: (Score:2)
Tyranny of the majority. Welcome it. With it, you can vote yourself a raise (along with everyone else) until the system is bankrupt.
This is why the Constitution was LIMITING governance, with prescribed roles and functions and everything else was left to the states, or to the people. But we've long bypassed that whole perspective.
It was one of the reasons we had representative governance (republic) democratically elected. And even that has been perverted by the artificial limitation of 435 Representatives fo
Tee El; Dee Arr. (Score:1)
Another option (Score:1)
To address the selection problem for the pool of reviewers and avoid having the self selection result in a bias, when an entry needs judging put the question to a random group of users. Have it pop up when they log in and require a vote of 'OK' or 'Not OK' before they finish logging in to their account.
How about (Score:1)
Re: (Score:1)
This sort of problem is what necessitates legislation and government oversight.
No it doesn't, this is bullshit. Here's a real wild idea: run your shit how you want to run it. Why the fuck would facebook need government regulation to tell them how to handle removing stuff from their site? Once something gets popular everybody should be able to use it and it must be about free speech? That's a flimsy argument and stupid too. Facebook runs the risk of removing more than they and their users would want, but that should not leave them open to any kind of law suit or regulation any more tha
Re: (Score:2)
How about they do what they are already doing and keep more of the money for themselves. if people really won't put up with the errors and start leaving for greener pastures then they can consider spending more money on it.
Hmm.. (Score:4, Insightful)
Re: (Score:2)
Not only fulfills the requirements, but almost in exactly the same way as proposed by the author and his crowd.
I don't really see the need for this survey. (Score:1)
I think it has more limitations with re. to lifting up of comments of value (though it also appears beneficial here), such as for slashdot comments, in comparison to stopping wrongful censure, where I really can't see a down side.
I don't see any practical way of gaming it. There are the threats of politically correct mindedness and herd-think, and perhaps some of the comments could be used
Metamoderation? (Score:2)
My personal opinion on the whole matter though, is that all the sewage of the world should be redirected into the Facebook da
Re: (Score:2)
Summary: he reinvented the jury system. (Score:2)
OP just reinvented the jury system.
It's not a bad idea, provided that people are actually willing to work for you for free. Usually, they aren't. It's been tried before, for spam filtering, but the reviewers were overwhelmed. (You'd get random messages containing spam which you have to rate. Right.) This approach sometimes works when the number of items to rate is much smaller than the number of raters, and when the user has to read the thing they're rating anyway, as with Slashdot.
Also, the annoyance
Re: (Score:2)
Actually, OKCupid does something like this, and it seems to work pretty well for them.
All "crowdsourced moderation" approaches Slashdot (Score:2)
suggested allowing jurors to anonymously review each other -- when a given juror is chosen for the "hot seat" (perhaps randomly, perhaps as a result of a history of skewed voting), other jurors are randomly selected from the voting pool, to review that juror's voting record and decide whether that juror has been voting honestly and judiciously, or not.
Have you metamoderated recently? In fact, the whole thing seems like what Slashdot's been doing for years and years. Randomly select moderators based on a number of factors, including previous history and past performance, and so on.
Sophisticated Censoring. (Score:2)
A business will only host material that is profitable. If it is unprofitable to host the material, the business will not host it. Unless they are protected by a more powerful government, a business will always remove material when directed to remove that material by a government. Google and Facebook are in this to make money for their shareholders.
In the sixties, a black man couldn't kiss a white woman on TV because many affiliates wouldn't carry the show. You still don't see it very much on TV. The TV
Start with better filtering of complaints (Score:2)
While setting up juries, etc, may be useful, it would be quite easy to use a Google-ish algorithm to discount abusive mob complaints in the first place. If you find that certain people have a high rate of complaining, their complaints probably should have a low weight. And you could use the age and activity level of the account to ignore some accounts (new accounts, and rarely used accounts do not represent Facebook users well). So if someone arranges a mob to get groups dealing with an issue removed, the m
It should be simpler (Score:2)
I still think there is a far simpler solution to this problem. Simply chose 1 in 10 people that _can_ vote. That is, 90% of Facebook members simply wouldn't be allowed to vote on any specific topic, on average they would have to try 10 different ones (or more) before they were allowed to vote.
That would make organized vote mobs very difficult to arrange, which is what we're trying to accomplish. Yet this version would not require any special effort on the part of the users, or the system. No setup either.
Great idea; author is not a lawyer (Score:1)
Remove robots from the mix (Score:1)
tl;dr (Score:1)
let's get started! (Score:1)