HuffPo, Identity, and Abuse - S01E02
First of two shows on Identity and Content Abuse Mitigation.
News: Huffington Post to require ID to post, says “It’s good for you!”
Topic: Does requiring “real” identity really prevent abuse and “raise the level of discourse”?
Tips: A free book chapter describing saving over $1m on social moderation costs, and Bryce gives three suggestions on user ID.
— Links —
Chapter 10 of Building Web Reputation Systems
Corporate Ratings Abuse and What to Do About it @ Building Web Reputation Systems: The Blog
RealID and WoW Forums: Classic Identity Design Mistake @ Habitat Chronicles
Goffman, Erving. Stigma: Notes on the management of spoiled identity. SimonandSchuster. com, 2009.
Resnick, Paul. “The social cost of cheap pseudonyms.” Journal of Economics & Management Strategy 10.2 (2001): 173-199.
Resnick, Paul et al. “The value of reputation on eBay: A controlled experiment.” Experimental Economics 9.2 (2006): 79-101.
Hogan, Bernie. “The presentation of self in the age of social media: distinguishing performances and exhibitions online.” Bulletin of Science, Technology & Society 30.6 (2010): 377-386.
— Transcription —
Randy: Welcome to episode two. I’m Randy Farmer.
Bryce: I’m Bryce Glass.
Marc: And I’m Marc Smith.
Randy: Today’s news item: it is the beginning of September, and Huffington Post is supposed to start requiring some form of real identities or better identity. The way they’ve been putting it is “no more anonymous posting on the Huffington Post”. The reasons listed were that people should stand by their words, but the truth is underneath it’s clear that they have an abuse mitigation problem with too many people posting stuff they don’t like. They have to pay forty people full time salaries to moderate the communities they have, and they find it very frustrating.
There will be links in the podcast notes to some of the specific articles, including a video with editor Jimmy Soni, where he told CNN that the idea was to promote civil discourse. What are your thoughts, guys?
Bryce: So this is I think something fairly common in thinking about social platforms and social products. It’s a very simplistic approach to what’s probably a much more nuanced problem. It seems like it’s preordained to fail across several different fronts. It’s an example of cargo cult design, looking of what others have done, and trying to emulate that to achieve an end, not really understanding whether that end will be achieved at not.
Randy: It’s an attempt at what we call abuse mitigation. The idea is to have less moderator required comments. The presumption is that using real identity will get you to behave better.
We’re going to talk about it today, but I think this is fundamentally wrong headed.
We’re likely to split this into two episodes. One focused on abuse mitigation. That’s this one. And a future episode focused on the uses of pseudonyms and anonymous posts, and the good qualities that they have.
Marc: I’m not entire delighted with the real name move on Huffington Post. I agree that they have legitimate motivation. They would like to reduce the amount of abusive content, no doubt. There is a lot of spam, and there can be content that is objectionable. But the remedy, the use of the so called real name, has itself a lot of negative consequences.
So I think that they can achieve their goals of reducing the objectionable content, while actually retaining some of the virtues of allowing pseudonymous content.
Randy: And on that note we’ll move into our main segment.
And now for our main segment, we’re going to talk about identity and abuse mitigation. I’ve seen this before, people saying that “boy, if people used their real names, that will just solve our quality problems associated with participation”, and part of that is based in some science.
So I’m going to pass to our scientist, Marc Smith.
Marc: Thanks, Randy. Research shows that bad behavior and anonymity do have a connection, and when people can change their identities very easily they’re inclined to behave in negative ways, and then change their identities and not pay any of the consequences.
And some people have argued that that means that we should insist on this so called real name, the government issued ID linked identity. Actually the research, which is interesting, and I’m thinking here of Paul Resnick, Professor Resnick at the University of Michigan, whose papers on the costs of cheap pseudonyms and the value of reputation in EBay, provide us with some guidance. In particular, Resnick shows that very, very cheap pseudonyms do cause problems, but that pseudonyms can be expensive. In other words, you can get the benefits of the so called real name, without using the real name. What you use is an identity that can accumulate value over time, value that people would be reluctant to lose.
Randy: That’s a very good point. We’ve seen this in lots of domains. EBay is an excellent domain in which most people aren’t using real names. They’re using company names, or even just arbitrary pseudonyms that are accumulating sales history.
There are other domains besides chat that are interesting. Bryce.
Bryce: Yeah, absolutely. I mean this is not just a problem with message boards or comment systems. This is a problem in online review systems. Yelp I’m sure deals with this.
Basically, how do you prevent astroturfing? How do you prevent sock puppet accounts? How do you prevent people with a vested interest from disrupting these systems that a lot of people rely on for commerce, or for maintaining their own reputation?
So the stakes are high, and obviously there’s a lot of interest in how best to moderate these systems.
You know, Marc, you mentioned pseudonyms with great value. I think anybody who has spent a lot of time on the hacker news community, think of Fake Grimlock, probably one of the more valued contributors to that community, obviously not his real name. He’s invested a lot of time in this kind of pseudonymity and building up a lot of value around this well-known identity.
Randy: Bryce, could you do me a quick favor and define “astroturfing” and “sock puppets”.
Bryce: Astroturfing is the coordinated effort of a small number of actors, sometimes even just one actor, to give the appearance that a viewpoint that they hold is a widely held one, and, also, to give the impression that it comes from a grassroots support mechanism.
Sock puppet is basically a fake account that someone creates to further some agenda. We’ve actually published on this issue of people trying to manipulate corporate ratings or reviews, and we’ll publish a link to that in the podcast web notes.
Randy: This has happened before, also, in another environment, which is gaming. The World of Warcraft in July 2010 tried to require real names for access to its bulletin boards. Over on Habitat Chronicles I have a post, again, we’ll link to this, which describes the main reasons that this is a problem.
Some of those will roll into the next episode on identity when we talk about the fact that, for example, all the girls would have quit the message boards if they had turned these on, because most of them were hiding their gender, and now their real names would be out.
But it specifically doesn’t prevent abuse, and this is the key point. That in some place where there’s some controversy, some conflict, Worlds of Warcraft has a fight, but Huffington Post carried political issue arguments, whether it’s gay marriage, or whether to not to bomb Syria.
In Yahoo, we worked on mitigation issues associated with message boards attached to Yahoo News, and there was a continual problem in any news articles about Israel. There are legitimate points of view associated with things like having your family and relative killed that it’s impossible to determine if they’re abusive, but it’s certainly not likely to produce a high level of discourse.
So what we found is that handing moderation over to the community, with an appropriate set of tools, is probably the best way to allow for pseudonymity, people protecting themselves, in exchange for producing high quality output.
When Bryce and I were at Yahoo, we implemented a user moderation system that eliminated all the spammers and trolls in a matter of weeks. Now, we didn’t raise the bottom. It didn’t make the jokes go away on Yahoo Answers. It wasn’t for that. It was to get rid of the troublemakers without giving up the pseudonymity and it also saved a million dollars a year in moderation costs.
Bryce: One thing I like about that approach, if you’ve said it once, you’ve said it a million times, is context is king in these online interactions. It truly is up to the community to decide what an appropriate level of discourse is for that particular community.
In the context of Huffington Post, it could vary from article to article. The subtle changes in context are so varied and so nuanced; it doesn’t surprise me that a team of forty moderators can’t keep up with that. It wouldn’t surprise me if a team of four thousand moderators couldn’t keep up with that
Randy: Then any small group of people can keep … we found when we were working on Yahoo Answers that there was a ten percent error rate in human moderation. The error rate when we dropped it to the community dropped to practically zero.
Marc, do you have any other comments?
Marc: I think that the argument that putting your real name on something makes everybody behave is misguided, and there are many cases where not putting your name on it is perfectly legitimate, and being forced to put your name on it is an erosion of the value of the service.
We should be exploring the kinds of tools that allow us to have some of the benefits of segmenting our identities on the internet, without enforcing the idea that we all have just one and only one identity, and all aspects of identity have to be seen at all times.
Randy: Totally agreed.
Bryce: It seems like a lot of this falls back to Facebook and their approach to real names. If you actually read Facebook’s TOS around real names, really what they’re trying to do is ensure that you use something that looks like a real name, something that isn’t visibly a nickname. There’s no guarantee that it’s your real name. It wouldn’t surprise me to find out that they do very little enforcement on real name stuff, except for stuff that gets flagged and reported by the community.
Randy: I have at least one pseudonym on Facebook, which I created for just this purpose back in 2010, and that I have a pseudonym for my hobby, which is not secret, but it is vaguely separate. If someone goes to the appropriate places on the internet, you’ll find out who my pseudonym is, and what my hobby is. But I should be allowed to present myself as my hobbyist self in my hobby community, and as my “real” self or my professional self, on things like my podcast.
Marc: Sociology gives us some reason to avoid this single real name identifier. Sociologists like Erving Goffman have talked about the fact that people maintain different identities for different groups of people. More recently the sociologist Bernie Hogan has talked about the idea of “context collapse”, and argued that this is not necessarily a good thing, that all of your identities in all the different facets of your life are all tied together.
Contradictions in “who you are” are forced into view. This is not necessarily a feature. This sounds like a bug.
Randy: That, Marc, sounds like a great teaser for part two on identity in a future podcast.
So with that, we’re going to move on to our next segment.
For our actionable tip this week, we have two things we want to share with you. One is chapter ten of Bryce and my book, Building Web Reputation Systems. There’s a link to it in the show notes. You can read it for free on the web, and it describes how we trusted the community to moderate Yahoo Answers, and removed spam and the worst of the trolls virtually overnight, and shut down over a million dollars in operating costs associated with human moderation.
Bryce: Here’s some actionable advice on techniques you can use on your site to help moderate content and mitigate abuse that fall short of requiring people’s real names. So the first tip is: have a strong notion of identity on your site. You don’t need to require a real name to ask people to invest a lot of work in maintaining a good identity on the site. Require them to register before they can participate.
Another is to establish karma. So keep user reputations, or karma, for the users on your site. Take note of when they’ve been good actors. Take note of when they’ve been bad actors. Keep that history around. You can make adjudications about it later.
And my third tip is: apply a heavy hammer, so when you do see abusive activity happening, delete the account flat out. Delete content that is contributed. Delete the entire history that it ever existed. What you want to create in these bad actors’ minds is the real threat that this one action when carried out, or a number of actions, could be the straw that breaks the camel’s back and deletes their history of participation.
Randy: Great. Thanks, Bryce, and, also, to you Marc, and thanks most of all to our listeners!