Apropos of nothing, I’d like to tell you a story. I’ve touched on this before, but this is the full version. It’s the story of hypothetical small-to-medium Internet community.

Stop me if you've heard this one

You create a little community for a thing you like. You give it a phpBB forum or something.

You want people to be nice, so you make a couple rules. No swearing. No spamming. Don’t use all caps.

You invite your friends, and they invite their friends, and all is well and good. There are a few squabbles now and then, but they get resolved without too much trouble, and everyone more or less gets along.

One day, a new person shows up, and starts linking to their website in almost every thread. Their website mostly consists of very mean-spirited articles written about several well-known and well-liked people in the group. When people ask them to stop, they lash out with harsh insults.

So you ban them.

There is immediate protest from a number of people, most of whom you strangely don’t recognize. The person didn’t break any of the rules — how dare you ban them? They never swore. They never used all caps. They never even spammed, because technically spam is unwanted and automated, and this was a real person linking their website which is related to the thing the community is about.

You can’t think of a good counter-argument for this, so you unban them. You also add a new rule, prohibiting linking to websites.

Now the majority of the community is affected, because they can’t link their own work any more. This won’t work. You repeal the previous rule, and instead make one that limits the number of website links to one per day.

The original jerk responds by linking their website once a day, and then making other posts that link to that first post they made. They continue to be abrasive towards everyone else, but they never swear, and you’re just not sure what to do about that.

A few other people start posting, seemingly just to make fun of the rest of you, but likewise never break any of your rules.

A preposterous arms race follows, with the rules becoming increasingly nitpicky as you try to distinguish overt antagonism from mundane and innocent behavior.

After a while, you notice that many of your friends no longer come around. And there seem to be a lot more jerks than there were before. You don’t understand why. Your rules are reasonable, and you enforced them fairly, right?

But it's not really a swear word

I’ve noticed that people really like to write rules that sound objective. Seems like a good enough idea, right? Lets everyone know exactly what the line is.

The trick is that human behavior, and especially human language, are very… squishy. We gauge each other based on a lot of unspoken context: our prior relationship, how both of us seem to be feeling, whether or not we skipped lunch today. When the same comment or action can mean radically different things in different circumstances, it’s hard to draw a fine distinction between what’s acceptable behavior and what’s not.

And rules are written in human language, which makes them just as squishy. Who decides what “swearing” is? If all caps aren’t allowed, how about 90%? Who decides what’s a slur? What, precisely, constitutes harassment? These things sound straightforward and concrete, but they can still be nitpicked to death.

We give people the benefit of the doubt and assume they’ll try to respect what we clearly mean, but there’s nothing guaranteeing that.

Have you ever tried to politely decline a request or invitation, and been asked why not? Then the other party starts trying to weasel around your reason, and now you’re somehow part of a debate about what you want? I’ve seen it happen with mundane social interactions, with freelance workers, and of course, with small online communities.

This isn’t to say that hunting for technicalities is a sign of aggressive malice; it’s human nature. We want to do a thing, we’re told me can’t because of X, and so we see X as an obstacle to overcome. Language is subjective, so it’s the easiest avenue of attack.

Fixing this in rules is a hard problem. The obvious approach is to add increasingly specific details, though then you risk catching innocent behaviors, and you can end up stuck in an almost comical game of cat-and-mouse where you keep trying to find ways to edit your own rules so you’re allowed to punish someone you’ve already passed judgment on.

I think we forget that even real laws are somewhat subjective, often hinging on intent. There are entire separate crimes for homicide, depending on whether it was intentional or accidental or due to clear neglect. These things get decided by a judge or a jury and become case law, the somewhat murky extra rules that aren’t part of formal law but are binding nonetheless.

(In an awkward twist, a lot of communities — especially very large platforms! — don’t explain their reasoning for punishing any particular behavior. That somewhat protects them from being “but technically“-ed, but it also means there’s no case law, and no one else can quite be sure what’s expected behavior.)

That’s why I mostly now make quasirules like “don’t be a dick” or “keep your vitriol to your own blog“. The general expectation is still clear, and it’s obvious that I reserve the right to judge individual cases — which, in the case of a small community, is going to happen anyway. Let’s face it: small communities are monarchies, not democracies.

I do have another reason for this, which is based on another observation I’ve made of small communities. I’ve joined a few where I didn’t bother reading the rules, made some conversation, never bothered anyone, and then later discovered that I’d pretty clearly violated a rule. But no one ever pointed it out, and perhaps no one even noticed, because I wasn’t being a dick.

So I concluded that, for a smaller community, the people who need the rules are likely to be people who you don’t want around in the first place. And “don’t be a dick” covers that just as well.

Evaporative cooling

There are some nice people in the world. I mean nice people, the sort I couldn’t describe myself as. People who are friends with everyone, who are somehow never involved in any argument, who seem content to spend their time drawing pictures of bumblebees on flowers that make everyone happy.

Those people are great to have around. You want to hold onto them as much as you can.

But people only have so much tolerance for jerkiness, and really nice people often have less tolerance than the rest of us.

The trouble with not ejecting a jerk — whether their shenanigans are deliberate or incidental — is that you allow the average jerkiness of the community to rise slightly. The higher it goes, the more likely it is that those really nice people will come around less often, or stop coming around at all. That, in turn, makes the average jerkiness rise even more, which teaches the original jerk that their behavior is acceptable and makes your community more appealing to other jerks. Meanwhile, more people at the nice end of the scale are drifting away.

And this goes for a community of any size, though it may take more jerks to significantly affect a very large platform.

It’s still hard to give someone the boot, though, because it just feels like a really harsh thing to do to someone, especially for an abstract reason like “preserving the feel of the community”. And a jerk is more likely to make a fuss about being made to leave, which makes it feel like a huge issue — whereas nice people generally leave very quietly, and you may not even notice until several of them have been gone for a while.

There’s a human tendency to measure peace as though it were the inverse of volume: the louder people get, the less peaceful it is. We then try to optimize for the least arguing. I’m sure you’ve seen this happen before: someone in a group points out that the group is doing something destructive, that causes an argument, and then onlookers blame the person who pointed out the problem for causing the argument to happen. You can probably think of some pretty high-profile examples in some current events.

(You may relatedly enjoy the tale of the missing stair.)

Have you ever watched one of those TV shows where a dude comes in to berate restaurant owners for all the ridiculous things they’ve been doing? One of the most common defenses is: “well, no one complained“.

In the age of the Internet, where it seems like everyone is always complaining about something, it’s easy to forget that by and large people don’t complain. Sure, they might complain on their Twitter or to their friends or whatever, but chances are, they won’t complain to you. Consider: either you’re aware of the problem and have failed to solve it, or you’re clueless for not noticing. Either way, complaining won’t help anything; it’ll just cause conflict, making them that person who “caused” an argument by pointing out the obvious.

Gamification

Some people are aware of the technicality game on some level, and decide to play it — deliberately. Maybe to get their way; maybe just for fun.

These are people who think “it’d be a shame if something happened to it” is just the way people talk. Layered thick with multiple levels of irony, cloaked in jokes and misdirection, up to its eyeballs in plausible deniability, but crystal clear to the right audience.

It’s a game that offers them a massive advantage, because even if you both know you’re playing it, they have much more experience. Oh, and chances are they don’t even truly care about whether they’re banned or not, so they have nothing to lose — whereas you’re stuck with an existential crisis, questioning everything you believe about free speech and community management, while your nicest peers sneak out the back door.

I remember a time when someone in a community I helped run decided they didn’t like me. They started making subtle jabs, and eventually built up to saying the most biting and personal things they could think to say. Those things weren’t true, but they didn’t know that, and they phrased everything in such a way that their friends could rationalize them as not really trying to be cruel. And they had quite a lot of friends in the community, which put me in a pretty awkward position. How do I justify banning them, if a significant number of people are sure they’re innocent? Am I fucking crazy for seeing this glaring pattern when no one else does?

I did eventually ban them, but it contributed to a complete schism where most of the more grating people left to form their own clubhouse. Win/win?

Or let’s say, hypothetically, that some miscreant constructs a fake tweet screenshot. It’s shared by a high-profile person and spreads like wildfire.

Should either of them be punished? Which one, and why? The faker probably regarded it as a harmless joke; if not for the sharer, it would’ve remained one. Yet the sharer’s only crime was being popular. Did the sharer know it was fake? Was the sharer trying to inflict harm, draw attention to troubling behavior, or share something that made them laugh? Are the faker and the sharer the same person? If you can’t be sure either way, does it matter?

What if, instead of the thing you may be thinking about, the forgery depicted Donald Trump plagiarizing Barack Obama’s tweet congratulating Michelle Obama for her speech? Does that change any of the answers?

This is really difficult in extremely large groups, where you most want to avoid doling out arbitrary punishment, yet where people who play this game can inflict the most damage. The people who make and enforce the rules may not even be part of the group any more, and certainly can’t form an impression of every individual person in the group, so how can anything be enforced consistently? How do you account for intention, sarcasm, irony, self-deprecating humor? How do you explain this clearly without subjecting yourself to an endless deluge of technicalities? You could refuse to explain yourself at all, of course, but then you leave yourself open for people to offer their own explanations: you’re a tyrant who bans anyone who contradicts you, or you hated them for demographic reasons, or you’re just plain irrational and do zany cruel things to people around you on a whim.

I don't have any good answers

I’m not sure there are any. Corralling people is a tricky problem. We still barely know how to do it in meatspace groups of half a dozen, let alone digital groups numbering in the hundreds of millions.

Our current approaches kinda suck, though.