Sextortion Is A Real & Serious Criminal Issue; Blaming Section 230 For It Is Not
Let’s say I told you a harrowing story about a crime. Criminals from halfway around the world used fraudulent means and social engineering to scam a teenager, causing them to effectively destroy their lives (at least in the mind of the teen). The person whose life was destroyed then took an easily accessible gun from their parent and shot and killed themselves. Law enforcement investigated the crime, tracked down the people responsible, extradited them to the US and tried them. Eventually, they were sentenced to many years in prison.
Who would you blame for such a thing?
Apparently, for some people, the answer is Section 230. And it makes no sense at all.
That, at least, is the takeaway from an otherwise harrowing, distressing, and fascinating article in Bloomberg Businessweek about the very real and very serious problem of sextortion.
The article is well worth reading, as it not only details the real (and growing) problem of sextortion, but shows how a momentary youthful indiscretion — coaxed by a skillful social engineer — can destroy someone’s life.
The numbers on sextortion are eye-opening:
It was early 2022 when analysts at the National Center for Missing & Exploited Children (NCMEC) noticed a frightening pattern. The US nonprofit has fielded online-exploitation cybertips since 1998, but it had never seen anything like this.
Hundreds of tips began flooding in from across the country, bucking the trend of typical exploitation cases. Usually, older male predators spend months grooming young girls into sending nude photos for their own sexual gratification. But in these new reports, teen boys were being catfished by individuals pretending to be teen girls—and they were sending the nude photos first. The extortion was rapid-fire, sometimes occurring within hours. And it wasn’t sexually motivated; the predators wanted money. The tips were coming from dozens of states, yet the blackmailers were all saying the same thing:
“I’m going to ruin your life.”
“I’m going to make it go viral.”
“Answer me quickly. Time is ticking.”
“I have what I need to destroy your life.”
As the article details, there is something of a pattern in many of these sextortion cases. There are even “training” videos floating around that teach scammers how to effectively social engineer the result: get control over an Instagram or Snapchat account of a young girl and start friending/flirting with teen boys.
After getting flirty enough, send a fake nude and ask for one in return. Then, the scammer goes straight into extortion mode the second the teen boy does the teen boy thing and sends a compromising photo, focused on promising to ruin the boy’s life:
Around midnight, Dani got flirtatious. She told Jordan she liked “playing sexy games.” Then she sent him a naked photo and asked for one in return, a “sexy pic” with his face in it. Jordan walked down the hallway to the bathroom, pulled down his pants and took a selfie in the mirror. He hit send.
In an instant, the flirty teenage girl disappeared.
“I have screenshot all your followers and tags and can send this nudes to everyone and also send your nudes to your family and friends until it goes viral,” Dani wrote. “All you have to do is cooperate with me and I won’t expose you.”
Minutes later: “I got all I need rn to make your life miserable dude.”
As the article notes, this is part of the “playbook” that is used to teach the scammers:
The Yahoo Boys videos provided guidance on how to sound like an American girl (“I’m from Massachusetts. I just saw you on my friend’s suggestion and decided to follow you. I love reading, chilling with my friends and tennis”). They offered suggestions for how to keep the conversation flowing, how to turn it flirtatious and how to coerce the victim into sending a nude photo (“Pic exchange but with conditions”). Those conditions often included instructions that boys hold their genitals while “making a cute face” or take a photo in a mirror, face included.
Once that first nude image is sent, the script says, the game begins. “NOW BLACKMAIL
!!” it tells the scammer, advising they start with “hey, I have ur nudes and everything needed to ruin your life” or “hey this is the end of your life I am sending nudes to the world now.” Some of the blackmail scripts Raffile found had been viewed more than half a million times. One, called “Blackmailing format,” was uploaded to YouTube in September 2022 and got thousands of views. It included the same script that was sent to Jordan DeMay—down to the typos.
The article mostly focuses on the tragic case of one teen, DeMay, who shot himself very soon after getting hit with this scam. The article notes, just in passing, that DeMay had access to his father’s gun. Yet, somehow, guns and easy access to them are never mentioned as anything to be concerned about, even as the only two suicides mentioned in the article both involve teen boys who seemed to have unsupervised access to guns with which to shoot themselves.
Apparently, this is all the fault of Section 230 instead.
Hell, even as the article describes how this was a criminal case, and (somewhat amazingly!) the FBI tracked down the actual scammers in Nigeria, had them extradited to Michigan, and even got them to plead guilty to the crime (along with a mandatory minimum of 15 years in prison). Apparently, this is still… an internet problem?
The reality is that this is a criminal problem, and it’s appropriate to treat it as such, where law enforcement has to deal with it (as they did in this case).
It seems like there are many things to blame here: the criminals themselves (who are going to prison for many years), the easy access to guns, even the failure to teach kids to be careful with who they’re talking to or what to do if they got into trouble online. But, no, the article seems to think this is all Section 230’s fault.
DeMay’s family appears to have been suckered by a lawyer into suing Meta (the messages to him came via Instagram):
In January, Jordan’s parents filed a wrongful death lawsuit in a California state court accusing Meta of enabling and facilitating the crime. That month, John DeMay flew to Washington to attend the congressional hearing with social media executives. He sat in the gallery holding a picture of Jordan smiling in his red football jersey.
The DeMay case has been combined with more than 100 others in a group lawsuit in Los Angeles that alleges social media companies have harmed children by designing addictive products. The cases involve content sent to vulnerable teens about eating disorders, suicide and dangerous challenges leading to accidental deaths, as well as sextortion.
“The way these products are designed is what gives rise to these opportunistic murderers,” says Matthew Bergman, founder of the Seattle-based Social Media Victims Law Center, who’s representing Jordan’s parents. “They are able to exploit adolescent psychology, and they leverage Meta’s technology to do so.”
Except all of that is nonsense. Yes, sextortion is problematic, but what the fuck in the “design” of Instagram aids it? It’s a communication tool, like any other. In the past, people used phones and the mail service for extortion, and no one sued AT&T or the postal service because of it. It’s utter nonsense.
But Bloomberg runs with it and implies that Section 230 is somehow getting in the way here:
The lawsuits face a significant hurdle: overcoming Section 230 of the Communications Decency Act. This liability shield has long protected social media platforms from being held accountable for content posted on their sites by third parties. If Bergman’s product liability argument fails, Instagram won’t be held responsible for what the Ogoshi brothers said to Jordan DeMay.
Regardless of the legal outcome, Jordan’s parents want Meta to face the court of public opinion. “This isn’t my story, it’s his,” John DeMay says. “But unfortunately, we are the chosen ones to tell it. And I am going to keep telling it. When Mark Zuckerberg lays on his pillow at night, I guarantee he knows Jordan DeMay’s name. And if he doesn’t yet, he’s gonna.”
So here’s a kind of important question: how would this story have played out any differently in the absence of Section 230? What different thing would Mark Zuckerberg do? I mean, it’s possible that Facebook/Instagram wouldn’t really exist at all without such protections, but assuming they do, what legal liability would be on the platforms for this kind of thing happening?
The answer is nothing. For there to be any liability under the First Amendment, there would have to be evidence that Meta employees knew of the specific sextortion attempt against DeMay and did nothing to stop it. But that’s ridiculous.
Instagram has 2 billion users. What are the people bringing the lawsuit expecting Meta to do? To hire people to read every direct message going back and forth among users, spotting the ones that are sextortion, and magically stepping in to stop them? That’s not just silly, it’s impossible and ridiculously intrusive. Do you want Meta employees reading all your DMs?
Even more to the point, Section 230 is what allows Meta to experiment with better solutions to this kind of thing. For example, Meta has recently announced new tools to help fight sextortion by using nudity detectors to try to prevent kids from sending naked photos of themselves.
Developing such a tool and providing such help would be riskier without Section 230, as it would be an “admission” that people use their tools to send nudes. But here, the company can experiment with providing better tools because of 230. The focus on blaming Section 230 is so incredibly misplaced that it’s embarrassing.
The criminals are actually responsible for the sextortion scam and the end results, and possibly whoever made it so damn easy for the kid to get his father’s gun in the middle of the night to shoot himself. The “problem” here is not Section 230, and removing Section 230 wouldn’t change a damn thing. This lawsuit is nonsense, and sure, maybe it makes the family feel better to sue Meta, but just because a crime happened on Instagram, doesn’t magically make it Instagram’s fault.
And, for good reason. As noted above, this was always a law enforcement situation. We shouldn’t ever want to turn private companies into law enforcement. Because that would be an extremely dangerous result. Let Meta provide its communications tools. Let law enforcement investigate crimes and bring people to justice (as happened here). Maybe we should focus on better educating our kids to be aware of threats like sextortion and how to respond to it if they happen to make a mistake and get caught up in it.
There’s lots of blame to go around here, but none of it belongs on Section 230.