John, a father of two in Colorado, had no idea what his 15-year-old son had gotten into, until one night last year when John walked into his home office. We're not using his last name to protect his son's privacy.
John saw a large pile of papers facedown next to his printer. He turned them over and found a copy of a notorious neo-Nazi propaganda book. "It's 'the white culture's in trouble, we are under attack by Jews, blacks, every other minority.' It was scary. It was absolutely frightening to even see that in my house. I was shaking, like, 'What in the world is this and why is it in my house?' "
John confronted his son angrily.
"I was through the roof." And then, "I went back into my room. I was crying. I felt like a failure that a child that I had raised would be remotely interested in that sort of stuff."
Almost every teen plays video games — 97 percent of boys, according to the Pew Research Center, and 83 percent of girls.
Increasingly, these games are played online, with strangers. And experts say that while it's by no means common, online games — and the associated chat rooms, livestreams and other channels — have become one avenue for recruitment by right-wing extremist groups.
At the time, John's son liked playing first-person shooter games, like Counterstrike: Global Offensive. Games like these are multiplayer — you must form teams with friends or strangers. You can chat in the game, over voice or text, or in separate chat rooms. Some of these are hosted by sites like Discord that make it easy for anyone to create a private chat.
John knew his son was spending time playing video games and chatting either out loud or over text, but there were no obvious red flags.
"There wasn't anything obvious to me at first because it's common. This is the norm for kids. Instead of hanging out at the drive-in they're all online," he said.
Yet it's exactly this way, John says, that his son started hanging out with avowed white supremacists.
These people became his son's friends. They talked to him about problems he was having at school, and suggested some of his African-American classmates as scapegoats. They also keyed into his interest in history, especially military history, and in Nordic mythology. Above all, they offered him membership in a hierarchy: whites against others.
"He started to feel like he was in on something. He was now in the in crowd with these guys. It provided some structure and identity that he was searching for at the time."
John learned his son had been drawn into conversation with at least one group that the Southern Poverty Law Center calls a Nazi terrorist organization. He searched online for help and found a man named Christian Picciolini.
Picciolini runs the Free Radicals project, which he calls "a global prevention network for extremism." He's a reformed skinhead himself.
"Thirty years ago, when I was involved in the white supremacist movement, it was very much a face to face interaction," he says. "You know, you had to meet somebody to be recruited, or you had a pamphlet or a flyer put on your car."
But today, he says, it's much more common for extremists to initially reach out online. And that includes over kids' headsets during video games. Picciolini describes the process: "Well typically, they'll start out with dropping slurs about different races or religions and kind of test the waters ... Once they sense that they've got their hooks in them they ramp it up, and then they start sending propaganda, links to other sites, or they start talking about these old kind of racist anti-Semitic tropes."
That's also what Joan Donovan has seen. She is the media manipulation research lead at Data and Society, a research institute, and she has been following white supremacists online for years. She says they've been highly innovative in using new online spaces, like message boards in the '90s, for recruitment.
"I saw how these groups communicated and spread out to other spaces online with the intent of not telling people specifically that they were white supremacists, but they were really trying to figure out what young men were angry about and how they could leverage that to bring about a broad-based social movement."
And violent first-person shooter games, she says, are one place to find angry young men. She calls "gaming culture" "one of the spaces of recruitment that must be addressed."
Donovan says that recruitment, and even the planning of harassment campaigns, happens not only during in-game chat, but during livestreaming of game play on platforms like Twitch and YouTube.
For example, there's a feature on YouTube called Super Chat, where fans can offer cash tips while gamers are playing.
"People will donate 14 dollars and 88 cents, which is a reference to ... a white nationalist slogan, as well as 88, which is most commonly found in prison tattoos, for Heil Hitler," Donovan said.
Game-related Reddit threads and chat sites like Discord also host similar conversations. Last year, a nonprofit media collective called Unicorn Riot published chat logs from Discord in which known white supremacists planned aspects of the Charlottesville "Unite The Right" rally.
Video games are a hundred billion dollar industry.
What are companies' responsibilities to ensure that young people won't encounter hate groups? We reached out to several game and chat companies for comment. Riot Games noted in a statement that it relies on volunteers to moderate game-related chats. And Discord, the chatroom site, forwarded a statement from the Southern Poverty Law Center, praising it for recently banning several far-right extremist communities.
Greg Boyd, who represents video game companies for the law firm Frankfurt Kurnit, says "toxic" behavior including hate speech, to say nothing of recruitment, is a key industry concern and a frequent topic of conversation. "If they could find it all they would get rid of it ASAP."
But it's a daunting technical challenge. The three biggest video game platforms — Microsoft, PlayStation and Steam — host 48 million, 70 million and 130 million monthly active players respectively, Boyd says. "That's the populations of Spain, France and Russia. And then imagine that you're monitoring all of their text chat ... all of their voice chat, in literally every language, dialect, and subdialect spoken in the world."
In the absence of sufficient resources for moderation, most game platforms rely on players to monitor and report each other.
Picciolini compares the companies to landlords with disruptive tenants "disrupting or damaging the building or threatening the other tenants. You know, they would take action."
But, Picciolini says, ultimately unwinding the influence of these groups over young people takes love and acceptance. He counseled John to find out what his son's personal struggles were — he calls them "potholes." As John put it, "He's missing something, so find out what those [potholes] are and fill those with something positive and healthy, and that's the way to steer them out of all this."
It took time, but lately, John says, his son, now 16, seems to have left these ideas behind. He's playing fewer networked shooter games, and on his own, he has started attending church.
AILSA CHANG, HOST:
All right, recent acts of violence - the shooting at a Pittsburgh synagogue, the explosives sent to critics of President Trump - happened in the real world, but the warnings took place online on social media. On this month's All Tech Considered, we look at toxic content - how it spreads and what should be done about it.
(SOUNDBITE OF ULRICH SCHNAUSS' "NOTHING HAPPENS IN JUNE")
CHANG: The Pew Research Center says 97 percent of boys and the vast majority of girls play video games. Increasingly, they're playing online with strangers. Although it's rare, experts say these games can become an avenue for recruitment by right-wing extremist groups. NPR's Anya Kamenetz has our story.
ANYA KAMENETZ, BYLINE: John's a father of two teenagers, and he said he never worried about his son's video game habit.
JOHN: I knew nothing about any of this until one night I'd walked into my home office.
KAMENETZ: We're not using John's last name to protect his family's privacy.
JOHN: I was shaking when I saw it. I'm like, what in the world is this? And why is this in our house?
KAMENETZ: He saw a pile of papers face-down next to his printer. He turned them over and found a copy of a notorious neo-Nazi propaganda book.
JOHN: It's white culture is in trouble, and we are under attack by Jews and blacks and every other minority. It was scary. It was absolutely frightening to even see that in my house.
KAMENETZ: John confronted his son angrily.
JOHN: And then I went back to my room. And I honestly was crying. I felt like a failure that a child that I had raised would even be remotely interested in this stuff.
KAMENETZ: Like millions of other kids, John's son loved first-person shooter games. It's increasingly popular to play these games online and form teams with friends or strangers.
JOHN: There wasn't anything obvious to me at first because it's common. Online gaming and chatting with friends online - that's the norm now instead of going out and hanging out at the drive-in or whatever.
KAMENETZ: And it's this way, John says, that his son started hanging out with avowed white supremacists. They keyed into his interests in military history and Nordic mythology.
JOHN: He was now in the in-crowd with these guys.
KAMENETZ: John learned his son had been drawn into at least one group that the Southern Poverty Law Center calls a Nazi terrorist organization. He searched online for help and found it.
CHRISTIAN PICCIOLINI: Christian Picciolini, founder of the Free Radicals Project, author of "White American Youth."
KAMENETZ: Picciolini is a heavily tattooed former skinhead. He's now dedicated his life to helping people move away from hate.
PICCIOLINI: We are assembling a team of people around the world to help people disengage from violent extremism.
KAMENETZ: Thirty years ago, he says...
PICCIOLINI: You had to meet somebody to be recruited. Or you had a pamphlet or a flyer put on your car, and you reached out.
KAMENETZ: Today, he says, it's much more common for extremists to reach out online, and that includes over kids' headsets during video games.
PICCIOLINI: Typically, they'll start out with dropping slurs about different races or religions and test the waters. Once they sense that they've got their hooks in them, they ramp it up. And then they start sending propaganda. They start sending links to other sites, or they start talking about these old racist, anti-Semitic tropes.
JOAN DONOVAN: Gaming culture is one of the spaces of recruitment that must be addressed.
KAMENETZ: Joan Donovan is with Data & Society, a research institute. For years, she's been tracking white supremacists from platform to platform online.
DONOVAN: They were really trying to figure out what young men were angry about and how they could leverage that to bring about broad-based social movement.
KAMENETZ: And first-person shooter games, chat rooms and video platforms, she says, are good places to find angry young men.
Video games are a hundred-billion-dollar industry. So what are companies' responsibilities to ensure that teens won't encounter hate groups? Greg Boyd is a lawyer who represents the game industry for the firm Frankfurt Kurnit. He says companies take the problem seriously and remove or ban people when they're flagged by other players, but the scale of the issue is daunting.
GREG BOYD: You're talking about Microsoft, PlayStation and Steam. You're talking about 48 million, 70 million and 130 million monthly active players or players that are playing, you know, probably on a weekly basis. I mean, that's the populations of Spain, France and Russia.
KAMENETZ: Imagine, he says, moderating all that chat, text and voice moment by moment.
BOYD: In literally every language dialect and subdialect spoken in the world.
KAMENETZ: While the industry struggles to contain the threat, experts say it's up to parents to keep an ear out and to step in if they notice something that concerns them. John tried. And lately, he says, his son, now 16, seems to have left these ideas behind. He's playing fewer online shooter games. And on his own, he started attending church. Anya Kamenetz, NPR News. Transcript provided by NPR, Copyright NPR.