Follow us on Twitter

Follow us on Facebook

Are governments and technology companies doing enough to counter the threat of online extremism?

Jul 8, 2019 | Studies & Reports

By Gordon Corera

Security correspondent

Far right activity on the internet threatens to spill over into violence in the real world, according to some observers. But are governments and tech companies doing enough to counter the danger?

Two days before the attack, Australian Brenton Tarrant tweeted images of the weapons he was going to use. Half an hour before, he outlined what he was going to do in an online forum. A few minutes before, he emailed a “manifesto” explaining why. And as he began, he switched on a Facebook live stream for people around the world to watch.The attack on mosques in Christchurch, New Zealand, in which more than 50 people were killed, was an act of violence foreshadowed online.Tarrant, who appeared in court in June to deny the charges, was a lone individual. But he was also someone who inhabited an internet-based international subculture, one whose ideology is moving from the darker reaches of the internet into the political mainstream.The fear is that security services and tech companies have been slow to deal with this growing new threat.

Travels

In late 2018, Tarrant was a long way from his New Zealand home, driving around Austria.He had spent the past few years travelling the world, venturing as far afield as North Korea and Pakistan.A reconstruction of his travels shows he spent most of his time crisscrossing Europe, from Spain to the Balkans and almost every country in between. There, he would often stop off to visit sites with links to particular historic battles.His 74-page manifesto is rooted in Europe and its history, with references to age-old European battles to attempt to justify an attack carried out on the other side of the world. In it, he also cites his time in France – where he describes seeing an “invasion” of immigrants – as having particularly affected him.

At the manifesto’s centre is the idea of something called the “great replacement” – a claim that “European” people are being replaced by Muslims (despite projections from independent bodies such as the Pew Forum saying there is no sign of this happening in the coming decades).Tarrant’s exact movements in Austria are unknown but they remain a focus of much interest.A tweet days before the Christchurch attack included a reference to “Vienna 1683”. This was the year the advance of Ottoman forces was stopped just outside the city. For some on the far-right, Austria has an almost mythical significance as the front line in the war centuries ago between Christian Europe and the Muslim world.

But there is another link between Tarrant and Austria. His manifesto is filled with in-jokes for those who inhabit a particular online subculture, with references to computer games and far-right memes. It’s a sign of a man who lived much of his life online. And through this world he had become an admirer of Martin Sellner, a young Austrian social media influencer who plays a major role in Europe’s new far-right ecosystem.In early 2018, Tarrant liked the Austrian enough to give him a 1,500 euro (£1,350; $1,690) donation.

The propagandist

I meet Martin Sellner in a café in a suburb of Vienna. With his sharp haircut and trainers, he looks nothing like the traditional image of a far-right activist.The 30-year-old is a leading figure in Generation Identity, the Austrian offshoot of the so-called identitarian movement, which is fiercely opposed to Muslim migrants, claiming that they threaten Europe’s identity and will eventually replace the indigenous populations.The movement began in France in 2012 and has expanded to nine countries including Germany, Italy and the UK.He is happy to be interviewed about his views, but the one issue he becomes uncomfortable talking about is his links to Brenton Tarrant.

Two weeks after the Christchurch attack, Sellner’s home was raided when details of the donation came to light. As part of an investigation into a possible link between Tarrant and Sellner, computers and bank cards were seized.Sellner says he never met Tarrant, but admits they exchanged emails after the donation.”If you ever come to Vienna, we have to go for a coffee or a beer,” Sellner wrote.”The same is true for you if you ever come to Australia or New Zealand,” replied Tarrant. “We have people in both countries who would like to welcome you to their home.”

Sellner rejects the idea that he and Tarrant share an ideology, stating that his aim is to bring about change through political activism and persuasion rather than violence. He condemned Tarrant as “misled” and “misguided” in a video soon after the attack.But those who worry about the far right argue that Tarrant’s actions were a result of an ideology spread by people like Sellner.His online postings often focus on the threat of multiculturalism and the idea that Muslims will take over Europe.Sellner tells me when he sees people in Vienna wearing headscarves, he doesn’t see Austrians but people with a different identity.

When I point out that Vienna has long been diverse – its grandeur coming from its time as the capital of the Austro-Hungarian Empire where many cultures and nationalities co-existed – Sellner’s fluency briefly fails him. It was culturally diverse, he acknowledges, but the real difference today, he says, is the birth rate.The phrase “it’s the birth rate” is one repeated at the opening of Tarrant’s manifesto (and one that demographers challenge as based on false assumptions). This convergence of ideas is what leads critics of Sellner to challenge his attempts to disassociate himself from the violence of Tarrant.

“I personally think he’s dangerous even though he’s not strictly speaking doing anything illegal,” says Prof Peter Neumann of King’s College, London, who founded the International Centre for the Study of Radicalisation. “I don’t think he’s personally involved in commissioning violence but the ideas he puts out there are being used by people to justify violence.”Sellner makes no secret of the fact that when he was a teenager, he was immersed in Austria’s neo-Nazi scene.But he says that as he matured, he realised how much it was driven by bitterness and hate, and so he embraced the new politics of identitiarianism. Over the past few years, it has grown into a Europe-wide far-right youth network.

Sellner asserts that he is not a racist but an “ethnopluralist” who believes that every culture has the right to maintain its own separate identity. In practice, this means separation.But critics are sceptical of his personal transition from neo-Nazi. They say he has been smart enough to realise that certain views are just too unpalatable to have widespread appeal.Inside Vienna’s old town hall sits the Documentation Centre of Austrian Resistance, detailing those who resisted Nazism and its victims. Bernard Weidinger, who tracks the modern far right, tells me that identitarians have simply rebranded to reach a new generation.

“Neo-Nazi ideology does not attract the masses in Austria anymore. So it made sense to modernise their appearance, to modernise the language and to some extent to also modernise their ideas.”They have replaced terms that have been considered historically stained by newer terms that are more appealing to a broader public. For example they don’t speak of mass deportation – they speak of ‘re-migration’. They say, ‘we’re not racist, we’re ethnopluralist.'”Weidinger says Austria has witnessed the normalisation of far-right language in the past few years. For example, “re-migration” has moved into mainstream political discourse even though mass deportations would inevitably involve large-scale use of force by the state.

Sellner is reluctant to explain how else large numbers of migrants would leave Austria.His group is close to Austria’s far-right Freedom Party, which became the junior partner in the governing coalition after achieving more than a quarter of the vote in 2017. “Our ideas have a lot of intersection with theirs,” Sellner says. The Freedom Party’s leader Heinz-Christian Strache, who became vice-chancellor after the election, has described how his party was fighting a “replacement” of the native population.In May, the coalition collapsed and the party is no longer part of the government.

Sellner has pioneered the use of social media in the German-speaking far-right world, and researchers say that his influence extends well beyond Austria.Joe Mulhall from the UK-based organisation HOPE not Hate describes Sellner as the de facto European leader for Generation Identity, since he speaks good English and has a feel for social media.”They have created a new language and lexicon for the European far right,” he says.He says that although Sellner disassociates himself from violence, there is a clear link. “If you spend your time talking about the replacement of a people and talk about us approaching a crunch time, you get people acting on it.”

As well as trying to shape political discourse by mainstreaming their ideas, identitarians also “plant the seeds” for violence through their rhetoric, Mulhall adds.Sellner, however, describes himself an “information warrior” whose ambition is to use the online world to leverage real-world political change.He and others on the far right have been agile at exploiting the freedoms of the online world.Identitarian groups are highly networked, circulating ideas, learning from each other and adapting those ideas to their own national context.Sellner is engaged to the alt-right vlogger Brittany Pettibone, who has appeared with him in his YouTube videos.

In June 2019, Sellner posted a letter from the UK Home Office stating that he had been permanently denied entry to the UK because Generation Identity “actively promotes anti-Islamic and anti-immigration narratives”. Sellner was deemed to pose a “serious threat” to countering extremism and protecting shared values.But he has been to the UK in the past.In the autumn of 2017, he attended the launch of the UK branch of Generation Identity. Sellner joined 15 people at a flat in Brixton, south London, to give a presentation and offer advice about tactics – particularly what to say if asked by the media if they were racists.Also present was another Austrian, a woman of roughly the same age as Sellner. Julia Ebner had secured an invitation after having been interviewed over Skype and in person about her political views. But the woman who turned up in the blonde wig that night was no identitarian. She was undercover.

Undercover

Ebner came to the UK in 2014 and began researching jihadist online propaganda. She switched focus after the murder in 2016 of the MP Jo Cox by a far-right loner. A court would later hear how he had researched Nazi and white supremacist groups online.Ebner quickly began to notice parallels between online jihadist groups and those of the far right, both in their tactics and their discussions of the inevitability of conflict between Muslims and non-Muslims.Going undercover has been important for her work with the Institute for Strategic Dialogue and research for her own book.In her office in central London, Ebner describes such situations as “quite scary” given how “hyper-suspicious” the groups are about infiltrators.

Most of her time, however, is spent infiltrating the online world, going beyond the open sites and delving into private chat rooms and closed conversations. “It’s definitely very disturbing,” she says, adding that she often needs a social media “detox” for a few days afterwards.She creates “avatar accounts” – false identities – to gain access to encrypted, invite-only channels. “Some of the conversations really take place in some very hidden corners of the internet,” she says.She has had to familiarise herself with cultural references and insider jokes to evade detection, and is often “vetted” – either by interview, or via social media accounts.

The more extreme the group, the tougher the vetting. Once she was asked to provide results of a genetic test to show her ancestry and prove that she was racially “pure”. She submitted fake results, she says.As Ebner takes me into the far-right online, links to dozens of groups from around the world fill the screen. These groups cooperate in what she calls “network nationalism”. A US group, for example, may create a content-sharing platform with a bank of anti-immigrant memes that can then be adapted by other groups in different countries.This ecosystem is inhabited by a bewildering array of groups – from national neo-Nazis, often defined by anti-semitism, to newer groups focusing on Western and European identity, which tend to identify Muslims as a threat.

YouTube and Instagram are used to try to reach a large mainstream audience. This is where the simply curious might come across far-right content. It is also where algorithms play an important role since they direct people to content judged likely to keep them on the platform.”You very quickly end up in the very extreme echo chamber,” says Ebner.Some of those who engage with extremist content will be offered a link to take them to the next level – often an invite-only encrypted messaging app. Telegram, for example, which was used by the Islamic State group, is popular with the far right.Such groups can also find a safe haven on libertarian sites, such as the message boards 4chan, 8chan, and Gab. A spokesperson for Gab said it protected all speech allowed under the US constitution: “Gab doesn’t condone hate speech, and our lives would be a lot easier if people wouldn’t post it on our site, but in order to adhere to our principle the site must allow it.”

Computer gaming sites are becoming increasingly attractive to far-right groups, according to Ebner. Discord, an application designed for gamers to communicate, is one that she has been particularly watching. Here, some groups have loose, decentralised chat groups, while others maintain strict hierarchies and control, echoing military structure. Membership of the various groups ranges from a handful to about 20,000.Ebner says that recruiters tailor their messages depending on where they are: “To recruit people from the gaming community, they would use a very “gamified” approach. When trying to recruit someone from the ultra-libertarian platforms, they would emphasise topics around freedom of speech. When trying to lure people from the conspiracy networks into their far-right channels, they would immediately reference those conspiracy theories.”

Generation Identity and the identitarians are the most dynamic and fastest-growing part of the new right, she says. And they demonstrate an understanding of their young audience. “They use a lot of gamification elements, but also make references to pop culture and develop their own insider jokes.”Online groups are increasingly having real-world effects. According to Ebner, this might involve influencing political debate by pushing a subject like migration into the top online trends in Germany. On the screen she shows me a Generation Identity-affiliated group discussion on Discord, about the printing of 40,000 fliers for a forthcoming election.”Some of these groups have been building up networks and they’re able to very quickly share pieces of disinformation and jointly launch bigger information operations in the online space, but also mobilise very quickly for offline.”And the evidence that this points towards violence, and not just political action, has been growing.

Pushing back

Since Christchurch, governments and tech companies have scrambled to deal with the threat from the far right. In May, tech companies and governments met in Paris and produced a “Call to Action”, which committed them to do more to curb the spread of terrorism and extremism online.YouTube has announced measures against white supremacist and neo-Nazi groups and individuals. Facebook has also permanently banned some individuals and groups, including the British National Party in the UK, and taken action against white supremacism and separatism.Sellner says he has been “deplatformed” on Instagram, Facebook, Kickstarter as well as other sites. His YouTube channel is still available.

“I know that tomorrow my digital existence could be vaporised on the official platforms,” he says.Because of this, he has been focusing his efforts on platforms such as Telegram, which are seen as less susceptible to political pressure, as well as building up email lists.In the US, Charlottesville was a turning point for tech companies. Discord, for one, began taking major steps afterwards, working with civil society groups that could help identify far-right content (although it has been slower to deal with material from other countries).Some argue there are risks to an exclusionary approach.

“A key part of the far-right narrative is about censorship,” says Adam Hadley of the group Tech Against Terrorism. “In banning accounts, you can validate the narrative, feeding the very thing you are trying to prevent.”Tech companies are trying to adapt systems built to deal with jihadist content to hunt for far-right content but there have always been limits to automation and the challenges are not always the same.Tarrant posted pictures of the guns he planned to use but without any accompanying text, in the knowledge that most of the systems are built around automated detection of keywords.He was suspended from Twitter and law enforcement was informed within minutes of the attack.

Christchurch also forced the issue of live streaming onto the agenda. Only about 200 people actually watched the live stream of the attack as it took place but the video was then rapidly distributed around the world (Facebook believes this was initially done by an 8chan user).One person with experience of dealing with the challenges of extremism on social media said they had never seen anything like it go so viral so quickly. In the first 24 hours after the attack, there were 1.5 million attempts to upload the video, about 1.2 million of which were successfully blocked. Some extremists shared tactics on how to modify the video to evade the filters designed to prevent it being uploaded.

Mainstream media and ordinary people, rather than just extremists, were also responsible for much of the distribution. In May, Facebook announced new restrictions on live streaming.Critics question whether tech giants have any interest in dealing with the deeper problem of algorithms designed to maximise the time people stay on their sites, even if that means being recommended and delivered more extremist content.According to YouTube, it launched changes to its recommendations systems in January 2019, which have limited the spread of harmful misinformation and what it calls “borderline content” by 50%.

The clearest warning of Christchurch was the posting on 8chan just before the attack. But spotting these types of posts requires real-time monitoring by police and security services, usually with undercover officers in the chat rooms. But it is not yet clear that enough resources or attention are being devoted to this.While independent researchers and civil society groups have been investigating, tracking and warning of the rise of the far right over a lengthy period, security services have been more reactive.Security sources say that while people talk about violence in chat rooms, the problem can be working out which ones are serious, and where they are.

In the UK, the murder of Jo Cox in 2016, perhaps surprisingly, did not lead to a shift in focus by the security service, MI5. For a long time, it resisted taking on responsibility for investigating far-right threats, partly because it felt it had its hands full tackling jihadist violence.The shift finally began in 2017 with a string of jihadist terrorist attacks accompanied by violence from the far-right. One UK intelligence source, who doesn’t want to be named, says that analysts then found the threat was “far wider and deeper than previously understood”.Another source says the issue of the far right is now always on the agenda in meetings of UK counter-terrorist officials. However, they say there is little focus on the international network or ideology behind it.

There is also a further complication for many governments since the issue potentially brings security services into monitoring politics in a way that dealing with jihadist violence does not. This is because of the way far-right ideas have entered the mainstream.The more mainstream these narratives become, the greater the tension will be over whether they really are extreme or whether they represent acceptable political discourse, and the views of a substantial number of real people. Far-right groups exploit this tension.But those who track groups like Sellner’s think not enough is being done to deal with people who spread the ideology that fuels events like Christchurch.

“The danger of Generation Identity lies in its potential to normalise an ideology which has ethnic separatism at its core,” says Mulhall.With jihadists, there was a long debate about whether to simply go after those planning violence, or whether to broaden the target to include “radicalisers”, who were potentially more dangerous and harder to deal with, since they were careful not to cross certain lines.A similar but more politicised debate may lie ahead with the far right.Looking for Brenton Tarrant is one matter. Investigating ideologues who claim to be non-violent, with links to political parties, may be harder.Christchurch may have been a turning point for how the interaction of online material and real-world violence is dealt with. But the fear in some quarters is that the full scale of this challenge is still not understood.

BBC

Related articles:

Follow us on Twitter

Follow us on Facebook