Toxic social media content is impossible to avoid, but companies are in hiding

Year 12 student Noah Cameron has never seen Andrew Tate pop up in his endlessly whirring social media feeds.

“But I can’t say that I have been able to avoid the thousands of videos that are put out there that are trying to emulate that style,” the 18-year-old said.

Tate, the British-American kickboxer turned social media influencer, has become a poster child for an ultra-masculinity many say is dangerous. But he’s far from alone.

“Anything that is created that generates some kind of interest spawns millions of copycats,” Noah told the ABC’s Q+A on Monday.

“It becomes pervasive.”

Noah is a school captain at Yarra Valley Grammar, a private Anglican school in the north-eastern suburbs of Melbourne that has generated headlines for all the wrong reasons.

In May, staff uncovered a spreadsheet created by a group of boys rating female classmates on their looks. Girls were classified as “wifeys” and “cuties” and even “unrapeable”.

But it wasn’t anything Noah hadn’t already seen online.

“It’s pretty impossible to not necessarily avoid the content itself, but avoid reference to the content through the algorithm,” he said.

“Things do come up on your feed that maybe you necessarily didn’t want to see, but given the extent that they are generating some kind of interest in other circles, they are pushed on you.”

Platform algorithms are ‘scary’

Alice Dawkins calls it “pernicious”.

“We know about rabbit holes, we know what the beginning of the youth radicalisation pipeline looks like,” Ms Dawkins, Executive Director of Reset.Tech Australia, said.

Reset.Tech, which researches the space and advocates for accountability, ran simulations of what younger Australians were seeing in their feeds.

“It was scary,” she said. “The shift happens very, very quickly.

“The content goes from diverse to not diverse rapidly. We have done experiments where the shift happened in minutes.

And algorithms at most social media companies are shrouded in mystery.

“This is a critical issue holding us back as a country to understand with empirical clarity what is going on,” Ms Dawkins said.

Frances Haugen — who worked at Facebook in the US before quitting and blowing the whistle on its corporate conduct — said platforms were in hiding.

“In the last couple of days, the most useful tool for looking at Instagram or Facebook was shut down by Facebook,” she told Q+A. “There will be nothing researchers can do to see what is the most popular content on the platforms.

“They want to make sure the US election has no transparency because they know how potentially dangerous it will end up.

“There are lots of tools they are using to become more opaque. And recalcitrant, frankly.”

Schools, parents need help

At Yarra Valley Grammar, students found to be involved in the sexist spreadsheet were disciplined. Two were expelled.

“These attitudes have always existed,” Principal Mark Merry told Q+A.

“It’s just that with social media, they’re amplified and the potential damage that can be caused is much greater.”

Dr Merry noticed a change after students returned to classrooms after COVID lockdowns.

“These young men have come back to school and some of them have attitudes which are very coarse and which they feel free to express publicly online,” he said.

Politicians from all major parties at the state and federal level are considering technology to restrict access to social media for kids under a certain age.

A YouGov/Q+A poll out on Monday shows most Australians — 61 per cent — support an age lockout. An overwhelming number — around eight in ten — want a regulator with the power to order the removal of dangerous content.

Dr Merry does not want to kick his students off platforms he knows most are addicted to.

“I agree it’s very useful,” he said. “It does create connections … it does create friendship groups and so forth.

And he thinks educators are unfairly burdened with the impact.

“It seems to fall to schools to regulate a lot of the behaviour of young people,” he said.

“There must be an option available for us to be able to draw upon the authorities or whomever to assist us.

“A lot of the behaviours aren’t actually in schools, they’re at homes, are behind closed doors, and we find ourselves at a distinct disadvantage.”

‘Regulation alone won’t help’

Julie Inman Grant, Australia’s eSafety Commissioner, agrees banning kids from social media — and potentially pushing them to platforms that are more dangerous and less regulated — isn’t the answer.

Her office has been empowered to target and remove dangerous content, recently taking the X platform to court to try and remove video of a Sydney church shooting (the legal action was ultimately abandoned).

“But regulation alone is not going to help,” she told Q+A.

“We have to bolster that with prevention on the front end … but also through proactive and systemic change.

“That also means anticipating technology trends and how they might be weaponised.”

She cited the rise of deepfakes — maliciously altering digital images — as one area where her office had been on the front foot.

Parents, too, have an important role to play.

“We need to build that respect, responsibility, digital resilience, but also critical reasoning skills,” Ms Inman Grant said.