Recent discussions with members of the OSA Network have focused on the approach to risk management being proposed by Ofcom in its consultation on the Online Safety Act illegal harms duties. Volume 3 of the suite of Ofcom documents covers this topic, including some initial proposals and evidence on governance and accountability.

Governance structures, along with robust risk assessment processes, are fundamental to influencing product design choices with a view to reducing the risk of harm. So Ofcom’s proposals here are crucial to the overall effectiveness of the Online Safety Act regime.

A recent Wired interview with Del Harvey - the former head of Trust and Safety at Twitter (now X) – underscores why this is important. Harvey talks about some of the things that concerned her during her time in her role. She gives the example of trying to escalate within the company the potential threat from a DM she had received suggesting that Twitter’s offices should be bombed: there was no route within the company to do this for such tweets. Harvey says:  

“It was the same issue that it always has been and always will be, which is resourcing. I made requests in 2010 for functionalities that did not get implemented, in many instances, till a decade-plus later.”

She also gives the following example: “Multiple account detection and returning accounts. If you’re a multiple-time violator, how do we make sure you stop? Without going down this weird path of, “Well, we aren’t sure if this is the best use of resources, so instead, we will do nothing in that realm and instead come up with a new product feature.” Because it was growth at all costs, and safety eventually.”

And here’s another, even bigger reason, why this matters in the context of the Ofcom consultation. In volume 3 of Ofcom’s consultation, they frequently refer to the responses received to their 2022 call for evidence and cite, in many instances, evidence provided to them by platforms – including X – that shows they are “already” doing much of what is required by the Act.

For example: “Responses to our 2022 Illegal Harms Call for Evidence demonstrated that several online services already have arrangements whereby they have a dedicated accountable staff member for regulatory compliance with online safety outcomes. This included Mojeek, Google, Trustpilot, X and Glassdoor, which all described overall ownership for online safety compliance at a senior manager level.” (para 8.58)

It is promising that many big platforms can point to existing governance structures (and there are likely to be plenty of platforms and smaller services who won’t be able to do this). But as yet we – or Ofcom – do not know whether these existing governance structures are working effectively. For example, Facebook has run into trouble in the past with investors about its oversight structures for risk.

In volume 4 of Ofcom’s consultation documents, which focuses on the codes of practice, they include: “Governance and accountability arrangements around the management of online safety risks, including senior management visibility of and accountability for key risks” and go on to “propose to recommend that all services establish clear accountability for compliance with their illegal content safety duty, complaints and risk assessment obligations, with additional expectations on large and multi-risk services.”

So how is Ofcom going to assess whether the structures, policies and accountability processes that already exist are sufficient? How will they measure their effectiveness? Is it enough for companies just to say they are doing it?

Let’s go back to Harvey’s interview: “When trust and safety is going well, no one thinks about it or talks about it. And when trust and safety is going poorly, it’s usually something that leadership wants to blame on policies. Quite frankly, policies are going to be a Band-Aid if your product isn’t being designed in a way that actually doesn’t encourage abuse. You’ve got to plan there, guys.” [emphasis added]

Getting the governance right from the start is crucial – there may even be a case for “governance by design” to provide a benchmark for companies that may either be missing the mark, or so new or small that they haven’t yet considered it.

On our initial reading of the proposals, we have concerns that Ofcom may not be quite there. We’ll be coming back to this topic again, in our discussions with our network and in written form before the consultation is over. 

Do get in touch with us with your views: [email protected]