Case study:
Global Social Media Platform

Protecting social media platform users from online harm and hate

Protecting the safe online experiences of users and advertising partners has always been priority No. 1 for our client - a global social media platform with more than a billion users worldwide, and a responsibility to advertisers who contribute revenue in excess of $4 billion annually to its business.

Early on, it established a Trust and Safety team who worked diligently, around the clock, to address illegal and harmful content appearing on its pages. However, this content increasingly included child predation, hateful commentary, terrorist radicalization and propaganda messages. The comments, imagery and videos began to flood the social media platform to promote these ideologies and recruit like-minded individuals.

Their team quickly realized that despite its best efforts, bad actors intent on doing harm to users - including children - would stop at nothing to forward their own malicious agendas.

It quickly became clear that a new approach was needed.

Their team quickly realized that despite its best efforts, bad actors intent on doing harm to users - including children - would stop at nothing to forward their own malicious agendas.

Challenges

Along with seeing a dramatic increase in harmful and hateful content on its social platform, our client also had to address the global realities of its business and influence on the world. This included the geopolitical context, cultural nuance and online colloquialisms behind bad actor behavior and their evasion techniques - a task that proved difficult for the dedicated internal team who was already consumed by present day threats.

In order to get control of their online environment, our client would have to overcome the following challenges:

Evolving evasion techniques

The more harmful content Trust and Safety teams addressed, the more bad actor groups watched, learned and adjusted their techniques to evade and avoid future countermeasures, creating a growing disadvantage and tougher adversary.

Context, culture and colloquialisms

Having more detailed intelligence on bad actor behavior became essential to understanding and preventing online harm at scale. This required a deeper understanding of their culture and the ways in which they communicated with one another.

Sheer scalability

The sheer growth of bad actors, their threats and the complexity of their ideologies, meant more trust and safety resources were needed to monitor the wellbeing of users 24/7/365 globally than was currently available.

Solution

Crisp quickly became an essential extension of the social platform’s Trust and Safety team, providing strategic intelligence, enhanced threat assessments, operational examples and policy development, enabling the social media platform to confidently protect their environment, both now and future-looking.

Strategic intelligence

Supporting the customer in understanding the threat at a macro level to inform future platform-wide policy and enforcement decisions.

Enhanced threat assessments

Tracking the evolution of various threats on a weekly basis to support the customer in developing an evolving response.

Operational examples

Crisp provided a daily set of new platform examples of each threat to support both machine and human decision-making.

Policy development

Supporting client policy development teams to understand new markets, new legislative considerations and new platform threat areas.

Today, our client’s Trust and Safety teams are much more informed about real-time harmful content, both on and off-platform, and associated bad actor evasion techniques. Our client now is equipped to make better data-driven decisions, including horizon-planning, user safety (including children) - covering all illegal and harmful online content types.

Crisp provides the world’s largest social media platforms with rapid intelligence to detect and prevent a full range of illegal and harmful content. To date, these efforts have contributed to the safe online experiences of over 2.2 billion users globally, including an estimated 450 million children.

15 years of online harm detection experience

2.2 billion safer user experiences globally

450 million safer user experiences covering children

The highest standard in Platform Trust and Safety Defense and Intelligence

We protect over $4.5 trillion of market capitalization by monitoring over a billion actors and their conversations to detect millions of harms, aimed at thousands of targets and prioritize hundreds of alerts within minutes, each and every 24 hours. Find out why Crisp is the highest standard in Platform Trust and Safety Defense and Intelligence today.