One-sentence Summary

The University of Copenhagen proposes a hate speech detection framework for Twitter that grounds its annotation criteria in critical race theory, annotates a corpus of more than 16k tweets, and analyzes the impact of extra-linguistic features combined with character n-grams for identifying hate speech.

Key Contributions

• To address inconsistent manual definitions of hate speech, the paper contributes a critical race theory-based set of criteria and a publicly available corpus of more than 16,000 tweets annotated for racist and sexist content.
• The work analyzes character n-grams together with extra-linguistic demographic and geographic features, showing that character-level features provide a strong baseline while most non-linguistic features improve detection little, except for gender.
• The study also introduces a dictionary of the most indicative words from the annotated data and evaluates which feature combinations yield the best hate speech identification performance.

Introduction

Online hate speech is a major content moderation challenge for social media platforms, where systems must identify abusive posts while balancing free speech and public safety. Prior work has been limited by vague definitions, reliance on profanity lists or narrow lexical cues, and little analysis of demographic factors or effective linguistic features. The authors introduce a 16k tweet hate speech corpus and analyze detection features, finding that character-level n-grams provide a strong baseline while demographic information beyond gender adds little practical benefit.

Dataset

• The authors construct a Twitter hate-speech corpus from tweets collected over 2 months, retrieving 136,052 tweets in total and annotating 16,914 tweets.
• The annotated set is intentionally unbalanced to preserve realistic class distributions. It contains 3,383 sexist tweets from 613 users, 1,972 racist tweets from 9 users, and 11,559 tweets labeled as neither sexist nor racist from 614 users. The released schema is planned as tweet IDs plus labels on GitHub.
• Corpus collection starts with manual searches for common slurs and terms associated with religious, sexual, gender, and ethnic minorities. From these results, the authors identify frequent hate-speech terms, referenced entities such as the #MKR hashtag, and a small set of prolific users, then use the public Twitter search API to collect a larger corpus.
• Tweets not written in English are filtered out. This collection strategy is designed to include non-offensive tweets that contain potentially offensive words in neutral contexts, so the label is based on use rather than presence of sensitive terms alone.
• Annotation follows an explicit hate-speech decision list. A tweet is considered offensive if it uses sexist or racial slurs, attacks or seeks to silence a minority, criticizes a minority without a well-founded argument, promotes hate speech or violent crime, uses straw man arguments, distorts truth about minorities, shows support for problematic hashtags, negatively stereotypes minorities, defends xenophobia or sexism, or is ambiguous but tied to an offensive screen name and a qualifying topic.
• To reduce annotator bias, the authors have an outside annotator review their labels. The inter-annotator agreement is kappa = 0.84. Most disagreements occur in sexist annotations, and most reviewer changes move tweets to the non-hate class, often because of context differences.
• For text processing, the authors remove stop words except “not”, remove retweet markers such as “RT”, screen names, and punctuation. They also derive the most frequent terms for each class, showing strong class-specific lexical patterns, including terms tied to Islam and Judaism in racist tweets and terms linked to televised competition in sexist tweets.
• The authors add simple quantitative features, including average and total tweet length and the length of user descriptions, as potential discriminators between hate-speech and non-hate-speech tweets.
• Gender is not provided directly by Twitter, so the authors infer it from profile text, display name, user name, pronouns, honorifics, and gender-specific nouns. The hate-speech users are heavily skewed toward men, but many users cannot be identified, limiting the usefulness of gender as a feature.
• Geographic information is sparse because only about 2% of users disclose location. The authors infer location from explicit location tags, time zones, and location mentions in tweets or user metadata. When possible, locations are mapped to longitude and latitude, and named locations such as “Sydney” can be used as classification features. The authors note that the location feature negatively affects F1 score.
• The authors use the annotated corpus as labeled data for hate-speech classification experiments, relying on lexical features, tweet and profile lengths, and weak demographic or geographic metadata derived from the corpus.

Experiment

The experiments use logistic regression with 10-fold cross-validation and grid search to compare feature sets for the classification task. The results show that character n-grams, especially up to length four, outperform word n-grams and form the most effective representation, likely because they are less sparse. Adding gender yields the best overall configuration but only a slight and statistically nonsignificant improvement, while location and length features reduce performance. Analysis of feature weights indicates that influential n-grams largely align with frequent discriminative terms, including gender-related slurs and appropriated ordinary words.

The gender analysis finds that men dominate the identified gender distribution across hate speech categories, while women are very rare. Many users remain unidentified, especially in racist content, which limits the usefulness of gender as a feature. The similar gender patterns in sexist and non-hate tweets suggest that gender alone is unlikely to strongly distinguish those groups. Men account for most of the identified gender labels across all hate categories. Racist documents have the largest share of unidentified users and no identified women.

The lexical distributions differ strongly between sexism and racism tweets. The racism list is dominated by Islam-related terms, while the sexism list mixes terms such as sexist, women, and dataset-specific markers from My Kitchen Rules. The term not appears prominently in both classes. Islam-related tokens are central to the racism class. #mkr and related names reflect a sampling effect in the sexism data.

Average character length is highest for racist content and lowest for non-hate content, with sexist content in between. Racist content has the tightest variability, whereas non-hate content shows the widest distribution and includes shorter minimum examples. The reported length ranges for all classes reach similar high extremes. Racist content has the greatest mean length and the narrowest spread. Non-hate content has the lowest mean length and includes shorter minimum examples.

Character n-gram features produce clearly stronger classifier performance than word n-gram features. Adding gender to the character n-gram model gives the best results, although the improvement is small. Adding location or length reduces performance, and the result with gender and location is the one identified as statistically significant. Character n-grams outperform word n-grams by a large margin. Gender is the only additional feature that improves the score, while location and length are detrimental.

Short character n-grams are the most indicative features for hate-speech detection, with distinct patterns for sexism and racism. Sexism-related cues mainly come from fragments of words such as sexist and related references to women, while racism-related cues mainly come from fragments associated with Islam and negative reappropriation. Character n-grams improve on word n-grams, and adding gender gives the strongest result, whereas location and length features reduce usefulness. Character n-grams outperform word n-grams for detecting hate speech. Sexism-related features are centered on fragments of sexist and women-related terms. Racism-related features are centered on fragments associated with Islam and negative discourse. Gender can slightly improve performance, while location and length features are not helpful.

The experiments examine whether user gender, lexical content, tweet length, and metadata provide useful signals for distinguishing hate-speech categories. Gender analysis shows that identified users are predominantly men, with many unknowns, especially in racist content, and little separation from non-hate or sexist material. Lexical and length patterns differ by category, with racist tweets centered on Islam-related terms and having longer average length, while sexist tweets reflect dataset-specific cues and women-related vocabulary. Classifier results indicate that character n-grams are most effective, short character fragments reveal distinct sexism and racism cues, and gender adds a slight improvement while location and length are generally unhelpful.