Friday, February 14, 2025

Identifying Crime Related Data from Anonymous Social Media with AI

Identifying Crime Related Data from Anonymous Social Media with AI:

koatnM21xGkL_ws-dc9ns9fPjlrtoxH-wQ6BRm70

While traditional adverse media screening tools rely on mainstream sources, anonymous forums remain largely untapped for crime intelligence. I recently explored classifying crimes mentioned in the Swedish forum, Flashback Forum
, with a locally hosted LLM and called the script Signal-Sifter

  1. Web Scraping: Utilizing Go Colly to extract thread titles from crime discussion boards and storing them in an SQLite database.
  2. LLM Classification: Passing thread titles through a locally hosted LLM (Llama 3.2 3B Instruct via GPT4ALL
  3. ) to determine if a crime was mentioned and categorize it accordinglgy
  4. Filtering & Analysis: Storing the LLM’s responses in a crime database for structured analysis of crime trends.⁠

Process of building and analysing corpus of data

Why apply LLM to Online Forums?

Anonymous forums like 4Chan and Flashback are often analysed for political sentiment, but their role in crime discussions is relatively underutilised.

These platforms host raw, unfiltered discussions where users openly discuss ongoing criminal cases, share unreported incidents, and sometimes even reveal details before they appear in mainstream media.

Given the potential of these forums, I set out to explore whether they could serve as a useful alternative data source for crime analysis. ⁠

Using Signal Sifter, I built a corpus of data from crime-related discussions on a well-known Swedish forum—Flashback.⁠

Building a Crime Data Corpus with Signal Sifter

My goal was to apply Signal Sifter to a popular site with regular traffic and extensive discussions on crime in Sweden. After some research, I settled on Flashback Forum, which contains multiple boards dedicated to crime and court cases. These discussions offer a unique, crowdsourced view of crime trends and incidents.

https://preview.redd.it/o9z8mpadg3ie1.png?width=1854&format=png&auto=webp&s=b0a5042ec292f78fb8b6d7b993f30732cadef1ec

Flashback, like 4Chan, is structured with boards that host various discussion threads. Each thread consists of posts and replies, making it a rich dataset for text analysis. By leveraging web scraping and natural language processing (NLP), I aimed to identify crime mentions in these discussions.

Data Schema and Key Insights

Crime-Related Data:

  • Crime type
  • Mentioned locations
  • Mentioned dates

Metadata:

  • Number of replies and views (proxy for public interest)
  • Sentiment analysis

By ranking threads based on views and replies, I assumed that higher engagement correlated with discussions containing significant crime-related information.

Evaluating LLM Effectiveness for Crime Identification

Once I had a corpus of 66,000 threads, I processed them using Llama 3.2B Instruct, running locally to avoid token costs associated with cloud-based models. However, hardware limitations were a major bottleneck—parsing 3,700 thread titles on my 8GB RAM laptop took over eight hours.

I passed a few examples to the prompt and made it as hard as possible for the bot to misunderstand:

# Example of data and output: EXAMPLES = """ Example 1: "Barnadråp i Gävle" -> Infanticide. """"" # Prompt f"{EXAMPLES}\nDoes the following Swedish sentence contain a crime? Reply strictly with the identified crime or 'No crime' and nothing else: {prompt}'"

Despite the speed limitations, the model performed well in classifying crime mentions. Notably:

  • It excelled at identifying when no crime was mentioned, avoiding false positives.
  • I was surprised by its ability to understand context and not so surprised that the model struggles with benign prompts (prompts where a word has two meanings). For example, it correctly identifies Narcoterrorism from "Narcos" and "explode" but misunderstands that explode means arrest in this context.
  • The model struggled with specificity, often labelling violent crimes like sexual assault and physical assault as generic "Assault." This is likely because the prompt was too narrow.

Sample Output

Thread Title Identified Crime
24-åring knivskuren i Lund 11 mars Assault
Gruppvåldtäkt på 13-åring Group sexual assault
Kvinna rånad och dödad i Malmö Homicide
Stenkastning i Rinkeby mot polisen Arson
Bilbomb i centrala London Bomb threat
Vem är dörrvakten? No crime
Narkotikaliga på väg att sprängas i Västerås. Narcoterrorism

Takeaways and Future Work

This experiment demonstrated that online forums can provide valuable crime-related insights. Using LLMs to classify crime discussions is effective but resource-intensive. Future improvements could include:

  • Fine-tuning the model for better crime categorisation.
  • Exploring more efficient LLM hosting solutions.
  • Expanding data collection to include post content beyond just thread titles.

Sweden’s crime data challenges persist, but alternative sources like anonymous forums offer new opportunities for OSINT and risk analysis. By refining these methods, we can improve crime trend monitoring and enhance investigative research.

This work is part of an ongoing effort to explore unconventional data sources for crime intelligence. If you're interested in OSINT, adverse media analysis, or data-driven crime research, feel free to connect!

Let's connect!
https://albintouma.com/

submitted by /u/ReadOrdinary3421
[link] [comments]
on