ICFD-31k: A Large-Scale Dataset and Benchmark for Real-Time Conversational Fraud Detection

Rishi Ahuja, Kumar Prateek, Simranjit Singh

ICFD-31k is a benchmark of more than 31,000 Indian English and Hinglish conversations for real-time fraud detection. Alongside conversation-level labels, it provides chunk-level streaming annotations and explanatory rationales, enabling systems to be evaluated before a conversation has ended.

  • IJCAI–AIJ grant

IJCAI–ECAI 2026 · AI for Social Good Special Track

Why this dataset

Most public conversational fraud datasets are small, concentrated in non-Indian settings, or labelled only at the conversation level. They can support retrospective classification, but they are not designed to study when a system can first identify a scam as it unfolds.

ICFD-31k addresses this gap by supporting both full-conversation classification and streaming evaluation across diverse fraud scenarios.

What is in the release

The dataset contains more than 31,000 scenario-grounded synthetic transcripts spanning ten fraud categories, including financial impersonation, job scams, and other common forms of conversational fraud.

Each transcript includes a final verdict, chunk-level streaming labels, and an explanatory rationale. Human evaluation reports κ = 0.534, indicating moderate inter-annotator agreement.

Models and evaluation

We fine-tune two RoBERTa baselines. M1 evaluates complete conversations, while M2 makes incremental predictions as new chunks arrive.

On the in-domain evaluation, M1 reaches 99.40% F1 and retains 92.97% F1 on held-out fraud types. These results provide reference baselines for the dataset; they should not be interpreted as evidence that real-time conversational fraud detection is solved.

I presented this work at IJCAI–ECAI 2026 in Bremen.

More research