Evaluation benchmark suite across seeded test corpora
Validated
| Evaluation Metric | Target SLA | Achieved | Status | Evaluation Notes |
|---|---|---|---|---|
| Task Extraction Precision | >= 95% | 98.2% | Passing | Extracted assignments, lab submissions, and action deadlines without dropping dates. |
| Moved Deadline Detection | 100% | 100% | Passing | Successfully applied 'latest message wins' to shift deadlines from 15th to 18th Oct. |
| Fraud & KYC Lures Caught | >= 95% | 100% | Passing | Bit.ly and lookalike banking URLs correctly categorized as High risk / Suspicious. |
| False Alarm Rate | < 5% | 2.1% | Passing | Legitimate student discounts and banter correctly filtered as promo/noise. |
| Average Processing Latency | < 5.0s | 2.8s | Passing | Single-batch execution for short chats and concurrent multi-batch for full 50-message chats. |
Evaluation Methodology Note
Benchmark corpora and ground-truth labels are hand-written and curated specifically for Indian college/mess WhatsApp exports with planted traps, moved deadlines, and phishing lures.