PRJ_002
002
Machine Learning · Computer Vision
Retail Fraud Detection System
Punch Digital Agency · San Francisco · Jan, 2022 – Jan, 2024
Overview

A production-grade deep learning pipeline designed to detect retail fraud in real time. The system handled image classification of transaction imagery and topic classification of unstructured merchant data, feeding into a combined risk scoring model deployed as a FastAPI microservice on AWS.

The Problem

Retail fraud detection at scale is not a binary problem — it's a distribution problem. The challenge was to distinguish genuine edge cases from adversarial patterns without overwhelming the fraud review team with false positives. The cost of a missed detection is asymmetric to the cost of a false alarm, but both erode trust.

Key Metrics
94%
Classification accuracy
<80ms
Inference latency
3×
Fraud catch rate improvement
2
Models in production
Process & Timeline
Phase 1
Data audit & labelling
Partnered with domain experts to build a labelled dataset. Discovered significant class imbalance — 94% legitimate transactions. Applied SMOTE and weighted loss functions to compensate.
Phase 2
Model architecture
Evaluated ResNet-50 vs EfficientNet for image streams. Built a parallel text classifier (BERT fine-tune) for merchant category signals. Ensembled outputs via a lightweight gradient boosted scorer.
Phase 3
Pipeline & deployment
Built FastAPI endpoints, containerised with Docker, deployed to AWS ECS. Set up CloudWatch monitoring for drift detection on incoming feature distributions.
Phase 4
Iteration & tuning
Model retraining scheduled weekly on fresh flagged samples. Implemented confidence thresholds — high-confidence scores auto-actioned, mid-range routed to human review.
Tech Stack
PyTorchTensorFlowFastAPIDockerAWS ECSCloudWatchSMOTEEfficientNetBERT
Critical Self-Evaluation
This was the project that taught me the difference between a model that performs and a model that ships. The preprocessing logic was too entangled with the model architecture — a mistake I'd never make again. I'd separate feature engineering into an independent service with its own versioning. The inference latency was a genuine achievement given the ensemble approach, but it came from hardware choices more than algorithmic elegance. I'd spend more time on the model monitoring layer — drift detection was reactive rather than proactive.