Smart Cart

Problem Context

This project focused on building a computer vision–powered smart cart that automatically identifies products during checkout and links them to consumer spending analysis. The challenge was to train a reliable object detection model on a small, custom dataset, while ensuring strong real-time performance suitable for retail deployment.

Data Collection

I built a dataset across five grocery categories (e.g., fruits, snacks, beverages, packaged goods, and household items).

• ~3,200 images collected via manual sampling and web scraping

• Annotated using LabelImg in Pascal VOC format for YOLO compatibility

Data Preparation

Images were cleaned and augmented to boost robustness.

• Preprocessing: cropping, resizing (416×416), and normalization

• Augmentations: random rotation, brightness shifts, horizontal/vertical flips

• Dataset split: 70% training, 20% validation, 10% testing

Baseline

Before moving to detection, I validated that categories were separable using a lightweight CNN classifier.

• Achieved baseline accuracy of ~72% classification across categories

• Proved dataset feasibility despite limited samples

Modeling

I trained a YOLOv4 custom object detection model on TensorFlow with GPU acceleration.

• Training: 100 epochs, batch size 16, learning rate 0.001 with cosine decay

• Achieved 84.6% detection accuracy (mAP@0.5)

• Average IoU = 0.71 across categories

• Per-class precision ranged 82–87%; recall ranged 78–85%

• Inference speed: ~25 FPS on Nvidia RTX GPU (real-time capable)

Evaluation

I benchmarked performance using industry-standard detection metrics.

• Metrics: mAP@0.5 = 84.6%, IoU = 0.71, Precision = 0.84, Recall = 0.81, F1 = 0.82

• Confusion matrices revealed minor overlaps between visually similar packaged items

• Validation loss plateaued after ~60 epochs, indicating early stopping potential

Refinement

To move toward deployment, I integrated the model with a retail analytics dashboard.

• Detected items were logged into spending categories

• Dashboard visualized frequency of purchases by category, spending share, and time-of-day patterns

• Designed with scalability: additional product classes could be added with incremental training

Conclusion

The Smart Cart model reached 84.6% accuracy and was able to categorize consumer purchases in real time. Beyond technical validation, the dashboard demonstrated how object detection can support shopping pattern analysis and provide insights into customer behavior.