OraScan — Oral Disease Detection

The Problem

Over 3.5 billion people worldwide have some form of oral disease, but most never get diagnosed because seeing a specialist is too expensive or too far away. A small camera at a dental kiosk should be enough to flag problems early — if the AI behind it is reliable enough to trust with real patients.

The Challenge

Oral diseases (caries, gingivitis, periodontal disease, oral cancer) affect over 3.5 billion people globally yet go undiagnosed in low-resource settings due to the cost of specialist consultations. The challenge was to build a classification model accurate enough to be clinically useful, compact enough to run inference on kiosk hardware without a GPU, and robust enough to generalise across varied lighting conditions, camera angles, and skin tones. The model had to handle class imbalance across 11 categories — from common (caries, calculus) to rare (mucocele, hypodontia) — without sacrificing recall on clinically dangerous classes like oral cancer.

Architecture & System Design

Desktop scanning application captures oral images and sends them to inference engine for disease classification. Inference system processes images locally without cloud connectivity. Analysis results and images stored in cloud database and file storage system for historical tracking and specialist review.

A custom EfficientNet-B0 backbone (4.67M parameters) was fine-tuned on a combined DENTEX + SMART-OM dataset of 78,058 labelled oral images. Training used PyTorch with AMD GPU acceleration (DirectML on Windows). The data pipeline applies aggressive augmentation: random crops, HSV jitter, horizontal flip, and cutmix. Post-training, the model is exported to ONNX and quantised to INT8 for edge deployment. A FastAPI inference server wraps the ONNX runtime and exposes a REST endpoint consumed by the kiosk scanning application. The Go backend stores session results in PostgreSQL and uploads image artefacts to AWS S3.

Code Walkthrough

3-step walk-through of the production implementation — file paths and intent shown above each block.

1. 01

   Step 1 of 3

   Mouth-open detection as a capture trigger

   OraScan-Facemesh/mouth.py

   The scanner shouldn't shoot at random — it should fire the moment the patient's mouth is open enough for a clean intraoral view. MediaPipe's face mesh gives us the inner upper/lower lip landmarks as normalised (y) coordinates, and the per-frame pixel delta is a clean-enough signal to gate acquisition without any ML at all.

   pythonCopy

   Takeaway

   Two landmarks and a threshold replace an entire 'press to capture' UX. The CV runs in a tight loop; the rest of the pipeline stays frame-agnostic.

2. 02

   Step 2 of 3

   Bounding hardware calls with a timeout decorator

   OraScan_Automated_Scanning/motor_driver.py

   GPIO and servo drivers hang. Not crash — hang, indefinitely, when the bus misbehaves or a pin is stuck. An ordinary Python call into a blocked driver freezes the scanner UI and the operator has to hard-reboot. A one-line decorator submits every hardware call into a thread, waits up to N seconds, and returns None on timeout, so the UI always stays responsive.

   pythonCopy

   Takeaway

   Hardware is the one place you should never trust a library's own timeout handling — wrap every motor/sensor call in a bounded executor, treat None as 'try again', and the UI is freed from the physical layer.

3. 03

   Step 3 of 3

   Desktop ↔ cloud patient reconciliation

   OraScan_backend/sync_handler.go

   The desktop scanner stores every patient locally in MySQL so scans work offline. When the operator reconnects, each record is POSTed to this handler for reconciliation against the cloud store. The important design decision: we refuse to auto-create patients from sync data — sync is reconciliation, not a registration shortcut. Bypassing the signup flow would skip consent capture and identity checks.

   goCopy

   Takeaway

   Offline-first apps have their own IDs; the cloud has its own. Sync handlers merge on a stable external key, audit every reconciliation, and reject rather than invent records that didn't come through the proper signup flow.

Results

The final model achieves 94.7% test accuracy and 95.2% best validation accuracy across all 11 disease classes. ONNX INT8 quantisation reduces model size by ~70% while keeping accuracy drop under 1%. Inference latency on the kiosk CPU is under 200ms per frame. The model is integrated into both the automated scanning application (MediaPipe FaceMesh-triggered) and the manual scanning desktop interface.

Gallery & Demos

AI Analysis Dashboard

Real-time disease classification results showing confidence scores across 11 oral disease categories (caries, gingivitis, cancer, etc.).

Desktop Sync Workflow

Offline-first architecture: local MySQL stores patient scans, cloud sync reconciles on reconnect matching by email, enforcing signup flow for consent.

Kiosk Hardware Layout

MediaPipe FaceMesh detects mouth-open trigger, motorized camera captures intraoral image, ONNX INT8 inference runs on CPU in <200ms.

Click any image or video to expand · ← → keys navigate