AI / ML · Working application with golden tests
ImageMismatch
Detect when a caption does not match its image, with a tested inference API.
In plain words. Upload a photo and a few captions and it tells you how well each caption matches the picture. It checks meaning only. It cannot tell you whether a caption is true.
The problem
Images and text can disagree, but ML demos often drift between notebook and server through different preprocessing.
What it does
Upload an image and up to five captions. The service returns a match decision, probability, raw similarity and a heatmap per caption, and handles printed text in the image through OCR.
Key decisions
- The API loads the same mismatch_core.py used in training and checks golden predictions within a numeric tolerance.
- Readiness is separate from liveness, with model warm-up before traffic.
- Inference is serialised and thread-limited for predictable latency on small CPUs.
Limits and status
- It measures semantic alignment, not truth. It cannot judge date, source or context.
- CLIP is weak on counting, colour and spatial relations; English only.
- Local run on 4 CPU cores / 3 GB: about 14 s startup, 1.36 s median for one caption, 3.73 s p95 for five captions plus OCR.