Multi-Modal Evidence Review
Hackathon / Multi-Modal AI | 2026
mustakim@portfolio: ~/projects/evidence-review
mustakim@portfolio:~$evidence-review verify --claim CLM-4471 --lang auto
[perception] per-image · claim-blind → 6 findings
[forensic] EXIF scan → 2 edited images · escalate, not reject
[policy] rules engine → 3 flags · 1 escalation
[adjudicate] frontier model → structured verdict
✓ verdict: REVIEW · confidence 0.71 · lang: hinglish
Overview
Verify the claim, don't just trust the image.
Format
Timed hackathon · HackerRank Orchestrate · ~20-hour build
A damage-claim verification system built under a tight time box. The task: decide whether a set of images actually supports an insurance damage claim. Instead of one monolithic vision-language call, I decomposed it into claim-blind per-image perception feeding a policy engine and an adjudication stage, so each part could be reasoned about and tuned independently.
Stack
Vision-Language Models · EXIF forensics · Ollama (local) · frontier model (final) · HackerRank Orchestrate
Model-agnostic by design: all development ran against local Ollama models, with a frontier model reserved only for the final scored output. The pipeline handled multilingual claims (English, Hindi, Hinglish, Spanish) and shipped with prompt-injection defenses on the untrusted claim text.
Approach
Decompose the judgment, then make it defensible.
Claim-blind perception.
Each image is described on its own, without knowing what the claim asserts, so the perception stage can't be led toward the answer. Those neutral findings feed a separate policy engine that applies the adjudication rules.
A deterministic forensic layer.
An
EXIF analysis pass escalates scrutiny on edited images rather than auto-rejecting them. The data showed edit signals on both fraudulent and legitimate claims, so “edited” is treated as a reason to look closer, not a verdict.Model-agnostic, local-first.
The whole system is provider-agnostic. Local Ollama models drove development and iteration; a frontier model was swapped in only for the final scored run, keeping the loop fast and cheap.
Multilingual + injection-hardened.
Claims arrive in English, Hindi, Hinglish, and Spanish. The claim text is untrusted input, so the prompts are hardened against injection attempts that try to talk the adjudicator into approving.
Retrospective
What the decomposition bought me.
Structure beat a single big prompt.
Splitting perception, policy, and adjudication meant each stage was debuggable on its own under time pressure, and the claim-blind boundary removed a whole class of the model talking itself into the claim.
The forensic insight was the differentiator.
Treating an edit signal as an escalation rather than an automatic rejection is the decision I'd keep. It came straight from looking at the data instead of assuming edited means fraudulent.