~/projects/document-claim-analysis / Case study
Document-grounded claim analysis
Every flagged claim should lead back to evidence.
I'm building a document analysis platform to help reviewers find unsupported claims, contradictions, and facts that may be out of date. It works with local PDF and Word documents and keeps the source passages alongside the review.
Proposals are the first use case: long documents that mix reused text, commitments, dates, and evidence from different teams.
- My work
- Architecture & implementation
- Document formats
- PDF & DOCX
- Project status
- In development
Application screenshots from September 2026, using the intentionally flawed Pinehaven demo document. They show the review interface, not a measured detection rate.
01 / The problem
Good wording can hide bad information.
An old client name, an unsupported promise, or two conflicting delivery dates can survive several rounds of editing. Finding those issues means checking the document against itself and against its references.
I wanted a reviewer to be able to follow an issue back to the exact wording, understand the concern, and decide what to change.
02 / The review
Show what was flagged, and why.
The workspace puts the findings list, document, and explanation together. A reviewer can inspect a passage, see why it was flagged, and dismiss an issue that does not apply.
The analysis combines concrete policy checks with local model work on claims and relationships between statements. Unsupported assertions, conflicting claims, and stale information are the review targets; each needs evidence and context before it becomes a useful finding.
A contradiction also needs more than two different numbers. A total and its components can agree. So can a completed count and a plan for the next item. I added narrow, reproducible checks for cases like these to reduce false alarms before model adjudication.
03 / Under the hood
The source stays authoritative.
PDF and DOCX ingestion preserves the original file, organizes its sections, and gives extracted text a stable source location. A reviewer confirms the section structure before analysis continues.
Models make bounded judgments about that material. The application validates and attaches the supporting source text itself, so a model's rewritten claim cannot silently replace the evidence.
Document versions, analysis runs, findings, and reviewer decisions have separate records. That makes it possible to trace a finding back to the file and run that produced it, and to revisit the work without losing its history.
04 / What I'm testing
A useful warning has to earn its place.
The current work includes document ingestion, section review, source-linked findings, and local analysis. I'm testing how well the semantic checks carry valid concerns through to the reviewer, while keeping false positives down.
A completed processing job does not automatically mean a complete review. Missing analysis, failed jobs, and incomplete coverage need to stay visible. Broad detection quality is still being evaluated.