Citations you can check: verifying every source twice, while the answer streams

A citation can fail in three ways. It can point at nothing. It can point at a source that has since been retired. Or it can point at a real, current source that simply does not say what the answer claims. A medical assistant has to catch all three, and it has to do it without making a clinician stare at a spinner.

This is how we built that for Hippocrates. The code below is a simplified sketch of the pattern, not the production source.

Mark sources inline, by id

The model cites as it writes, by putting a source’s id in square brackets after the sentence it supports. Ids rather than numbers mean every marker maps to exactly one row in the knowledge base, with no renumbering to get wrong.

After the answer finishes, we split it into sentences and collect, for every source id, the sentences that cite it. Those are the claims each source is on the hook for.

MARKER = re.compile(r"\[([0-9a-f-]{36})\]")

def claims_by_source(answer: str) -> dict[str, list[str]]:
    claims: dict[str, list[str]] = {}
    for sentence in split_sentences(answer):
        for source_id in MARKER.findall(sentence):
            claims.setdefault(source_id, []).append(sentence.strip())
    return claims

Check one: does the source exist, and is it live?

One query loads every cited source at once. Each citation gets a status: confirmed if the source exists and is active, flagged if it is missing or retired.

The important part is what we do not do: we never drop a bad citation silently. Every attempt, flagged or not, is kept with the answer, so the team can see when and how often the model reaches for a source it should not have.

Never drop a citation silently. Tag it, keep it, and show only what passed.

Check two: does the excerpt actually say that?

A source that exists can still be cited for something it never said. So the confirmed citations go through a second check: for each source, does its excerpt support at least one of the claims attached to it? Paraphrase is fine. Invention is not.

We batch every source into one structured model call per answer, rather than one call per citation, and ask for a plain true or false per source. A source that fails is marked ungrounded and hidden from the reader, but kept in the record.

class GroundingCheck(BaseModel):
    source_id: str
    grounded: bool  # supports at least one claim; contradicting all of them is false

class Grounding(BaseModel):
    checks: list[GroundingCheck]

result = await asyncio.wait_for(
    model.with_structured_output(Grounding).ainvoke(prompt),
    timeout=12,
)

The check runs against a hard time limit. If it times out or errors, the confirmed citations stay visible. A missing check is not evidence of a bad source, and it should never erase one that passed the first test.

Stream first, verify in parallel

None of this should slow the answer down. The text streams as the model writes it. The moment the answer finishes, verification starts as a background task and the interface shows “Checking citations…”. The final event carries only the citations that passed both checks.

Hold back the last few characters

Streaming has its own problem: once text is on the screen, it is on the screen. So every chunk passes through a small guard before release. The guard always holds back a short tail, long enough that any pattern the safety check looks for can be caught before any part of it reaches the reader.

The same tail keeps half-written citation markers off the screen. A clinician never sees a stray opening bracket and half an id flicker in, then vanish.

class StreamGuard:
    def __init__(self, hold: int):
        self.text, self.released, self.hold = "", 0, hold

    def feed(self, delta: str) -> str:
        self.text += delta
        if unsafe(self.text):
            return ""  # stop releasing; the caller retracts what is out
        end = max(self.released, len(self.text) - self.hold)
        chunk = strip_partial_marker(self.text[self.released:end])
        self.released += len(chunk)
        return chunk

If a check does trip after some text is already out, the server sends a reset event and the interface clears it. The same reset fires when the model decides mid-answer to look something up: the half-sentence it wrote before calling a tool is withdrawn, and the real answer streams in after.

What the reader sees

• Thinking: what the assistant is doing, such as searching the knowledge base
• Sources: previews of what it found, before the answer arrives
• Text: the answer, a short step behind the model
• Reset: a retraction, when something has to be withdrawn
• End: the final answer with only verified citations attached

The result is an assistant that is slower to sound sure and quicker to show its work, which is the trade you want when the reader is a clinician.