Source Grounding¶

→ Documentation index

Runnable example: examples/13_source_grounding.py Tests: tests/test_grounding.py, tests/test_grounding_integration.py, tests/test_grounding_source.py

LLMs sometimes invent plausible-looking values — a vendor name, an ID, a total — that are not present in the document. Mark a field as grounded and SAIDEX verifies, after Pydantic validation, that the extracted value actually appears in the source text. A value it cannot find is rejected and the existing retry loop asks the model to correct it, exactly like a Pydantic failure.

Grounding is driven by markers in your schema. By default the source text is taken transiently from the human-turn message(s) of the run (it is not retained — that is what capture_source_text controls). When the user message carries more than the document, pass grounding_source= to choose what counts as evidence — see Grounding against part of the prompt.

Two ways to mark a field¶

Both surfaces are equivalent — they attach the same Grounded check to the field's metadata — so pick whichever reads best. Grounded fields stay obvious right where they are declared.

from typing import Annotated
from pydantic import BaseModel, Field
from saidex import Grounded, GroundedField, GroundedStr

class Invoice(BaseModel):
    vendor: Annotated[str, Grounded()]                 # Annotated marker
    invoice_no: str = GroundedField(description="No.")  # Field helper
    alias: GroundedStr                                  # ready-made alias
    note: str = Field(description="Free summary")       # not grounded

Optional fields

Put the check on the field itself for optional values: GroundedStr | None or GroundedStr | MISSING is not grounded because Annotated metadata inside a union is not picked up. Use Annotated[str | None, Grounded()] or field_check(Grounded(), default=MISSING); see Fields that may be missing.

How it fits the retry loop¶

LLM produces output
        │
        ▼
  Pydantic validation
        │
        ├─ fails → field-level error fed back, retry ◄────────────┐
        │                                                          │
        └─ passes                                                  │
              │                                                    │
              ▼                                                    │
     grounding checks (per field)                                  │
              │                                                    │
              ├─ value not in text → fed back, retry ──────────────┤
              │                                                     │
              └─ all present                                        │
                    │                                               │
                    ▼                                               │
              validator(instance) (if any)                          │
                    │                                               │
                    ├─ rejects → fed back, retry ───────────────────┘
                    │
                    └─ accepts → returned ✅

A grounding failure:

consumes one validation retry (stats.primary_retries / validation_retries),
records a structured FieldIssue with category="grounding" and the field path (e.g. items -> 2 -> vendor),
triggers the fallback model when the primary exhausts its retries,
sets stats.failure_reason = "validation_exhausted" if all attempts fail.
keeps the rejected instance on stats.last_candidate — see External Validators → last_candidate.

Grounding runs before an external validator and only on structurally valid instances.

Matching semantics¶

The default mode="normalized" compares case-, whitespace- and diacritics-insensitively (so acme gmbh matches ACME GmbH, and Müller matches Muller). Non-breaking and thin spaces are treated as ordinary spaces.

Grounded()                              # normalized (default)
Grounded(mode="exact")                  # require the verbatim str(value)
Grounded(mode="fuzzy", threshold=0.85)  # approximate — tolerate OCR/typos

Grounding applies to scalar leaf fields (str, numbers, dates); container and nested-model fields are walked through to reach the scalars inside them.

Fuzzy matching (`mode="fuzzy"`)¶

Scanned or OCR'd documents introduce small character-level noise — Corporation read as Corporatlon, a word split as Corpor-\nation — that breaks an otherwise-correct match. mode="fuzzy" accepts a value when its best match in the text is similar enough:

vendor: Annotated[str, Grounded(mode="fuzzy", threshold=0.85)]

Similarity is 1 - edit_distance / len(value) against the best-matching substring of the source (an in-house Levenshtein variant — no extra dependency). Insertions, deletions and substitutions each cost one edit, so hyphenation and stray OCR characters are tolerated.

threshold (default 0.85) is length-sensitive: one edit in a 4-character value already drops the ratio to 0.75, while one edit in a 16-character value still scores 0.94. Lower the threshold for short values; raise it to demand a closer match.

Retry vs. flag (`on_mismatch`)¶

By default a grounding failure re-enters the retry loop so the model can correct the value. Set on_mismatch="flag" for advisory grounding: the extracted value is kept, no retry is consumed, and the mismatch is only recorded as a FieldIssue(category="grounding") for later inspection.

class Invoice(BaseModel):
    vendor: Annotated[str, Grounded()]                      # "retry" (default)
    summary: str = GroundedField(on_mismatch="flag")        # advisory only

Both modes record a FieldIssue; only "retry" produces feedback to the model and consumes an attempt. A schema can mix the two — retry failures drive the correction prompt, flag failures ride along as recorded issues.

Locale-aware values¶

A canonical value rarely matches its surface form in the document, and the difference is locale dependent:

Value	Surface forms matched
`1234.5`	`1.234,50` (de) · `1,234.50` (en) · `1 234,50` (fr) · `1'234.50` (ch)
`-1234.5`	`-1.234,50` · `(1.234,50)` · `1.234,50-`
`"2024-04-05"`	`05.04.2024` · `5. April 2024` · `April 5, 2024`
`2` (int 0–20)	`two` (en) · `zwei` (de) · `deux` (fr)
`True` / `False`	`yes`/`no` (en) · `ja`/`nein` (de) · `oui`/`non` (fr) · check glyphs `✓ ✔ ☑ x X` (True)

Tell grounding which locale to render for:

class Invoice(BaseModel):
    country: str                                          # e.g. "DE"
    total: float = GroundedField(locale_field="country")  # dynamic: from a sibling
    tax: float = GroundedField(locale="de")               # static: fixed hint

locale_field reads the hint from a sibling field at runtime (a CountryCodeStr or LanguageCodeStr works well); it can read a sibling because checks run over the fully validated instance.
locale is a fixed hint for when the language is known up front.
The hint accepts a country or language code. When it is missing or unknown, grounding falls back to a broad set of conventions, so a wrong hint never causes a false rejection.

Currency symbols need no special handling: the number 1234.5 rendered as 1.234,50 is found as a substring of €1.234,50 / EUR 1.234,50.

Number-words are localised for whole integers 0–20; larger or fractional values fall back to numeric forms only.

Short surface forms match loosely

Grounding uses a normalised substring test, so short lexical forms — "x", "ja", "no", a spelled-out "one" — can also match inside unrelated words ("Jacke", "nowhere"). This mirrors how a bare "2" is found inside "2024". It rarely matters because grounding is opt-in per field, but prefer on_mismatch="flag" (advisory) over "retry" when grounding very short boolean or number-word fields.

Grounding against part of the prompt¶

A common prompt layout puts reference data and the source document into the same user message:

Customer data (JSON): {"orders": [{"order_number": 32008}, {"order_number": 28870}]}

<mail>
Subject: Monitor broken
My monitor from order 32008 has dead pixels.
</mail>

With the default source (the whole human message), a hallucinated order_number="28870" passes: it appears in the customer data, just not in the mail. Pass grounding_source to check against the mail only:

class Concern(BaseModel):
    order_number: Annotated[str | None, Grounded()]  # must be stated in the mail

result, stats = await extract_data_from_text(
    llm, Concern, prompt,
    grounding_source=f"{mail.subject}\n{mail.body}",  # only the mail counts as evidence
)

Now 28870 is rejected and fed back to the model, while 32008 passes. Every entry point accepts grounding_source — extract_data*, extract_data_list*, extract_data_with_tools, run_extractor_agent and all *_sync wrappers.

Named sources¶

When different fields need different evidence, pass a mapping and name the source on the field:

class Concern(BaseModel):
    order_number: Annotated[str | None, Grounded(source="mail")]
    product_name: Annotated[str | None, Grounded(source="catalog")]
    note: Annotated[str | None, Grounded()]          # default source

await extract_data_from_text(
    llm, Concern, prompt,
    grounding_source={"mail": mail_text, "catalog": catalog_text},
)

GroundedField(source="mail") works the same way. The default source — used by Grounded() without a name and exposed as ctx.source_text — is resolved like this:

`grounding_source`	Default source	Named sources
`None` (default)	human-turn text	none
`"…"`	that string	none
`{"mail": …, …}`	the `"default"` key if present, else the human-turn text	every key

A rejection names the source, e.g. "… which does not appear in the source text 'mail'", so the model understands why a value it can see in the prompt is not accepted.

Configuration errors fail fast

A field that names a source the mapping does not provide raises a ValueError before any model call — also when grounding_source is a plain string or omitted, because named sources only resolve against a mapping. A schema with Grounded(source="mail") never silently falls back to the whole prompt. A grounding_source that is not a str, a mapping of str to str, or None raises a TypeError.

Empty sources

An explicit source is used as given, even when empty: with grounding_source="" or {"mail": ""} (an empty mail body) no value can be grounded, so every non-None grounded value is rejected and the model is pushed towards None. Without grounding_source, an empty human turn (e.g. image-only input) still skips grounding as before.

Privacy: like the human-turn text, grounding_source is added to the chain-run trace inputs (key "grounding_source") only when capture_source_text=True. stats.source_text and event.source_text keep holding the human-turn text.

Build your own check — the `FieldCheck` pattern¶

Grounding is the first built-in check on a reusable mechanism. Any FieldCheck subclass can be attached the same two ways — as an Annotated marker, or via the field_check helper — and SAIDEX runs it over the validated instance with an ExtractionContext (source text, the containing instance, the field path).

from typing import Annotated
from pydantic import BaseModel
from saidex import FieldCheck, ExtractionContext, field_check

class InCatalogue(FieldCheck):
    def check(self, value, ctx: ExtractionContext) -> str | None:
        if str(value) not in KNOWN_SKUS:
            return f"'{value}' is not a known SKU."
        return None

class Order(BaseModel):
    sku: Annotated[str, InCatalogue()]            # marker form
    alt_sku: str = field_check(InCatalogue())     # helper form

A check returns None to accept or an LLM-friendly message to reject; rejections re-enter the same retry loop as grounding. Return None when ctx.source_text is None so the check is skipped during plain model construction.

Named sources passed via grounding_source={...} are available as ctx.sources (a read-only mapping). Give your check a source: str attribute to pick one — the engine then also validates the name before the first model call, exactly as for Grounded(source=...):

from dataclasses import dataclass

@dataclass(frozen=True)
class MentionedIn(FieldCheck):
    source: str = "mail"

    def check(self, value, ctx: ExtractionContext) -> str | None:
        text = ctx.sources.get(self.source)
        if value is None or text is None:
            return None
        return None if str(value) in text else f"'{value}' is not in the {self.source}."

Availability¶

Grounding works across every extraction entry point and both extraction modes (TOOL_CALLING and JSON), including the batch variants (checked per item) and the agent loop (checked on the final answer), plus all the *_sync wrappers. Marking the fields is enough; grounding_source= is only needed to ground against part of the prompt or against named sources.

Out of scope (tracked separately)¶

Phone-number surface forms are tracked as a follow-up issue on the project tracker.