Failure reasons¶
When an extraction ends without a result, stats.success is False and
stats.failure_reason says why. The value is a saidex.FailureReason, a
StrEnum: it compares equal to its plain string, so both styles work.
from saidex import FailureReason
result, stats = await extract_data(llm, Invoice, messages)
if stats.failure_reason is FailureReason.LLM_ERROR:
schedule_retry_later()
elif stats.failure_reason == "validation_exhausted": # same thing, as a string
send_to_review_queue()
The reasons¶
| Value | Meaning | Typical cause | Schema-valid candidate possible? | Retry later? |
|---|---|---|---|---|
validation_exhausted |
The last attempt produced parseable output, but it failed schema validation, source grounding or the external validator, and no retries were left. | The model cannot satisfy a strict rule; the value is not in the source text; ambiguous field descriptions. | Yes. Grounding and external-validator rejections always follow a schema-valid instance. Check stats.last_candidate. |
Rarely with the same input and model. Improve the prompt or schema, use a stronger fallback model, or relax the validator. In the agent loop it is also reported when max_iterations ran out before a valid final answer (for example only helper-tool calls): raise max_iterations. |
parse_error |
The last attempt's output could not be parsed into arguments: malformed JSON or an invalid tool call. | Small or reasoning models, truncated output (max_tokens), provider quirks. |
Yes, if an earlier attempt produced one. Check stats.last_candidate. |
Sometimes. Truncation is often transient; consider a larger max_tokens or JSON mode. |
llm_error |
The model call failed after all network retries, or the provider rejected the tool binding. | Outage, authentication, a rate limit beyond the retry budget, unsupported tool-calling flags. stats.llm_error_kind says why — see below. |
Yes, if an earlier attempt produced one. Check stats.last_candidate. |
Yes for transient outages and rate limits. No if the tool binding was rejected even after automatic relaxation (or with auto_relax=False): fix tool_config. |
no_tool_call |
Agent loop with tool-calling final answers only: the model replied with plain text instead of calling a tool, and no retries were left. | The model ignores tools; the prompt does not ask for the final-answer tool. | Yes, if an earlier attempt produced one. Check stats.last_candidate. |
Rarely. Adjust the prompt, or use final_answer_mode=ExtractionMode.JSON. |
A reason describes the last attempt only
The retry loop keeps going after a rejection, so the run's reason is the
outcome of its last attempt. Attempt 1 may return a schema-valid
instance that the external validator rejects, and attempt 2 may then return
malformed JSON: the run ends with parse_error even though a schema-valid
candidate existed.
stats.last_candidate is the authoritative check — see
External Validators.
Why did the model call fail? — llm_error_kind¶
When failure_reason is llm_error, stats.llm_error_kind is a
saidex.ModelErrorKind, independent of the provider:
| Kind | Typical cause | Retried by default? |
|---|---|---|
rate_limit |
HTTP 429 | yes (rate-limit backoff) |
timeout |
request timed out | yes (provider/SDK exception types; see Retry & Fallback) |
connection |
provider unreachable | yes (provider/SDK exception types; see Retry & Fallback) |
server_error |
HTTP 5xx | yes |
authentication |
HTTP 401, bad API key | no |
permission_denied |
HTTP 403 | no |
invalid_request |
HTTP 400, or tool-binding flags the provider rejects | no |
not_found |
HTTP 404, unknown model name | no |
context_overflow |
input larger than the context window | no |
other |
anything else | no |
The kind is derived by saidex.classify_model_error(exc) from langchain-core's
standard model exceptions (langchain-core ≥ 1.6) and, for older versions, from the
openai / httpx exception types. A salvaged run (on_exhausted) has no kind.
Primary and fallback model¶
When a fallback model is configured and the primary fails, the fallback gets
fresh attempts. If both fail, the combined reason is the fallback's. It is
the primary's only when the fallback reported none, e.g. because it was
configured with zero attempts. The same rule applies to extract_data*, the
batch variants and the agent loop.
Salvaged runs¶
A run salvaged by on_exhausted reports success=True, failure_reason=None and salvaged=True; the original rejection stays in last_rejection.