Exception Resolution¶

Purpose¶

Exception clusters are created but never resolved today — decision_status is written once as "pending" and nothing updates it. This doc defines the resolve endpoint, its payload, and the file structure behind it.

Structure¶

exception_service/
├── utils.py                sample_transaction_date / window_from_clusters / index_known_row_uids —
│                            reusable across banking/, commercial/, and resolution/ (see below)
├── banking/
├── commercial/
└── resolution/            ← sibling to both, not nested inside either
    ├── schema.py            exception_resolution table adapter
    ├── models.py            ResolveRequest / ResolveResponse
    ├── repository.py        exception_resolution audit table + bulk cluster reads/updates
    ├── service.py           resolve(request) — two independent lookups (below), OUTCOME_HANDLERS registry,
    │                        the bulk generic path (everything except not_found_gl/not_found_bank)
    └── outcomes/
        └── not_found/            special: partner-wide bulk MERGE re-match, no classification rerun.
            ├── __init__.py           thin re-export: `from .handler import resolve`, so callers keep
            │                         writing `not_found.resolve` unchanged even though this is a package
            ├── handler.py            orchestration — resolve()'s dispatcher + every vertical-agnostic
            │                         helper (bulk status flip, row_uid indexing, date-window math, audit
            │                         bookkeeping); branches to a vertical-specific function by `request.domain`
            └── matching/             bulk-MERGE matching primitives, one module PER VERTICAL — this
                │                     subfolder is what grows as verticals are added, handler.py doesn't
                └── nip.py            NIP's own MERGE tiers (reference_no/settlement_session_id),
                                       mirroring the standard pipeline's shape. A future matching/atm.py
                                       would sit right next to it, its own join keys, never touching this one

routes/exceptions.py         POST /exceptions/resolve — the HTTP layer, same split as
                              routes/reconciliation.py (route here, logic in the service above)

Why sibling, not nested in banking/ or commercial/: one endpoint has to reach across both domains' tables. Nesting it in either domain would force the other domain to reach into a sibling's package — backwards.

Why not_found/ is a package, not a flat file: separates two concerns that grow independently — orchestration (handler.py, one file, shared across every vertical since it's the same dispatch-and-audit logic regardless of which vertical actually matches) from matching mechanics (matching/, one file per vertical, since NIP's join keys — reference_no/settlement_session_id — share nothing with ATM's — stan/auth_id/terminal_id). Adding ATM support means adding matching/atm.py and one elif branch in handler.py's resolve() — never editing matching/nip.py, and never splitting handler.py itself into per-vertical files (confirmed design choice: this package is the one home for the whole not_found_gl/not_found_bank family, not a proliferation of top-level modules).

There is no outcomes/generic.py — the default (non-not_found) path is inlined directly in service.py as a bulk UPDATE + bulk audit INSERT, not a per-row function. See "A resolve call is a BULK action" below for why.

exception_service/utils.py — genuinely shared helpers live at this top level, not inside resolution/, because they operate on cluster-row fields present on both banking_exception_cluster and commercial_exception_cluster (sample_gl_transaction_date/sample_bank_transaction_date, gl_record_ids/bank_record_ids) and aren't specific to one vertical, one cluster_type family, or even to resolution — sample_transaction_date's fallback idiom was already duplicated verbatim in both banking/backend_notifier.py and commercial/backend_notifier.py before this existed. handler.py uses window_from_clusters/index_known_row_uids for both NIP's and banking's not_found_gl/not_found_bank handlers — the same functions, no vertical-specific fork — index_known_row_uids takes its cluster_type -> field mapping as a parameter for exactly this reason, so it carries no not_found-specific assumptions. deterministic_ingestion_session_id (also here) is the third shared helper — originally NIP-only, promoted here once banking became a second real consumer of the same retry-safety trick.

A resolve call is a BULK action by default — optionally narrowed to specific clusters¶

decision applies to every cluster matching an exact scope: partner_id + tenant_id + domain + cluster_type (+ the right decision_status band — pending for every decision except reopened, which selects the opposite: already-resolved rows, to reopen them). There is no session_id, and no branch_id, anywhere in this feature — a resolve call acts across every branch under the given partner/tenant (see "Branch scoping" below for the one place branch still matters internally).

This fell out of two decisions made together:

Resolution scope had already become partner-wide (see "Scope" below) — matching no longer needs, or trusts, a specific session_id.
Once the search itself is partner-wide, requiring the caller to pre-know a specific cluster_id just to say "retry matching everything pending for this partner" (or "reject every pending X exception for this partner") is unnecessary indirection. cluster_type is the selector — same role domain already plays for disambiguating verticals sharing one physical table.

A resolve call with cluster_type="missing_identifier", decision="rejected" and no cluster_ids rejects every pending missing_identifier cluster in that partner/tenant/branch/domain in one call — not one specific exception a human picked.

Targeted resolution (single-unit or small-batch)¶

not_found's bulk model works because there's no per-cluster judgment involved — it's mechanical ("does this new document contain a match, yes/no"). Other cluster_types — possible_bank_fee, date_difference, etc. — don't fit that shape: resolving one of these plausibly requires gathering intelligence first (narration analysis, historical pattern lookup, maybe an LLM call) before a decision is even possible. That's inherently a "pick specific ones, look closer, then decide" workflow, not "blindly approve every pending bank_fee for this partner" — the bulk-by-cluster_type selector alone is too coarse for it.

ResolveRequest.cluster_ids: Optional[List[str]] narrows the bulk scope to exactly those cluster_ids instead of every pending cluster of cluster_type. Implemented in get_clusters_for_bulk_resolve (resolution/repository.py) as AND cluster_id IN UNNEST(@cluster_ids) appended to the same query — still ONE SELECT regardless of whether the list has 1 id or 500, so the constant-BigQuery-calls principle (see below) holds; only the selector changes from "all of type X" to "these specific ones." Omitting it (or passing an empty list — treated identically, since an accidentally-empty list is far more likely to be a caller bug than a deliberate "resolve zero clusters" request) preserves the original bulk-by-cluster_type behavior exactly.

Consulted by the generic bulk path (service.py) and _bulk_status_flip (non-reconcile decisions — including decision="approved" — for any cluster_type, including not_found_gl/not_found_bank). Not consulted by not_found's own reconcile-and-matched pipeline (outcomes/not_found/handler.py's _resolve_nip_match/_resolve_banking_match), which fetches its pending row set via get_pending_clusters_by_type — partner-wide by design, never scoped to individual clusters. ResolveRequest._validate_cluster_ids_scope fails loudly (a 422) if cluster_ids is supplied together with decision="reconcile" and cluster_type in {not_found_gl, not_found_bank}, rather than silently ignoring it there.

Intelligence gathering stays OUT of resolve() itself. Investigation (fetching context, running an LLM, checking history) is a read concern with different failure semantics than resolve's deliberate "fail loudly, direct action" contract (see "Logging & failure behavior" below) — it's a separate step (existing tooling, or a new read-only endpoint) that produces the note/decision a caller then feeds into resolve() via cluster_ids + decision, not something resolve() does inline. That separate investigation step itself is not built — only the cluster_ids targeting mechanism is.

Not implemented yet — recorded here so the shape is settled before it's built.

Every code path costs a CONSTANT number of BigQuery calls, never one-per-cluster¶

This is load-bearing, not a nice-to-have. An earlier version of not_found/handler.py looped over every pending cluster one at a time, doing several sequential BigQuery round-trips per cluster. Tested against a real session with 4,212 pending clusters, that produced over 10,000 sequential BigQuery calls for a single resolve request — confirmed still running, not crashed, over an hour later, with zero progress visible in BigQuery. BigQuery jobs carry real per-query scheduling overhead (roughly 1–3 seconds) regardless of how little data they touch, so a fully sequential per-row loop cannot be made fast by any amount of retrying — it's O(N) BigQuery jobs by construction.

The fix models this feature directly on how the standard reconciliation pipeline (orchestrator.py's run_nip_reconciliation) already processes an unbounded number of rows in a small, constant number of BigQuery calls: land the file (one load job), run a couple of bulk MERGE passes (mirroring matching/session_id_match.py's shape — identify-and-write-back in one DML statement, no Python-side row list), then one bulk status update and one bulk audit write. 10 pending clusters or 10,000 costs exactly the same number of round trips. See outcomes/not_found/matching/nip.py and the approved-path breakdown below for the concrete call count.

Domain values¶

domain is the actual vertical — "mfb" | "nip" | "atm" | ... — the same ServiceType vocabulary CommercialReconciliationRouter already routes incoming requests on. Not a generic "commercial" bucket with a separate field disambiguating verticals underneath it — that was tried and reverted, since it just meant keeping two fields in sync for information domain alone can already carry.

Multiple domain values can map to the same physical table — "nip" and "atm" both resolve to commercial_exception_cluster (repository.py's _TABLE_BY_DOMAIN), disambiguated by the domain column value itself, not by which table they're in. Every query filters on the domain value explicitly, not just on which table it picked, so two verticals sharing a table never see each other's rows. Only "nip" is actually wired today — "mfb" and "atm" raise UnsupportedDomainError on purpose, rather than guessing at a shape that doesn't exist yet.

domain is computed in code at the point a cluster gets classified — CommercialSessionContext.domain="nip" is set explicitly at the NIP hook-in (services/commercial/nip_reconciliation/reconciliation.py), not derived from anything stored. A future ATM exception module would do the same with domain="atm" at its own hook-in point.

Routing — two independent lookups, not one branch¶

domain picks the repository/table (and, for verticals sharing one, the row-level scope too).
(domain, cluster_type), both straight off the request, picks the outcome handler:

OUTCOME_HANDLERS = {
    ("nip", "not_found_gl"):   not_found.resolve,
    ("nip", "not_found_bank"): not_found.resolve,
    # everything else, including a future ("atm", "not_found_gl"), → the bulk
    # generic path inlined in service.py, until ATM gets its own registered
    # handler — it must never silently reuse NIP's.
}

Keying on the pair, not cluster_type alone, is deliberate: if a future ATM type ever collided with one of NIP's names, keying on cluster_type alone would route it into NIP's handler — which hardcodes calls into NIP's own ETL and matching functions — silently. Keying on (domain, cluster_type) makes that impossible; an unregistered pair always falls through to the bulk generic path, never to the wrong vertical's special-cased one.

not_found.resolve is called once per request with the whole request object (not once per row) — it needs to manage its own row set internally (both not_found_gl and not_found_bank pending clusters together, to catch incidental matches). The default path is also one bulk call, not a loop: service.py fetches the matching row set (get_clusters_for_bulk_resolve) and issues one bulk_update_decision_status + one bulk_write_resolution_rows covering every row in it.

Payload¶

{
  "partner_id": "partner-123",
  "tenant_id": "tenant-123",
  "customer_type": "commercial_bank",
  "service": "nip",
  "cluster_type": "not_found_gl",
  "decision": "reconcile",
  "note": "Confirmed after review",
  "metadata": {},
  "gl_file_url": [],
  "bank_file_url": []
}

Banking payloads use "customer_type": "mfb" and omit service entirely (not meaningful — banking has no vertical split).

Required: partner_id, tenant_id, customer_type (ReconciliationCustomerTypeEnum — same enum routes/reconciliation.py's ReconcileRequest already takes), service (ServiceType — same enum CommercialReconciliationRouter already routes on; required when customer_type="commercial_bank", ignored for "mfb"), cluster_type (the selector — see above, no longer optional/client-asserted-only; validated against a ClusterType enum, the union of every cluster_type any rule engine actually produces — a typo now fails as a 422 instead of silently matching zero rows), decision (DecisionType enum — "approved" | "rejected" | "escalated" | "reopened" | "reconcile" — an unknown value is a 422). "reconcile" is distinct from the other four: it's the one value that means "take the newly supplied information (gl_file_url/bank_file_url) and perform an action (ingest + re-match)" — see the outcomes/not_found/ section below. The other four are plain status decisions on existing data, with no re-matching, "approved" included.

There is no branch_id field — deliberately, see "Branch scoping" below. There is no actor_id/actor_type field — deliberately. Neither was ever verified against a real identity (see the removed "unverified" limitation this section used to note), and actor_type had exactly one real use: deriving whether an audit row's resolution_method was "manual" or "auto". That's now determined structurally instead — every external caller (this route, the Pub/Sub trigger) always produces "manual"; only exception_resolution/auto_resolve.py's own internal calls ever produce "auto", via an explicit resolution_method argument on ResolutionService.resolve() that no external payload can set. Optional: resolution_request_id (idempotency key for the Pub/Sub resolve trigger — mandatory on that path specifically, backfilled with a fresh UUID on the legacy HTTP route; see "Idempotency" below), note, metadata, gl_file_url/bank_file_url (only meaningful for not_found_gl/not_found_bank + decision="reconcile"), cluster_ids (narrows the bulk scope to specific clusters — see "Targeted resolution" above; not accepted together with decision="reconcile" for not_found_gl/not_found_bank), matching_method/amount_tolerance/recon_time_search_forward/recon_time_search_backward (banking/mfb-only — tune the core matcher's customer_settings for a given call; see docs/banking_exception_resolution.md, meaningless for NIP).

Branch scoping¶

There is no branch_id anywhere in ResolveRequest — a resolve call acts across every branch under the given partner_id/tenant_id, not one. This is inert for NIP/commercial (which has no branch concept at all — see schema.py's own comment on commercial_exception_cluster) and for resolution_intelligence's learned-pattern key (patterns now pool across every branch under a partner/tenant, rather than staying separate per branch).

It is NOT inert for banking's (mfb) not_found_gl/not_found_bank matching, though: reconciliation-engine's own server-side candidate-fetch for banking matching is genuinely branch-scoped internally (a separate microservice, out of scope for this contract), and the document-ingestion ETL tags EVERY row of an uploaded file with one branch_id, uniformly — there is no per-row derivation from file content. A single uploaded document therefore only ever belongs to ONE branch's worth of transactions; re-ingesting it once per branch (to cover a partner-wide, multi-branch pending set) would insert duplicate copies of the same rows under multiple, mostly wrong, branch tags — a real data-correctness bug, not just an efficiency one.

So outcomes/not_found/handler.py's _resolve_banking_match runs exactly ONE ingest → match → recheck pass per resolve call, scoped to whichever branch has the MOST pending clusters among the fetched (partner+tenant-wide) set — a resolve call's uploaded document realistically belongs to that branch's own backlog already, since that's how the pending clusters themselves originally got created. Pending clusters belonging to any OTHER branch are left untouched this pass; _finalize still sees the full pending set, so they land in the same "still pending" bucket a genuine no-match already does, and get picked up on a later resolve call once their own document is uploaded.

There is no session_id field. See "A resolve call is a BULK action" above.

customer_type/service, not a single domain field, on the wire. This matches the shape every other caller in this platform already uses — routes/reconciliation.py's request models, which CommercialReconciliationRouter reads directly (data.service == ServiceType.NIP) — rather than inventing a resolution-specific shape. Internally, ResolveRequest.domain is a computed property, not a stored field: customer_type="mfb" → domain="mfb"; customer_type="commercial_bank" → domain=service.value ("nip"/"atm"/...). Everything downstream of the request (BigQuery's domain column, OUTCOME_HANDLERS, _TABLE_BY_DOMAIN) still keys on that single derived value, completely unchanged — this is a payload-shape adjustment, not a reversal of the earlier decision to keep domain as the one stored/routed-on value (see "Domain values" above). Only "mfb"/"commercial_bank" are accepted today — "payment_provider"/"lender"/"fund_management" fail loudly with a 422 at request-parse time, same "not wired yet" posture "atm" already has one level down (accepted by the model, still not registered in _TABLE_BY_DOMAIN).

No resolution_method field either — it's backend-stamped, not client-supplied. resolution_method ("manual" | "auto", written into exception_resolution audit rows) records how a cluster actually got resolved: "manual" for the cluster_type the request explicitly targeted, "auto" for anything only resolved as an incidental side effect of not_found's partner-wide sweep. A client can't honestly self-report this — it could claim "auto" for something it triggered directly, or the reverse — so every place that writes an audit row stamps this itself rather than trusting a request field for it.

The default (generic) path — inlined in `service.py`¶

get_clusters_for_bulk_resolve(domain, partner_id, tenant_id, cluster_type, decision) fetches every cluster in scope (pending, or non-pending for decision="reopened") — no LIMIT, since a bulk resolve action's whole point is resolving all of them, not a subset — one SELECT. Then one bulk_update_decision_status UPDATE and one bulk_write_resolution_rows INSERT cover every row in that set. Three BigQuery calls total, regardless of set size. No side effects, no pipeline calls. Covers every cluster_type except not_found_gl/not_found_bank. (A WARNING log fires above _LARGE_FETCH_WARNING_THRESHOLD — 5,000 — purely as a "this backlog is unusually large, worth a look" signal; it never drops anything — see Open Items.)

Recurring-pattern intelligence (`resolution_intelligence`)¶

A cluster_type can learn that a given partner/tenant consistently deviates by roughly the same amount — e.g. "this counterparty's settlement always posts ~10 days late," or "this partner's fee clusters are always off by about the same few dollars" — and auto-resolve future matching clusters without a human. Two cluster_types are wired today, both through the exact same generic mechanism: date_difference (the original) and amount_mismatch (banking-only, since only banking/rules/amount_mismatch.py produces that cluster_type today).

Registration — service.py's INTELLIGENCE_SOURCE dict is the single switch: {ClusterType: {"intelligence_type": ..., "value_field": ..., "kind": "numeric"}}. value_field names the already-existing cluster-row column carrying the raw deviation value (date_delta_days, amount_delta — both unsigned magnitudes, populated by the classification rule itself, nothing resolution-specific). intelligence_type is the pattern's own identifier in the resolution_intelligence table. Adding a cluster_type here is the ONLY step needed — every piece below is generic over this dict, nothing branches on a specific cluster_type by name.

Learn path (write) — a human resolve call with decision="approved" and is_recurring=true on a registered cluster_type (ResolveRequest.is_recurring, ignored for unregistered types) tags that cluster's own raw value_field onto its audit row's metadata.learned_value, then service.py's _bulk_resolve calls repository.py's refresh_resolution_intelligence once per request (never once per cluster). That function re-aggregates from EVERY historical exception_resolution row ever tagged is_recurring for this (tenant_id, partner_id, domain, intelligence_type) — not an incremental blending formula, not a duplicated raw-history table — via one aggregate SELECT (median/mean/min/max, domain included in both the filter and the MERGE key so two verticals sharing a cluster_type spelling never silently pool into one pattern) and one MERGE upsert. Outliers are excluded via Tukey's IQR fence (_IQR_FENCE_MULTIPLIER=1.5, skipped below _MIN_POINTS_FOR_OUTLIER_FILTER=4 raw points) BEFORE the final aggregate, so one wild confirmation can't permanently poison a pattern's confidence.

Confidence — _numeric_confidence = occurrence_weight × consistency, where occurrence_weight = min(occurrence_count/5, 1) and consistency = 1 - (max-min)/median. _confidence_level buckets the score: candidate (just starting) → learning (≥0.4) → eligible (≥0.7, the only level auto-resolve ever acts on).

Read path (consume) — two trigger points, one shared mechanism, auto_resolve.py's sweep_pattern_backlog:

After a new classification run — auto_resolve_session_clusters, called once per classification session right after new clusters are written (both verticals' orchestrators, before notify_exceptions). A cheap in-memory pre-filter (which registered cluster_types does THIS session's own candidate list contain — a query-free no-op for most sessions) decides whether to bother checking a pattern at all.
"On the spot" — service.py's _bulk_resolve, right after a human's decision="approved", is_recurring=true resolution refreshes a pattern (the only moment a pattern's eligibility could have just changed). Its own isolated try/except, separate from refresh_resolution_intelligence's own — a sweep failure must never be conflated with, or mask, a refresh failure, or affect the resolution that already committed.

Either trigger, once a registered cluster_type's pattern needs checking: get_eligible_pattern (a point lookup filtering confidence_level='eligible' in SQL) → a staleness gate (_STALENESS_WINDOW_DAYS=60 — a pattern whose last_seen_at is older than that is treated as if no eligible pattern existed, routing to a human instead of trusting a possibly-drifted pattern) → sweep_pattern_backlog fetches the whole still-pending backlog for that exact (partner_id, tenant_id, domain, cluster_type) via get_pending_clusters_by_type — not session- or request-scoped, so a cluster that's been sitting pending since long before the pattern became eligible gets picked up too, not just brand-new ones. Each candidate cluster's own raw value_field must fall within the pattern's historically observed [min, max] (no extrapolation) → a synthetic ResolveRequest(decision="approved", is_recurring=False, cluster_ids=[...]) is built and resolved with resolution_method="auto" explicitly (the one place in the codebase that genuinely IS the system deciding, so it says so structurally). is_recurring is never set True on the synthetic request — an auto-resolution must never feed back into its own learned pattern, only genuine independent human confirmations do that. No volume cap: an eligible, fresh pattern auto-resolves however many matching clusters exist in the backlog. A backlog with nothing qualifying is a true no-op — no resolve() call, no publish.

No mutual exclusion between the two trigger points — closed at the row level, not with a lock. The two triggers can still fire for the SAME pattern close enough together (a human's approval landing right as a classification session for the same partner finishes) that both pass their own eligibility/staleness checks and both call sweep_pattern_backlog for the same (domain, partner_id, tenant_id, cluster_type) concurrently. A previous pass added a Redis-backed claim (ActiveClaimLock) around this exact window; it was removed as added complexity that didn't fully solve the problem (it only covered sweep-vs-sweep, not a sweep colliding with an unrelated concurrent resolve touching the same cluster) — see git history on this file/shared/core/redis.py if reviving the idea. What replaced it: _bulk_update_clusters (backing bulk_update_decision_status) and bulk_enrich_and_approve_clusters both guard their write on decision_status = 'pending' still being true for each row at write time, and both return only the cluster_ids THIS call actually flipped — a strict subset of the input whenever a concurrent caller claimed some first. Every caller (the generic bulk path's _bulk_resolve, and not_found's matched-and-approved branch) builds its audit rows from that returned set, never from the original request — so two overlapping calls (sweep-vs-sweep, or a sweep colliding with an unrelated concurrent resolve) can never both flip, or both audit, the same cluster. This closes the race at individual-cluster granularity, with no lock at all — see _bulk_update_clusters'/bulk_enrich_and_approve_clusters' own docstrings (resolution/repository.py) for the updated_at-marker attribution mechanism this relies on.

Publishing the outcome — after a non-empty sweep resolves, sweep_pattern_backlog publishes the result to exception-resolution-results-{env}, the exact same Pub/Sub publisher (shared/pubsub/publisher.py's get_resolution_result_publisher) every normal resolve() call already uses to tell the frontend about an approved resolution (see "Idempotency" below / pubsub_trigger.py's _publish_resolution_result, whose payload shape this mirrors exactly). A sweep has no inbound Pub/Sub request to key off, so it synthesizes its own resolution_request_id ("auto-sweep-{uuid4()}") and publishes directly — it does not touch resolution_request_log/claim_or_inspect/mark_resolved/mark_published, since that machinery exists purely for inbound-request redelivery idempotency, which doesn't apply here. The publish step is isolated in its own try/except: a publish failure must never wipe out the already-computed, already-committed set of resolved clusters this function reports back to its caller.

`outcomes/not_found/` — the special path¶

Covers ("nip", "not_found_gl")/("nip", "not_found_bank") and ("mfb", "not_found_gl")/("mfb", "not_found_bank") — registered per (domain, cluster_type), not cluster_type alone (see Routing above). Banking's matching mechanics are genuinely different from NIP's (no hard join key like reference_no/settlement_session_id — banking reuses the real core reconciliation matcher, BatchLLMReconciliationMatcher, directly and read-only) — see docs/banking_exception_resolution.md for the full design. ResolveRequest's optional matching_method/amount_tolerance/recon_time_search_forward/recon_time_search_backward fields tune that matcher's customer_settings for a given call; meaningless for NIP. Landing a NEW document into gl_transactions/bank_statements (_ingest_banking_document) is fully implemented — it reuses the platform's own LLM-based column-mapping inference plus the real MicroFinanceBankReconciliationETL/BaseReconciliationETL.process_both_streams file parser (the same one every normal banking upload goes through), followed by a tagging UPDATE so resolution-ingested rows are excluded from ordinary reconciliation runs; see that doc's "Implementation status" section for the full design.

Only decision = "reconcile" triggers a re-match. approved/rejected/escalated/reopened are a plain bulk status flip on exactly cluster_type — get_clusters_for_bulk_resolve + one bulk_update_decision_status + one bulk_write_resolution_rows, same shape as the default path above, no re-match logic at all. "approved" on a not_found cluster is therefore a simple override ("we already know this is fine, close it," no new document needed) — it does NOT ingest or match anything. "reconcile" is the one value that means "take the newly supplied information and perform an action — try to actually resolve these," and the resulting decision_status per cluster reflects the real outcome, not blindly the caller's intent:

Match found → decision_status = "approved".
No match found → stays "pending" — the attempt is still logged in exception_resolution, but the cluster isn't falsely marked resolved.

The response only ever reports what got approved. Still-pending clusters (no match found, or no document provided at all) are fully audited in exception_resolution — that history isn't lost — but never appear in the results list the caller gets back. The caller's only concern is what actually got resolved; making it filter out success=False noise from every response would just be needless complexity for a value nobody asked for.

Every result carries its own session_id. A resolve call itself has no session_id (see "There is no cluster_id..." above — resolution is partner-wide), but each ClusterResolutionResult in results does: it's the resolved cluster's own session_id, read straight off the row that was fetched. Since a single bulk call can resolve clusters that originally landed under different sessions (the whole point of going partner-wide instead of session-scoped), this lets a caller map each result back to the session it actually came from without a second lookup.

Which document is expected depends on cluster_type, since the naming reflects which side the orphan record sits on, not which side is missing:

`cluster_type`	Orphan sits on	Missing side	Expected field
`not_found_gl`	GL	Bank	`bank_file_url`
`not_found_bank`	Bank	GL	`gl_file_url`

No file provided for the expected field → no-op for that cluster_type's clusters — each still gets a "still pending" audit row (not in the response — see above), but nothing is ingested or matched, and no BigQuery calls happen beyond the one fetch + one bulk audit write.

Scope and matching — partner-wide, bulk MERGE, constant BigQuery calls¶

session_id is just a per-upload batch id, not a stable business concept — the real join keys (reference_no, settlement_session_id) are set by the source data and can legitimately span multiple ingestion batches for the same partner. There is no request session_id to inherit from, and no cluster_id to fetch a "target" cluster from either. The whole reconcile-path pipeline, in order:

Fetch pending clusters — get_pending_clusters_by_type(domain, partner_id, tenant_id, cluster_types=["not_found_gl", "not_found_bank"]), one SELECT, partner+tenant-wide. If empty, return [] immediately — no ingestion, no ad hoc session, no further calls. (Banking picks the majority-branch subset of this fetched set before the rest of this pipeline — see "Branch scoping" above; NIP does not, since it has no branch concept.)
Index known row_uids — pure Python, no BigQuery: split the fetched clusters into gl_known_row_uids (from not_found_gl clusters' gl_record_ids) and bank_known_row_uids (from not_found_bank clusters' bank_record_ids), and build a row_uid -> cluster map for later.
Ingest — parse the document (_etl.run), land it under an ad hoc session_id — purely to satisfy the REQUIRED session_id column on cba_transactions/nip_transactions. Never used to scope a search or a match. One load job (per side, gathered). This id is deterministic, not random: _deterministic_ingestion_session_id hashes tenant_id/partner_id/branch_id/file_urls (sha256, truncated) — branch_id is "" for NIP (no branch concept) and the majority branch's real value for banking (ingested exactly once per resolve call, never per branch — see "Branch scoping" above) — so a retried resolve call (client timeout, double-click, upstream proxy retry) for the identical file reuses the exact same id. That matters because NipReconciliationRepository's composite_key dedup guard hashes session_id in as one of its components and scopes its existence check to that same session_id — a random id per call would produce a different composite_key on every retry and let the same transaction rows land twice; the deterministic id lets the guard (unmodified) recognize and skip the already-ingested rows.
Bulk MERGE, reference_no pass — not_found_matching.bulk_match_by_reference_no: one call, internally up to 2 MERGE statements gathered concurrently (one per table, skipped entirely if that side's known-row_uid list is empty). Target side restricted to row_uid IN UNNEST(@known_row_uids) — only the specific rows already tracked as pending clusters. Counterpart side is not restricted to freshly-uploaded rows — any still-unreconciled row in tenant/partner/branch scope is a valid counterpart, since a real counterpart may already sit under an older session (the same partner-wide reasoning that motivated this whole redesign). match_id/match_method computed per-row inside the MERGE, same deterministic-hash shape as matching/session_id_match.py's own tiers, just without a session_id term.
Bulk MERGE, settlement_session_id + amount pass (only for whatever pass 4 left unmatched) — not_found_matching.bulk_match_by_session_amount, same shape, one call.
Recheck — not_found/handler.py's _recheck_reconciled: one call (2 SELECTs gathered, one per side) to find which known row_uids are now is_reconciled=TRUE. Returns the per-side row_uid lists (not a merged set — step 7 needs them split by table). Mapped back to cluster_id via the dict from step 2 — pure Python.
Unmatched cleanup + cluster bookkeeping, run concurrently (_finalize, via asyncio.gather) — two independent branches, no data dependency between them:
nip_repo.remove_from_unmatched(gl_resolved_row_uids, bank_resolved_row_uids): one call (2 DELETEs gathered, one per table, each skipped if its list is empty) — removes exactly the newly-reconciled row_uids from unmatched_cba/unmatched_nip so they stop showing up as unmatched immediately. See "Reconciliation flags" below for why this can't just reuse refresh_unmatched_tables.
Bulk status update, then bulk audit insert (these two stay sequential relative to EACH OTHER — an audit row must never claim "approved" if the UPDATE that made it true never landed): one bulk_enrich_and_approve_clusters UPDATE covering every resolved cluster_id at once (sets decision_status='approved' AND is_reconciled=TRUE/agent_invoked=TRUE — see below), then one bulk_write_resolution_rows INSERT covering every resolved cluster (full audit row) plus every still-pending cluster of the requested cluster_type (a "still pending" audit row — logged, but never part of the response; see "The response only ever reports what got approved" above). Both bulk_enrich_and_approve_clusters and bulk_update_decision_status run via run_dml_with_retry (BigQueryRepository, shared by every repository in this codebase) — BigQuery allows only one DML mutation in flight per table at a time, so a second concurrent UPDATE against the same cluster table (e.g. two resolve calls racing for the same partner, or a retry racing the original) fails outright with a "concurrent update" error; run_dml_with_retry retries that specific, transient error with exponential backoff instead of dying on the first collision — safe because the UPDATE's WHERE + SET is idempotent no matter how many times it lands.

Total: 8 logical BigQuery call groups, independent of how many clusters are pending — this is the whole point (see "Every code path costs a CONSTANT number of BigQuery calls" above).

Reconciliation flags on the cluster row — `is_reconciled` and `agent_invoked`¶

Both banking_exception_cluster and commercial_exception_cluster carry is_reconciled (new) and agent_invoked (pre-existing, previously always False). Neither is the same thing as decision_status:

decision_status is a classification decision — a human or automated call on how to handle the exception (approved/rejected/escalated/pending).
is_reconciled/agent_invoked reflect whether an actual transaction-level match was confirmed. That only ever happens in not_found/handler.py's matched-and-approved branch (step 8 above, via bulk_enrich_and_approve_clusters — the only place either flag is ever set TRUE). A cluster can be decision_status='approved' via the generic bulk path (e.g. a human approving a missing_identifier exception) with no underlying transaction match at all — is_reconciled/agent_invoked correctly stay FALSE for that.
reopened (via bulk_update_decision_status, used by every non-matching path) additionally resets is_reconciled=FALSE when the new status is pending — a cluster the not_found pipeline previously matched-and-approved shouldn't keep claiming to be reconciled once reopened. agent_invoked is left alone on reopen — "an automated process touched this at some point" stays true as a historical fact even after reopening.
banking_exception_cluster's is_reconciled column is now populated by banking's own resolution path (_resolve_banking_match), same as NIP's — set TRUE via bulk_enrich_and_approve_clusters once a signal-key match is confirmed. (Document ingestion itself isn't wired yet — see the not_found section above — so this only fires once a caller lands rows into gl_transactions/bank_statements some other way in the meantime.)

Why unmatched_cba/unmatched_nip need their own cleanup call (step 7): refresh_unmatched_tables (the mechanism the normal reconciliation pipeline uses to keep those two tables in sync) is scoped to one session_id — but a resolution match can span several different original sessions in the same call. remove_from_unmatched sidesteps that entirely by deleting by row_uid alone, which is already treated elsewhere in this codebase as a genuine globally-unique identity.

Why not reuse the shared session-scoped tiers (matching/session_id_match.py) directly: they hard-require session_id equality on both MERGE sides, and they filter to ingestion_purpose='reconciliation' only (NORMAL_RECON_INGESTION_FILTER) — which would silently exclude the newly-uploaded exception_resolution-flagged rows this feature depends on. matching/nip.py's functions have the same MERGE shape, deliberately different scope.

Date window: no single target cluster's date to anchor on — the window is the union across every pending cluster fetched in step 1 (earliest sample date minus 30 days, to latest sample date plus 30 days). Bounds the query, avoids matching something ancient and coincidental.

Which clusters get an audit row on a miss: only clusters whose cluster_type equals the request's cluster_type — those are the explicit subject of this call. An incidentally-swept cluster of the other not_found type that also failed to match is a silent bystander; it gets no new audit row. If it does match incidentally, it gets a full audit row with resolution_method="auto" and an auto-generated note.

Any other, unrelated records the new document happens to contain are not this action's concern — they're not read, matched, or reported on here. They'll surface normally on the next real reconciliation run if relevant.

Hard rule: resolution never calls the classification orchestrator (commercial_exception_run / banking_exception_run) or the full matching-tier pass across a session. Both are reserved for the normal recurring reconciliation pipeline — never a side effect of resolving exceptions.

Idempotency¶

There's no separate idempotency check — the bulk query itself only ever selects rows already in the correct starting state (pending, or non-pending for decision="reopened"). A cluster not in that state for the requested scope is simply absent from the row set; nothing rejects it, because nothing ever tried to act on it.

`auto_resolved` on each result¶

ClusterResolutionResult.auto_resolved: bool (default False) is True only when that specific cluster was resolved via the recurring-pattern sweep (resolution_method == "auto", see "Recurring-pattern intelligence" above) — never for a manually-triggered call, not_found reconcile matches included. It's a plain boolean signal to a caller/UI ("did a human do this, or did the system"), distinct from — and narrower than — resolution_method in the exception_resolution audit row: resolution_method can also be "auto" for an incidentally-swept other-cluster_type row inside a manually-triggered not_found call (see "Which clusters get an audit row on a miss" above), but auto_resolved on the RESPONSE only ever reflects whether the whole resolve() invocation itself was the sweep, not whether an individual result happened to piggyback on it.

Summary regeneration after a successful resolve¶

Every session touched by a successful resolve() outcome gets its stored reconciliation summary regenerated — as a new row, never an in-place update, so both summary (banking) and nip_summary (commercial) accumulate a full audit history of every recalculation. This fires for every successful outcome, not just not_found/reconcile matches: a plain approved/rejected/escalated/reopened bulk flip regenerates just as much as a matched-and-approved not_found cluster, since either kind of change can make a session's previously-stored summary stale.

Wiring — service.py's ResolutionService._finish(request, response) is the single hook both of resolve()'s exit points return through (the not_found dispatch path and the generic bulk path alike). It collects {r.session_id for r in response.results if r.session_id and r.success} and awaits exception_resolution/summary_generation/regenerator.py's regenerate_summaries_for_sessions(session_ids, tenant_id=, partner_id=, domain=) inside its own try/except — a regeneration failure is logged and never turns an already-successful resolution into an error response, and never delays or blocks the response beyond its own await.

Domain routing — regenerate_summaries_for_sessions no-ops for anything but "mfb"/"nip" (ATM has no summary mechanism yet). For each session_id it fans out concurrently (asyncio.gather(..., return_exceptions=True)) to one of two ReconEngineClient methods — regenerate_banking_summary ("mfb") or regenerate_nip_summary ("nip") — each a POST to reconciliation-engine's own /v1/internal/summary/regenerate/{banking,nip} endpoint (routes/internal_match.py). One session's failure is logged and never blocks another session's regeneration in the same call.

Why an HTTP hop into reconciliation-engine, for both verticals — exception-service and reconciliation-engine are separately deployed microservices with separate dependency sets; only shared/ code is importable from both. NIP's regeneration genuinely needs the hop: its logic (recon_engine.domain.commercial.nip_reconciliation) is private to reconciliation-engine, not shared. Banking's regeneration mostly could run directly in exception-service instead (ReconciliationRepository/GeminiLLMService already live in shared/) — it stays a reconciliation-engine endpoint today for symmetry with NIP's genuine hop, not because it strictly needs to be one; collapsing it into exception-service directly remains an open, unconfirmed simplification (see Open Items).

Per-vertical mechanics (reconciliation-engine/src/recon_engine/internal_match/summary_nip.py / summary_banking.py): - Both recover the session's scope (branch_id + date window, plus NIP's direction/cba_opening_balance) from that session's most recent stored summary row — get_nip_summary_scope/get_summary_scope — since a resolution-triggered regeneration has none of its own; a session with no prior summary raises NoStoredSummaryError (surfaced as a 500 to exception-service, logged there, never raised back to the original resolve() caller). - NIP re-runs the same standalone build_reconciliation_summary (nip_reconciliation/summary.py) the original end-of-run trigger uses, then persists via insert_nip_summary — a plain INSERT, not a MERGE (see "NIP nip_summary is now append-only" below for why the write path itself had to change to support this). No separate delivery step: NIP has never sent its summary anywhere outside BigQuery, so the new row IS the delivery. - Banking re-derives current match/unmatched stats from BigQuery (ReconciliationRepository.get_reconciliation_totals — known caveat: doesn't recover the original run's matching_method, see below), generates a fresh narrative via the same Gemini call the original run uses, writes it as a new summary row (already insert-only, no write-path change needed), then delivers it the same way the original run does: an HTTP PATCH to the loader service (LoaderClient.update_reconciliation). - Both modules hold their repository (NipReconciliationRepository/ReconciliationRepository) as a module-level singleton, constructed once at import time rather than per call — both classes open a ThreadPoolExecutor in __init__ (ReconciliationRepository opens its own plus a nested BigQueryRepository's, 12 threads total) that is never explicitly closed, so constructing one per regeneration call would leak threads indefinitely.

NIP nip_summary is now append-only. Reusing one table for both the original end-of-run write and this resolution-triggered write meant the existing write path also had to change: upsert_nip_summary's MERGE errors at runtime if its ON key ever matches more than one target row, which a second row for the same key would immediately trigger. It's now insert_nip_summary, a plain INSERT — recalculation_count comes from a COUNT(*) + 1 subquery instead of the MERGE's T.recalculation_count + 1, and the old COALESCE(@cba_opening_balance, T.cba_opening_balance) fallback is gone (redundant with build_reconciliation_summary's own upstream get_stored_cba_opening_balance fallback, which already resolves the final value before this method is ever called). Every existing nip_summary reader that assumed one row per key (get_stored_cba_opening_balance) was audited and given an explicit ORDER BY created_at DESC to keep returning the latest value now that more than one row can share a key.

Known gap — banking's matching_method is not recoverable. regenerate_banking_summary calls get_reconciliation_totals without a matching_method, silently defaulting to non-bulk treatment. For a session whose original run used matching_method="bulk" (one match row covers many GL/bank records via details.gl_count/details.bank_count), a regeneration would understate matched counts and could log a false "accounting gap" warning. matching_method is not persisted anywhere queryable today — it's purely an orchestration-level concept (WorkWindow.matching_method) that never reaches a table. Non-bulk sessions are unaffected. Not fixed — needs an explicit decision (accept as a documented limitation, or persist it going forward).

Logging & failure behavior — one aggregate summary, never one line per cluster¶

At the scale this feature targets (thousands of pending clusters per call), logging one line per cluster is useless noise. not_found/handler.py's _finalize logs exactly one summary line per resolve() call, built entirely from data already in hand (no extra BigQuery calls) — resolved-cluster count grouped by each cluster's own stored session_id, plus the still-pending count:

[partner=...] not_found.resolve summary: 214 approved across 3 session(s)
  {'sess-A': 140, 'sess-B': 52, 'sess-C': 22}; 1,621 still pending (cluster_type=not_found_bank).

This is the direct answer to "how many got approved, across which original sessions" — the business question that motivated cutting the per-cluster logging in the first place.

Detection's notify_exceptions is deliberately best-effort: it must never fail the BigQuery write that already succeeded, so it swallows its own errors and just logs a warning. Resolution is the opposite case. A resolve call is a direct action a human or system explicitly took — if it fails, silently swallowing that is exactly how a reviewer ends up believing something is resolved when it isn't, or how a not_found re-match silently stops working for weeks before anyone notices. So, across every file in resolution/:

Fail loudly, not gracefully. Unlike backend_notifier.py's try/except-and-log pattern, errors in service.py/outcomes/* should propagate as real errors back to the caller (a clear HTTP error from routes/exceptions.py), not get caught and downgraded to a log line while returning a false "success."
Log at ERROR level for anything unexpected (an unsupported domain, a BigQuery write failure) — not WARNING. These need to be noticeable in log monitoring, not blend into routine noise.
not_found's re-match step specifically — since it touches document parsing and BigQuery match queries, any failure there must raise clearly rather than leaving clusters in an ambiguous state (neither "approved" nor a logged failed attempt).
This loud-logging contract was silently broken until fixed: every logger.error(msg, exc_info=True) call in this feature (service.py, handler.py, routes/exceptions.py) previously raised a TypeError instead of logging, because the shared Logger.error only accepted an exc= kwarg, not stdlib logging's exc_info=. That TypeError replaced the real exception and kept propagating — service.py's per-cluster failure fallback never ran, and routes/exceptions.py's own try/except (meant to turn any escaping error into a clean HTTPException(500, ...)) crashed the same way, so callers got an opaque, generic 500 with the real cause masked. Fixed at the source (shared/config/logger.py's Logger.error now accepts exc_info too) rather than editing every call site — this was a pre-existing, codebase-wide bug (3 more call sites outside this feature), not something introduced by resolution.

Testing¶

Required, not optional, before this ships:

Routing: domain → correct repository (mfb/atm → UnsupportedDomainError, since neither is wired; nip → the shared commercial cluster table); an unregistered (domain, cluster_type) pair falls through to the bulk generic path, never a KeyError; a genuinely different-vertical pair sharing a cluster_type name (e.g. a hypothetical ("atm", "not_found_gl"), once ATM is wired) must route to the generic path, never to not_found.resolve. Also test that every bulk method filters by the domain value itself, not just picks a table.
Constant call count — the core regression test: construct pending-cluster sets of very different sizes (e.g. 5 vs 500, mocked) and assert the mocked repository/BigQuery call counts (get_pending_clusters_by_type, bulk_enrich_and_approve_clusters, bulk_write_resolution_rows, remove_from_unmatched, the bulk MERGE functions) stay exactly the same as the cluster count grows. This is the regression test for the exact defect the bulk redesign fixes.
Default path: each decision value flips decision_status correctly for every row the bulk query returns, via one UPDATE + one INSERT.
not_found/handler.py — the highest-value tests, since this is the one path with real logic:
Pairing table: not_found_gl + bank_file_url proceeds; not_found_gl + gl_file_url (wrong side) no-ops; neither file field provided no-ops (only for clusters of the requested cluster_type).
Match found in the new document → decision_status="approved", both the underlying transaction row AND the cluster row get is_reconciled=TRUE (cluster row also gets agent_invoked=TRUE, via bulk_enrich_and_approve_clusters specifically, never bulk_update_decision_status); remove_from_unmatched called with exactly that row_uid.
Match NOT found → cluster stays "pending", is_reconciled/agent_invoked untouched, attempt still written to exception_resolution for the requested cluster_type; an incidentally-swept other-type cluster that also missed gets no audit row.
Incidentally-resolved other-type clusters get resolution_method="auto"; the primary-type ones get "manual" — both backend-stamped, never taken from the request (there is no resolution_method field on ResolveRequest to take it from).
reference_no path and settlement_session_id path are independent — a target with reference_no IS NULL must fall through to the session-amount path, and vice versa.
Isolation: a same-reference_no row in a different tenant_id/partner_id must never match (NIP has no branch_id to isolate on — see "Branch scoping" above).
Non-reconcile decisions (including approved) are a plain bulk status flip scoped to exactly request.cluster_type, not a re-match attempt; reopened resets is_reconciled=FALSE (via bulk_update_decision_status's SQL, not just the Python call args — assert the actual UPDATE text contains the reset).
Summary log fires once per call, not once per cluster — assert the logger mock's call count directly (the shared Logger wrapper sets propagate=False, so pytest's caplog won't see these records).
Banking branch scoping: pending clusters spanning 2+ branch_ids must produce exactly ONE ingest+match pass, scoped to the majority branch — never a pass per branch (which would re-ingest the same uploaded document under multiple branch tags). The minority branch's clusters stay pending (audited as "still pending"), not silently dropped from the response.
Hard-rule regression test: assert commercial_exception_run/banking_exception_run (the classification orchestrators) are never called anywhere in the resolve path — a mock/spy that fails the test if either is invoked, since this is exactly the kind of thing a future edit could reintroduce by accident.

Open items¶

Collapse regenerate_banking_summary out of reconciliation-engine, into exception-service directly? Proposed, not confirmed. Unlike NIP's genuine need for the HTTP hop (its logic is private to reconciliation-engine), banking's regeneration only uses shared/-resident code (ReconciliationRepository, GeminiLLMService) — it could run directly in exception-service with no hop at all. It stays a reconciliation-engine endpoint today purely for symmetry with NIP's, not necessity.
Auto-resolve / "learn from resolution history" — deliberately deferred, not designed yet.
ATM's resolution needs — once ATM detection exists.
Communication with Tyrone (outbound status-change notification) — out of scope for now.
Finer-than-cluster_type targeting is now implemented — see "Targeted resolution (single-unit or small-batch)" above (ResolveRequest.cluster_ids). The still-missing piece is the intelligence-gathering step itself (narration analysis, historical pattern lookup, LLM-assisted review) that would decide which specific cluster_ids to pass — deliberately out of scope for resolve(), not designed yet.
exception_clusters (the pre-split legacy table name) is already fully gone from application code (schema.py, every constant) — confirmed by direct search. If a stale physical copy still exists in any BigQuery dataset, it's an infra cleanup (DROP TABLE), not a code change; needs bq/gcloud auth to check.
No auth on this endpoint yet (POST /exceptions/resolve has no Depends(...), same posture as routes/reconciliation.py) — a platform-wide auth-strategy decision (API key? JWT? mTLS?), not something to bolt onto one router alone. Flagged because this endpoint's blast radius (bulk-approve financial reconciliation, mark ledger rows reconciled) is larger than most.
No read/history API. exception_resolution is write-only from the API's perspective — no GET endpoint anywhere queries it. A dashboard or reporting need would require a direct BigQuery query outside this service until one is designed.
No metrics/alerting. Structured logging (via the shared Logger) is the only observability signal in this feature — no counters/StatsD/Prometheus/Cloud Monitoring emission anywhere. A resolve call silently matching zero clusters for days (e.g. a broken file format) would have no signal beyond someone reading logs.
Concurrent/retried-call audit-row duplication — closed for overlapping-scope calls, still open for reopened retries. Two simultaneous "approved" (or matched-and-approved) calls for the same scope can no longer each write their own audit row for the same cluster: _bulk_update_clusters/bulk_enrich_and_approve_clusters guard their write on decision_status = 'pending' and return only the cluster_ids THIS call actually flipped, and every caller audits strictly from that returned set — see "No mutual exclusion between the two trigger points" above. Still open: a retried decision="reopened" call selects the already-non-pending band (not guarded the same way, since re-reopening an already-pending row is a harmless no-op, not a race to close), so a genuine double-send of a reopened request can still write two audit rows for the same event. State stays correct either way — this remaining case is an audit-trail duplication risk, not a state-correctness one.
is_reconciled backfill nuance. The schema migration adds is_reconciled as a bare nullable column (BigQuery rejects adding a defaulted column to an existing table via a schema patch), so pre-existing cluster rows get NULL, not FALSE. The bulk MERGE SQL already defensively COALESCEs this, so it's cosmetic today, but worth knowing if anything ever queries is_reconciled directly without the same guard.
ATM-wiring reality check — resolved for _recheck_reconciled/_finalize, still open for the rest. OUTCOME_HANDLERS/_TABLE_BY_DOMAIN genuinely support registering a new ("atm", ...) entry with zero changes to NIP's or banking's code, as designed. _recheck_reconciled is now generic (table names + id_column — "row_uid" for NIP, "record_id" for banking — as parameters, not hardcoded), and _finalize already worked for any vertical via duck typing on vertical_repo.remove_from_unmatched(gl_ids, bank_ids) — both confirmed by banking becoming a second real consumer, not just theorized. A future ATM implementation still needs its own _resolve_atm_match, its own matching/atm.py, and its own remove_from_unmatched method on whatever repository ATM uses — those three remain genuinely vertical-specific.
Banking not_found matching is now chunk-isolated, not all-or-nothing. Previously, _resolve_banking_match called reconciliation-engine's core-engine match ONCE for the entire pending-cluster sweep; any single failure (a transient recon-engine blip, an LLM timeout) aborted every cluster in the batch, even ones unrelated to whatever failed — a real resilience regression from when matching was per-row, and previously undocumented here. It's now chunked (_BANKING_MATCH_CHUNK_SIZE, default 200, via the new cluster_ids scoping param on BankingMatchRequest/fetch_known_row_uids) — but ONLY once a sweep exceeds that threshold: a batch at or under 200 pending clusters still makes exactly ONE call with cluster_ids=None, byte-identical in cost/behavior to before this feature existed. This matters because each chunk re-runs the matcher's full candidate-pool fetch (only the known row_uid set being searched for is scoped down), so N chunks is genuinely ~N× the BigQuery/LLM work of one call — a real cost, not a free lunch, so it's deliberately reserved for sweeps large enough that one giant call's blast radius is worse than that redundant cost. A failed chunk's clusters simply fall into "still pending" (the same bucket a genuine no-match already lands in) while the rest of the sweep continues; every chunk failing still raises, treated as a total/infra failure rather than a false "nothing matched." Cross-chunk double-claim risk, closed. The matcher's "one candidate claimed at most once" guarantee is enforced by in-memory state cleared PER CALL (greedy claim-and-remove sets in mfb_match_engine.py) — it has no memory across independent chunk calls, so two different orphans landing in different chunks could each independently claim the SAME counterpart record against the same (unscoped, full-date-window) candidate pool. _dedupe_cross_chunk_matches closes this: after all chunks return, it keeps only the higher-confidence match for any gl/bank record_id claimed more than once and drops the loser entirely (never partially) — that orphan falls into "still pending" and is retried on the next resolve() call, rather than the same transaction silently getting marked reconciled against two different counterparts. A no-op on the common unchunked path (0 or 1 total matches skip it outright; within-call uniqueness already holds for a single chunk).

NIP's not_found path is NOT chunked the same way — its pipeline does a one-time document ingest per call, and chunking would mean re-ingesting the same file per chunk, which is wrong. fetch_known_row_uids (the shared helper both verticals call) supports the cluster_ids scoping generically, but only banking's call site actually uses it today; run_nip_match still derives the known set unscoped, all-or-nothing, same as before this fix. - Unbounded cluster fetch — deliberately NOT capped with a LIMIT. get_pending_clusters_by_type/get_clusters_for_bulk_resolve have no row cap: a bulk resolve action's whole contract is "resolve every cluster in scope," and a LIMIT would silently leave clusters past the cutoff stuck pending forever with nothing in the API response to show it happened — worse than the scale risk it would guard against. (An earlier version of this fix did add a 5,000-row LIMIT; that was wrong and was reverted — BigQuery's actual request-size ceiling is 10MB, and even 100k UUID cluster_ids is only ~3.6MB, so there's no real hard limit being protected against at this feature's scale.) What exists instead: a WARNING log (_LARGE_FETCH_WARNING_THRESHOLD, 5,000) that fires purely as a "this backlog is unusually large" signal, with every row still returned and resolved. If a single-call payload ever does need to be split up for real (a much higher row count, or non-UUID identifiers pushing past the 10MB ceiling), that's proper pagination/chunking — looping until exhausted, batching the UPDATE/audit-insert across multiple calls — which hasn't been built and would need its own design.