v1.1.0: headerless mode for audits printed without row/column headings
Sniffs the format on page 1: printed row numbers -> gridded mode (unchanged); UUID-keyed rows -> new headerless mode, which anchors rows on per-record UUIDs and takes the column map from the page-1 header labels. Handles fused cells (ID+Name+Org+networks), mid-value line wraps, two-timestamp Time Frame ranges, wrapped header labels, and per-export schema drift. Validated against three monthly JC PD audits: 120/120 cross-engine spot checks, 0 anomalies, field-identical to the standalone converter it replaces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012bQ28wRBiQYcSR9FzV9U7c
This commit is contained in:
1 parent
74b06be742
commit
c9c90920cc
6 files changed
+394
-65
No files matched your search
@@ -61,3 +61,38 @@ diffing against known-good output or by a check line going red:
|
||||
absent: the export was truncated at the ceiling, and roughly 17% of the
|
||||
original records are estimated to be missing. All totals from the file
|
||||
are floors.
|
||||
|
||||
# Headerless mode (v1.1.0)
|
||||
|
||||
Validated against three monthly Johnson City TN PD audit PDFs printed via
|
||||
"Microsoft Print to PDF" without headings (48/74/69 pages; 2,032 / 2,437 /
|
||||
2,607 rows).
|
||||
|
||||
Checks passed on all three:
|
||||
|
||||
1. **Cross-engine spot check.** 40 random rows per file re-verified
|
||||
against pdftotext: 120/120 after making the probes wrap-tolerant (a
|
||||
cell that wraps mid-value in the PDF never appears as one contiguous
|
||||
string in any text extractor's output — probe fragments, not lines).
|
||||
2. **Range sanity.** Every two-timestamp Time Frame parses and runs
|
||||
start <= end (7,074 ranges, 0 disordered).
|
||||
3. **Anomaly counters at zero**: no bad UUIDs, no unplaced runs, no
|
||||
strays in the Name/Org windows.
|
||||
|
||||
Bugs the checks caught during development:
|
||||
|
||||
- **Wrapped header labels** ('Total Networks ' / 'Searched' on stacked
|
||||
runs) made a whole column invisible to name-matching; its values landed
|
||||
in the Org Name window and were silently overwritten. Fixed by merging
|
||||
stacked header runs; guarded by the strays counter.
|
||||
- **A false-alarm verification failure**: 43 spot-checks "failed" because
|
||||
the probe assumed cells appear contiguously in pdftotext output. Glyph
|
||||
level inspection proved the CSV was right and the check was naive.
|
||||
Kept here as a reminder: verify the verifier before trusting a red
|
||||
result.
|
||||
|
||||
Schema drift is real: of the three sibling exports, one has all 13
|
||||
columns, two lack License Plate entirely, and Total Networks Searched
|
||||
moves between its own column and being fused into the ID run. Headerless
|
||||
mode takes each file's schema from its own header and reports what's
|
||||
absent.
|
||||
Reference in new issue
Block a user