v1.1.0: headerless mode for audits printed without row/column headings

Sniffs the format on page 1: printed row numbers -> gridded mode
(unchanged); UUID-keyed rows -> new headerless mode, which anchors rows
on per-record UUIDs and takes the column map from the page-1 header
labels. Handles fused cells (ID+Name+Org+networks), mid-value line
wraps, two-timestamp Time Frame ranges, wrapped header labels, and
per-export schema drift. Validated against three monthly JC PD audits:
120/120 cross-engine spot checks, 0 anomalies, field-identical to the
standalone converter it replaces.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012bQ28wRBiQYcSR9FzV9U7c

README.md

@@ -1,5 +1,7 @@
 # pinion-pdf
 
+*Those who would sacrifice essential Balls to preserve a little Ligma deserve Deez Nuts.* — Dave Adams
+
 Someone took a spreadsheet, hit "Print", and handed out the PDF. pinion
 turns that PDF back into a CSV — and then proves the conversion is right.
 
@@ -12,7 +14,7 @@ pip install pymupdf
 python3 -m pinion input.pdf output.csv
 ```
 
-Or install it properly and get a `pinion` command:
+Or install it and get a `pinion` command:
 
 ```
 pip install .
@@ -24,59 +26,79 @@ the first 50 batches first.
 
 ## Why you can't just use a normal PDF tool
 
-The obvious tools (pdftotext, pdfplumber, Camelot, Tabula) fail on printed
-spreadsheets, for three separate reasons.
+The obvious tools (pdftotext, pdfplumber, Camelot, Tabula) have some common
+trappings.
 
-**Trap 1 — text spills over the column next door.** Some columns are too
-narrow for what's in them, so the text is drawn straight over the top of
-the neighbouring cell. You, being a human with a brain, can see the issue;
-a position-based sort cannot. pdfplumber genuinely returns this for one
-cell:
+### Trap 1 — Text Spillover
+Some columns can be too narrow for what's in them, and text spills into neighbors.
+You, being a human with a brain, can see the issue - OCR or position-based sort cannot.
 
-```
-'l,o 0o5k:u5p6 :-0 M0 oAbMil eUTC'
-```
+Something like `pdfplumber` will interleave spillover text, giving you unrecoverable nonsense.
 
-That is `lookup - Mobile` and `05:56:00 AM UTC` interleaved letter by
-letter — unrecoverable once mixed. The fix: place text by where it
-**starts**, never by where it sits. A cell's text always begins at its own
-column's left edge, even when the tail spills past the far edge.
+Our approach measure start points and then uses the PDF's embedded information about write
+order to trace the text.
 
-**Trap 2 — blacked-out cells are invisible to text extraction.** A
-redacted cell has a black rectangle drawn over it. Rectangles are graphics,
-not text, so every text extractor reports the cell as simply empty — and
-you can no longer tell "nobody filled this in" from "somebody removed
-this". pinion finds the rectangles itself and emits a `reason_redacted`
-TRUE/FALSE column plus the bar's width in points (a rough proxy for how
-much text was removed). It also checks whether any text survives *under*
-each bar — if a redaction is cosmetic rather than destructive, it says so
-loudly.
+### Trap 2 - Redactions Get Lost
+Blacked out cells (which are graphics) can be invisible to text extractors.
 
-**Trap 3 — counting lines gives the wrong total.** Every page carries an
-"A B C D" strip labelling the columns, which looks exactly like a row of
-data to a line-counting script. Count lines naively and you invent one
-phantom row per page.
+This might conflate 'redacted' with 'empty', but we generally want to preserve this context.
+
+Our approach searches for redaction boxes, tracks their size, and adds some extra csv fields
+in case this information is relevant later. It also picks up on bad redactions
+(text survival under the boxes).
+
+
+### Trap 3 - Line Counts are Janky
+
+Every page in the typical export format carries an "A B C D ..." strip labeling columns.
+
+To a eegular old parser, that's a row...but it's obviously not.
+
+We avoid phantom rows by stripping those tracking rows from the output.
+
+
+## Two print formats
+
+pinion sniffs page 1 and picks the right strategy on its own:
+
+**Gridded** — Excel printed with row/column headings on. Printed row
+numbers and the letter strip anchor everything; this is the mode described
+below.
+
+**Headerless** — the same audits printed via "Microsoft Print to PDF" with
+headings off. No gridlines, no row numbers, adjacent cells fused into
+single text runs, cells wrapping mid-value, and a schema that drifts
+between exports (one month has a License Plate column, the next doesn't).
+Here pinion anchors rows on the UUID that starts every record and takes
+the column map from the file's own page-1 header labels. Columns absent
+from a given export are reported loudly and left empty — never guessed.
+
+If a file is neither, pinion says so and stops.
 
 ## How the pages are laid out
 
-A wide spreadsheet prints in vertical slices: page 1 holds columns A–D for
-rows 1–32, page 2 holds E–F for those same rows, and so on until the
-letters start over at A and the next 32 rows begin. One set of those pages
-is a "batch"; pinion works out the batch size itself by watching for the
-letter strip to restart.
+A wide spreadsheet prints in vertical slices. In the original data used to build this tool,
+page 1 held columns A-D for 32 rows, page 2 held E-F, etc. 
 
-Rebuilding one spreadsheet row therefore means stitching together pieces
-from several pages — which is why the printed row numbers in the left
-margin matter so much. pinion anchors every row to Excel's own numbering
-and verifies the chain runs 1..N with no gaps or repeats.
+For nomenclature, one set of those pages make a 'batch'. The pattern then repeats.
 
-## Checking its own working
+Rebuilding a row naturally requires stiching together several pages.
+
+We track our calculated rows against the reported row numbers to make sure we haven't
+lost the plot as we process things. Any misses or phantoms -> explosion.
+
+## How We Verify Accuracy
+
+Every run contains internal consistency checks to make sure things haven't gone off the rails.
+
+We track and log any row number deviation, unplaceable text, unplaceable redaction bars,
+and any time we made an assumption regarding a row number, etc because of a redaction bar overlap.
+
+If a layout assumption fails during processing, we stop and yell about it.
 
-pinion never hands you plausible-looking output without evidence. Every run
-ends with:
 
 ```
-CHECKING OUR OWN WORKING
+OUTPUT VALIDATION
 --------------------------------------------------------------
   rows written ............... 1,048,575
   rows expected .............. 1,048,575   ok
@@ -90,7 +112,7 @@ CHECKING OUR OWN WORKING
   blackout bars found ........ 2,714
   rows affected .............. 1,549
   which columns .............. {'G': 2702, 'H': 3, 'I': 1, '?': 8}
-  text readable under bars ... none (deleted, not merely covered)
+  text readable under bars ... none
 ```
 
 If a layout assumption fails, pinion stops with a specific complaint
@@ -98,25 +120,22 @@ instead of guessing. Wrong-but-plausible output is the one failure mode
 this tool must never have. See [VERIFICATION.md](VERIFICATION.md) for how
 the reference conversion was validated.
 
-## A warning about truncated data
+## Data Truncation Warning
 
-Excel tops out at 1,048,576 rows; anything past that was thrown away
-before the PDF existed. pinion warns when a sheet lands exactly on that
-ceiling — data that stops precisely at the maximum was almost certainly
-cut off there, so every total from such a file is a floor, not a full
-count. We can warn about it, but we can't recover data that isn't in the
-PDF.
+Excel tops out at 1,048,576 rows.
 
-## Layout of the code
+If you get a result with 1,048,575 rows + a header row, you must either immediately purchase
+a lottery ticket or consider the possibility of data truncation.
+
+We issue a warning, but the truncated data is an issue in the source PDF.
+
+## General Code Layout
 
 ```
-pinion/page.py     — reading what the PDF drew on a page
-pinion/measure.py  — measuring the layout (one function per fact)
-pinion/convert.py  — rebuilding the rows
-pinion/cli.py      — run it, then check our own working
+pinion/page.py       — Read the PDF page layout
+pinion/measure.py    — Measure the gridded layout
+pinion/convert.py    — Rebuilt rows using our measurements
+pinion/headerless.py — The no-headings variant (UUID + header-label anchors)
+pinion/cli.py        — Format sniffing, execution harness + inline sanity checks
 ```
 
----
-
-*Those who would sacrifice essential Ligma to preserve a little Balls
-deserve Deez Nuts.* — Dave Adams