[veget-able]

Refs #5198. This complements #5213, which re-enables quad corrections for the line-based finder. It addresses text grouping when corrections are off, including the layout path.

I use line coordinates derived from font metrics to group and order cell text, while keeping the existing glyph boxes for cell membership and detection. This keeps decimal points, spaces and other small glyphs with the text they belong to, without a second text extraction or a change to the quad-correction setting.

Why this happens

find_tables() enables TOOLS.set_small_glyph_heights(True) so characters on the same line share a common top coordinate. This depends on quad corrections being enabled. When they are disabled, JM_char_quad() returns the raw glyph quad instead, which is the tight glyph box when TEXT_ACCURATE_BBOXES is set, as textpage_flags() does when the layout package is available.

The layout path deliberately disables corrections before get_layout(). Importing pymupdf4llm also disables them. In these cases, the . in 0.74 can start about 7 pt below the digits, outside the 3 pt y tolerance used by extract_text(), so it becomes a separate line.

Empty glyph boxes for spaces cause similar grouping problems. Small text can lose spaces, as in ClassA/BOvercollateralization in test_4017. On rotated pages, the same geometry can change word order or split numbers into text such as 12 . 5 mg.

Thomas traced the cause in #5198 and suggested grouping by baseline. This follows that approach, with font size taken into account.

Changes

make_chars() records a line_top for each character, derived from its baseline and the span's font metrics using the same normalisation and glyphless-font fallback as quad correction.

For horizontal text, the calculation is origin.y - size * asc / (asc - dsc). With corrections off, the reported ascender and descender defaults of 0.8 and -0.2 give origin.y - 0.8 * size.

For vertical lines on pages rotated by 90 or 270 degrees, it records the corresponding x edges as line_x0 and line_x1. The distance between these edges equals the font size. Oblique lines, which the extractor already treats as vertical, retain their glyph box edges.

WordExtractor.iter_sort_chars(), char_begins_new_word() and extract_text() use these coordinates in place of glyph box edges for line grouping and ordering. Words carry the minimum line_top and line_x0, and the maximum line_x1, of their characters.

With corrections on, the derived coordinates match the corrected box edges for ordinary text. I also checked this on a rotated page with mixed font sizes. There is one known difference for glyphs entirely below the baseline, such as underscores. The flip test in JM_char_quad() treats these as upside-down text and places their boxes below the rest of the line. Using line_top keeps Prem_Written together instead of producing Prem Written\n_, matching the span text.

Using the baseline alone would still split raised footnote markers. In the test case, a marker raised by 3.5 pt falls outside the 3 pt tolerance, producing a\nTotal. Its font-metric line top is within 0.3 pt of the main text, so it stays on the same line as Totala, matching the 1.28 result.

char_in_bbox, has_text and header detection continue to use the existing glyph boxes. The text strategy (words_to_edges_h/v) and the layout=True text map consume the regrouped words, so their output can change when previously split words are joined.

Tests

• Updated seven layout-branch expectations in tests/test_tables.py that recorded split punctuation or missing spaces. The updated results agree with the non-layout expectations and were checked both with and without pymupdf4llm installed.
• Combined the two layout expectations in test_4017. Since Lazy Import of heavy weight modules #5205, find_tables() disables corrections before get_layout() in both environments, and both produce the same four tables.
• Added coverage in tests/test_5198.py for low glyphs, word-level grouping, decimal points with corrections on and off, all four page rotations on the line-based path, the layout path, raised footnote markers and spaces between 8 pt words. Nine of the sixteen cases fail on main.

Validation

I compared current main (16c664fd, whose table code is what I measured, using the 2.0 binary) with the same environment plus this branch's table.py.

For direct find_tables() calls, I sampled every third page of the ParseBench table group, giving 168 pages. “Lone punctuation” counts cells containing a line made only of punctuation such as ., , or -.

The correction setting below is the initial process setting. The layout path disables corrections internally in both cases.

Initial corrections	Mode	Pages with tables	Grids changed	Pages with changed cells	Cells changed	Lone-punctuation cells, before → after
Off, pymupdf4llm imported	Layout	152	0	151	8,923	7,185 → 179
Off	No layout	121	0	116	4,478	3,335 → 138
On, plain import	Layout	152	0	151	8,923	7,185 → 179
On	No layout	121	0	2	6	138 → 138

The 179 remaining flagged cells contain text such as S.p.A. or S.c.a.r.l.. The six changed cells with corrections on are the underscore cases described above.

On the sample's 270-degree page, the no-layout output changes from THERIFLEMAN 00 Feb19/20 and M -F/3-4P to THE RIFLEMAN 0.0 Feb19/20 and M-F/3-4P.

I also compared corrections on and off using the same code in no-layout mode. On main, 115 pages and 4,501 cells differ between the settings. With this change, that falls to 2 pages and 36 cells. The remaining differences concern glyphs near cell borders, where membership still depends on the glyph boxes. The five table boxes and one grid that differ between the settings on main still differ with this change alone, since detection reads the boxes. With #5213 applied as well, there are no differences.

strategy="text" can produce different grids because it derives edges from words. In the layout-mode run with corrections off, 56 of 152 grids change, and lone-punctuation cells fall from 10,273 to 577. With corrections on, 3 grids change, on pages where underscores inside identifiers were split into separate words before.

As a separate non-regression check, I ran all 503 ParseBench table pages through pymupdf4llm Markdown extraction with HTML tables. The score remains 0.758617, with no Markdown changes on any page. That path builds cell text through extract_cells from the TextPage and did not show this symptom.

Scope

The extra empty column also reported in #5198 is unrelated to the correction setting and is not addressed here.

#5204 addresses the punctuation symptom by obtaining character boxes from a second extraction. This approach instead changes the coordinates used for grouping and ordering, while preserving the boxes used for cell membership and detection.