mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 06:50:53 +00:00
docs+fix: evidence-driven container admission + figure-number-inference spec/plan
_extract_visual_container_rects: - Replace flat width<100/height<50 gate with evidence-driven admission - Add line-like veto (<=3pt, pure decorative separators) - Add page-sized veto (>=60% page + near full edge = background/clip) - Add vertical component merge (same x-range, gap <=5pt) - Keep existing fill/border/thin-gray filter logic spec: 2026-06-26-figure-number-fallback-by-sequence-design - Fix integration point (after _promote_sequence_matches) - Add 3-tier legend_bbox lookup - Add _FRONTMATTER_VISUAL_VETO keyword list - Add structured _extract_figure_marker() return - Add skip-reason contract to inventory plan: 2026-06-26-figure-number-inference-implementation - 6 tasks, ~95min, single-commit PR
This commit is contained in:
parent
190ce2440a
commit
c9ca8e2657
4 changed files with 638 additions and 10 deletions
|
|
@ -0,0 +1,249 @@
|
|||
# Figure Number Inference — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: use `subagent-driven-development` or `executing-plans`.
|
||||
|
||||
**Goal:** Add `_infer_missing_main_figure_numbers()` to `build_figure_inventory()` that fills `figure_number: None` for matched main-sequence figures when a single leading gap (Figure 1) can be inferred with high confidence. Motivation: N6XCZD25 lost "Figure 1." text in OCR, producing `figure_unknown_005`.
|
||||
|
||||
**Primary Spec:** `docs/superpowers/specs/2026-06-26-figure-number-fallback-by-sequence-design.md`
|
||||
|
||||
---
|
||||
|
||||
## 1. Stage Boundary
|
||||
|
||||
This plan:
|
||||
- Adds one new function: `_infer_missing_main_figure_numbers()`
|
||||
- Adds one new helper: `_extract_figure_marker()` (structured marker parser)
|
||||
- Adds one new helper: `_resolve_legend_bbox()` (3-tier lookup)
|
||||
- Adds one module-level constant: `_FRONTMATTER_VISUAL_VETO`
|
||||
- Mutates only `matched_figures` and `figure_legends` entries in the inventory dict
|
||||
- Accepts `inventory` (already assembled) and returns updated `inventory`
|
||||
|
||||
This plan does NOT:
|
||||
- Touch the main matching loop
|
||||
- Touch `_promote_sequence_matches()`
|
||||
- Touch renderer or reader
|
||||
- Touch `compute_figure_legend_completeness()`
|
||||
|
||||
---
|
||||
|
||||
## 2. Non-Negotiable Constraints
|
||||
|
||||
- Do NOT mutate `marker_signature` on any legend entry.
|
||||
- Do NOT infer supplementary/extended_data namespaces.
|
||||
- Do NOT infer middle gaps, multiple leading gaps, or end gaps — first release only handles the single leading `[1]` gap.
|
||||
- Do NOT skip inference silently — always write `inventory["figure_number_inference"]` with reason.
|
||||
- If `legend_bbox` is not resolvable after 3-tier fallback, do NOT infer.
|
||||
|
||||
---
|
||||
|
||||
## 3. File Map
|
||||
|
||||
| File | Responsibility |
|
||||
|------|---------------|
|
||||
| `paperforge/worker/ocr_figures.py` | New function + helpers + constant + integration point in `build_figure_inventory()` |
|
||||
| `tests/test_ocr_figures.py` | All new tests |
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Tasks
|
||||
|
||||
### Task 1: Add `_FRONTMATTER_VISUAL_VETO` constant
|
||||
|
||||
**File:** `paperforge/worker/ocr_figures.py`
|
||||
|
||||
```python
|
||||
_FRONTMATTER_VISUAL_VETO = (
|
||||
"graphical abstract",
|
||||
"toc",
|
||||
"table of contents",
|
||||
"highlights",
|
||||
"available with this article",
|
||||
"supplementary data",
|
||||
"supporting information",
|
||||
"video abstract",
|
||||
"visual abstract",
|
||||
)
|
||||
```
|
||||
|
||||
Module-level, near top of file with other constants.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add `_extract_figure_marker()` helper
|
||||
|
||||
**File:** `paperforge/worker/ocr_figures.py`
|
||||
|
||||
Returns a structured dict:
|
||||
|
||||
```python
|
||||
def _extract_figure_marker(text: str) -> dict:
|
||||
return {
|
||||
"namespace": "main" | "supplementary" | "extended_data",
|
||||
"number": int | None,
|
||||
"raw_prefix": str,
|
||||
"has_s_prefix": bool,
|
||||
"marker_text": str,
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Match against existing `_FIGURE_NUMBER_PATTERN` (or equivalent regex)
|
||||
- If text contains `supplementary`, `supporting`, `additional file`, `appendix` → namespace=supplementary
|
||||
- If text contains `extended data`, `extended figure` → namespace=extended_data
|
||||
- If matched prefix has `S` immediately before digit (case-insensitive) → has_s_prefix=True, namespace=supplementary
|
||||
- Otherwise namespace=main
|
||||
|
||||
This REPLACES `_extract_figure_number()` for building the known-number set in the inference pass. The old function can remain for other callers.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Add `_resolve_legend_bbox()` helper
|
||||
|
||||
**File:** `paperforge/worker/ocr_figures.py`
|
||||
|
||||
```python
|
||||
def _resolve_legend_bbox(
|
||||
matched_item: dict,
|
||||
structured_blocks: list[dict],
|
||||
inventory: dict,
|
||||
) -> list[float] | None:
|
||||
```
|
||||
|
||||
Lookup order:
|
||||
1. `matched_item.get("legend_bbox")` — direct field
|
||||
2. Scan `structured_blocks` for a block whose `block_id == matched_item["legend_block_id"]` and `page == matched_item.get("legend_page")` → return its `bbox`
|
||||
3. Scan `inventory["figure_legends"]` for entry whose `block_id == matched_item["legend_block_id"]` → return its `bbox`
|
||||
4. Return `None` (ineligible)
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Implement `_infer_missing_main_figure_numbers()`
|
||||
|
||||
**File:** `paperforge/worker/ocr_figures.py`
|
||||
|
||||
```python
|
||||
def _infer_missing_main_figure_numbers(
|
||||
inventory: dict,
|
||||
structured_blocks: list[dict],
|
||||
) -> dict:
|
||||
```
|
||||
|
||||
Algorithm (detailed steps in spec):
|
||||
|
||||
1. **Build known set**: `matched_figures` with `isinstance(figure_number, int)`, namespace=main (via `_extract_figure_marker` on their text), not has_s_prefix, unique numbers.
|
||||
|
||||
2. **Build eligible unknown set**: `matched_figures` with `figure_number is None`, namespace=main, legend_block_id, asset_block_ids non-empty, settlement_type in accepted set, not vetoed by frontmatter keywords, legend_bbox resolvable.
|
||||
|
||||
3. **Early skip if no eligible unknowns** → `reason="no_eligible_unknowns"`
|
||||
|
||||
4. **Early skip if known set empty or min != 2** → `reason="known_min_not_2"`
|
||||
|
||||
5. **Exactly one eligible unknown required** → else `reason="multiple_eligible_unknowns"`
|
||||
|
||||
6. **Order check**: compute `figure_order_key = (min(asset_pages or [page]), legend_page, bbox_y, bbox_x)`. Unknown must be before first_known. Else → `reason="unknown_not_before_first_known"`.
|
||||
|
||||
7. **Noise check**: scan `matched_figures`, `held_figures`, `ambiguous_figures` with order_key between unknown and first_known. If any exists → skip (conservative).
|
||||
|
||||
8. **Infer**: Update matched item, update corresponding figure_legends entry, write `"accepted"` status.
|
||||
|
||||
9. **Write `inventory["figure_number_inference"]`** with full metadata.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Wire into `build_figure_inventory()`
|
||||
|
||||
**File:** `paperforge/worker/ocr_figures.py`
|
||||
|
||||
Find the exact location where `inventory` is assembled and `_promote_sequence_matches()` is called. Insert:
|
||||
|
||||
```python
|
||||
inventory = _promote_sequence_matches(inventory, structured_blocks)
|
||||
|
||||
# NEW: infer missing main figure numbers
|
||||
inventory = _infer_missing_main_figure_numbers(inventory, structured_blocks)
|
||||
|
||||
# Existing completeness check follows
|
||||
inventory["figure_legend_completeness"] = compute_figure_legend_completeness(
|
||||
structured_blocks, inventory,
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Tests
|
||||
|
||||
**File:** `tests/test_ocr_figures.py`
|
||||
|
||||
#### 6a. `test_infer_figure1_leading_gap`
|
||||
|
||||
Simulate N6XCZD25: matched_figures with known=[2,3,4,5,6], one eligible unknown before Figure 2.
|
||||
|
||||
Assert:
|
||||
- `inference["status"] == "accepted"`
|
||||
- matched item `figure_number == 1`, `figure_id == "figure_001"`
|
||||
- corresponding `figure_legends` item has `inferred_figure_number == 1`
|
||||
- `inventory["figure_number_inference"]["reason"] == "accepted"`
|
||||
|
||||
#### 6b. `test_infer_frontmatter_veto`
|
||||
|
||||
known=[2,3,4], two unnumbered — one has text "Graphical Abstract".
|
||||
|
||||
Assert:
|
||||
- Only the non-vetoed one gets inferred
|
||||
- Vetoed item unchanged
|
||||
|
||||
#### 6c. `test_infer_main_supplementary_isolation`
|
||||
|
||||
known_supplementary=[1], known_main=[2,3], one eligible main unknown before Figure 2.
|
||||
|
||||
Assert:
|
||||
- Unknown gets number 1
|
||||
- S1 not in known_main set
|
||||
|
||||
#### 6d. `test_infer_no_eligible_unknowns`
|
||||
|
||||
All matched_figures have numbers. Assert skip, reason `no_eligible_unknowns`.
|
||||
|
||||
#### 6e. `test_infer_known_min_not_2`
|
||||
|
||||
known=[1,3,4], one unknown before 3. Assert skip, reason `known_min_not_2`.
|
||||
|
||||
#### 6f. `test_infer_multiple_eligible_unknowns`
|
||||
|
||||
known=[2,3], two unknown before 2. Assert skip, reason `multiple_eligible_unknowns`.
|
||||
|
||||
#### 6g. `test_infer_missing_legend_bbox`
|
||||
|
||||
Matched item without legend_bbox, structured_blocks empty, figure_legends empty.
|
||||
|
||||
Assert skip, reason `missing_legend_bbox`.
|
||||
|
||||
#### 6h. `test_infer_unknown_not_before_first_known`
|
||||
|
||||
known=[2,3], unknown after 2. Assert skip, reason `unknown_not_before_first_known`.
|
||||
|
||||
---
|
||||
|
||||
## 5. Test Harness Note
|
||||
|
||||
Tests that need `structured_blocks` can use a minimal stub list. Tests that need a realistic inventory structure can construct a small dict inline. Do NOT need to open a real PDF.
|
||||
|
||||
Each test must verify:
|
||||
1. The `inventory["figure_number_inference"]` block (status + reason)
|
||||
2. The specific mutations on matched_figures and figure_legends
|
||||
3. That unmatching entries are untouched
|
||||
|
||||
---
|
||||
|
||||
## 6. Execution Order
|
||||
|
||||
```
|
||||
Task 1: _FRONTMATTER_VISUAL_VETO constant → 5 min (one PR comment)
|
||||
Task 2: _extract_figure_marker() → 15 min
|
||||
Task 3: _resolve_legend_bbox() → 10 min
|
||||
Task 4: _infer_missing_main_figure_numbers() → 30 min (core logic)
|
||||
Task 5: Wire into build_figure_inventory() → 5 min
|
||||
Task 6: Tests (8 test cases) → 30 min
|
||||
```
|
||||
|
||||
Total: ~95 min. Single commit PR (or squashed into one commit per the pomodoro-log pattern).
|
||||
|
|
@ -0,0 +1,329 @@
|
|||
# Figure Number Inference from Sequence Gaps
|
||||
|
||||
**Date:** 2026-06-26
|
||||
**Status:** Draft (Revised 2026-06-26, round 2)
|
||||
**Motivation:** PaddleOCR sometimes fails to extract "Figure N." text when the label is rendered inside a figure image or in a narrow visual gap. In N6XCZD25, Figure 1's "Figure 1." text was lost, producing `figure_unknown_005` in the matched inventory while Figures 2-6 were correctly numbered.
|
||||
|
||||
## Problem
|
||||
|
||||
`figure_inventory.json` > `matched_figures` may contain items with `figure_number: null` even when the legend has been correctly matched to an asset. The renderer falls back to `figure_unknown_{N}.md` for these, producing a dead embed in the fulltext — visually placed but no cross-reference number.
|
||||
|
||||
## Proposal
|
||||
|
||||
Add `_infer_missing_main_figure_numbers()`, called inside `build_figure_inventory()` in `ocr_figures.py`, **after** `inventory` is assembled and after `_promote_sequence_matches()`, but **before** `compute_figure_legend_completeness()`.
|
||||
|
||||
### Terminology
|
||||
|
||||
| Term | Meaning |
|
||||
|------|---------|
|
||||
| `_infer_missing_main_figure_numbers()` | New function: infers missing main-sequence figure numbers for matched_figures entries with `figure_number: None` |
|
||||
| `_promote_sequence_matches()` | **Existing** logic: promotes ambiguous figures to matched based on adjacent numbering (they already have a number of their own) |
|
||||
| sequence number inference | **New**: matched figures without a number get one from gap analysis |
|
||||
|
||||
These are distinct concerns — *sequence_match* promotes match status, *inference* fills a missing number.
|
||||
|
||||
### Eligibility
|
||||
|
||||
Candidate must satisfy **all**:
|
||||
|
||||
```text
|
||||
bucket == matched_figures
|
||||
figure_number is None
|
||||
legend_block_id exists and len > 0
|
||||
asset_block_ids non-empty
|
||||
settlement_type in {"same_page", "group_sequential", "cross_page_forward",
|
||||
"cross_page_backward", "composite_parent"}
|
||||
namespace == main (see namespace isolation below)
|
||||
NOT vetoed by frontmatter visual keywords (see below)
|
||||
legend_bbox resolvable (see order contract)
|
||||
```
|
||||
|
||||
Non-candidates (never inferred):
|
||||
- `held_figures`, `ambiguous_figures`, `unmatched_legends`, `rejected_legends`, `unresolved_clusters`
|
||||
- Asset-matched legends that turn out to be frontmatter images, graphical abstracts, TOC images, publisher previews
|
||||
|
||||
### Frontmatter Visual Veto
|
||||
|
||||
```python
|
||||
_FRONTMATTER_VISUAL_VETO = (
|
||||
"graphical abstract",
|
||||
"toc",
|
||||
"table of contents",
|
||||
"highlights",
|
||||
"available with this article",
|
||||
"supplementary data",
|
||||
"supporting information",
|
||||
"video abstract",
|
||||
"visual abstract",
|
||||
)
|
||||
```
|
||||
|
||||
A candidate is vetoed if its `text`, its corresponding legend block text, or nearby container text contains any of these keywords (case-insensitive, substring match).
|
||||
|
||||
### First Release Scope
|
||||
|
||||
**Only one scenario: leading Figure 1 gap.**
|
||||
|
||||
```text
|
||||
known main figures: [2, 3, 4, ...] (min == 2)
|
||||
eligible unknown: exactly 1
|
||||
order check: unknown.figure_order_key < first_known.figure_order_key
|
||||
noise check: no other unnumbered matched/ambiguous/held legend
|
||||
intervenes between unknown and first_known
|
||||
```
|
||||
|
||||
Scenarios explicitly deferred:
|
||||
- Middle gaps (`known=[1,3,4]`, unknown=`[2]`)
|
||||
- Multiple leading gaps (`known=[3,4]`, unknown=`[1,2]`)
|
||||
- End gaps (`known=[1,2,3]`, unknown=`[4]`)
|
||||
|
||||
### Order Contract
|
||||
|
||||
`figure_order_key` is a tuple `(page, legend_page, y, x)` used to sort matched figures for gap detection.
|
||||
|
||||
**Do NOT assume `legend_bbox` is present on the matched_figures item.** Lookup order:
|
||||
|
||||
```python
|
||||
def _resolve_legend_bbox(matched_item, structured_blocks, inventory) -> list[float] | None:
|
||||
# 1. Prefer direct field
|
||||
if matched_item.get("legend_bbox"):
|
||||
return matched_item["legend_bbox"]
|
||||
# 2. Look up legend block by (legend_page, legend_block_id) in structured_blocks
|
||||
legend_block = _find_legend_block(matched_item, structured_blocks)
|
||||
if legend_block and legend_block.get("bbox"):
|
||||
return legend_block["bbox"]
|
||||
# 3. Look up corresponding item in inventory["figure_legends"]
|
||||
for fl in inventory.get("figure_legends", []):
|
||||
if fl.get("block_id") == matched_item.get("legend_block_id"):
|
||||
if fl.get("bbox"):
|
||||
return fl["bbox"]
|
||||
# 4. Not resolvable → skip inference for this item
|
||||
return None
|
||||
```
|
||||
|
||||
If `legend_bbox` is not resolvable after all three fallbacks, the item is **not eligible** for inference.
|
||||
|
||||
```python
|
||||
figure_order_key = (
|
||||
min(matched_item.get("asset_pages") or [matched_item.get("page", 1)]),
|
||||
matched_item.get("legend_page") or matched_item.get("page", 1),
|
||||
legend_bbox[1], # y0 (top-to-bottom)
|
||||
legend_bbox[0], # x0 (left-to-right)
|
||||
)
|
||||
```
|
||||
|
||||
For leading gap: `unknown.order_key < first_known.order_key`
|
||||
|
||||
### Namespace Isolation
|
||||
|
||||
`_extract_figure_marker()` returns a structured dict:
|
||||
|
||||
```python
|
||||
_extract_figure_marker(text: str) -> dict:
|
||||
return {
|
||||
"namespace": "main" | "supplementary" | "extended_data",
|
||||
"number": int | None, # parsed digit, excluding S prefix
|
||||
"raw_prefix": str, # e.g. "Figure", "Fig.", "Figs."
|
||||
"has_s_prefix": bool, # "Fig. S1" → True
|
||||
"marker_text": str, # the matched portion
|
||||
}
|
||||
```
|
||||
|
||||
Heuristic:
|
||||
- Text containing `supplementary`, `supporting`, `additional file`, `appendix` → namespace=supplementary
|
||||
- Text containing `extended data`, `extended figure` → namespace=extended_data
|
||||
- `Fig. S1` / `Figure S1` / `Figure S1C` (capital `S` immediately before digit) → namespace=supplementary, has_s_prefix=True
|
||||
- Everything else → namespace=main
|
||||
|
||||
Only `namespace == "main" and not has_s_prefix` entries participate in inference.
|
||||
|
||||
This parser REPLACES the old `_extract_figure_number()` for building the known-number set. The old function strips `S` and returns a bare int — it must not be used for this pass.
|
||||
|
||||
### Skip Reasons
|
||||
|
||||
The function writes a metadata block to inventory for every invocation, even on no-op:
|
||||
|
||||
```python
|
||||
inventory["figure_number_inference"] = {
|
||||
"status": "skipped" | "accepted",
|
||||
"method": "leading_gap",
|
||||
"reason": "no_eligible_unknowns" | "known_min_not_2"
|
||||
| "multiple_eligible_unknowns" | "unknown_not_before_first_known"
|
||||
| "namespace_not_main" | "frontmatter_visual_veto"
|
||||
| "missing_legend_bbox" | "accepted",
|
||||
"eligible_unknown_count": int,
|
||||
"known_main_numbers": list[int],
|
||||
"inferred_figure_number": int | None,
|
||||
}
|
||||
```
|
||||
|
||||
Reason catalog:
|
||||
|
||||
| reason | meaning |
|
||||
|--------|---------|
|
||||
| `no_eligible_unknowns` | No matched_figures items with figure_number=None |
|
||||
| `known_min_not_2` | known_main empty or min != 2 |
|
||||
| `multiple_eligible_unknowns` | >1 eligible unknown before first_known |
|
||||
| `unknown_not_before_first_known` | Eligible unknown but order_key >= first_known |
|
||||
| `namespace_not_main` | Eligible unknown but namespace != main |
|
||||
| `frontmatter_visual_veto` | Eligible unknown but vetoed by keyword |
|
||||
| `missing_legend_bbox` | Eligible unknown but legend_bbox not resolvable |
|
||||
| `accepted` | Inferred successfully |
|
||||
|
||||
### Mutation Contract
|
||||
|
||||
**Do not mutate `marker_signature`.** It is observed evidence. Instead:
|
||||
|
||||
```python
|
||||
# On matched_figures item:
|
||||
matched_item["figure_number"] = 1
|
||||
matched_item["figure_id"] = "figure_001"
|
||||
matched_item["figure_namespace"] = "main"
|
||||
matched_item["number_inference"] = {
|
||||
"status": "accepted",
|
||||
"method": "leading_gap",
|
||||
"inferred_number": 1,
|
||||
"known_numbers": [2, 3, 4, 5, 6],
|
||||
}
|
||||
|
||||
# On corresponding figure_legends item:
|
||||
legend["inferred_figure_number"] = 1
|
||||
legend["figure_number_source"] = "sequence_gap_inference"
|
||||
```
|
||||
|
||||
Reader-facing output reads `figure_number` / `figure_id` from the matched item, which is already the canonical source — no additional reader changes needed.
|
||||
|
||||
### Integration Point
|
||||
|
||||
In `ocr_figures.py`, `build_figure_inventory()` — the existing matching loop builds `matched_figures` incrementally, and `inventory` is assembled after the loop. The correct insertion point is:
|
||||
|
||||
```python
|
||||
# 1. Assemble inventory (existing code, already after full matching loop)
|
||||
inventory = {
|
||||
"figure_legends": figure_legends,
|
||||
"matched_figures": matched_figures,
|
||||
"held_figures": held_figures,
|
||||
"ambiguous_figures": ambiguous_figures,
|
||||
"unmatched_legends": unmatched_legends,
|
||||
...
|
||||
}
|
||||
|
||||
# 2. Existing sequence promotion
|
||||
inventory = _promote_sequence_matches(inventory, structured_blocks)
|
||||
|
||||
# 3. NEW: infer missing main figure numbers
|
||||
inventory = _infer_missing_main_figure_numbers(inventory, structured_blocks)
|
||||
|
||||
# 4. Existing completeness check
|
||||
inventory["figure_legend_completeness"] = compute_figure_legend_completeness(
|
||||
structured_blocks, inventory,
|
||||
)
|
||||
|
||||
return inventory
|
||||
```
|
||||
|
||||
Do NOT place inference before the main matching loop. At that point `matched_figures` does not exist yet.
|
||||
|
||||
### Algorithm
|
||||
|
||||
```
|
||||
_infer_missing_main_figure_numbers(inventory, structured_blocks) -> inventory
|
||||
|
||||
1. Build known set:
|
||||
- matched_figures with integer figure_number
|
||||
- namespace == main (via _extract_figure_marker on their text)
|
||||
- not has_s_prefix
|
||||
- unique numbers only
|
||||
|
||||
2. Build eligible unknown set:
|
||||
- matched_figures with figure_number is None
|
||||
- namespace == main
|
||||
- legend_block_id exists
|
||||
- asset_block_ids non-empty
|
||||
- settlement_type in accepted set
|
||||
- not vetoed by frontmatter keywords
|
||||
- legend_bbox resolvable
|
||||
|
||||
3. If len(eligible_unknowns) == 0:
|
||||
→ write "no_eligible_unknowns" skip reason, return
|
||||
|
||||
4. If len(known_main) == 0 or min(known_main) != 2:
|
||||
→ write appropriate skip reason, return
|
||||
|
||||
5. Eligible unknown count check:
|
||||
- exactly 1 eligible unknown → OK
|
||||
- > 1 → write "multiple_eligible_unknowns", return
|
||||
|
||||
6. Order check:
|
||||
- compute order_key for unknown and first_known
|
||||
- if unknown.order_key >= first_known.order_key:
|
||||
→ write "unknown_not_before_first_known", return
|
||||
|
||||
7. Noise check:
|
||||
- scan matched_figures, held_figures, ambiguous_figures between
|
||||
unknown and first_known by order_key
|
||||
- if any exists → skip (first release conservatism)
|
||||
|
||||
8. Infer:
|
||||
- update matched_figures item with number_inference metadata
|
||||
- update corresponding figure_legends item
|
||||
- write "accepted" status
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
1. **N6XCZD25 regression**:
|
||||
- `matched_figures[i]` for Figure 1 has `figure_number == 1`, `figure_id == "figure_001"`
|
||||
- No `figure_unknown_*` in matched_figures
|
||||
- `inventory["figure_number_inference"]["status"] == "accepted"`
|
||||
- Reader produces `reader_figure_id == "figure_001_reader"` for that asset
|
||||
- Reader `figure_number == 1`
|
||||
- Reader `strict_source == "matched_figures"`
|
||||
- Fulltext render does not contain `figure_unknown`
|
||||
- Fulltext render does not contain unresolved `visual_group_{page}_{asset}_reader` for that asset
|
||||
|
||||
2. **Frontmatter veto**:
|
||||
- known=[2,3,4], two unnumbered — one is graphical abstract (text contains "graphical abstract")
|
||||
- Only the non-vetoed eligible matched figure gets number 1
|
||||
- Vetoed item unchanged
|
||||
|
||||
3. **Supplementary isolation A**:
|
||||
- known supplementary=[1], known main=[2,3], eligible main unknown=1 before Figure 2
|
||||
- → infer main Figure 1; S1 does not enter known main set
|
||||
|
||||
4. **Supplementary isolation B**:
|
||||
- known supplementary=[1], known main=[2,3], eligible main unknown=0
|
||||
- → no-op; supplementary S1 does not affect main sequence
|
||||
|
||||
5. **Mid-gap deferral**:
|
||||
- known=[1,3,4], one unnumbered between 1 and 3
|
||||
- First release: skip, reason `known_min_not_2`
|
||||
|
||||
6. **legend_bbox fallback**:
|
||||
- matched item without `legend_bbox` → lookup succeeds via structured_blocks → inference proceeds
|
||||
- matched item where all 3 lookups fail → skip, reason `missing_legend_bbox`
|
||||
|
||||
7. **Strict/reader consistency**:
|
||||
- `matched_figures.figure_number`, `matched_figures.figure_id`, reader `figure_number`, `reader_figure_id` all reflect inferred number
|
||||
- Fulltext embed no longer shows `figure_unknown`
|
||||
|
||||
### Implementation Notes
|
||||
|
||||
- `_infer_missing_main_figure_numbers(inventory, structured_blocks) -> dict` — returns updated inventory
|
||||
- No new dependencies
|
||||
- `_extract_figure_marker()` can be adapted from existing `_FIGURE_NUMBER_PATTERN`, but must NOT strip `S` prefix silently
|
||||
- `_resolve_legend_bbox()` is a new helper that tries 3 fallbacks
|
||||
- `_FRONTMATTER_VISUAL_VETO` is a module-level tuple for easy extension
|
||||
|
||||
### Changes vs Original Spec
|
||||
|
||||
| Original | Revised |
|
||||
|----------|---------|
|
||||
| Integration: after inventory write | Integration: inside `build_figure_inventory()`, after `_promote_sequence_matches()`, before `compute_figure_legend_completeness()` |
|
||||
| Mutation: write into `marker_signature.number` | Mutation: add `figure_number` + `number_inference` metadata, leave `marker_signature` untouched |
|
||||
| Gap: any gap, any position | Gap: leading `[1]` only |
|
||||
| Candidate: any unnumbered legend | Candidate: `matched_figures` only, with asset+legend block IDs, no frontmatter veto |
|
||||
| Supplementary: brief note | Supplementary: explicit `_extract_figure_marker()` returning structured dict with `has_s_prefix` |
|
||||
| Order: assumed `legend_bbox` on item | Order: 3-tier lookup via structured_blocks/figure_legends, skip if unresolvable |
|
||||
| Name: sequential gap-fill fallback | Name: `_infer_missing_main_figure_numbers()` (no collision with `_promote_sequence_matches`) |
|
||||
| No skip logging | `inventory["figure_number_inference"]` with status+reason for every invocation |
|
||||
|
|
@ -116,15 +116,26 @@ def _extract_visual_container_rects(page: Any) -> list:
|
|||
- noticeable fill color (not white/transparent), OR
|
||||
- visible border/outline
|
||||
|
||||
Uses evidence-driven admission (not absolute size thresholds):
|
||||
1. Reject line-like noise (≤3pt)
|
||||
2. Reject page-sized clips/crops
|
||||
3. Accept candidates with fill or visual border
|
||||
4. Merge vertically adjacent components sharing same x-range
|
||||
|
||||
Returns list of fitz.Rect objects for each detected container.
|
||||
"""
|
||||
|
||||
rects: list = []
|
||||
import fitz
|
||||
|
||||
try:
|
||||
drawings = page.get_drawings()
|
||||
except Exception:
|
||||
return rects
|
||||
return []
|
||||
|
||||
page_rect = page.rect
|
||||
page_w, page_h = page_rect.width, page_rect.height
|
||||
|
||||
candidates: list = []
|
||||
for drawing in drawings:
|
||||
fill = drawing.get("fill")
|
||||
color = drawing.get("color")
|
||||
|
|
@ -133,7 +144,20 @@ def _extract_visual_container_rects(page: Any) -> list:
|
|||
if not rect:
|
||||
continue
|
||||
|
||||
if rect.width < 100 or rect.height < 50:
|
||||
# 1. Line-like: pure decorative separator, not a container
|
||||
if rect.height <= 3 and rect.width <= 3:
|
||||
continue
|
||||
|
||||
# 2. Page-sized: background fill / drawing frame / crop clip
|
||||
# near entire page + thin gray border = layout noise
|
||||
area_ratio = (rect.width * rect.height) / (page_w * page_h)
|
||||
near_full_page = (
|
||||
rect.x0 <= page_w * 0.05
|
||||
and rect.y0 <= page_h * 0.05
|
||||
and rect.x1 >= page_w * 0.95
|
||||
and rect.y1 >= page_h * 0.95
|
||||
)
|
||||
if area_ratio >= 0.6 and near_full_page:
|
||||
continue
|
||||
|
||||
is_filled = False
|
||||
|
|
@ -142,7 +166,7 @@ def _extract_visual_container_rects(page: Any) -> list:
|
|||
brightness = (r + g + b) / 3
|
||||
is_filled = brightness < 0.95
|
||||
|
||||
has_border = bool(color) and (isinstance(color, (list, tuple))) and stroke_width > 0
|
||||
has_border = bool(color) and isinstance(color, (list, tuple)) and stroke_width > 0
|
||||
|
||||
# Skip thin-bordered, unfilled rectangles in dark near-gray color —
|
||||
# these are PDF layout/drawing frame lines (text area borders, figure
|
||||
|
|
@ -158,9 +182,35 @@ def _extract_visual_container_rects(page: Any) -> list:
|
|||
continue
|
||||
|
||||
if is_filled or has_border:
|
||||
rects.append(rect)
|
||||
candidates.append(rect)
|
||||
|
||||
return rects
|
||||
if not candidates:
|
||||
return []
|
||||
|
||||
# 3. Merge vertically adjacent components sharing x-range.
|
||||
# ponytail: greedy 1-pass, assumes non-overlapping candidates
|
||||
# upgrade: full union-rect if multi-box callouts need deeper layout reconstruction
|
||||
candidates.sort(key=lambda r: (r.y0, r.x0))
|
||||
merged: list = []
|
||||
for rect in candidates:
|
||||
merged_with = False
|
||||
for i, existing in enumerate(merged):
|
||||
x_overlap = min(rect.x1, existing.x1) - max(rect.x0, existing.x0)
|
||||
if x_overlap > 0:
|
||||
gap = rect.y0 - existing.y1
|
||||
if 0 < gap <= 5:
|
||||
merged[i] = fitz.Rect(
|
||||
min(rect.x0, existing.x0),
|
||||
existing.y0,
|
||||
max(rect.x1, existing.x1),
|
||||
rect.y1,
|
||||
)
|
||||
merged_with = True
|
||||
break
|
||||
if not merged_with:
|
||||
merged.append(rect)
|
||||
|
||||
return merged
|
||||
|
||||
|
||||
def backfill_span_metadata_from_pdf(
|
||||
|
|
|
|||
|
|
@ -226,8 +226,8 @@ def test_extract_visual_container_keeps_black_thin_border() -> None:
|
|||
assert len(rects) == 1, "Black thin border should be detected"
|
||||
|
||||
|
||||
def test_extract_visual_container_skips_small_thin_border() -> None:
|
||||
"""A small thin-border rectangle below min size should be skipped."""
|
||||
def test_extract_visual_container_accepts_small_filled() -> None:
|
||||
"""A small filled rect is a candidate (not line-like, not page-sized)."""
|
||||
import fitz
|
||||
|
||||
from paperforge.worker.ocr_pdf_spans import _extract_visual_container_rects
|
||||
|
|
@ -235,14 +235,14 @@ def test_extract_visual_container_skips_small_thin_border() -> None:
|
|||
doc = fitz.open()
|
||||
page = doc.new_page(width=612, height=792)
|
||||
shape = page.new_shape()
|
||||
shape.draw_rect(fitz.Rect(100, 100, 150, 130)) # 50x30 < 100x50 minimum
|
||||
shape.draw_rect(fitz.Rect(100, 100, 150, 130)) # 50x30, filled red
|
||||
shape.finish(fill=(1, 0, 0), color=None, width=0)
|
||||
shape.commit()
|
||||
|
||||
rects = _extract_visual_container_rects(page)
|
||||
doc.close()
|
||||
|
||||
assert len(rects) == 0, "Sub-minimum rect should be skipped"
|
||||
assert len(rects) == 1, "Filled rect should be detected regardless of size"
|
||||
|
||||
|
||||
def test_backfill_span_metadata_with_real_pdf() -> None:
|
||||
|
|
|
|||
Loading…
Reference in a new issue