mirror of
https://github.com/lllin000/PaperForge.git
synced 2026-07-22 06:50:53 +00:00
docs: add convergence architecture spec/plans and dense-parent hardening spec/plan
This commit is contained in:
Research Assistant
parent
d7e7821c9c
commit
147f5b77e1
4 changed files with 1606 additions and 5 deletions
|
|
@ -0,0 +1,357 @@
|
|||
# Dense Composite Parent Candidate Hardening Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** 让 `VFS8CBW2` 这类 dense fragmented page 稳定地产生 audit-visible `composite_parent_candidates`，而不改变 ordinary same-page arbitration、sidecar、persisted buckets 或 direct settlement paths。
|
||||
|
||||
**Architecture:** 保持现有 atomic grouping 和 live `composite_parent` arbitration path 不动，只增强 Layer 2 candidate construction：把 unresolved visual mass 与 dense-page visual envelope 纳入 parent candidate 构造。这个 ticket 只解决“能不能看见 parent candidate”，不解决“最终谁赢”。
|
||||
|
||||
**Tech Stack:** Python, `pytest`, PaperForge OCR pipeline (`paperforge/worker/ocr_figures.py`), live-paper audit corpus (`VFS8CBW2`, `2UIPV93M`, `3FDT9652`, `24YKLTHQ`).
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope And Hard Boundaries
|
||||
|
||||
This plan implements:
|
||||
|
||||
- `docs/superpowers/specs/2026-06-23-dense-composite-parent-candidate-hardening-design.md`
|
||||
|
||||
It must remain inside these boundaries:
|
||||
|
||||
1. Do **not** add or modify any `settlement_type`
|
||||
2. Do **not** change ordinary same-page arbitration precedence
|
||||
3. Do **not** modify sidecar behavior
|
||||
4. Do **not** widen `_cluster_semantic_page_assets()` thresholds
|
||||
5. Do **not** add literal label blacklists
|
||||
6. Do **not** rewrite panel-title suppression; you may only read already-existing suppression results
|
||||
7. Do **not** append dense parent candidates back into ordinary `candidate_groups`
|
||||
8. Do **not** change persisted bucket semantics (`matched_figures`, `ambiguous_figures`, `unresolved_clusters`, etc.)
|
||||
|
||||
Success condition for this ticket:
|
||||
|
||||
```text
|
||||
VFS8CBW2-class dense pages emit usable composite_parent_candidates,
|
||||
while ordinary pages do not regress into page-wide mega-merge.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Files Expected To Change
|
||||
|
||||
Primary implementation:
|
||||
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
|
||||
Primary tests:
|
||||
|
||||
- Modify: `tests/test_ocr_figures.py`
|
||||
|
||||
Optional logging/doc update if findings materially sharpen strategy:
|
||||
|
||||
- Modify: `PROJECT-MANAGEMENT.md`
|
||||
|
||||
Real-paper verification targets:
|
||||
|
||||
- `VFS8CBW2`
|
||||
- `2UIPV93M`
|
||||
- `3FDT9652`
|
||||
- `24YKLTHQ`
|
||||
|
||||
---
|
||||
|
||||
## 3. Task 0: Baseline Candidate Construction Audit
|
||||
|
||||
**Files:**
|
||||
- Modify: none
|
||||
- Verify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: none yet
|
||||
|
||||
- [ ] **Step 1: Record current dense-parent construction inputs**
|
||||
|
||||
Read and enumerate the current candidate-construction sources around:
|
||||
|
||||
```python
|
||||
_build_candidate_figure_groups_from_assets(...)
|
||||
_build_composite_parent_figure_groups_visual_only(...)
|
||||
```
|
||||
|
||||
Record explicitly:
|
||||
|
||||
1. whether unresolved clusters are currently visible to parent construction
|
||||
2. whether parent construction depends only on atomic groups
|
||||
3. whether panel-title suppression output is readable at this stage
|
||||
|
||||
- [ ] **Step 2: Capture current VFS8CBW2 failure signature**
|
||||
|
||||
Run a lightweight inspection script or test helper showing, for pages `31/32/39/41`:
|
||||
|
||||
```text
|
||||
matched_figures on page
|
||||
composite_parent_candidates on page
|
||||
unresolved_clusters on page
|
||||
```
|
||||
|
||||
Expected baseline:
|
||||
|
||||
```text
|
||||
composite_parent_candidates absent or insufficient on the worst dense pages
|
||||
```
|
||||
|
||||
This step is evidence-only. No code changes.
|
||||
|
||||
---
|
||||
|
||||
## 4. Task 1: Write Failing Synthetic Dense-Candidate Tests
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/test_ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Add a failing test for dense page parent candidate emission**
|
||||
|
||||
Add a synthetic page with:
|
||||
|
||||
1. one formal numbered figure caption
|
||||
2. multiple visual fragments (`>= 4`)
|
||||
3. one ordinary same-page local group that would only explain part of the page
|
||||
4. extra unresolved visual mass in the same visual envelope
|
||||
|
||||
The test should assert:
|
||||
|
||||
```python
|
||||
def test_dense_fragmented_page_emits_composite_parent_candidate() -> None:
|
||||
from paperforge.worker.ocr_figures import build_figure_inventory
|
||||
|
||||
blocks = [
|
||||
# one numbered legend
|
||||
# several figure-like fragments arranged as one composite page
|
||||
]
|
||||
|
||||
inv = build_figure_inventory(blocks)
|
||||
dense_parents = [
|
||||
p for p in inv.get("composite_parent_candidates", [])
|
||||
if p.get("parent_subtype") == "dense_composite"
|
||||
]
|
||||
|
||||
assert dense_parents, "Dense fragmented page must emit a dense composite parent candidate"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add a failing ordinary-page guard test**
|
||||
|
||||
Add a second synthetic page with two independent numbered figures and assert no page-wide dense parent is emitted:
|
||||
|
||||
```python
|
||||
def test_ordinary_multi_figure_page_does_not_emit_dense_parent() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add a failing unresolved-cluster visibility test**
|
||||
|
||||
Assert the parent candidate records `unresolved_cluster_ids` when unresolved visual mass participates:
|
||||
|
||||
```python
|
||||
def test_dense_parent_candidate_records_unresolved_cluster_ids() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the focused tests and confirm they fail for the right reason**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "dense_parent_candidate" -v --tb=short
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
FAIL because current candidate construction does not emit or annotate dense composite parent candidates strongly enough
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Task 2: Strengthen Dense Candidate Construction Only
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Extend parent construction inputs without touching arbitration**
|
||||
|
||||
Enhance `_build_composite_parent_figure_groups_visual_only(...)` so it can incorporate:
|
||||
|
||||
1. atomic group envelopes
|
||||
2. unresolved-cluster envelopes
|
||||
3. dense-page fragment counts / compactness / grid-like structure
|
||||
|
||||
Implementation rule:
|
||||
|
||||
```text
|
||||
The function may return richer composite_parent candidates,
|
||||
but must not append to matched_figures, must not consume ownership,
|
||||
and must keep ownership_enabled = False.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add dense-page trigger gating**
|
||||
|
||||
Only construct dense parent candidates when the page looks like a true dense composite page, using structural signals only.
|
||||
|
||||
Minimum signals to encode:
|
||||
|
||||
1. same-page numbered legend exists
|
||||
2. fragment count high enough
|
||||
3. unresolved / atomic fragments form compact local envelope
|
||||
4. candidate does not look like page-wide scatter
|
||||
|
||||
- [ ] **Step 3: Add required candidate fields**
|
||||
|
||||
Ensure emitted parent candidates carry at least:
|
||||
|
||||
```python
|
||||
parent_subtype = "dense_composite"
|
||||
unresolved_cluster_ids = [...]
|
||||
fragment_count = ...
|
||||
atomic_child_count = ...
|
||||
unresolved_child_count = ...
|
||||
compactness = ...
|
||||
grid_score = ...
|
||||
construction_reason = [...]
|
||||
ownership_enabled = False
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Re-run the focused synthetic tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "dense_parent_candidate" -v --tb=short
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
PASS
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Task 3: Real-Paper Verification
|
||||
|
||||
**Files:**
|
||||
- Modify: none unless test scaffolding is needed
|
||||
- Verify: live artifacts and/or real-paper regression tests
|
||||
|
||||
- [ ] **Step 1: Verify VFS8CBW2 emits dense parent candidates on hot pages**
|
||||
|
||||
Use either fixture-backed replay or live artifact inspection to confirm pages `31/32/39/41` now contain non-empty `composite_parent_candidates`.
|
||||
|
||||
- [ ] **Step 2: Verify 2UIPV93M does not regress**
|
||||
|
||||
Confirm page 18 still keeps:
|
||||
|
||||
```text
|
||||
图3 = same_page
|
||||
图4 = composite_parent
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify ordinary papers do not page-merge**
|
||||
|
||||
Check at least:
|
||||
|
||||
1. `3FDT9652`
|
||||
2. `24YKLTHQ`
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
No new page-wide mega-merge behavior
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the focused verification commands**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "dense_parent_candidate or composite_parent" -v --tb=short
|
||||
python -m paperforge --vault "D:/L/OB/Literature-hub" ocr rebuild VFS8CBW2
|
||||
python -m paperforge --vault "D:/L/OB/Literature-hub" ocr rebuild 2UIPV93M
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
VFS8CBW2 gains visible composite_parent_candidates on dense pages
|
||||
2UIPV93M remains stable
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Task 4: Logging And Stop Check
|
||||
|
||||
**Files:**
|
||||
- Modify: `PROJECT-MANAGEMENT.md`
|
||||
|
||||
- [ ] **Step 1: Record what changed and what did not**
|
||||
|
||||
Add one entry documenting:
|
||||
|
||||
1. the dense candidate-construction gap
|
||||
2. the visual-only parent candidate hardening
|
||||
3. that this ticket did **not** change arbitration precedence or sidecar
|
||||
4. which real papers were used as success / guardrail checks
|
||||
|
||||
- [ ] **Step 2: Explicitly record stop-condition outcome**
|
||||
|
||||
Log whether any of these happened:
|
||||
|
||||
1. atomic grouping thresholds widened
|
||||
2. new settlement path introduced
|
||||
3. sidecar changed
|
||||
4. bucket semantics changed
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
all remain unchanged
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Self-Review Of Plan Against Spec
|
||||
|
||||
Coverage check:
|
||||
|
||||
1. Candidate construction only, not arbitration: covered by Tasks 0/2
|
||||
2. Keep separate from ordinary `candidate_groups`: covered by Task 2 constraints
|
||||
3. Use unresolved visual mass as input: covered by Task 1/2/3
|
||||
4. Ordinary pages must stay stable: covered by Task 1 guard + Task 3 real-paper checks
|
||||
5. No sidecar / bucket / settlement path rewrite: enforced in Scope And Hard Boundaries and Task 4 stop check
|
||||
|
||||
Placeholder scan:
|
||||
|
||||
1. No `TODO` / `TBD`
|
||||
2. All test additions name concrete assertions
|
||||
3. All verification steps include explicit commands
|
||||
|
||||
Type consistency:
|
||||
|
||||
1. Candidate subtype name fixed as `parent_subtype = "dense_composite"`
|
||||
2. Candidate remains `group_type = "composite_parent"`
|
||||
3. `ownership_enabled = False` preserved throughout the plan
|
||||
|
||||
No spec gap remains for this narrow ticket.
|
||||
|
||||
---
|
||||
|
||||
## 9. Execution Recommendation
|
||||
|
||||
This is already a small, executable ticket.
|
||||
|
||||
Recommended execution mode:
|
||||
|
||||
**1. Subagent-Driven (recommended)**
|
||||
- one implementer
|
||||
- one spec-boundary reviewer
|
||||
- one code-quality reviewer
|
||||
|
||||
Do **not** combine this ticket with panel-title suppression changes, sidecar changes, or dense arbitration rollout.
|
||||
|
|
@ -0,0 +1,736 @@
|
|||
# Figure Ownership Arbitration Convergence Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Converge OCR-v2 figure ownership onto a single arbitration model so dense composite, panel-title, sidecar, and sequence-shell cases stop adding new direct settlement behavior.
|
||||
|
||||
**Architecture:** Keep the current OCR-v2 figure pipeline and persisted inventory buckets, but add an internal convergence layer: caption evidence normalization, candidate-source normalization, page-level arbitration metadata, and accounting semantics. This plan does not replace the previously approved visual-grammar roadmap; it constrains how the next `P1B/P2/health`-class work is implemented.
|
||||
|
||||
**Tech Stack:** Python, `pytest`, PaperForge OCR pipeline (`paperforge/worker/ocr_figures.py`, `ocr_figure_reader.py`, `ocr_render.py`), existing real-paper audit fixtures and live-paper regression corpus.
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope And Relationship To Existing Roadmap
|
||||
|
||||
This plan implements the architecture constraints defined in:
|
||||
|
||||
- `docs/superpowers/specs/2026-06-23-figure-ownership-arbitration-convergence-design.md`
|
||||
|
||||
This plan does **not** replace:
|
||||
|
||||
- `docs/superpowers/specs/2026-06-23-ocr-visual-grammar-hardening-design.md`
|
||||
- `docs/superpowers/plans/2026-06-23-ocr-visual-grammar-hardening-implementation.md`
|
||||
|
||||
Instead, it adds the missing convergence layer so later `P1B`/`P2`/accounting work cannot keep growing by direct fallback path.
|
||||
|
||||
This is a **convergence scaffolding plan first**.
|
||||
|
||||
Rule:
|
||||
|
||||
```text
|
||||
It must not implement dense parent ownership arbitration unless P1A/P1B visual grammar foundations are already landed and explicitly verified.
|
||||
```
|
||||
|
||||
Hard rules for this plan:
|
||||
|
||||
1. Do not create a new direct `settlement_type` path.
|
||||
2. Do not remove existing persisted inventory buckets in this pass.
|
||||
3. Do not widen atomic semantic grouping thresholds in `_cluster_semantic_page_assets()`.
|
||||
4. Do not hardcode paper ids, page ids, or literal short labels such as `RND` / `COL II`.
|
||||
5. Do not let `sequence_match` without real asset payload remain in `matched_figures`.
|
||||
6. Treat dense composite as a subtype of `composite_parent`, not a parallel matcher family.
|
||||
7. `Task 4`, `Task 5`, and `Task 7` are gated tasks and may not run unless `P1A` diagnostic-only composite-parent candidates already exist on the target branch.
|
||||
|
||||
---
|
||||
|
||||
## 2. Files Expected To Change
|
||||
|
||||
Primary implementation files:
|
||||
|
||||
- `paperforge/worker/ocr_figures.py`
|
||||
- `paperforge/worker/ocr_figure_reader.py`
|
||||
- `paperforge/worker/ocr_render.py` (only if accounting labels need render-safe treatment)
|
||||
|
||||
Primary tests:
|
||||
|
||||
- `tests/test_ocr_figures.py`
|
||||
- `tests/test_ocr_figure_reader.py`
|
||||
- `tests/test_ocr_render.py`
|
||||
|
||||
Reference docs/logging:
|
||||
|
||||
- `PROJECT-MANAGEMENT.md`
|
||||
- `project/current/ocr-v2-generalization-boundary.md` (read for context only unless the implementation changes strategic conclusions)
|
||||
|
||||
Real-paper validation targets:
|
||||
|
||||
- `VFS8CBW2`
|
||||
- `6FGDBFQN`
|
||||
- `2UIPV93M`
|
||||
- `RKSLQRIM`
|
||||
|
||||
---
|
||||
|
||||
## 3. Task 0: Baseline Convergence Inventory
|
||||
|
||||
**Files:**
|
||||
- Modify: none
|
||||
- Verify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`, `tests/test_ocr_figure_reader.py`, `tests/test_ocr_render.py`
|
||||
|
||||
- [ ] **Step 1: Record current settlement producers**
|
||||
|
||||
Inspect and list every code site that appends to `matched_figures` or emits match-like states in `paperforge/worker/ocr_figures.py`.
|
||||
|
||||
Expected producer list should include current landed producers.
|
||||
|
||||
If `composite_parent` is not yet a live producer on the branch, record it as a planned candidate source rather than as a missing baseline seam.
|
||||
|
||||
Typical producer list may include:
|
||||
|
||||
```python
|
||||
same_page
|
||||
composite_parent
|
||||
sidecar
|
||||
cross_page_backward
|
||||
cross_page_forward
|
||||
legend_bundle
|
||||
group_sequential
|
||||
sequential
|
||||
sequence_match
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Record current bucket semantics**
|
||||
|
||||
Write down the active persisted buckets and where they are consumed:
|
||||
|
||||
```python
|
||||
matched_figures
|
||||
ambiguous_figures
|
||||
held_figures
|
||||
unmatched_legends
|
||||
unmatched_assets
|
||||
rejected_legends
|
||||
unresolved_clusters
|
||||
```
|
||||
|
||||
Verify reader/render assumptions before making any internal convergence change.
|
||||
|
||||
- [ ] **Step 3: Run baseline tests before changes**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py tests/test_ocr_figure_reader.py tests/test_ocr_render.py -q
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
current baseline remains green before convergence changes begin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Task 1: OwnershipDecision Adapter Layer
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for internal-to-persisted mapping**
|
||||
|
||||
Add focused tests asserting that internal ownership-decision metadata can be attached to existing persisted entries without replacing current bucket semantics.
|
||||
|
||||
Tests must use public registry APIs rather than direct mutation of `registry.asset_states`.
|
||||
|
||||
Use tests shaped like:
|
||||
|
||||
```python
|
||||
def test_ownership_decision_metadata_attaches_without_replacing_buckets() -> None:
|
||||
from paperforge.worker.ocr_figures import _ownership_decision_metadata
|
||||
|
||||
meta = _ownership_decision_metadata(
|
||||
"provisional",
|
||||
"same_page_partial",
|
||||
strong=False,
|
||||
reason="dense_page_leftovers",
|
||||
)
|
||||
|
||||
assert meta["ownership_decision"] == "provisional"
|
||||
assert meta["decision_provenance"] == "same_page_partial"
|
||||
assert meta["strong_ownership"] is False
|
||||
assert meta["decision_reason"] == "dense_page_leftovers"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the new mapping test to confirm failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "ownership_decision_metadata" -v --tb=short
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
FAIL because the helper does not exist yet
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add minimal adapter helpers**
|
||||
|
||||
In `paperforge/worker/ocr_figures.py`, add a minimal internal metadata helper like:
|
||||
|
||||
```python
|
||||
def _ownership_decision_metadata(decision: str, provenance: str, *, strong: bool, reason: str = "") -> dict:
|
||||
return {
|
||||
"ownership_decision": decision,
|
||||
"decision_provenance": provenance,
|
||||
"strong_ownership": strong,
|
||||
"decision_reason": reason,
|
||||
}
|
||||
```
|
||||
|
||||
Do **not** introduce new persisted buckets in this task.
|
||||
Do **not** remove current bucket writes in this task.
|
||||
|
||||
- [ ] **Step 4: Re-run the focused tests**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "ownership_decision_metadata" -v --tb=short
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
PASS
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Task 2: Provisional Soft Reservation Semantics
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for provisional reservation behavior**
|
||||
|
||||
Add tests asserting that provisional ownership blocks legacy fallback consumption during arbitration but may still be superseded by a stronger candidate.
|
||||
|
||||
Use tests shaped like:
|
||||
|
||||
```python
|
||||
def test_provisional_reservation_blocks_legacy_fallback() -> None:
|
||||
from paperforge.worker.ocr_figures import FigureOwnershipRegistry
|
||||
|
||||
registry = FigureOwnershipRegistry()
|
||||
registry.soft_reserve_assets([(1, "a1")], owner_id="legend_1", reason="partial_dense_local")
|
||||
|
||||
assert registry.can_consume_assets([(1, "a1")]) is False
|
||||
|
||||
|
||||
def test_soft_reservation_does_not_update_final_used_sets_until_finalized() -> None:
|
||||
from paperforge.worker.ocr_figures import FigureOwnershipRegistry
|
||||
|
||||
registry = FigureOwnershipRegistry()
|
||||
registry.soft_reserve_assets([(1, "a1")], owner_id="legend_1", reason="partial_dense_local")
|
||||
|
||||
assert (1, "a1") not in registry.used_asset_page_ids
|
||||
|
||||
|
||||
def test_release_soft_reservation_reopens_assets_for_fallback() -> None:
|
||||
from paperforge.worker.ocr_figures import FigureOwnershipRegistry
|
||||
|
||||
registry = FigureOwnershipRegistry()
|
||||
registry.soft_reserve_assets([(1, "a1")], owner_id="legend_1", reason="partial_dense_local")
|
||||
registry.release_soft_reservation([(1, "a1")], owner_id="legend_1")
|
||||
|
||||
assert registry.can_consume_assets([(1, "a1")]) is True
|
||||
|
||||
|
||||
def test_stronger_candidate_may_supersede_soft_reservation() -> None:
|
||||
from paperforge.worker.ocr_figures import FigureOwnershipRegistry
|
||||
|
||||
registry = FigureOwnershipRegistry()
|
||||
registry.soft_reserve_assets([(1, "a1")], owner_id="legend_1", reason="partial_dense_local")
|
||||
registry.finalize_soft_reservation([(1, "a1")], owner_id="legend_2", owner_family="figure")
|
||||
|
||||
assert registry.asset_states[(1, "a1")]["owner_id"] == "legend_2"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the provisional tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "provisional_reservation or soft_reservation" -v --tb=short
|
||||
```
|
||||
|
||||
Expected: fail because `soft_reserved` semantics do not exist yet.
|
||||
|
||||
- [ ] **Step 3: Implement minimal soft reservation support**
|
||||
|
||||
Extend `FigureOwnershipRegistry` in `paperforge/worker/ocr_figures.py` with the smallest safe additions:
|
||||
|
||||
```python
|
||||
def soft_reserve_assets(self, asset_ids: list[tuple[int, str]], *, owner_id: str, reason: str) -> None:
|
||||
...
|
||||
|
||||
def finalize_soft_reservation(self, asset_ids: list[tuple[int, str]], *, owner_id: str, owner_family: str) -> None:
|
||||
...
|
||||
|
||||
def release_soft_reservation(self, asset_ids: list[tuple[int, str]], *, owner_id: str) -> None:
|
||||
...
|
||||
```
|
||||
|
||||
Constraint:
|
||||
|
||||
```text
|
||||
soft reservations are internal arbitration state only; they must not directly emit matched_figures entries
|
||||
soft reservations must not update final used sets until finalized
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Re-run the provisional tests**
|
||||
|
||||
Run the same command and confirm green.
|
||||
|
||||
---
|
||||
|
||||
## 6. Task 3: Panel-Title Suppression As Structural Demotion
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for structural panel-title suppression**
|
||||
|
||||
Add tests for the exact contract from the spec:
|
||||
|
||||
```python
|
||||
def test_short_unnumbered_in_figure_text_does_not_compete_with_numbered_caption() -> None:
|
||||
inventory = build_figure_inventory([...])
|
||||
assert all(
|
||||
h.get("legend_block_id") != "panel_title_1"
|
||||
for h in inventory["local_pairing_hypotheses"]
|
||||
)
|
||||
|
||||
|
||||
def test_suppressed_panel_title_remains_embedded_not_body() -> None:
|
||||
inventory = build_figure_inventory([...])
|
||||
# Assert it is still represented as figure-internal evidence, not lost.
|
||||
```
|
||||
|
||||
Use synthetic text such as `RND` / `COL II` only as test fixtures, not as implementation literals.
|
||||
|
||||
- [ ] **Step 2: Run the suppression tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "panel_title or embedded_figure_text" -v --tb=short
|
||||
```
|
||||
|
||||
Expected: fail because no formal suppression layer exists yet.
|
||||
|
||||
- [ ] **Step 3: Implement structural demotion only**
|
||||
|
||||
Add a helper in `paperforge/worker/ocr_figures.py` such as:
|
||||
|
||||
```python
|
||||
def _should_suppress_panel_title_candidate(block: dict, *, page_has_numbered_legend: bool, visual_envelopes: list[dict]) -> bool:
|
||||
...
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
1. no numbered marker
|
||||
2. short text span
|
||||
3. inside likely figure visual envelope
|
||||
4. same page already has a strong numbered figure caption
|
||||
5. no literal string blacklist
|
||||
|
||||
Input requirement:
|
||||
|
||||
```text
|
||||
`visual_envelopes` must come from a visual-only prepass.
|
||||
```
|
||||
|
||||
Allowed sources:
|
||||
|
||||
1. atomic group envelopes
|
||||
2. diagnostic composite-parent envelopes if `P1A` already exists
|
||||
|
||||
Forbidden source:
|
||||
|
||||
```text
|
||||
caption text identity must not define the visual envelope
|
||||
```
|
||||
|
||||
Output behavior:
|
||||
|
||||
```text
|
||||
demote from formal matching
|
||||
retain as in-figure / embedded text evidence
|
||||
never promote into body paragraphs
|
||||
```
|
||||
|
||||
Add an audit-visible surface such as:
|
||||
|
||||
```python
|
||||
inventory["suppressed_caption_candidates"] = [
|
||||
{
|
||||
"block_id": "...",
|
||||
"suppression_reason": "panel_title_inside_visual_envelope",
|
||||
"retained_as": "embedded_figure_text",
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Re-run the suppression tests**
|
||||
|
||||
Run the same test slice and confirm green.
|
||||
|
||||
---
|
||||
|
||||
## 7. Task 4: Dense Composite Parent Candidate Normalization
|
||||
|
||||
**Gate:** This task may only run if `P1A` diagnostic-only composite-parent candidates already exist on the branch.
|
||||
|
||||
If they do not exist, stop and execute the `P1A` visual-grammar roadmap task first.
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for dense parent as a composite-parent subtype**
|
||||
|
||||
Add tests that enforce the spec’s subtype rule:
|
||||
|
||||
```python
|
||||
def test_dense_parent_candidate_is_composite_parent_subtype() -> None:
|
||||
candidate = _build_dense_parent_candidate_fixture(...)
|
||||
assert candidate["group_type"] == "composite_parent"
|
||||
assert candidate["parent_subtype"] == "dense_composite"
|
||||
|
||||
|
||||
def test_dense_parent_candidate_not_appended_into_ordinary_candidate_groups() -> None:
|
||||
inventory = build_figure_inventory([...])
|
||||
assert all(g.get("group_type") != "dense_composite_parent" for g in inventory.get("candidate_groups", []))
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the dense parent tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "dense_parent_candidate or composite_parent_subtype" -v --tb=short
|
||||
```
|
||||
|
||||
Expected: fail because subtype normalization is not explicit yet.
|
||||
|
||||
Rule:
|
||||
|
||||
```text
|
||||
This task may normalize existing parent candidate schema, but must not secretly implement parent generation if none exists.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Normalize dense parent candidate schema**
|
||||
|
||||
Adjust parent candidate generation in `paperforge/worker/ocr_figures.py` so dense composite candidates are represented as:
|
||||
|
||||
```python
|
||||
{
|
||||
"group_type": "composite_parent",
|
||||
"parent_subtype": "dense_composite",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Do not add a new settlement path.
|
||||
Do not append them into the ordinary atomic `candidate_groups` stream.
|
||||
|
||||
- [ ] **Step 4: Re-run dense parent tests**
|
||||
|
||||
Run the same test slice and confirm green.
|
||||
|
||||
---
|
||||
|
||||
## 8. Task 5: Trigger/Scoring Separation For Dense Pages
|
||||
|
||||
**Gate:** This task may only run after `P1A` diagnostic-only composite-parent candidates already exist and Task 4 schema normalization is complete.
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for construction-time vs arbitration-time signals**
|
||||
|
||||
Add tests that enforce:
|
||||
|
||||
```text
|
||||
construction-time candidate generation may not require already-built unresolved buckets
|
||||
arbitration-time scoring may use coverage gain / leftover mass
|
||||
```
|
||||
|
||||
Example test shape:
|
||||
|
||||
```python
|
||||
def test_dense_parent_candidate_can_be_constructed_from_visual_fragment_count_only() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_dense_parent_arbitration_uses_leftover_mass_to_outrank_partial_same_page() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the trigger/scoring tests to confirm failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "dense_parent_arbitration or leftover_mass" -v --tb=short
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Separate the two phases in code**
|
||||
|
||||
Refactor the parent logic into two helpers in `paperforge/worker/ocr_figures.py`:
|
||||
|
||||
```python
|
||||
def _build_dense_composite_parent_candidates(...):
|
||||
... # visual-only, construction-time
|
||||
|
||||
|
||||
def _score_dense_parent_candidate_against_local_ownership(...):
|
||||
... # arbitration-time, may consider coverage gain and unresolved reduction
|
||||
```
|
||||
|
||||
Do not allow the build helper to depend on already-finalized unresolved buckets.
|
||||
|
||||
- [ ] **Step 4: Re-run the trigger/scoring tests**
|
||||
|
||||
Run the same slice and confirm green.
|
||||
|
||||
---
|
||||
|
||||
## 9. Task 6: Unified Sequence-Shell Accounting
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`, `paperforge/worker/ocr_figure_reader.py`
|
||||
- Test: `tests/test_ocr_figures.py`, `tests/test_ocr_figure_reader.py`, maybe `tests/test_ocr_render.py`
|
||||
|
||||
- [ ] **Step 1: Write failing tests for assetless shell demotion**
|
||||
|
||||
Add tests shaped like:
|
||||
|
||||
```python
|
||||
def test_assetless_sequence_shell_not_emitted_to_matched_figures() -> None:
|
||||
inventory = _promote_sequence_matches({...}, blocks=[])
|
||||
assert all(m.get("settlement_type") != "sequence_match" for m in inventory["matched_figures"])
|
||||
|
||||
|
||||
def test_assetless_sequence_shell_emits_ambiguous_hold_reason() -> None:
|
||||
inventory = _promote_sequence_matches({...}, blocks=[])
|
||||
assert any(a.get("hold_reason") == "assetless_sequence_shell" for a in inventory["ambiguous_figures"])
|
||||
```
|
||||
|
||||
If the current branch already keeps assetless sequence shells out of `matched_figures`, do not force a behavior rewrite.
|
||||
Use this task to add explicit shell labeling and accounting only.
|
||||
|
||||
- [ ] **Step 2: Run the sequence-shell tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py tests/test_ocr_figure_reader.py -k "assetless_sequence_shell or sequence_match" -v --tb=short
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Demote shell outcomes without payload**
|
||||
|
||||
In `_promote_sequence_matches(...)`, enforce:
|
||||
|
||||
```text
|
||||
no asset payload -> stay out of matched_figures
|
||||
emit shell status into ambiguous/diagnostic surface instead
|
||||
do not increment official_figure_count
|
||||
```
|
||||
|
||||
Update reader handling only as needed so this new shell state remains render-safe and audit-visible.
|
||||
|
||||
- [ ] **Step 4: Re-run the sequence-shell tests**
|
||||
|
||||
Run the same slice and confirm green.
|
||||
|
||||
---
|
||||
|
||||
## 10. Task 7: VFS8CBW2-Class Dense Page Arbitration Regression
|
||||
|
||||
**Gate:** This is behavior-changing dense arbitration work and must be executed as a separate follow-up ticket.
|
||||
|
||||
Preconditions:
|
||||
|
||||
1. `P1A` diagnostic-only parent candidates are already stable
|
||||
2. Task 1 metadata exists
|
||||
3. Task 2 soft reservations exist
|
||||
4. Task 4 and Task 5 have landed
|
||||
|
||||
If those conditions are not met, stop here.
|
||||
|
||||
**Files:**
|
||||
- Modify: `paperforge/worker/ocr_figures.py`
|
||||
- Test: `tests/test_ocr_figures.py` and real-paper validations
|
||||
|
||||
- [ ] **Step 1: Write failing synthetic dense-page arbitration tests**
|
||||
|
||||
Add the minimum family regressions:
|
||||
|
||||
```python
|
||||
def test_dense_composite_parent_collects_large_fragment_set() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_dense_parent_does_not_swallow_neighboring_ordinary_figure() -> None:
|
||||
...
|
||||
|
||||
|
||||
def test_partial_same_page_claim_becomes_provisional_when_large_same_zone_leftovers_remain() -> None:
|
||||
...
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the synthetic dense-page tests to verify failure**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py -k "dense_composite or provisional_when_large_leftovers" -v --tb=short
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement the arbitration rule, not a new settlement path**
|
||||
|
||||
Update same-page + parent arbitration flow so:
|
||||
|
||||
```text
|
||||
partial local same-page ownership on dense pages is provisional
|
||||
strong dense composite parent may outrank it if safe
|
||||
the final accepted output still lands in existing matched_figures bucket
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Re-run synthetic dense-page tests**
|
||||
|
||||
Run the same slice and confirm green.
|
||||
|
||||
---
|
||||
|
||||
## 11. Task 8: Real-Paper Validation
|
||||
|
||||
**Files:**
|
||||
- Modify: none unless a narrowly justified follow-up is required
|
||||
- Test: `tests/test_ocr_real_paper_regressions.py`
|
||||
|
||||
- [ ] **Step 1: Run focused real-paper validation for capability families**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_real_paper_regressions.py -k "VFS8CBW2 or 6FGDBFQN or 2UIPV93M" -v --tb=short
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
no regression on 2UIPV93M scoped arbitration
|
||||
no regression on 6FGDBFQN mixed grammar
|
||||
VFS8CBW2 improves or at minimum stops overstating ownership via shell matches
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run core figure stack verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m pytest tests/test_ocr_figures.py tests/test_ocr_figure_reader.py tests/test_ocr_render.py -q
|
||||
```
|
||||
|
||||
Expected: green.
|
||||
|
||||
- [ ] **Step 3: Rebuild one live paper only after tests are green**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python -m paperforge --vault "D:/L/OB/Literature-hub" ocr rebuild VFS8CBW2
|
||||
```
|
||||
|
||||
Expected manual checks afterward:
|
||||
|
||||
```text
|
||||
fewer unresolved clusters on representative dense pages
|
||||
panel-title blocks no longer compete as formal captions
|
||||
no assetless sequence shell counted as strong match
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. Task 9: Documentation And Logging
|
||||
|
||||
**Files:**
|
||||
- Modify: `PROJECT-MANAGEMENT.md`
|
||||
|
||||
- [ ] **Step 1: Add a convergence entry after the code is verified**
|
||||
|
||||
Document:
|
||||
|
||||
```text
|
||||
problem
|
||||
root cause
|
||||
fix
|
||||
result
|
||||
tests
|
||||
```
|
||||
|
||||
Mention explicitly that this plan added convergence constraints rather than a new one-paper path.
|
||||
|
||||
---
|
||||
|
||||
## 13. Self-Review Against Spec
|
||||
|
||||
Coverage check:
|
||||
|
||||
1. OwnershipDecision adapter/mapping -> Task 1
|
||||
2. provisional soft reservation semantics -> Task 2
|
||||
3. panel-title suppression -> Task 3
|
||||
4. dense parent as composite-parent subtype -> Task 4
|
||||
5. trigger/scoring separation -> Task 5
|
||||
6. assetless sequence-shell demotion -> Task 6
|
||||
7. unified dense-page arbitration -> Task 7
|
||||
8. relationship to existing roadmap -> enforced in Scope and Task constraints
|
||||
|
||||
Placeholder scan:
|
||||
|
||||
1. No `TBD` / `TODO`
|
||||
2. Every task has exact file targets and commands
|
||||
3. No step says “similar to previous task”
|
||||
|
||||
Type consistency check:
|
||||
|
||||
1. `ownership_decision` is internal metadata, not a replacement bucket
|
||||
2. `composite_parent` remains the parent family; `dense_composite` is subtype only
|
||||
3. `assetless_sequence_shell` is not allowed into `matched_figures`
|
||||
|
||||
---
|
||||
|
||||
## 14. Execution Recommendation
|
||||
|
||||
This plan should be executed in **small tickets**, not as one uninterrupted rewrite.
|
||||
|
||||
Suggested ticket order:
|
||||
|
||||
1. `Ticket A: Task 0 + Task 1 + Task 2 + Task 6`
|
||||
2. `Ticket B: Task 3`
|
||||
3. `Ticket C: Task 4 + Task 5` (only if `P1A` already exists)
|
||||
4. `Ticket D: Task 7` (separate behavior-changing follow-up)
|
||||
5. `Ticket E: Task 8 + Task 9`
|
||||
|
||||
This keeps the convergence work measurable and reversible.
|
||||
|
|
@ -0,0 +1,381 @@
|
|||
# Dense Composite Parent Candidate Hardening Design
|
||||
|
||||
> **Date:** 2026-06-23
|
||||
> **Status:** Proposed
|
||||
> **Scope:** 只增强 dense composite page 上的 `composite_parent` 候选构造能力，不改变 ordinary same-page arbitration、sidecar 路径、persisted bucket 契约，也不引入新的 direct settlement path。
|
||||
|
||||
---
|
||||
|
||||
## 1. 为什么要单独开这个 spec
|
||||
|
||||
当前分支里的 `composite_parent` 已经不是纯诊断 seam，而是一个**真实参与 ownership 的 producer**。
|
||||
|
||||
证据已经很清楚：
|
||||
|
||||
1. `2UIPV93M` page 18 已经能产出 `settlement_type="composite_parent"`
|
||||
2. `VFS8CBW2` page 31/32/39/41 则完全产不出可用 parent candidate
|
||||
3. 因而当前瓶颈不是 arbitration 是否允许 parent 胜出，而是：
|
||||
|
||||
```text
|
||||
dense fragmented page 上，parent candidate 根本没有被稳定构造出来
|
||||
```
|
||||
|
||||
所以这里必须先把问题切窄：
|
||||
|
||||
```text
|
||||
先修 candidate construction
|
||||
再谈 parent arbitration
|
||||
```
|
||||
|
||||
这个 spec 的目标不是解决全部 `VFS8CBW2` ownership，而是先让系统在这类页面上**看见**候选 parent。
|
||||
|
||||
---
|
||||
|
||||
## 2. 当前问题的真实形态
|
||||
|
||||
`VFS8CBW2` 暴露的是下面这条链：
|
||||
|
||||
```text
|
||||
一个大 composite figure
|
||||
-> 被拆成很多 media_asset / figure_asset / panel-title text
|
||||
-> ordinary local matcher 只抓住一小块
|
||||
-> 大量剩余碎片变成 unresolved_clusters
|
||||
-> 个别 caption 再被 sequence_match 壳子补编号
|
||||
```
|
||||
|
||||
这不是 sidecar overwrite 主导的问题。
|
||||
|
||||
这也不是单纯 Chinese / OCR 文本识别问题。
|
||||
|
||||
这是一个更具体的 candidate-generation 缺口：
|
||||
|
||||
```text
|
||||
系统现在能稳定构造 atomic groups
|
||||
但不能在 dense composite page 上把这些 fragments 提升成 scoped parent candidate
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. 本 spec 不做什么
|
||||
|
||||
为了防止又长出新的 ad hoc path，这个 spec 明确排除以下内容：
|
||||
|
||||
1. **不** 新增 `settlement_type`
|
||||
2. **不** 改 sidecar 逻辑
|
||||
3. **不** 改 current persisted buckets（`matched_figures` / `ambiguous_figures` / `unresolved_clusters` 等）
|
||||
4. **不** 通过放宽 `_cluster_semantic_page_assets()` 阈值来把整页直接 mega-merge
|
||||
5. **不** 用 paper id / page id / literal string 黑名单硬编码 `VFS8CBW2`
|
||||
6. **不** 在这一轮直接重排 parent vs ordinary same-page arbitration
|
||||
7. **不** 把 dense parent candidate 塞回 ordinary `candidate_groups`
|
||||
|
||||
一句话：
|
||||
|
||||
```text
|
||||
这里只增强“能不能构造出 parent candidate”，不增强“谁最后赢”。
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 与现有架构的关系
|
||||
|
||||
本 spec 必须服从两份上游文档：
|
||||
|
||||
1. `docs/superpowers/specs/2026-06-23-ocr-visual-grammar-hardening-design.md`
|
||||
2. `docs/superpowers/specs/2026-06-23-figure-ownership-arbitration-convergence-design.md`
|
||||
|
||||
关系如下：
|
||||
|
||||
### 4.1 对 visual-grammar hardening 的位置
|
||||
|
||||
它相当于把原先 `P1A` 里“diagnostic-only composite parent detector”这件事切得更具体。
|
||||
|
||||
也就是说：
|
||||
|
||||
```text
|
||||
这不是新的大路线
|
||||
而是对 P1A 的一次收窄和落地约束
|
||||
```
|
||||
|
||||
### 4.2 对 convergence 的位置
|
||||
|
||||
它属于 Layer 2 candidate generation，而不是 Layer 3 arbitration。
|
||||
|
||||
因此必须满足 convergence 约束：
|
||||
|
||||
1. visual-only construction
|
||||
2. separate from ordinary `candidate_groups`
|
||||
3. dense parent 作为 `composite_parent` 的 subtype，而不是新 family
|
||||
|
||||
允许的表达方式仍然是：
|
||||
|
||||
```python
|
||||
{
|
||||
"group_type": "composite_parent",
|
||||
"parent_subtype": "dense_composite",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. 设计目标
|
||||
|
||||
本轮只追求三个目标：
|
||||
|
||||
### 5.1 让 dense page 稳定地产生 parent candidate
|
||||
|
||||
对于 `VFS8CBW2` 这类页面，系统应该至少能产出 audit-visible 的 `composite_parent_candidates`，而不是完全没有 parent candidate。
|
||||
|
||||
### 5.2 不伤 ordinary page
|
||||
|
||||
普通双图页、普通 caption-below 页、已经稳定的 same-page figure 页，不应因为这次增强而被错误合并。
|
||||
|
||||
### 5.3 给后续 arbitration 提供可用输入
|
||||
|
||||
候选需要包含足够的信息，让后续阶段判断：
|
||||
|
||||
1. coverage 是否显著提升
|
||||
2. 是否跨越别的 numbered caption boundary
|
||||
3. 是否只是 page-wide scatter 而不是 scoped composite
|
||||
|
||||
---
|
||||
|
||||
## 6. 新的 candidate construction 思路
|
||||
|
||||
### 6.1 不改变 atomic grouping 的职责
|
||||
|
||||
`atomic semantic groups` 继续保持“局部视觉事实”的角色。
|
||||
|
||||
它的职责是：
|
||||
|
||||
```text
|
||||
我能确定哪些小块在局部上属于一起
|
||||
```
|
||||
|
||||
它**不**负责：
|
||||
|
||||
```text
|
||||
整页 dense composite parent 的最终组织
|
||||
```
|
||||
|
||||
所以本 spec 不允许把 dense-page 需求反向压回 atomic grouping 阈值。
|
||||
|
||||
### 6.2 dense parent candidate 的输入
|
||||
|
||||
构造输入允许来自：
|
||||
|
||||
1. existing atomic `candidate_groups`
|
||||
2. 同页 `unresolved_clusters` 的几何信息
|
||||
3. 同页 figure-like assets 的 envelope statistics
|
||||
4. 同页 structured blocks 中的正文/表格/section interruption 信息
|
||||
|
||||
不允许作为 construction-time 输入的：
|
||||
|
||||
1. caption 文本编号本身
|
||||
2. legend 文本语义
|
||||
3. arbitration 之后才知道的 coverage truth
|
||||
|
||||
### 6.3 dense page trigger
|
||||
|
||||
只有页面满足下列条件时，才进入 dense parent candidate construction：
|
||||
|
||||
1. 同页存在至少一个 formal numbered figure caption
|
||||
2. 同页 figure-like visual fragments 数量较高，例如 `>= 4`
|
||||
3. 同页 atomic groups + unresolved clusters 共同显示出明显碎片化
|
||||
4. 这些 fragments 在局部包络内足够紧凑，而不是整页散落
|
||||
|
||||
目标是区分：
|
||||
|
||||
```text
|
||||
true dense composite page
|
||||
vs
|
||||
ordinary multi-figure page
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Dense Parent Candidate Contract
|
||||
|
||||
本轮生成的 candidate 至少要带这些字段：
|
||||
|
||||
```python
|
||||
{
|
||||
"group_id": str,
|
||||
"group_type": "composite_parent",
|
||||
"parent_subtype": "dense_composite",
|
||||
"page": int,
|
||||
"child_group_ids": list[str],
|
||||
"unresolved_cluster_ids": list[str],
|
||||
"asset_block_ids": list[str],
|
||||
"embedded_text_block_ids": list[str],
|
||||
"cluster_bbox": list[float],
|
||||
"fragment_count": int,
|
||||
"atomic_child_count": int,
|
||||
"unresolved_child_count": int,
|
||||
"visual_mass": float,
|
||||
"compactness": float,
|
||||
"grid_score": float,
|
||||
"construction_reason": list[str],
|
||||
"crosses_caption_boundary": bool,
|
||||
"ownership_enabled": False,
|
||||
}
|
||||
```
|
||||
|
||||
说明：
|
||||
|
||||
### 7.1 `ownership_enabled` 必须继续为 `False`
|
||||
|
||||
因为这一轮只解决 candidate construction，不直接把它升级成更激进的 live arbitration。
|
||||
|
||||
### 7.2 `unresolved_cluster_ids` 必须显式记录
|
||||
|
||||
这是本轮最重要的新信息之一。
|
||||
|
||||
如果一个 dense parent candidate 无法说明自己吸收了哪些 unresolved visual mass，那么它对后续 arbitration 的价值就不够。
|
||||
|
||||
### 7.3 `embedded_text_block_ids` 先允许为空
|
||||
|
||||
这一轮不要求彻底解决 panel-title suppression 与 parent consumption 的绑定。
|
||||
但 contract 上要给这个槽位留出来，避免后面又生出并行结构。
|
||||
|
||||
---
|
||||
|
||||
## 8. Construction-Time Scoring Signals
|
||||
|
||||
这里说的是**构造候选时**可以使用的视觉信号，不是最终 arbitration score。
|
||||
|
||||
允许信号：
|
||||
|
||||
1. fragment count
|
||||
2. unresolved child count
|
||||
3. bbox compactness
|
||||
4. row/column grid regularity
|
||||
5. child bbox size similarity
|
||||
6. page-local visual mass density
|
||||
7. body/table interruption absence
|
||||
|
||||
禁止信号：
|
||||
|
||||
1. caption 文本内容
|
||||
2. “这个 caption 解释得好不好”
|
||||
3. same-page ownership 之后剩多少已确认 solved mass
|
||||
|
||||
原因很简单：
|
||||
|
||||
```text
|
||||
这一层必须仍然是 visual-only construction
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. 与 panel-title suppression 的边界
|
||||
|
||||
panel-title suppression 是支持性约束，但不是本 spec 的主修复，也不是这个 ticket 的实现目标。
|
||||
|
||||
本 spec 只要求 candidate construction **能够容忍** 这些短文本存在，而不是依赖 suppression 先彻底清空页面。
|
||||
|
||||
实现边界：
|
||||
|
||||
```text
|
||||
本 ticket 可以读取已经存在的 suppression 结果，
|
||||
但不得为了让 dense parent candidate 通过而新增或重写 suppression 规则。
|
||||
```
|
||||
|
||||
因此这里的边界是：
|
||||
|
||||
1. dense parent construction 不应因为存在短 panel-title text 就完全失效
|
||||
2. 若页面同时存在 numbered legend 与短 panel-title candidates，dense parent construction 仍应依据 visual fragments / unresolved clusters 自身成立
|
||||
3. suppression 负责减少 caption 竞争；parent construction 负责恢复 visual object
|
||||
|
||||
换句话说：
|
||||
|
||||
```text
|
||||
suppression 是减噪
|
||||
parent construction 是补 object
|
||||
```
|
||||
|
||||
两者不要互相替代。
|
||||
|
||||
---
|
||||
|
||||
## 10. 成功标准
|
||||
|
||||
这轮不要求直接“解决 VFS8CBW2 所有 ownership”。
|
||||
|
||||
它的成功标准应该更窄：
|
||||
|
||||
### 10.1 候选可见性
|
||||
|
||||
`VFS8CBW2` page 31/32/39/41 这类 dense composite pages 上，`figure_inventory` 中必须能看到 audit-visible 的 `composite_parent_candidates`，而不是空。
|
||||
|
||||
### 10.2 不污染 ordinary pages
|
||||
|
||||
`2UIPV93M`、`3FDT9652`、`24YKLTHQ` 这类当前已部分稳定的普通/混合页面，不能因为 dense parent construction 扩张而出现 page-wide mega-merge。
|
||||
|
||||
### 10.3 unresolved mass 纳入 parent 候选视野
|
||||
|
||||
候选必须明确吸收部分 unresolved cluster 作为 parent construction input，而不是继续只盯着已成形 atomic groups。
|
||||
|
||||
### 10.4 不新增 settlement path
|
||||
|
||||
任何结果都仍然应该通过现有 arbitration/ownership path 落地，而不是偷偷长出 `dense_parent_settlement` 或类似新分支。
|
||||
|
||||
---
|
||||
|
||||
## 11. 最低测试要求
|
||||
|
||||
至少应覆盖下面几类：
|
||||
|
||||
1. `dense page emits composite parent candidates`
|
||||
- dense multi-panel synthetic page
|
||||
- ordinary local same-page matcher只能抓到部分
|
||||
- 仍应产生 `composite_parent_candidates`
|
||||
|
||||
2. `ordinary multi-figure page does not emit page-wide parent`
|
||||
- 两个独立 numbered figures 同页
|
||||
- 不得因为 fragment 数量上来就误造 dense parent
|
||||
|
||||
3. `dense candidate may include unresolved cluster ids`
|
||||
- unresolved visual mass 必须进入 candidate contract
|
||||
|
||||
4. `panel-title noise does not prevent candidate construction`
|
||||
- 即使同页有短 panel-title-like text，candidate 仍可构造
|
||||
- 这里测试的是 candidate construction 的鲁棒性，不是 suppression 新行为
|
||||
|
||||
5. `current working paper remains stable`
|
||||
- 至少校验 `2UIPV93M` page 18 不回退
|
||||
|
||||
---
|
||||
|
||||
## 12. Stop Condition
|
||||
|
||||
如果实现过程中发现必须做下面任何一件事，说明 scope 已经越界：
|
||||
|
||||
1. 放宽 atomic grouping 阈值去吞整页
|
||||
2. 引入新的 direct settlement type
|
||||
3. 依赖 literal label blacklist
|
||||
4. 必须重写 sidecar
|
||||
5. 必须重排 entire arbitration precedence
|
||||
|
||||
一旦出现这些信号，应停止实现，回到上层 spec/plan 重新切分。
|
||||
|
||||
---
|
||||
|
||||
## 13. 结论
|
||||
|
||||
当前分支的 `composite_parent` 不是不存在，而是：
|
||||
|
||||
```text
|
||||
已经 live
|
||||
但 candidate construction 对 dense fragmented pages 还不够强
|
||||
```
|
||||
|
||||
因此下一步不应直接做更大的 dense arbitration rollout，
|
||||
而应先完成：
|
||||
|
||||
```text
|
||||
dense composite parent candidate hardening
|
||||
```
|
||||
|
||||
把 `VFS8CBW2` 这类页面从“根本看不见 parent”推进到“至少稳定地产生可审计 parent candidate”，
|
||||
后续再谈谁在 arbitration 里最终胜出。
|
||||
|
|
@ -139,6 +139,27 @@ rejected
|
|||
`settlement_type` may continue to exist as provenance.
|
||||
It must no longer be treated as equivalent to truth strength.
|
||||
|
||||
### 4.2.1 OwnershipDecision is internal before bucket migration
|
||||
|
||||
In the first convergence pass, `OwnershipDecision` is an internal arbitration result.
|
||||
It does **not** replace the persisted inventory buckets immediately.
|
||||
|
||||
Required mapping:
|
||||
|
||||
```text
|
||||
accepted -> matched_figures
|
||||
provisional -> ambiguous_figures (or a future provisional audit surface), but not strong solved ownership
|
||||
unresolved -> unresolved_clusters / unmatched_assets / unmatched_legends depending on object type
|
||||
rejected -> rejected_legends / rejected_candidates
|
||||
```
|
||||
|
||||
Phase-1 rule:
|
||||
|
||||
```text
|
||||
The system may add `ownership_decision` or equivalent metadata onto existing entries,
|
||||
but it must not remove the current persisted buckets until reader/render/health/object consumers are migrated.
|
||||
```
|
||||
|
||||
### 4.3 Partial local ownership is not automatically protected
|
||||
|
||||
Any same-page result that only explains a small subset of the local visual mass on a dense page is not automatically strong ownership.
|
||||
|
|
@ -151,6 +172,21 @@ provisional
|
|||
|
||||
until arbitration confirms it is the best ownership explanation.
|
||||
|
||||
### 4.3.1 Provisional ownership uses soft reservation semantics
|
||||
|
||||
`provisional` must have explicit reservation behavior.
|
||||
Otherwise it either blocks stronger repair paths too early or remains too weak to prevent duplicate fallback consumption.
|
||||
|
||||
Required semantics:
|
||||
|
||||
```text
|
||||
provisional candidates may place soft reservations during arbitration
|
||||
soft reservations block legacy fallback from consuming the same assets during the arbitration window
|
||||
soft reservations may be superseded by a stronger candidate before finalization
|
||||
only accepted decisions finalize ownership into matched_figures truth
|
||||
rejected or unresolved outcomes must release or downgrade the reservation according to the audit policy
|
||||
```
|
||||
|
||||
### 4.4 Assetless sequence shells are not solved matches
|
||||
|
||||
If a `sequence_match` has no real asset payload, it is not a real ownership win.
|
||||
|
|
@ -158,6 +194,22 @@ If a `sequence_match` has no real asset payload, it is not a real ownership win.
|
|||
It may be visible as a shell outcome for traceability.
|
||||
It must not count as a strong solved match in audit or health reporting.
|
||||
|
||||
Required output rule:
|
||||
|
||||
```text
|
||||
assetless sequence shells must not be emitted into matched_figures
|
||||
```
|
||||
|
||||
They may instead appear as:
|
||||
|
||||
1. `ambiguous_figures` with `hold_reason = "assetless_sequence_shell"`
|
||||
2. a separate `sequence_shells` audit bucket
|
||||
|
||||
They must not increment:
|
||||
|
||||
1. `official_figure_count`
|
||||
2. solved ownership metrics in health / audit summaries
|
||||
|
||||
---
|
||||
|
||||
## 5. Unified Object Model
|
||||
|
|
@ -198,6 +250,32 @@ A proposed explanation that a given legend owns a specific visual grouping.
|
|||
|
||||
This is where same-page, composite-parent, sidecar-rescue, and sequence-shell provenance belong.
|
||||
|
||||
Minimum contract shape in the convergence direction:
|
||||
|
||||
```python
|
||||
{
|
||||
"candidate_id": str,
|
||||
"candidate_source": str,
|
||||
"legend_block_id": str | None,
|
||||
"visual_group_id": str | None,
|
||||
"asset_block_ids": list[str],
|
||||
"embedded_text_block_ids": list[str],
|
||||
"page": int,
|
||||
"legend_page": int | None,
|
||||
"asset_pages": list[int],
|
||||
"bbox": list[float],
|
||||
"evidence": list[str],
|
||||
"conflicts": list[str],
|
||||
"coverage_score": float,
|
||||
"caption_score": float,
|
||||
"visual_score": float,
|
||||
"arbitration_score": float | None,
|
||||
}
|
||||
```
|
||||
|
||||
The exact field names may vary.
|
||||
The shape must remain unified across candidate sources.
|
||||
|
||||
### 5.5 `OwnershipDecision`
|
||||
|
||||
The final arbitrated state:
|
||||
|
|
@ -294,6 +372,29 @@ dense_composite_parent_candidate
|
|||
This must be generated as part of candidate generation.
|
||||
It must **not** be implemented as a new direct settlement path.
|
||||
|
||||
### 7.2.1 Dense parent is a composite-parent subtype
|
||||
|
||||
`dense_composite_parent_candidate` is not a new ownership family parallel to `composite_parent`.
|
||||
|
||||
It is a subtype or evidence profile of composite parent candidacy.
|
||||
|
||||
Required interpretation:
|
||||
|
||||
```text
|
||||
dense_composite_parent_candidate is a subtype of composite_parent_candidate,
|
||||
not a separate ownership path and not a new settlement mechanism
|
||||
```
|
||||
|
||||
One acceptable representation is:
|
||||
|
||||
```python
|
||||
{
|
||||
"group_type": "composite_parent",
|
||||
"parent_subtype": "dense_composite",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
### 7.3 Dense parent construction constraints
|
||||
|
||||
Dense parent candidates must be:
|
||||
|
|
@ -310,18 +411,30 @@ Caption alignment is allowed later in arbitration.
|
|||
|
||||
The system should only build dense parent candidates in pages that look like real dense-composite pages.
|
||||
|
||||
Typical trigger signals:
|
||||
Construction-time trigger signals:
|
||||
|
||||
1. page contains a formal numbered figure caption
|
||||
2. same local visual zone contains many fragments
|
||||
3. visual fragment count is high, e.g. `>= 4`
|
||||
4. unresolved clusters are already present, e.g. `>= 2`
|
||||
5. ordinary same-page ownership explains only part of the zone
|
||||
6. the candidate does not cross another numbered caption interval boundary
|
||||
4. the candidate does not cross another numbered caption interval boundary
|
||||
5. local fragments show compact visual structure rather than page-wide scatter
|
||||
|
||||
This is not page-wide mega-merge.
|
||||
This is scoped visual-parent candidacy.
|
||||
|
||||
Arbitration-time scoring signals may additionally use:
|
||||
|
||||
1. coverage gain over ordinary local ownership
|
||||
2. unresolved visual mass reduction
|
||||
3. penalty for partial local ownership that explains only a small subset of the zone
|
||||
4. sibling-caption conflict penalties
|
||||
|
||||
Rule:
|
||||
|
||||
```text
|
||||
Layer 2 construction may not depend on values that only exist after Layer 3/4 finalization.
|
||||
```
|
||||
|
||||
### 7.5 Dense parent candidate contract
|
||||
|
||||
Minimum suggested fields:
|
||||
|
|
@ -425,6 +538,20 @@ partial ordinary local ownership on dense page
|
|||
< strong dense parent candidate
|
||||
```
|
||||
|
||||
### 8.6 Relationship to existing roadmap
|
||||
|
||||
This convergence spec does **not** replace the already-approved visual-grammar hardening roadmap.
|
||||
|
||||
It acts as an architecture constraint layer for that roadmap.
|
||||
|
||||
Required relationship:
|
||||
|
||||
```text
|
||||
P0 close-out fixes still land first
|
||||
P1A diagnostic-only parent detection still lands before ownership-bearing parent arbitration
|
||||
P1B / P2 / P3 and later dense-page work must follow the convergence rules in this spec
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Layer 4 - Render and Accounting
|
||||
|
|
@ -446,7 +573,7 @@ Required distinction:
|
|||
|
||||
Shell outcomes must not be counted as strong matched ownership in health and audit summaries.
|
||||
|
||||
Suggested status label:
|
||||
Suggested shell label:
|
||||
|
||||
```text
|
||||
assetless_sequence_shell
|
||||
|
|
|
|||
Loading…
Reference in a new issue