obsidian-plugin/lllin000_PaperForge
1
0
Fork 0
mirror of https://github.com/lllin000/PaperForge.git synced 2026-07-22 06:50:53 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1538 commits 4 branches 19 tags 51 MiB
Commit graph

138 commits

Author SHA1 Message Date
Research Assistant
b1fdbc30eb feat: P0 fixes (ref sort, caption insert) + P1 figure containment render hygiene 
P0:
- Fix 2: _ref_number_sort_key handles [N] bracket format (ocr_render.py:481)
- Fix 4: remove figure_caption from _INSERT_CANDIDATE_ROLES (ocr_document.py:3846)
- Fix 5: filter demoted body_paragraph from figure legends (ocr_figures.py:2950)

P1:
- figure containment render-hygiene pass after table inventory writeback
- 6 helpers: _cluster_bboxes_by_proximity, _is_contained,
  _highly_overlaps_any_matched_region, _figure_region_bbox,
  _matched_asset_keys, tag_figure_contained_text
- Wire call site in postprocess_ocr_result() (ocr.py:1891)

Infra:
- Sync workspace fulltext after rebuild (asset_index.py)
- Fix UnboundLocalError on index-refresh (sync_service.py)
- Add missing frontmatter fields to field registry (field_registry.yaml)
- 283 + 19 = 302 tests passing
 2026-06-27 22:56:22 +08:00
Research Assistant
653d0925c6 docs: apply 4 must-fix review changes to container admission spec+plan 
1. _has_container_text() gets pdf_page param for real _map_ocr_bbox_to_pdf_rect()
2. Call site rewritten as per-page three-phase loop (clear/ compute/ mark)
3. page-sized/crop-like features excluded from grouping_pool entirely
4. Merged rects replace children; accepted rects sorted by area descending
+ fix Stage Boundary count (5->9 helpers)
+ add 11th test for non-identity coordinate mapping
+ add _bbox_overlap_ratio zero-area/empty handling
+ add yellow flag note for black-border plot frame risk
 2026-06-26 22:25:23 +08:00
Research Assistant
9b07d68c10 docs: revise figure-number-inference plan with 9 review fixes 
- Add supersede note for spec's old integration wording
- Replace _FIGURE_NUMBER_PATTERN with new _FIGURE_MARKER_PATTERN regex (captures S prefix)
- Add _coerce_int_figure_number() helper (handles int/float/str from JSON)
- Add duplicate_known_main_numbers skip reason
- _FRONTMATTER_VISUAL_VETO: remove bare 'toc' substring risk
- Add _has_frontmatter_visual_veto() helper with word-boundary match
- Noise check: resolvable order key failure strategy
- Stage Boundary: include inventory[figure_number_inference] in mutation scope
- Use _format_figure_id() not literal string
- Add test_infer_duplicate_known_numbers_skips test case
- Frontmatter veto test: veto item must otherwise be eligible
 2026-06-26 20:29:15 +08:00
Research Assistant
c9ca8e2657 docs+fix: evidence-driven container admission + figure-number-inference spec/plan 
_extract_visual_container_rects:
- Replace flat width<100/height<50 gate with evidence-driven admission
- Add line-like veto (<=3pt, pure decorative separators)
- Add page-sized veto (>=60% page + near full edge = background/clip)
- Add vertical component merge (same x-range, gap <=5pt)
- Keep existing fill/border/thin-gray filter logic

spec: 2026-06-26-figure-number-fallback-by-sequence-design
- Fix integration point (after _promote_sequence_matches)
- Add 3-tier legend_bbox lookup
- Add _FRONTMATTER_VISUAL_VETO keyword list
- Add structured _extract_figure_marker() return
- Add skip-reason contract to inventory

plan: 2026-06-26-figure-number-inference-implementation
- 6 tasks, ~95min, single-commit PR
 2026-06-26 20:22:43 +08:00
Research Assistant
b4152abe10 A2: check table ownership (consumed_table_block_keys) before _SKIPPED_BODY_ROLES in render loop 
Moves the block_id fetch and consumed_table_block_keys check before
_SKIPPED_BODY_ROLES so table note removal is a contract-based decision
(ownership) not a role-based accident (footnote role).

Part of table ownership contract refactor.
 2026-06-25 22:27:35 +08:00
Research Assistant
d018a68fd6 fix: harden early OCR frontmatter-side zone inference 2026-06-25 15:57:53 +08:00
Research Assistant
1f7bd743ce fix: close out OCR merge-gate figure blockers 2026-06-24 01:09:38 +08:00
Research Assistant
147f5b77e1 docs: add convergence architecture spec/plans and dense-parent hardening spec/plan 2026-06-23 14:29:18 +08:00
Research Assistant
fb28eca1fe fix: converge figure ownership arbitration and add VFS8CBW2 dense-composite diagnostics 2026-06-23 02:42:22 +08:00
Research Assistant
224882f697 fix: govern local figure pairing and ownership fallbacks 2026-06-22 22:15:39 +08:00
Research Assistant
410151ea8e fix: reuse rebuild metadata and object extraction work 2026-06-22 22:15:25 +08:00
Research Assistant
5bbe40d08b fix: reject weak isolated layout clusters 2026-06-22 22:15:12 +08:00
Research Assistant
6522dc9aef docs: update PROJECT-MANAGEMENT with 2HEUD5P9 rebuild findings 2026-06-22 02:50:37 +08:00
Research Assistant
db69961c7c feat: convert body-zone footnotes with distinct font to callout 2026-06-22 01:51:10 +08:00
Research Assistant
69f6a82b52 docs: update PROJECT-MANAGEMENT with figure merge refactor + heading/backmatter fixes + cover page plan 2026-06-22 01:44:52 +08:00
Research Assistant
4cd0cf3d54 fix: heading detection, backmatter, table consumed, footnote reorder, figure crop + audit fixtures 2026-06-22 01:28:55 +08:00
Research Assistant
da2e018b19 feat: figure merge clustering spec + fixes for caption dedup and structural gate 
- Add global-distance-clustering figure merge design spec
- Fix ocr_roles.py: body_zone body_paragraph no longer promoted to
  figure_caption_candidate (prevents body mentions from entering legend pipeline)
- Fix ocr_document.py: preserve subsection_heading, reference_item,
  table_caption from structural-gate demotion
- Update PROJECT-MANAGEMENT.md with edge-case analysis
- Add ocr-vision-audit-master.md for CODE vs VISION separation
- Update SKILL.md dispatch rules with trust boundaries
 2026-06-21 21:32:30 +08:00
Research Assistant
70a7978de8 feat: complete OCR-v2 readiness gates + blind audit validation 
- Gate 1: completeness signals for page text coverage
- Gate 2: DW figure ownership generalization with strict regression
- Gate 3: reference boundary enforcement from structure
- Gate 4: layout coverage taxonomy in audit/coverage_ledger.json
- diff_audit.py: canonical role alias mapping + auto-normalize truth_role
- ocr_structural_gate.py: table caption text-evidence fallback
- ocr-canonical-roles.md: canonical role list for audit workflow
- ocr-truth-audit.md: pre-flight checklist enforces canonical roles
- Blind audit: 5 unseen papers across domains, all PASS with coverage
 2026-06-19 00:35:21 +08:00
Research Assistant
7e0b69b3ff docs: add OCR real-paper regression design 2026-06-14 21:13:39 +08:00
Research Assistant
123234b93b docs: add OCR-v2 closure gap remediation plan 2026-06-14 11:41:05 +08:00
Research Assistant
711bb3c86a docs: add OCR real-paper regression harness implementation plan 2026-06-13 15:22:45 +08:00
Research Assistant
c65ff39c4d docs: add fixture layout, debug bundle, segment-order, and consumption refinements 2026-06-13 15:20:17 +08:00
Research Assistant
9cb8fb7b04 docs: apply 6 spec refinements for agent execution readiness 2026-06-13 15:16:54 +08:00
Research Assistant
df82820e66 docs: add consumption assertion field to fixture schema 2026-06-13 15:11:45 +08:00
Research Assistant
da00c083e6 docs: add audited OCR-v2 failure families 2026-06-13 15:04:02 +08:00
Research Assistant
e1ecf125ce docs: add OCR production-path regression design 2026-06-13 14:40:15 +08:00
Research Assistant
d9f76917e9 style: ruff format after OCR role gate implementation 2026-06-12 15:11:33 +08:00
Research Assistant
03e3966183 docs: document OCR structural role gate 2026-06-12 15:09:59 +08:00
Research Assistant
eb3b73ae78 docs: prune superseded OCR worktree plans 2026-06-09 12:55:05 +08:00
Research Assistant
b33ac572e4 docs: add real-paper audit report — pre-proof, author matching, inner text, ambiguous figures 2026-06-08 18:47:26 +08:00
Research Assistant
0aab860cf2 feat: OCR risk convergence — scored figure/table matching, layout confidence, insert scorer, health uncertainty metrics 
- Figure matching: nearest-asset → scored multi-candidate selection with ambiguous state
- Table matching: vertical-nearest → score-based selection with prev/next page window
- Layout confidence: PageLayoutProfile gains confidence/evidence; eligible-body block filter
- Tail reorder: guarded by layout + tail confidence (low confidence preserves raw order)
- Structured insert: direct promotion → scorer + candidate + fallback state
- Health: adds 14 uncertainty metrics (ambiguous matches, unresolved clusters, forced inserts, low tail confidence)
- Audit: docs/ocr/high-risk-rule-audit.md documents 8 hard-rule decisions mapped to plans
- Spec + 6 implementation plans for agentic execution
 2026-06-08 13:43:48 +08:00
Research Assistant
acd28d321e test: make OCR real-paper regressions reproducible 2026-06-08 12:13:11 +08:00
Research Assistant
a3c8dab389 docs: audit OCR production route and mark experimental modules 2026-06-08 12:13:07 +08:00
Research Assistant
c974ba7480 fix: remove open-ended authors text heuristics, anchor to page-1 zone only 2026-06-06 13:28:46 +08:00
Research Assistant
43087401c1 docs: mark completed tasks in document analysis plan, keep only delta tasks 2026-06-06 12:23:08 +08:00
Research Assistant
687332d126 docs: correct span_metadata source to PDF (not PaddleOCR), revise plan 2026-06-06 11:42:14 +08:00
Research Assistant
bdb3eae8ac docs: add unified span metadata spec and integration plan 2026-06-06 11:06:39 +08:00
Research Assistant
7c278fbc60 fix(ocr): properly cluster multi-panel figures — role assignment, cluster bbox crop, render visibility 2026-06-06 10:08:42 +08:00
Research Assistant
c235dbdc68 feat(ocr): journal layout generalization, OCR diagnostics, and debug scripts 2026-06-06 00:46:49 +08:00
Research Assistant
f9b5fb0541 feat(ocr): journal layout generalization — structured boundary, container regime, legend validation 
- Task 1: TailBoundary namedtuple with body_end/backmatter_start/references_start
- Task 2: _classify_backmatter_form — 'container' (PeerJ) vs 'flat' (Frontiers) regime
- Task 3: _attach_container_bodies, _container_child flag for ### child headings
- Task 4: Style-family classification already complete
- Task 6: _is_formal_legend threshold raised to 100, more axis_words added,
  low-confidence fallback with legend_uncertain flag for rejected captions
- 2GN9LMCW: authors correct, furniture suppressed, heading hierarchy,
  container child headings, Figure 4 uncertain
- 7C8829BD: 26/26 markers, correct tail order, no regression
- 296/296 tests pass
 2026-06-05 23:48:56 +08:00
Research Assistant
d52ea37e92 feat(ocr): tail regime remediation + style-aware heading profiles + boundary detection 
- Task: Bidirectional body/backmatter boundary detection (forward/backward spine + tail spread reconciliation)
- Task: Style-aware heading profiles from PDF span_metadata (extract, cluster, disambiguate)
- Task: Lock tests for tail-candidate overreach, cross-page continuation, style-aware heading detection
- 296/296 tests pass, real-paper 7C8829BD verified
 2026-06-05 22:47:13 +08:00
Research Assistant
1779b8b0ff docs: add legacy OCR backfill and sync integration plan 2026-06-05 12:49:01 +08:00
Research Assistant
5f31d2924f docs: add OCR phase4.1 cleanup and phase5 plan 2026-06-05 12:00:22 +08:00
Research Assistant
f3fe86038e docs: add OCR structured pipeline phase4 plan 2026-06-05 10:56:54 +08:00
Research Assistant
1963a091fc docs: add OCR structured pipeline phase3 plan 2026-06-05 01:01:50 +08:00
Research Assistant
cca983d517 docs: add OCR structured pipeline phase2 plan 2026-06-04 23:44:12 +08:00
Research Assistant
feb6129cd2 docs: add OCR structured pipeline phase1 plan 2026-06-04 22:49:01 +08:00
Research Assistant
bef8360cbd docs: add OCR structured pipeline design spec 2026-06-04 22:41:52 +08:00
Research Assistant
2ab524aecf feat: ocr reading-order fixes, single-source fulltext, redo closed-loop, release notes & manual 2026-06-01 18:12:31 +08:00
Research Assistant
5abdd51ca7 docs: describe layered OCR reading-order pipeline 2026-06-01 00:02:45 +08:00