obsidian-plugin/lllin000_PaperForge
1
0
Fork 0
mirror of https://github.com/lllin000/PaperForge.git synced 2026-07-22 17:00:23 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
1112 commits 4 branches 19 tags 51 MiB
Commit graph

49 commits

Author SHA1 Message Date
Research Assistant
94de2dbbc7 feat: persist role_span_profiles.json and wire into rebuild pipeline 2026-06-06 11:16:43 +08:00
Research Assistant
f9b5fb0541 feat(ocr): journal layout generalization — structured boundary, container regime, legend validation 
- Task 1: TailBoundary namedtuple with body_end/backmatter_start/references_start
- Task 2: _classify_backmatter_form — 'container' (PeerJ) vs 'flat' (Frontiers) regime
- Task 3: _attach_container_bodies, _container_child flag for ### child headings
- Task 4: Style-family classification already complete
- Task 6: _is_formal_legend threshold raised to 100, more axis_words added,
  low-confidence fallback with legend_uncertain flag for rejected captions
- 2GN9LMCW: authors correct, furniture suppressed, heading hierarchy,
  container child headings, Figure 4 uncertain
- 7C8829BD: 26/26 markers, correct tail order, no regression
- 296/296 tests pass
 2026-06-05 23:48:56 +08:00
Research Assistant
d52ea37e92 feat(ocr): tail regime remediation + style-aware heading profiles + boundary detection 
- Task: Bidirectional body/backmatter boundary detection (forward/backward spine + tail spread reconciliation)
- Task: Style-aware heading profiles from PDF span_metadata (extract, cluster, disambiguate)
- Task: Lock tests for tail-candidate overreach, cross-page continuation, style-aware heading detection
- 296/296 tests pass, real-paper 7C8829BD verified
 2026-06-05 22:47:13 +08:00
Research Assistant
38663913d5 fix: keep images/ for page render blocks, add path_map for compatibility migration 2026-06-05 15:27:43 +08:00
Research Assistant
332eafe4be fix: write page block images to assets/ primary, mirror to images/ compat 2026-06-05 15:25:54 +08:00
Research Assistant
8e0b4bf7c9 fix: sync images/ to assets/ for structured truth 2026-06-05 15:24:29 +08:00
Research Assistant
cff4107633 fix: stabilize object notes, render heading sanity, health/index detection 2026-06-05 15:13:33 +08:00
Research Assistant
0d9725808f fix: strengthen tests, clean object-note format, fix assets_path compat 2026-06-05 15:04:21 +08:00
Research Assistant
5d243c8e72 feat: add OCR role-based search index 2026-06-05 12:13:18 +08:00
Research Assistant
633a59f998 feat: classify OCR raw and derived version state 2026-06-05 11:14:24 +08:00
Research Assistant
5cfb58668d feat: add OCR structured health report 2026-06-05 10:26:42 +08:00
Research Assistant
15e2de1377 feat: render OCR fulltext from structured artifacts 2026-06-05 10:20:15 +08:00
Research Assistant
1de9cc281b feat: emit OCR figure and table objects 2026-06-05 00:43:39 +08:00
Research Assistant
cb79942044 feat: add OCR table inventory 2026-06-05 00:26:07 +08:00
Research Assistant
28f0772adc feat: add OCR figure inventory 2026-06-05 00:20:21 +08:00
Research Assistant
514c045e0a feat: add OCR resolved metadata artifact 2026-06-05 00:02:26 +08:00
Research Assistant
cfc8c973a7 refactor: preserve OCR compatibility while adding phase1 artifacts 
- Fix postprocess_ocr_result to use resolved vault path from paths dict
- Add test_sync_reads_enriched_meta_without_breaking_ocr_status
- Add test_postprocess_preserves_compatibility_outputs
- Fix test isolation: pre-import ocr module before mock patches
- Add vault key to mock paths in test_selection_sync_pdf
 2026-06-04 23:29:32 +08:00
Research Assistant
86569fe01c feat: emit OCR structured block artifacts 2026-06-04 23:20:49 +08:00
Research Assistant
b20d2c0cff feat: emit canonical OCR raw blocks 2026-06-04 23:16:10 +08:00
Research Assistant
d9168f81f9 feat: persist OCR raw metadata and version payloads 2026-06-04 23:12:31 +08:00
Research Assistant
2ab524aecf feat: ocr reading-order fixes, single-source fulltext, redo closed-loop, release notes & manual 2026-06-01 18:12:31 +08:00
Research Assistant
b240e27797 fix: reset ocr_redo as boolean false after redo 2026-06-01 13:51:37 +08:00
Research Assistant
48560df7b6 fix: consolidate ocr redo logic into worker.ocr_redo_papers with validation, --dry-run, and proper frontmatter updates 2026-06-01 12:52:09 +08:00
Research Assistant
82fffc6ffe fix: address code review issues — fulltext_md_path clear, key validation, redo no auto-run, ribbon button 2026-06-01 11:36:29 +08:00
Research Assistant
9c4d150da6 feat: add paperforge ocr redo command for re-running OCR on marked papers 2026-06-01 11:29:06 +08:00
Research Assistant
c1c5b9b8ba feat: bump PaddleOCR model from VL-1.5 to VL-1.6 2026-06-01 00:32:46 +08:00
Research Assistant
dde9c903fe feat: wire layered OCR pipeline into render_page_blocks + fixture-backed integration tests 2026-06-01 00:10:04 +08:00
Research Assistant
7406689746 feat: add OCR layered modules (roles, body spine, layout, attach) 2026-06-01 00:01:05 +08:00
Research Assistant
b58205940e fix(ocr): use block_order primary + bbox validation to fix column interleaving 
- block_sort_key() now sorts by block_order (from PaddleOCR API) as primary key,
  falling back to Y coordinate only when block_order is unavailable.
- Added validate_block_order() to verify sort results with bbox geometry:
  - Detects excessive column switches (more than 3) and falls back to
    column-major reading order (left col then right col, each sorted by Y).
  - Within each column, re-sorts blocks by Y if block_order places them
    in non-monotonic order.
- Fixes 2-column PDFs where text blocks from left and right columns were
  interleaved due to Y-coordinate sorting (affected 85% of OCR'd papers).
 2026-05-31 19:02:29 +08:00
Research Assistant
19c28a8b91 chore: commit session work before hotfix switch 2026-05-18 22:47:30 +08:00
Research Assistant
fe36dcfe10 fix: reset nopdf OCR status to pending when paper re-enters queue 2026-05-14 18:43:48 +08:00
Research Assistant
bae753aa5e fix(ocr): return real exit code, add real-time progress output to stdout, show Obsidian Notices 2026-05-12 00:55:30 +08:00
Research Assistant
290f67c6f4 fix: update ocr auto_analyze and ld_deep search for {key}.md filenames with fallback 2026-05-11 21:13:27 +08:00
Research Assistant
64e73067fb style: ruff format pass across paperforge/ 2026-05-09 22:31:08 +08:00
Research Assistant
dd71bb0f79 fix: v1.4.17rc2 — OCR quote fix, 4-stage lifecycle, .env fallback, Codex support, deep-reading anchors, auto-update pip 2026-05-08 01:26:04 +08:00
Research Assistant
e742df7888 fix(merge-gate): upgrade migration safeguards, index path integrity, and Python version alignment 
- migrate_to_workspace: preserve legacy flat notes even without canonical index
- migrate_to_workspace: reconcile legacy library-record do_ocr/analyze flags including false overrides
- migrate_to_workspace: handle non-canonical legacy filenames via frontmatter title
- asset_index: only advertise fulltext_path/deep_reading_path when files exist
- asset_index: prefer note frontmatter over legacy library-records, fall back to legacy
- setup_wizard: require Python >=3.10 instead of >=3.8
- plugin main.js: align Python version messaging to 3.10+
- ocr.py: auto_analyze_after_ocr reads from formal note paths, not library-records
 2026-05-07 21:51:58 +08:00
Research Assistant
b2e446eb21 fix: harden repair paths, slugify edge case, OCR meta resilience, venv python detection, and repair exit handling 2026-05-06 22:09:29 +08:00
Research Assistant
c412f82390 feat(23-03): switch OCR post-processing to incremental index refresh 
- Add refresh_index_entry import from asset_index module
- Replace full rebuild (_sync.run_index_refresh) with incremental refresh per completed OCR key
- Capture done keys before queue filter to preserve them for incremental refresh
- Add ImportError fallback to full rebuild for pre-migration safety
- When no OCR completed, still run full rebuild for sync changes
 2026-05-04 00:49:59 +08:00
Research Assistant
1af4924c52 chore: bump version to v1.4.13 2026-05-01 11:14:59 +08:00
Research Assistant
d06654ef70 fix: continuous OCR batch processing + real-time dashboard progress + fix loading flicker 2026-04-30 16:05:02 +08:00
Research Assistant
3aee514fd5 refactor: extract pipeline_paths into _utils.py — remove 7 copies, 3 of which were dead code 2026-04-28 23:42:00 +08:00
Research Assistant
61b72a467d refactor: extract load_domain_config into single _domain.py module — remove 7 copies, full rebuild on every call 2026-04-28 23:31:31 +08:00
Research Assistant
d82fc52521 feat: persistent OCR poll until completion; fix OpenCode commands for pip install; fix test hangs; add command_files to package 2026-04-27 22:52:03 +08:00
Research Assistant
93713be21e feat(UX-01): add auto_analyze_after_ocr config and OCR hook 
- Add auto_analyze_after_ocr: false to paperforge.json (opt-in, default off)
- Insert hook in ocr.py after ocr_status=done to set analyze:true when
  config is enabled
- Wrapped in try/except so a single failure does not abort OCR batch
- Uses read_json (already imported) and re.sub (already imported)
- Regex pattern matches existing frontmatter convention
 2026-04-27 17:57:33 +08:00
Research Assistant
7cccf4eefc refactor(17-dead-code-precommit): remove delegation wrappers and dead code, run ruff cleanup 
- Remove def load_vault_config delegation wrappers from all 7 worker modules
- Add from paperforge.config import load_vault_config, paperforge_paths at module level
- Remove unused system_dir/resources_dir/control_dir extraction in pipeline_paths
- Fix sync.py intra-function imports (run_selection_sync, run_index_refresh)
- Fix repair.py import path for _resolve_formal_note_path (direct from _utils)
- Add ruff and pre-commit as test dependencies in pyproject.toml
- Configure per-file ruff ignores for pre-existing issues
- Run ruff check --fix + ruff format (353 + 58 issues auto-fixed)
- All 203 tests pass, ruff check zero warnings
 2026-04-27 17:40:47 +08:00
Research Assistant
c08f86de0f feat(17-dead-code-precommit): add ruff config, pre-commit hooks, Check 5, and OCR error context 
- Add [tool.ruff] section to pyproject.toml (target-version py310, rules E/F/I/UP/B/SIM)
- Create .pre-commit-config.yaml with 7 hooks (ruff lint+format, check-yaml, check-toml, end-of-file-fixer, trailing-whitespace, consistency-audit)
- Add Check 5 (check_duplicate_utils) to scripts/consistency_audit.py
- Add library_record field to OCR error meta.json (4 error paths in ocr.py)
 2026-04-27 17:29:13 +08:00
Research Assistant
00692ef667 feat(13-logging-foundation): add verbose param to worker function signatures 
- run_selection_sync(vault) -> run_selection_sync(vault, verbose=False)
- run_index_refresh(vault) -> run_index_refresh(vault, verbose=False)
- run_ocr(vault) -> run_ocr(vault, verbose=False)
- run_doctor(vault) -> run_doctor(vault, verbose=False)
- run_status(vault) -> run_status(vault, verbose=False)
- Backward compatible: default False preserves existing call sites
 2026-04-27 15:15:43 +08:00
Research Assistant
e0dbfeccec feat(13-logging-foundation): add module-level logger to all 12 worker and command modules 
- Added import logging and logger = logging.getLogger(__name__) to:
  worker: repair, deep_reading, ocr, sync, status, base_views, update
  commands: ocr, repair, deep, sync, status
- 12 of 13 planned files completed (paperforge/update.py does not exist on disk - skipped)
- Verified all logger instances resolve to correct hierarchical names
 2026-04-27 15:07:32 +08:00
Research Assistant
498a9edfe5 feat(phase-12): extract worker modules from literature_pipeline.py 
- Create paperforge/worker/ package with sync.py, ocr.py, repair.py,
  status.py, deep_reading.py, update.py, base_views.py
- Update paperforge/commands/*.py to import from new worker modules
- Update paperforge/cli.py _import_worker_functions for new paths
- Update paperforge/config.py ld_deep_script resolution with fallback
- Update setup_wizard.py for new worker/skills paths
- Add function-level imports to break circular dependencies
 2026-04-24 22:32:25 +08:00