Contributing to PaperForge

Thank you for your interest in contributing to PaperForge! This document provides setup instructions, workflow guidelines, and code conventions to help you get started.

Development Setup

# Clone the repository
git clone https://github.com/LLLin000/PaperForge.git
cd PaperForge

# Editable install with test dependencies
pip install -e ".[test]"

# Verify installation
python -m paperforge --version

Requirements

Python 3.10+
Zotero + Better BibTeX plugin (for end-to-end testing)
Obsidian (for Base view generation)
PaddleOCR API Key (for OCR pipeline — configure in .env)

Pre-commit Hooks

Before committing, install the pre-commit hooks to ensure code quality:

pip install pre-commit
pre-commit install

Hooks are configured in .pre-commit-config.yaml and include:

Ruff lint (ruff check --fix) — catches common errors and style issues
Ruff format (ruff format) — enforces consistent code formatting
check-yaml — validates YAML file syntax
check-toml — validates TOML file syntax
end-of-file-fixer — ensures files end with a single newline
trailing-whitespace-fixer — removes trailing whitespace
Consistency audit (scripts/consistency_audit.py) — detects duplicate utility functions across modules

Note: Pre-commit hooks run automatically on git commit. Do not bypass them. If a hook fails, fix the issue and re-commit. The Ruff configuration in pyproject.toml suppresses E501 (line-too-long) and pre-existing simplifications via per-file-ignores — these are not blockers.

Test Workflow

# Run all tests, stop on first failure
pytest tests/ -x

# Verbose output
pytest tests/ -v

# Filter by keyword (e.g., OCR-related tests)
pytest tests/ -k "ocr"

# Run with coverage report
pytest tests/ --cov

# Collect tests only (no execution) — useful for import validation
pytest tests/ --collect-only

Test Requirements

All tests must pass before merging. The current baseline is 203+ tests, 0 failures, 0 errors (2 pre-existing skips for PDF-dependent tests).
Lint must pass: ruff check . && ruff format --check .
Do not introduce new test failures or lint violations.
Tests must not make live API calls (PaddleOCR, etc.). Use sandbox fixtures in tests/sandbox/.

Test Structure

Tests are in tests/ with a shared conftest.py providing fixtures. The tests/sandbox/ directory contains generated deterministic fixtures for OCR, Zotero exports, and vault layouts. Fixture generation scripts are in tests/sandbox/generate_*.py — run these only when fixtures need updating.

Architecture Overview

PaperForge uses a two-layer design:

Worker Layer (automated, deterministic, CLI-triggered)
    paperforge/worker/
    ├── _utils.py        — Leaf module: shared utilities (read_json, write_json, etc.)
    ├── sync.py          — Zotero sync (selection-sync, index-refresh)
    ├── ocr.py           — OCR pipeline (upload, poll, postprocess)
    ├── repair.py        — State and path repair
    ├── status.py        — Doctor and status commands
    ├── deep_reading.py  — Deep-reading queue management
    ├── base_views.py    — Obsidian Base view generation
    ├── update.py        — Auto-update system
    ├── _progress.py     — Progress bar utilities
    └── _retry.py        — Retry with backoff utilities

Agent Layer (interactive, reasoning-driven, user-triggered)
    paperforge/skills/literature-qa/
    ├── scripts/ld_deep.py      — /pf-deep implementation
    ├── prompt_deep_subagent.md  — Agent prompt for deep reading
    └── chart-reading/           — 14 chart-type review guides

Key Architecture Rules

_utils.py is a leaf module — imports only from stdlib and paperforge/config.py. Must never import from paperforge.worker.* or paperforge.commands.*. This prevents circular dependencies.
Worker modules do not import each other directly — use _sync.run_selection_sync(vault) (module-reference pattern) in ocr.py to break the sync↔ocr cycle.
Config flows one way: paperforge/config.py → _utils.py → worker modules. No reverse imports.
Re-exports preserve backward compatibility: Moved functions retain # Re-exported from _utils.py comments in original modules.

Key Files

File	Purpose
`paperforge/config.py`	Path resolution and config loading
`paperforge/cli.py`	CLI entry point and command routing
`paperforge/__main__.py`	`python -m paperforge` entry point
`paperforge.json`	User-facing configuration (paths, update settings, feature flags)
`setup_wizard.py`	Interactive first-time setup
`scripts/consistency_audit.py`	Duplicate utility function detection

Code Conventions

No new print() calls in worker modules — use logger = logging.getLogger(__name__) with logger.info(), logger.debug(), etc. print() is reserved for user-facing stdout output in CLI commands (paperforge/commands/).
Import shared utilities from _utils.py — do not copy-paste read_json, write_json, load_journal_db, etc. into new modules. If you need a utility, add it to _utils.py first.
_utils.py is a leaf module — must never import from paperforge.worker.* or paperforge.commands.*. Violations cause circular import errors.
Requirement IDs in commit messages — link commits to requirements using the format: feat(UX-01): add auto_analyze_after_ocr. Requirement IDs are tracked in REQUIREMENTS.md and .planning/.
Pre-commit hooks run on git commit — do not bypass them with --no-verify. Fix hook failures rather than skipping them.
Forward slashes in paths — use Path.as_posix() for all path-to-string conversions. This ensures consistent wikilinks regardless of OS.
Structured logging — use logging.getLogger(__name__) at module level. Log levels: DEBUG for detailed diagnostics, INFO for normal operations, WARNING for recoverable issues, ERROR for failures.
No live API calls in tests — all tests must use deterministic sandbox fixtures. Mock external services where necessary.
Type hints — use from __future__ import annotations at the top of new modules for PEP 604-style type hints.
Backward compatibility — API breaks are unacceptable within a major version. Use function-level imports and re-exports when refactoring.

Pull Request Process

Ensure all tests pass and lint is clean.
Update CHANGELOG.md under [Unreleased] with the changes.
Update documentation (AGENTS.md, command docs) if user-facing behavior changes.
Create a pull request with a clear description of changes and motivation.
Reference requirement IDs where applicable.

Getting Help

Open an issue on GitHub: https://github.com/LLLin000/PaperForge/issues
Run paperforge doctor for system diagnostics
Review docs/INSTALLATION.md and docs/ARCHITECTURE.md for detailed setup and design docs

Thank you for contributing to PaperForge — Building a better literature workflow, underground!

7.2 KiB Raw Blame History