anthonyfitzpatrick_manuscript-compiler/DEVELOPER_GUIDE.md

Manuscript Compiler Developer Guide

Repository overview

Manuscript Compiler is a mobile-safe Obsidian Community Plugin that turns a reviewed vault structure into six native, in-memory export formats. The repository deliberately separates physical vault discovery, author-controlled structure, semantic manuscript construction, export projection, byte generation, validation, delivery, and history.

The central maintenance rule is:

The prepared semantic Book is the only manuscript source for preview, validation, and export.

No exporter may scan the vault, reread notes, rerun cleaning, infer structure, or rebuild order. No delivery code may write a manuscript export into the vault or use Node/Electron filesystem APIs.

Start with ARCHITECTURE.md, then read src/compile-preparation.ts, src/semantic-document.ts, src/export-coordinator.ts, and src/workspace/compile-workspace-controller.ts.

Prerequisites

• Node.js 24, matching CI.
• npm with lockfile support.
• A development Obsidian vault for manual checks.
• Target applications for release interoperability checks: Word/Vellum where available, LibreOffice, EPUB readers, browsers, text editors, and an XML-aware tool.

Install exactly the locked development graph:

npm ci

fflate is the only production dependency. New runtime packages require exceptional justification, a security/license review, updated third-party notices, and explicit product approval.

Building and running

Development watch build:

npm run dev

Production build:

npm run build

The build writes the ignored main.js bundle. For local Obsidian testing, this repository can itself be the vault plugin directory; reload the plugin after rebuilding. Do not add hot-reload, network, or another-plugin dependencies merely for development convenience.

Type checking and linting

npm run typecheck
npm run lint

TypeScript is strict. The official Obsidian ESLint plugin is part of the development toolchain and must report zero errors and zero warnings. Do not suppress a lint rule without documenting why the repository cannot comply through supported Obsidian APIs.

Test suites

npm test
npm run test:docx
npm run test:odt
npm run test:epub
npm run test:html
npm run test:markdown
npm run test:xml
npm run test:exports
npm run benchmark:large

• tests/run.ts covers detection, parsing, cleaning, migration, workspace state, privacy, accessibility-oriented source assertions, and repository hygiene.
• tests/docx-integration.ts inspects generated WordprocessingML and protects Vellum-oriented semantics.
• tests/exports.ts exercises all exporter/validator registries, complete generated packages, browser download cleanup, deterministic output, and forbidden content.
• tests/large-manuscript-benchmark.ts prepares one large Book and measures all six exporters without a strict machine-specific threshold.
• tests/golden/ contains deterministic Markdown expectations for sample structures.
• .test-build/export-artifacts/ contains ignored manual-inspection artifacts generated by export tests. They must never be packaged or committed.

Behavioral changes require focused regression coverage plus the relevant combined suite. Documentation-only changes still run typecheck, lint, main tests, build, and git diff --check at minimum.

Architecture summary

Obsidian folder
  → mechanical VaultScanner
  → inferred/corrected ContentPlan
  → ManuscriptParser + ContentCleaningPipeline
  → semantic Book
  → PreparedCompileSession + fingerprints
  → SemanticDocument
  → selected exporter
  → selected validator
  → BrowserDownloadService
  → bounded privacy-safe history

Important distinctions:

• ScannedBook describes physical files and folders. It is never exportable.
• ContentPlanItem[] captures detection plus explicit author corrections.
• Book is the publication model and source of truth.
• PreparedCompileSession binds that Book to source and input fingerprints.
• SemanticDocument is a format-oriented projection, not another parser or Book model.
• GeneratedExport contains validated in-memory bytes and no filesystem path.

Adding an exporter

Adding a format is a product change, not a local class addition. Preserve this sequence:

1. Add the lowercase format to ExportFormat/EXPORT_FORMATS in src/export-types.ts.
2. Add label, extension, exact MIME type, and description to EXPORT_FORMAT_DETAILS.
3. Implement ManuscriptExporter using only ManuscriptExportContext.document, resolved formatting options, and portable filename.
4. Register exactly one exporter in EXPORTERS.
5. Add a structural validator and register it exhaustively in EXPORT_VALIDATORS.
6. Add the format card and only controls with a meaningful effect.
7. Update filename correction, history types if required, test commands, combined suites, benchmark, documentation, and packaging assertions.
8. Prove browser delivery occurs once and no vault write occurs.

Never give an exporter Vault, TFile, TFolder, parser, scanner, or source paths. Never invoke a converter or external executable. ZIP formats must use controlled entry paths and the existing fflate dependency.

Adding a validator

Validators receive generated bytes and, when needed, the same export context. They run before download dispatch and return deterministic error messages. A validator should:

• reject empty or malformed output;
• check the minimum package/schema invariants promised by the product;
• verify forbidden manuscript content where appropriate;
• avoid claiming full standards conformance when it performs structural checks only;
• avoid repairing bytes or changing output;
• never read the vault or call the network.

A validation failure must prevent download and must be recorded as a failed operation, not a successful export.

Adding settings

Use one settings path:

1. Add the field to the appropriate interface in src/settings.ts.
2. Add a safe default.
3. Repair missing/malformed persisted values in repairSettings.
4. Migrate historical schema only when the old representation requires translation.
5. Preserve explicit user values and make migration idempotent.
6. Resolve the value into the existing request/profile/formatting object.
7. Include semantic settings in the compile-input signature so stale previews are invalidated.
8. Persist through the existing coordinator/controller boundary.
9. Add first-pass and second-pass migration tests.

Historical compatibility fields can remain storage-only. Never reactivate obsolete vault-output, external-executable, or conversion fields.

Adding a migration

Migration and repair are different:

• Migration translates a known older schema while preserving meaning.
• Repair treats persisted data as untrusted and restores bounded, type-safe current values.

Both must be idempotent. Test missing values, malformed values, explicit false/zero values, nested arrays/objects, and a second pass whose serialized result is unchanged. Never reset custom formatting merely because a new default exists.

Adding or changing workflow pages

The authoritative workspace has exactly three stages: Manuscript, Contents, Create file. CompileWorkspaceController owns mutable state and operations; step modules render DOM and call controller methods.

When adding a control:

• use documented Obsidian DOM/Setting APIs;
• use sentence-case labels;
• retain native keyboard behavior and visible focus;
• use scoped classes in styles.css, not inline UI styles;
• ensure narrow panes and mobile layouts reflow;
• invalidate preparation only if the value affects semantic output;
• preserve focus/scroll for local tree edits;
• hide controls that cannot affect the selected format.

Do not put scanning, parsing, export generation, history, or download code into a step renderer.

State, cancellation, and stale previews

OperationStateController permits one active preparation/export operation. Cancellation uses AbortSignal checkpoints and the recognisable CompilationCancelledError. Once download finalisation begins, cancellation is locked because dispatch cannot be truthfully rolled back.

CompileWorkspaceController deduplicates concurrent preparation/export promises. Mutations affecting the semantic result clear the prepared session. Filename and selected-format changes do not rebuild the Book. Before export, ExportCoordinator recalculates source and input fingerprints; mismatches block output and require refresh.

Never return a partial Book after cancellation. Always release operation state in finally paths. Never record cancellation as failure or success.

Debugging

1. Reproduce with the smallest sample under samples/ or a privacy-safe fixture.
2. Identify the boundary where observed state first differs: scan, plan, Book, SemanticDocument, bytes, validation, or delivery.
3. Inspect prepared statistics and warnings before inspecting exporter bytes.
4. For ZIP formats, unzip test bytes in memory with fflate and inspect the referenced content/style files together.
5. For stale-preview issues, compare source paths, source fingerprints, input signatures, and controller invalidation calls.
6. For UI issues, test keyboard navigation, focus restoration, a narrow pane, popout ownership, and mobile-safe loading.
7. Use the redacted diagnostics generator for support information; never log manuscript prose or raw metadata.

Do not add temporary network logging, telemetry, filesystem exports, or unsafe innerHTML. Remove any local debugging statements before review.

Security model

The plugin is offline and local:

• note reads use Obsidian Vault APIs;
• manuscript generation occurs in memory;
• delivery uses Blob/object URL and one temporary anchor;
• no destination path is observed or persisted;
• no manuscript export is written into the vault;
• no Electron, Node filesystem, shell, child_process, external process, remote asset, or network request is allowed;
• diagnostics and histories exclude prose, absolute paths, metadata values, Blob URLs, and environment details.

See SECURITY.md for reporting and user-facing guarantees.

Performance expectations

Preparation is linear in discovered manuscript size within bounded concurrent note reads. Exporters consume the same prepared Book and SemanticDocument; they must not repeat parsing. Avoid nested full-document searches inside per-paragraph loops. Benchmarks are informational across machines, but large-manuscript generation must complete without runaway memory, repeated scanning, or quadratic behavior.

If a change introduces caching, document ownership and invalidation. Never cache raw manuscript prose in persistent settings or diagnostics.

Coding and documentation standards

• Explain architectural purpose, ownership, invariants, failure behavior, cancellation, and privacy—not syntax.
• Add JSDoc to exported classes and functions, including side effects and assumptions.
• Keep internal comments for non-obvious algorithms, transforms, or state transitions.
• Prefer small pure helpers around security-sensitive escaping and repair.
• Use explicit exhaustive registries for formats.
• Preserve deterministic output and exactly defined MIME/extension behavior.
• Avoid any; narrow unknown persisted/error input defensively.
• Do not use promotional, conversational, or filler wording in code comments.

Obsidian standards

• Register commands, events, and views through documented lifecycle APIs.
• Avoid global app; use injected/owned application references.
• Use registerEvent, registerDomEvent, or component-owned event lifecycles where applicable.
• Scope CSS to plugin roots and retain theme variables, focus-visible, reduced-motion, and high-contrast behavior.
• Keep UI sentence case and accessible.
• Keep manifest.json identity and minimum version stable unless the product release explicitly changes them.
• Release exactly main.js, manifest.json, and styles.css.

CI and release process

CI installs with npm ci and runs repository checks from .github/workflows/ci.yml. Before release, run the complete matrix in RELEASE_READINESS.md, then complete every applicable unchecked manual gate in MANUAL_TESTING.md.

Packaging:

npm run package
npm run package:validate

The ZIP is optional; attach main.js, manifest.json, and styles.css individually to the GitHub release. The tag must exactly match manifest.json without a v prefix. Do not package tests, fixtures, source, diagnostics, data files, or inspection artifacts.

Common mistakes

• Exporting from ScannedBook instead of the prepared Book.
• Re-reading notes inside an exporter.
• Treating the selected root or transparent containers as structural headings.
• Rebuilding preparation when only format or filename changes.
• Forgetting to invalidate preparation after a semantic setting changes.
• Adding a format to the UI but not both exhaustive registries.
• Defining a bold ODT style that generated paragraphs do not reference.
• Styling generic HTML headings instead of manuscript-specific classes.
• Simulating Markdown indentation with spaces or HTML.
• Adding presentation attributes to XML.
• Recording a successful history entry before download dispatch starts.
• Persisting absolute paths, warning details, or manuscript text.
• Adding an inert control to a format where it has no meaning.

Extension points

Safe extension points are intentionally narrow:

• new pure content-plan inference rules with fixtures;
• new cleaning rules limited to structured authoring syntax;
• new SemanticDocument block presentation mappings;
• new validators that do not mutate bytes;
• new view-model projections that do not own state;
• new documentation and manual interoperability matrices.

Any new semantic structure belongs in the Book model only through an explicit product design, migration plan, and cross-format contract. Do not smuggle new structure into one exporter.

Future roadmap

The plugin is feature complete. Appropriate future work is maintenance-led:

• track Obsidian API and lint changes;
• expand real-application interoperability records;
• add privacy-safe fixtures for newly observed vault layouts;
• improve accessibility based on verified testing;
• tighten validators when a real malformed-output case is identified;
• profile large manuscripts when a measured regression appears.

Future work must preserve offline operation, one semantic Book, one prepared-session architecture, browser delivery, deterministic exports, and the runtime dependency policy.