callumalpass_tasknotes/docs/development/data-flow.md

# TaskNotes Data Flow

This document records the intended data-flow model for the refactor.

## Source of Truth

Task data is stored in Markdown files with frontmatter inside the user vault.

Primary source of truth:

- markdown file content
- Obsidian metadata cache for indexed read access

TaskNotes should avoid introducing a competing internal database for core task state.

## Settings Startup Path

1. `TaskNotesPlugin` asks `settingsPersistence` to read plugin `data.json`.
2. If Obsidian returns `null` while the existing data file is present, the read
   is retried before defaults are trusted.
3. A compromised startup read blocks settings saves for that session so
   existing user configuration is not overwritten by in-memory defaults.
4. Legacy settings and nested defaults are normalized in
   `settingsPersistence` before `TaskNotesPlugin.settings` is assigned.
5. Settings-only saves write the known settings keys back to plugin data while
   preserving other persisted plugin state already in `data.json`.

## Read Path

1. Vault files change.
2. Obsidian metadata cache updates.
3. `TaskManager` and related adapters read task state from metadata cache.
4. Task frontmatter is identified through `taskIdentification`, then mapped
   task data is completed through `taskInfoAssembly`.
5. Filter/query behavior receives user-field settings, i18n, link resolution,
   and subtask lookup through the `FilterService` runtime contract rather than
   reaching for the full plugin object.
6. Query predicate evaluation runs through `filterPredicateEvaluation`, which
   owns recursive group semantics, user-field coercion, project-link matching,
   `hasSubtasks`, and `status.isCompleted` decisions.
7. Domain services and views consume normalized `TaskInfo` representations.

## Write Path

1. A view, modal, command, or API endpoint requests a task mutation.
2. The request flows through `TaskService` or a dedicated mutation collaborator.
3. Single-property updates are normalized and planned through
   `taskPropertyUpdate` before frontmatter is written.
4. Bulk task updates are normalized and planned through `taskUpdatePlanning`
   before mapped frontmatter or body writes happen.
5. Frontmatter/body changes are written to the markdown file.
6. `taskPropertyChangeSideEffects` refreshes the task cache, emits task update
   events, and runs webhook, calendar-sync, and auto-archive side effects.
7. Obsidian metadata cache emits update events.
8. Event listeners invalidate caches and refresh views.

Task creation payload assembly should happen before the write boundary:

- `TaskCreationService` and `TaskUpdateService` receive narrow runtime
  contracts, not the full plugin type, so the task write path documents which
  settings, app, cache, event, mapper, and sync capabilities it can use.
- Bases view filters are converted into frontmatter defaults through
  `basesFilterDefaults` before the creation payload is assembled.
- Bases `New` frontmatter is normalized through `basesTaskCreation`.
- Kanban column and swimlane creation defaults are planned through
  `kanbanCreationDefaults` before the creation payload is assembled.
- Modal user fields are normalized through `taskModalUserFields`.
- Modal creation defaults and pre-populated values are normalized through
  `taskCreationFormState` before `TaskCreationModal` renders or saves.
- Modal post-save subtask project writes are planned through
  `taskCreationSubtasks`.
- Modal edit defaults and cached user-field values are normalized through
  `taskEditFormState` before `TaskEditModal` renders.
- Modal edit writes read existing frontmatter and assemble change input through
  `taskEditChangeState` before `TaskEditModal` calls `TaskService`.
- Modal edit subtask additions/removals are planned through `taskEditSubtasks`
  before child task project links are updated.
- NLP/API creation payloads use the shared parsed-data assembly path.
- Convert-current-note command payloads use `currentNoteConversion` to coerce
  existing frontmatter, apply status/priority defaults, and extract the body
  before opening the edit modal.
- Task creation defaults are applied through `taskCreationDefaults`.
- Task title sanitization for filenames and stored frontmatter values is applied
  through `taskTitleSanitizer`.
- Single-property task updates use `taskPropertyUpdate` to normalize incoming
  values and apply frontmatter mutation rules.
- Bulk task updates use `taskUpdatePlanning` to sanitize time-entry updates,
  plan recurrence date/DTSTART changes, apply mapped frontmatter and explicit
  field-removal rules, merge custom frontmatter, and assemble the returned task
  state.
- Archive toggles use `taskArchivePlanning` to plan archive state, frontmatter
  tag changes, and optional archive/tasks folder moves.
- Time-tracking start/stop/delete writes use `taskTimeTrackingPlanning` to
  sanitize legacy entries, choose the active entry to close, validate deleted
  entry indexes, and apply `timeEntries` frontmatter mutations.
- Recurring completion/skip writes use `taskRecurringPlanning` to select the
  action date, update complete/skipped instance arrays, adjust recurrence
  anchors, and advance scheduled/due dates before side effects run.
- Blocking relationship updates use `taskBlockingRelationships` to deduplicate
  changed blocked-task paths and plan reverse `blockedBy` add/remove patches
  before child tasks are written.
- Property-change side effects are coordinated through
  `taskPropertyChangeSideEffects` after the frontmatter write has happened.

Callers should pass typed task creation data into `TaskService`; they should not
reimplement frontmatter filtering, default application, or user-field coercion.

## Cache Principles

Current intent:

- metadata cache remains the primary read accelerator
- plugin-local caches should be derived and invalidatable
- derived caches must never become a second source of truth
- task-file identification and final `TaskInfo` assembly stay in focused
  helpers so cache reads, frontmatter matching, and display defaults are tested
  independently

Refactor rule:

- every plugin-local cache should document:
  - owner
  - key
  - invalidation trigger
  - whether stale reads are acceptable

## Event Flow

Current event propagation uses plugin/task-manager events plus view-level refresh handling.

Refactor direction:

- distinguish domain events from UI refresh events
- normalize event payload types
- keep low-level file events from leaking too far into UI modules

## Bases Boundary

Bases values and entries are not a stable internal domain model. They should be adapted into local TaskNotes types before the rest of the code depends on them.

Refactor rule:

- use local adapter types for the subset of Bases APIs the plugin actually uses
- prefer `unknown` plus narrowing over `any`
- keep Bases property mapping dependent on the narrow `FieldMapper` contract
  instead of the full plugin object
- keep Bases-specific value extraction out of general rendering code unless the
  renderer is explicitly responsible for displaying a native Bases value
- keep native conversion and display stringification for Bases value wrappers
  behind `basesValueConversion`
- keep cheap Bases entry property assembly from frontmatter/properties and file
  metadata behind `basesEntryProperties`
- keep filter-expression defaults for Bases-created tasks behind
  `basesFilterDefaults`, including task-tag exclusion, mapped core fields,
  custom user-field keys, list properties, and current-file link defaults
- keep Bases `createFileForView` task-creation data assembly behind
  `basesCreateFileForView`; the view should only open the modal and refresh
  after creation
- keep default Base file creation and overwrite policy behind a startup/settings
  helper rather than inside the plugin shell
- unregister custom Bases views synchronously during plugin unload so reloads
  cannot remove newly registered view types after startup
- keep shared Bases selection-mode classes, selected-card visuals, keyboard
  shortcuts, click-selection decisions, indicator behavior, and default
  visible-task path extraction behind `basesSelectionUi`
- keep shared Bases copy/export table assembly, value stringification, TSV/CSV
  escaping, and export file-name sanitization behind `basesExport`
- keep shared Bases task update/delete relevance decisions, path-rename refresh
  planning, and rendered-card path extraction behind `basesUpdateEvents`
- keep shared Bases task-update/delete listener registration, relevant-path
  cache mutation, and listener cleanup behind `basesTaskUpdateListeners`
- keep shared Bases config-change refresh hooks, data-update render debouncing,
  and explicit refresh debouncing behind `basesRefreshLifecycle`
- keep shared Bases toolbar DOM integration for the TaskNotes New button behind
  `basesToolbar`
- keep shared Bases result-menu task copy action construction and current-view
  clipboard task assembly behind `basesTaskCopyActions`
- keep shared Bases-to-TaskCard visible property mapping and display-label
  override assembly behind `basesVisibleProperties`
- keep shared Bases search controls, no-results state, and rendering behind
  `basesSearchUi`

Current adapter seams include:

- calendar task, property, and external-event builders
- calendar mutation planning
- calendar config snapshot construction for refresh/recreate decisions
- calendar data-signature property selection and value normalization
- calendar initial-date and navigation-state planning for recreate decisions
- Kanban task grouping, column/swimlane ordering, drag planning, and
  task-creation default planning
- Task List drag/drop insertion geometry
- Task List grouped render planning and sub-property grouping
- Bases value conversion and group-key display strings
- Bases entry property assembly from frontmatter and file metadata
- Bases `createFileForView` task-creation data assembly
- shared Bases formula evaluation and path-property map assembly
- shared Bases selection UI state and click/keyboard selection decisions
- shared Bases copy/export table assembly
- shared Bases update/delete event relevance planning
- shared Bases task-update listener lifecycle handling
- shared Bases refresh lifecycle scheduling
- shared Bases toolbar New button integration
- shared Bases result-menu task copy actions
- shared Bases visible-property and display-label mapping
- shared Bases search control creation and no-results rendering
- Bases-backed task creation assembly
- TaskCard property access
- TaskCard relationship expansion rendering
- TaskCard metadata assembly
- TaskCard metadata-line toggle wiring
- TaskCard render-state and class assembly
- TaskCard completion-state visual refresh
- TaskCard status/priority indicator rendering
- TaskCard secondary badge and dependency-toggle rendering
- TaskCard title and link rendering
- TaskCard context-menu integration
- TaskCard quick-action service calls
- TaskCard badge and interactive-control helpers
- TaskModal details-editor adapter wiring
- TaskModal title-input behavior
- TaskModal details/right-column layout transitions
- TaskModal mobile focus and keyboard scroll guards
- TaskModal field rendering and visibility dispatch
- TaskModal metadata field construction
- TaskModal organization field construction
- TaskModal project list rendering
- TaskModal subtask list rendering
- TaskModal dependency list state and candidate filtering
- TaskModal task-selector opening for dependency and subtask pickers
- TaskModal action menu-state snapshots and icon-state assembly
- TaskCreationModal initial form-state defaults and pre-populated values
- TaskCreationModal post-save subtask assignment planning
- TaskEditModal initial form-state defaults and cached user-field values
- TaskEditModal frontmatter-cache reads and edit-change input assembly
- TaskEditModal subtask add/remove planning and child project-link updates
- Bases filter defaults for task creation from filtered Base views
- default Bases file creation and regeneration writes

## Date and Recurrence Flow

Date handling is a historically sensitive area.

Refactor rule:

- parsing and coercion should happen at the boundary
- internal logic should use explicit date semantics
- recurrence expansion should remain centralized rather than duplicated in views
- date rollover detection should stay behind `dateChangeDetection`, which owns
  the minute poll, next-midnight timeout, timer registration, and event trigger

Calendar views should build events through normalized event builders before
touching FullCalendar APIs. Drag/drop and resize handlers should ask planning
helpers for explicit mutation plans before performing side effects. Event-mount
list-card rendering and Bases-backed task enrichment belong in
`calendarEventMount`. Initial-date option normalization, property-backed
navigation candidates, and navigation-state snapshots belong in
`calendarInitialDate`; Calendar views keep FullCalendar callback wiring, hover
previews, context menus, and refresh orchestration.

Agenda date sections should ask `agendaTaskSelection` whether a task belongs to
a date or overdue section. `FilterService` keeps task loading and query
filtering, while recurrence expansion, completed-overdue hiding, and
due/scheduled date eligibility live in the helper.

Task sorting should run through `filterTaskSorting`. `FilterService` keeps
metadata-cache access and orchestration while the helper owns sort-key
comparison, time-aware date ordering, natural fallback ordering, tag ordering,
and user-field sort comparison.

Filter predicates should ask `filterPredicateEvaluation` whether a task matches
a query node. `FilterService` keeps cache reads, query planning, grouping,
labels, and option discovery while the helper owns recursive group
evaluation, dynamic user-field coercion, project matching, subtask lookup, and
completion-state semantics.

Filter option assembly should run through `filterOptions`. `FilterService`
keeps option caching and invalidation while the helper owns task-folder
extraction and dynamic user-property definition construction.

Filter query state assembly should run through `filterQueryState`. `FilterService`
keeps validation, cache reads, and execution orchestration while the helper owns
default query construction, quick-toggle query mutation, and saved-query
normalization.

## UI State Flow

TaskCard and TaskModal rendering should consume already-normalized state.

Refactor rule:

- property lookup belongs in access helpers
- input parsing belongs in modal state helpers
- relationship filtering/sorting belongs in relationship helpers
- relationship expansion DOM containers should use injected card renderers
- visible-property metadata assembly belongs in metadata helpers
- metadata-line blocked-by toggle wiring belongs in metadata-line helpers
- target-date, completion, and class-name state belongs in state helpers
- status-click completion refresh, checkbox sync, title completion sync, and
  stale status/priority/project class cleanup belong in completion-state helpers
- status/priority dot visibility, colors, icons, and insertion belong in
  primary-indicator helpers
- recurrence, reminder, project, chevron, and dependency badge rendering belongs
  in secondary-badge helpers
- title text resolution, link rendering, and completion class sync belong in
  title helpers
- context-menu button wiring and native file-menu fallback belong in
  context-menu helpers
- quick-action service calls belong in action helpers
- reusable badge/control DOM behavior belongs in indicator helpers
- Task List drag/drop segmenting, insertion-slot resolution, and drop-target
  reconstruction belong in `taskListDragGeometry`; `TaskListView` keeps DOM
  measurement, drag event handling, and persistence
- Task List grouped-drop frontmatter mutation, status-derived fields, and
  side-effect task snapshots belong in `taskListDropPlanning`; `TaskListView`
  keeps sort-order queueing, vault writes, notices, and refresh orchestration
- Task List grouped render items, sub-property groups, grouped sort-scope paths,
  formula-backed property maps, and group-value stringification belong in
  `taskListGrouping`; `TaskListView` keeps render mode switching, virtualization,
  and DOM updates
- custom user-field input/toggle DOM behavior belongs in user-field control
  helpers
- creation-modal defaults, pre-populated values, default reminders, project
  strings, and user-field defaults belong in creation form-state helpers
- creation-modal subtask project assignment belongs in creation subtask helpers
- edit-modal defaults, identifying tag filtering, recurrence/reminder copies,
  and cached user-field values belong in edit form-state helpers
- edit-modal frontmatter-cache reads and task-change input assembly belong in
  edit change-state helpers
- edit-modal subtask add/remove planning and child project-link mutation belong
  in edit subtask helpers
- modal action-button bar construction and save disabled-state handling belongs
  in action-button helpers
- modal compact action-icon construction and the shared core action-icon set
  belongs in action-bar helpers
- modal compact action-icon active-state, tooltip, and color updates belong in
  action-icon state helpers
- modal date, status, priority, recurrence, and reminder action-menu callbacks
  belong in action-menu helpers
- modal action menu-state snapshots, due/scheduled setter routing, recurrence
  anchor preservation, and icon-state assembly belong in action-state helpers
- modal default status/priority and recurrence action-label derivation belong
  in action-value helpers
- title textarea construction, newline normalization, Enter-key policy, and
  dynamic title height belong in title-input helpers
- details/right-column collapse, expansion, and animation classes belong in
  layout helpers
- title-focus scroll restoration, mobile-like environment checks, and mobile
  keyboard scroll nudges belong in focus-guard helpers
- details-editor markdown lifecycle, tab focus policy, and value sync belong in
  details-editor helpers
- modal field grouping, core-field dispatch, and user-field fallback belong in
  field-rendering helpers
- contexts, tags, and time-estimate field construction belongs in metadata field
  helpers
- project, subtask, and dependency list-field shells belong in organization
  field helpers
- project-string parsing, project item deduplication, project list rendering, and
  project value serialization belong in project state helpers
- subtask candidate filtering, selected-subtask deduplication, removal, and
  cached/fallback list rendering belong in subtask state helpers
- dependency duplicate checks, add/remove behavior, and blocked-by/blocking
  candidate filtering belong in dependency state helpers
- task-selector task loading, empty-state notices, cancellation handling, and
  selector failure logging belong in the task-selector helper; modals keep the
  selected-item state mutation
- filename-safe and storage-safe task title normalization belongs in
  `taskTitleSanitizer`; callers should not strip filename-unsafe characters or
  control characters inline
- single-property normalization and frontmatter mutation rules belong in
  `taskPropertyUpdate`; callers should not duplicate status coercion,
  completed-date derivation, date removal, dependency serialization, or
  date-modified writes inline
- bulk task-update sanitation, recurrence-update planning, mapped frontmatter
  mutation, custom-frontmatter merge/delete semantics, explicit mapped-field
  removals, and returned task-state assembly belong in `taskUpdatePlanning`;
  `TaskUpdateService` keeps vault writes, renames, body writes, cache updates,
  events, webhooks, calendar sync, and auto-archive side effects
- archive tag mutation and archive/tasks move path construction belong in
  `taskArchivePlanning`; `TaskService` keeps the vault write, rename, cache,
  calendar, and webhook side effects
- time-entry sanitization and start/stop/delete frontmatter mutation rules
  belong in `taskTimeTrackingPlanning`; `TaskService` keeps active-session
  checks, vault writes, cache updates, events, and webhooks
- recurring instance action-date selection, complete/skipped array mutation,
  DTSTART updates, and next-occurrence advancement belong in
  `taskRecurringPlanning`; `TaskService` keeps vault writes, body checkbox
  resets, cache updates, events, webhooks, and calendar sync
- blocking relationship path deduplication, reverse `blockedBy` mutation, and
  dependency metadata preservation belong in `taskBlockingRelationships`;
  `TaskService` keeps task cache reads and child task writes
- post-write cache refresh, dependent-task refresh events, webhooks, calendar
  sync, and auto-archive routing belong in `taskPropertyChangeSideEffects`
- Calendar drag/resize frontmatter mutation decisions belong in
  `calendarMutationPlanning`; Calendar views keep event dispatch, vault file
  lookup, and provider/task-service side effects
- Kanban drop frontmatter mutation, status-derived fields, and post-write
  side-effect task snapshots belong in `kanbanDragUtils`; Kanban final
  coordinate drop-target reconstruction and optimistic DOM card movement also
  belong there. Kanban column/swimlane task-creation defaults belong in
  `kanbanCreationDefaults`, while Kanban views keep event wiring, active drag
  state, sort-order service calls, vault writes, notices, and modal entry
  points
- task-frontmatter identification belongs in `taskIdentification`; callers
  should not duplicate tag/property matching, metadata-cache `#` stripping, or
  boolean-like identifier comparison
- mapped task-info defaults, computed tracked time, and blocking flags belong in
  `taskInfoAssembly`; `TaskManager` keeps metadata-cache access, FieldMapper
  calls, and dependency-cache lookups
- custom user-field filter coercion, list-token normalization, sort comparison,
  and group bucket selection belong in `userFieldValues`; `FilterService` keeps
  metadata-cache reads and query orchestration
- filter predicate evaluation belongs in `filterPredicateEvaluation`; callers
  should not duplicate recursive group rules, user-field filter coercion,
  project wikilink matching, `hasSubtasks`, or `status.isCompleted` semantics
- task grouping, project/tag fan-out, date bucket labels, completed-date labels,
  and group header ordering belong in `filterTaskGrouping`; `FilterService`
  injects translation, locale, status, priority, project-resolution, and
  frontmatter-value access
- task sort-key comparison, time-aware date ordering, fallback ordering, tag
  ordering, and user-field sort comparison belong in `filterTaskSorting`;
  `FilterService` injects status, priority, and frontmatter-value access
- filter option assembly, task-folder extraction, and dynamic user-property
  definitions belong in `filterOptions`; `FilterService` keeps cache
  invalidation and source orchestration
- default query construction, quick-toggle condition mutation, and partial query
  normalization belong in `filterQueryState`
- DOM functions should wire controls, render values, and call services

This keeps DOM tests focused on interaction while pure unit tests cover state
rules such as custom field filtering, list parsing, dependency normalization,
and inherited view ordering.

## Operational Principle

When debugging a behavior problem, the first question should be:

"Is this a source-data issue, an adapter issue, a domain-logic issue, or a view-refresh issue?"

The architecture should make that answer obvious. New diagnostics should use
`TaskNotesLogCategory` values so logs preserve the same distinction:

- source-data problems usually map to `validation` or `stale-data`
- adapter/API problems usually map to `provider` or `configuration`
- write-path failures usually map to `persistence`
- unexpected view/service failures usually map to `internal`