Create comprehensive Bases architecture design document

2026-07-22 06:45:14 +00:00 · 2025-08-26 08:35:52 -05:00 · 2025-08-26 08:35:52 -05:00 · 8fae03a1b4
commit 8fae03a1b4
parent df80581209
1 changed files with 379 additions and 0 deletions
--- a/wiki/Design.md
+++ b/wiki/Design.md
@ -0,0 +1,379 @@
+# Obsidian MCP Plugin - Architecture Design
+
+## ADR-001: Bases Architecture Modernization
+
+**Status**: Proposed  
+**Date**: 2025-08-26  
+**Requirement IDs**: Performance optimization, architectural compliance, extensibility requirements
+
+## Context
+
+The current Bases implementation, while functional, exhibits several architectural anti-patterns that limit scalability, maintainability, and performance:
+
+### Current Architecture Issues
+1. **Monolithic BasesAPI Class**: 440+ lines handling file parsing, query execution, export formatting, and property extraction
+2. **Tight Coupling**: ExpressionEvaluator directly depends on Obsidian concrete types
+3. **Performance Bottlenecks**: Full vault scans for every query, no caching, synchronous processing
+4. **Security Concerns**: Direct eval() usage in expression evaluation without proper sandboxing
+5. **Limited Extensibility**: Hard-coded export formats and view types
+
+### SOLID Violations
+- **SRP**: BasesAPI has multiple responsibilities (file ops, queries, formatting)
+- **OCP**: Adding new view types or export formats requires modifying existing code
+- **ISP**: No separation between different operational concerns
+- **DIP**: Direct dependencies on Obsidian concrete implementations
+
+## Decision
+
+Implement a layered architecture with clear separation of concerns following SOLID principles:
+
+### Layer 1: Core Interfaces (Domain Layer)
+```typescript
+interface IBaseRepository {
+  findAll(): Promise<BaseConfig[]>;
+  findByPath(path: string): Promise<BaseConfig>;
+  create(config: BaseConfig): Promise<void>;
+  update(path: string, config: BaseConfig): Promise<void>;
+  delete(path: string): Promise<void>;
+}
+
+interface IQueryEngine {
+  execute(query: BaseQuery): Promise<QueryResult>;
+  count(query: BaseQuery): Promise<number>;
+  explain(query: BaseQuery): QueryPlan;
+}
+
+interface IViewRenderer {
+  render(result: QueryResult, config: ViewConfig): Promise<RenderedView>;
+  supports(viewType: string): boolean;
+}
+
+interface IExportStrategy {
+  export(result: QueryResult): Promise<string>;
+  supports(format: string): boolean;
+}
+```
+
+### Layer 2: Service Layer
+```typescript
+class BaseService {
+  constructor(
+    private repository: IBaseRepository,
+    private queryEngine: IQueryEngine,
+    private viewRenderers: Map<string, IViewRenderer>,
+    private exportStrategies: Map<string, IExportStrategy>
+  ) {}
+
+  async listBases(): Promise<BaseInfo[]> { /* delegates to repository */ }
+  async queryBase(path: string, viewName?: string): Promise<BaseQueryResult> { /* orchestrates query + view */ }
+  async exportBase(path: string, format: string): Promise<string> { /* orchestrates query + export */ }
+}
+```
+
+### Layer 3: Infrastructure Layer
+```typescript
+class ObsidianBaseRepository implements IBaseRepository {
+  constructor(private api: ObsidianAPI) {}
+  // Direct integration with vault operations
+}
+
+class IndexedQueryEngine implements IQueryEngine {
+  constructor(
+    private indexService: IIndexService,
+    private evaluator: IExpressionEvaluator
+  ) {}
+  // Optimized query execution with indexing
+}
+```
+
+## Consequences
+
+### Positive
+- **Testability**: Each component can be unit tested in isolation
+- **Performance**: Dedicated query engine with indexing and caching
+- **Extensibility**: New view types and export formats via plugin pattern
+- **Security**: Sandboxed expression evaluation with controlled context
+- **Maintainability**: Clear separation of concerns following SOLID principles
+
+### Negative
+- **Complexity**: More interfaces and classes to manage
+- **Initial Development Cost**: Significant refactoring required
+- **Learning Curve**: Team needs to understand layered architecture
+
+### Neutral
+- **Memory Usage**: Slight increase due to additional abstraction layers
+- **Bundle Size**: Minimal impact on plugin size
+
+## Alternatives Considered
+
+1. **Incremental Refactoring**: Gradually improve existing code
+   - Rejected: Doesn't address fundamental architectural issues
+2. **External Query Library**: Use existing database/query libraries
+   - Rejected: Adds dependencies and doesn't integrate well with Obsidian patterns
+3. **Event-Driven Architecture**: Use pub/sub for loose coupling
+   - Deferred: Would add complexity without clear benefits for current use cases
+
+## Implementation Notes
+
+### Phase 1: Core Architecture (2-3 weeks)
+- Extract interfaces and core domain types
+- Implement repository pattern for base file operations
+- Create basic query engine with filtering support
+
+### Phase 2: Performance Optimization (2 weeks)
+- Add indexing service for frequently queried properties
+- Implement result caching with cache invalidation
+- Add query optimization and execution planning
+
+### Phase 3: Extensibility (1-2 weeks)  
+- Plugin system for view renderers and export strategies
+- Security hardening of expression evaluation
+- Advanced query features (aggregation, joins)
+
+---
+
+## ADR-002: Expression Evaluation Security Model
+
+**Status**: Proposed  
+**Date**: 2025-08-26  
+**Requirement IDs**: Security hardening, safe expression evaluation
+
+## Context
+
+Current expression evaluation uses `new Function()` with dynamic code generation, creating potential security vulnerabilities. The evaluation context has access to the full Obsidian API and file system.
+
+## Decision
+
+Implement a sandboxed expression evaluation system:
+
+```typescript
+interface IExpressionEvaluator {
+  evaluate(expression: string, context: EvaluationContext): Promise<any>;
+  validate(expression: string): ValidationResult;
+}
+
+class SandboxedExpressionEvaluator implements IExpressionEvaluator {
+  private allowedGlobals: Set<string>;
+  private functionWhitelist: Set<string>;
+  
+  constructor() {
+    this.allowedGlobals = new Set(['Math', 'Date', 'String', 'Number']);
+    this.functionWhitelist = new Set(['date', 'now', 'today', 'iff', 'choice']);
+  }
+  
+  async evaluate(expression: string, context: EvaluationContext): Promise<any> {
+    // Parse AST and validate against whitelist
+    const ast = this.parseExpression(expression);
+    this.validateAST(ast);
+    
+    // Execute in controlled environment
+    return this.executeInSandbox(ast, context);
+  }
+}
+```
+
+### Security Boundaries
+1. **AST-based evaluation**: Parse expressions into Abstract Syntax Tree for validation
+2. **Function whitelist**: Only allow pre-approved functions and operations
+3. **Context isolation**: Limit access to necessary properties only
+4. **Resource limits**: Prevent infinite loops and excessive memory usage
+
+## Consequences
+
+### Positive
+- **Security**: Eliminates code injection vulnerabilities
+- **Reliability**: Prevents expressions from crashing the plugin
+- **Predictability**: Clear boundaries on what expressions can do
+
+### Negative
+- **Performance**: AST parsing adds overhead compared to direct eval
+- **Functionality**: Some advanced expressions may not be possible
+
+---
+
+## ADR-003: Query Engine Architecture
+
+**Status**: Proposed  
+**Date**: 2025-08-26  
+**Requirement IDs**: Performance optimization, scalability
+
+## Context
+
+Current query implementation scans all files for every query, leading to O(n) performance that doesn't scale with large vaults. No caching or indexing strategy exists.
+
+## Decision
+
+Implement a multi-tier query engine with indexing and caching:
+
+```typescript
+interface IIndexService {
+  createIndex(property: string): Promise<void>;
+  dropIndex(property: string): Promise<void>;
+  getIndexedValues(property: string): Promise<Set<any>>;
+  findByIndexedValue(property: string, value: any): Promise<string[]>;
+}
+
+class QueryEngine implements IQueryEngine {
+  constructor(
+    private indexService: IIndexService,
+    private cacheService: ICacheService
+  ) {}
+
+  async execute(query: BaseQuery): Promise<QueryResult> {
+    // 1. Check cache for identical query
+    const cacheKey = this.generateCacheKey(query);
+    const cached = await this.cacheService.get(cacheKey);
+    if (cached) return cached;
+
+    // 2. Optimize query using available indexes
+    const optimizedQuery = this.optimizeQuery(query);
+    
+    // 3. Execute optimized query
+    const result = await this.executeOptimizedQuery(optimizedQuery);
+    
+    // 4. Cache result with TTL
+    await this.cacheService.set(cacheKey, result, { ttl: 300000 }); // 5 min
+    
+    return result;
+  }
+
+  private optimizeQuery(query: BaseQuery): OptimizedQuery {
+    // Use indexes for equality filters
+    // Push down filters to reduce dataset early
+    // Optimize sort operations using indexed properties
+  }
+}
+```
+
+### Performance Strategies
+1. **Property Indexing**: Build indexes for frequently queried properties
+2. **Query Result Caching**: Cache expensive query results with invalidation
+3. **Filter Pushdown**: Apply filters as early as possible in query pipeline
+4. **Incremental Updates**: Only re-process changed files on vault updates
+
+## Consequences
+
+### Positive
+- **Performance**: O(log n) query performance for indexed properties
+- **Scalability**: Handles large vaults efficiently
+- **Responsiveness**: Cached results provide instant responses
+
+### Negative
+- **Memory Usage**: Indexes and caches consume additional memory
+- **Complexity**: Index management and cache invalidation logic
+- **Startup Time**: Initial index building may slow plugin initialization
+
+---
+
+## ADR-004: Integration with ObsidianAPI Abstraction
+
+**Status**: Proposed  
+**Date**: 2025-08-26  
+**Requirement IDs**: Architecture consistency, plugin integration
+
+## Context
+
+The plugin maintains an ObsidianAPI abstraction layer for all vault operations. The new Bases architecture must integrate seamlessly with this pattern while maintaining the performance benefits of direct API access.
+
+## Decision
+
+Extend the ObsidianAPI abstraction with Bases-specific operations while maintaining the facade pattern:
+
+```typescript
+// ObsidianAPI Extensions
+interface IObsidianAPI {
+  // Existing methods...
+  
+  // Bases operations
+  getBases(): Promise<BaseInfo[]>;
+  getBase(path: string): Promise<BaseConfig>;
+  createBase(path: string, config: BaseConfig): Promise<void>;
+  updateBase(path: string, config: BaseConfig): Promise<void>;
+  deleteBase(path: string): Promise<void>;
+  
+  // Optimized data access for Bases
+  getFileMetadataCache(): MetadataCache;
+  getFilesByTag(tag: string): TFile[];
+  getFilesByFolder(folder: string): TFile[];
+  getFileFrontmatter(file: TFile): Record<string, any>;
+}
+
+// Dependency Injection Setup
+class BasesModule {
+  static create(api: IObsidianAPI): BaseService {
+    const repository = new ObsidianBaseRepository(api);
+    const indexService = new ObsidianIndexService(api);
+    const cacheService = new MemoryCacheService();
+    const queryEngine = new IndexedQueryEngine(indexService, cacheService);
+    
+    const viewRenderers = new Map([
+      ['table', new TableViewRenderer()],
+      ['cards', new CardViewRenderer()],
+      ['list', new ListViewRenderer()]
+    ]);
+    
+    const exportStrategies = new Map([
+      ['csv', new CSVExportStrategy()],
+      ['json', new JSONExportStrategy()],
+      ['markdown', new MarkdownExportStrategy()]
+    ]);
+    
+    return new BaseService(repository, queryEngine, viewRenderers, exportStrategies);
+  }
+}
+```
+
+## Consequences
+
+### Positive
+- **Consistency**: Maintains established architectural patterns
+- **Testability**: Easy to mock ObsidianAPI for testing
+- **Performance**: Direct access to Obsidian APIs where needed
+- **Flexibility**: Can switch between HTTP and direct API implementations
+
+### Negative
+- **Additional Abstraction**: One more layer between Bases logic and Obsidian
+- **API Surface**: ObsidianAPI interface grows larger
+
+## Implementation Notes
+
+- Preserve exact method signatures from current implementation for backward compatibility
+- Add performance monitoring to identify bottlenecks during migration
+- Implement feature flags to enable gradual rollout of new architecture
+- Maintain fallback to current implementation if issues arise
+
+---
+
+## Migration Strategy
+
+### Phase 1: Foundation (Week 1-2)
+1. Extract core interfaces and domain types
+2. Implement repository pattern
+3. Create basic service layer
+4. Maintain existing API compatibility
+
+### Phase 2: Performance (Week 3-4)
+1. Add indexing service
+2. Implement caching layer
+3. Optimize query execution
+4. Add performance metrics
+
+### Phase 3: Extensibility (Week 5-6)
+1. Plugin system for views and exports
+2. Security hardening
+3. Advanced query features
+4. Complete testing coverage
+
+### Testing Strategy
+- **Unit Tests**: Each service and component in isolation
+- **Integration Tests**: End-to-end query workflows
+- **Performance Tests**: Benchmark against current implementation
+- **Security Tests**: Validate expression evaluation safety
+
+### Risk Mitigation
+- **Feature Flags**: Enable gradual rollout
+- **Fallback Mechanism**: Revert to current implementation if needed
+- **Performance Monitoring**: Track metrics during migration
+- **User Feedback**: Beta testing with BRAT users
+
+This architectural redesign addresses all identified issues while maintaining backward compatibility and establishing a foundation for future enhancements.