obsidian-plugin/qf3l3k_obsidian-data-fetcher
1
0
Fork 0
mirror of https://github.com/qf3l3k/obsidian-data-fetcher.git synced 2026-07-22 05:43:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

release: v1.0.6

Browse source
This commit is contained in:
qf3l3k 2026-03-02 15:50:28 +01:00
parent 6bdfdb89be
commit c25f3011ea
9 changed files with 169 additions and 124 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
CHANGELOG.md Normal file
View file

@ -0,0 +1,19 @@
# Changelog
All notable changes to this project will be documented in this file.
## [1.0.6] - 2026-03-02
### Fixed
- Issue #3: improved error block readability in dark/light themes by switching to normal text color with a clear error border.
- Fixed command refresh flow by implementing the missing `data-fetcher:refresh-query` event handler for active notes.
- Fixed code block section lookup used by "Save to Note" to correctly resolve source positions.
### Changed
- Replaced Node `crypto` cache-key dependency with a platform-safe in-code hash to stay compatible with desktop and mobile.
- Reduced noisy debug logging in plugin runtime output.
- Updated README examples and added explicit network/data disclosure notes.
### Metadata
- Bumped plugin version to `1.0.6`.
- Updated `manifest.json`, `versions.json`, `package.json`, and `package-lock.json` for release consistency.

129
README.md
View file

@ -1,45 +1,41 @@
# Data Fetcher
A plugin for [Obsidian](https://obsidian.md) that allows users to fetch data from multiple sources (REST APIs, GraphQL, gRPC, RPC) and insert the results into notes.
A plugin for [Obsidian](https://obsidian.md) that fetches data from external endpoints (REST, GraphQL, RPC, gRPC via proxy) and renders the results in notes.
## Features
- Support for multiple data sources:
- Fetch from:
- REST APIs
- GraphQL endpoints
- gRPC services (via REST proxies)
- RPC services
- Two modes of operation:
- Directly define queries within notes
- Predefine endpoints in settings and reference them in notes
- Automatic caching of query results to reduce redundant requests
- Cache expiration settings
- Manual refresh capabilities
- Customizable headers for authentication
- RPC endpoints
- gRPC services via REST proxy
- Two query modes:
- Inline query definition in a `data-query` code block
- Endpoint aliases configured in plugin settings
- Query result caching with configurable expiration
- Manual refresh, copy, and save-to-note actions
- Per-endpoint request headers for authentication and API keys
## Installation
### From Obsidian Community Plugins
### Community Plugins
1. Open Obsidian Settings
2. Go to Community Plugins
3. Search for "Data Fetcher"
4. Click Install, then Enable
1. Open Obsidian Settings.
2. Go to Community Plugins.
3. Search for `Data Fetcher`.
4. Install and enable.
### Manual Installation
### Manual
1. Clone the latest release from the [plugin repository page](https://github.com/qf3l3k/obsidian-api-fetcher)
2. Copy or move cloned folder into your Obsidian vault's `.obsidian/plugins` folder
3. Enable the plugin in Obsidian settings
1. Download the latest release from the [repository](https://github.com/qf3l3k/obsidian-api-fetcher).
2. Place the release files in your vault at `.obsidian/plugins/data-fetcher`.
3. Enable the plugin in Obsidian settings.
## Usage
### Method 1: Direct Query Definition
### 1. Inline Query
Create a code block with the language set to `data-query` and define your query in JSON format:
```
​```data-query
```data-query
{
"type": "rest",
"url": "https://api.example.com/data",
@ -48,91 +44,52 @@ Create a code block with the language set to `data-query` and define your query
"Authorization": "Bearer your-token"
}
}
​```
```
### Method 2: Using Aliases
### 2. Endpoint Alias
1. First, define an endpoint alias in the plugin settings
2. Then reference it in your notes:
1. Configure an endpoint alias in plugin settings.
2. Reference it in a note:
```
​```data-query
```data-query
@my-api-alias
body: {"id": 123}
​```
```
## Configuration
Go to Settings > Data Fetcher to configure:
- Cache duration
- Pre-defined endpoint aliases
- Default headers
## Examples
### REST API Example
```
​```data-query
{
"type": "rest",
"url": "https://jsonplaceholder.typicode.com/posts/1",
"method": "GET"
}
​```
body: {"id":123}
```
### GraphQL Example
```
​```data-query
```data-query
{
"type": "graphql",
"url": "https://api.spacex.land/graphql",
"query": "{ launchesPast(limit: 5) { mission_name launch_date_local } }",
"variables": {}
}
​```
```
### Using Aliases
## Configuration
```
​```data-query
@github-api
query: query { viewer { repositories(first: 5) { nodes { name } } } }
​```
```
Open `Settings -> Data Fetcher` to manage:
- Cache duration
- Endpoint aliases
- Endpoint headers
- Cache cleanup
## TODO
- [X] Endpoints list in Settings
- [X] Query cache
- [X] Pulling data from RPC
- [X] Pulling data from API
- [X] Pulling data from GraphQL
- [ ] Pulling data from gRPC
- [ ] Cache browser
- [X] Cache cleaner
- [X] Save query results as static text
- [ ] ... some other not yet known features
## Data Disclosure
This plugin communicates with external services and stores response data:
- Network usage: Sends HTTP(S) requests to endpoint URLs defined in notes or settings.
- External dependencies: Uses Obsidian's built-in `requestUrl` API; no third-party analytics SDK is used.
- Data sent: Request URL, method, headers, and optional body/query you configure.
- Data stored locally: Cached responses in vault folder `.data-fetcher-cache` and plugin settings in Obsidian plugin data.
- Data shared externally: Only with endpoint services you explicitly configure.
## Support
If you encounter any issues or have feature requests, please file them on the [GitHub issues page](https://github.com/qf3l3k/obsidian-api-fetcher/issues).
If you like this plugin you can support it's development and ...
<a href="https://www.buymeacoffee.com/qf3l3k" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style=" height: 60px !important;width: 217px !important;" ></a>
- Issues and feature requests: [GitHub Issues](https://github.com/qf3l3k/obsidian-api-fetcher/issues)
## License
MIT
MIT

103
main.ts
View file

@ -1,4 +1,4 @@
import { App, Editor, MarkdownView, Modal, Notice, Plugin, PluginSettingTab, Setting } from 'obsidian';
import { App, Editor, EventRef, MarkdownView, Modal, Notice, Plugin, PluginSettingTab, Setting } from 'obsidian';
import { DataFetcherSettings, DEFAULT_SETTINGS } from './src/settings';
import { parseDataQuery, executeQuery, QueryParams, QueryResult } from './src/queryEngine';
import { CacheManager } from './src/cacheManager';
@ -10,8 +10,6 @@ export default class DataFetcherPlugin extends Plugin {
private queryButtonMap: WeakMap<HTMLElement, QueryParams> = new WeakMap();
async onload() {
console.log('Loading Data Fetcher plugin');
await this.loadSettings();
this.cacheManager = new CacheManager(this.app, this);
@ -45,10 +43,88 @@ export default class DataFetcherPlugin extends Plugin {
}
});
// Handle refresh events triggered by command or other plugin actions.
const workspaceEvents = this.app.workspace as unknown as {
on(name: string, callback: (...args: unknown[]) => unknown, ctx?: unknown): EventRef;
};
this.registerEvent(workspaceEvents.on('data-fetcher:refresh-query', async () => {
await this.refreshQueriesInActiveNote();
}));
// Add settings tab
this.addSettingTab(new DataFetcherSettingTab(this.app, this));
}
private extractDataQueryBlocks(markdown: string): string[] {
const queryBlocks: string[] = [];
const dataQueryRegex = /```data-query[^\n]*\n([\s\S]*?)```/g;
let match: RegExpExecArray | null;
while ((match = dataQueryRegex.exec(markdown)) !== null) {
queryBlocks.push(match[1].trim());
}
return queryBlocks;
}
private rerenderActiveView(view: MarkdownView): void {
const previewMode = (view as any).previewMode;
if (previewMode && typeof previewMode.rerender === 'function') {
previewMode.rerender(true);
return;
}
// Fallback that still refreshes markdown render output.
this.app.workspace.trigger('layout-change');
}
private async refreshQueriesInActiveNote(): Promise<void> {
const activeView = this.app.workspace.getActiveViewOfType(MarkdownView);
const activeFile = activeView?.file;
if (!activeView || !activeFile) {
new Notice('No active markdown file to refresh');
return;
}
const content = await this.app.vault.cachedRead(activeFile);
const queryBlocks = this.extractDataQueryBlocks(content);
if (queryBlocks.length === 0) {
new Notice('No data-query blocks found in the current note');
return;
}
let refreshedCount = 0;
let failedCount = 0;
for (const querySource of queryBlocks) {
try {
const query = parseDataQuery(querySource, this.settings);
const result = await executeQuery(query);
await this.cacheManager.saveToCache(query, result);
if (result.error) {
failedCount++;
} else {
refreshedCount++;
}
} catch (error) {
failedCount++;
console.error('Failed to refresh query block:', error);
}
}
this.rerenderActiveView(activeView);
if (failedCount === 0) {
new Notice(`Refreshed ${refreshedCount} data quer${refreshedCount === 1 ? 'y' : 'ies'}`);
return;
}
new Notice(`Refreshed ${refreshedCount}; ${failedCount} failed`);
}
renderResult(result: QueryResult, container: HTMLElement, query?: QueryParams, ctx?: any) {
// First clear the container
container.empty();
@ -69,7 +145,7 @@ export default class DataFetcherPlugin extends Plugin {
if (ctx && ctx.sourcePath && ctx.getSectionInfo) {
try {
// Store section info for this code block in the container's dataset
const sectionInfo = ctx.getSectionInfo(ctx.getSectionInfo().lineStart);
const sectionInfo = ctx.getSectionInfo(container);
if (sectionInfo) {
container.dataset.sourcePath = ctx.sourcePath;
container.dataset.lineStart = String(sectionInfo.lineStart);
@ -139,7 +215,6 @@ export default class DataFetcherPlugin extends Plugin {
// Add event listener for save to note button with proper data
saveToNoteBtn.addEventListener('click', () => {
console.log("Save to Note button clicked");
// Get the data as a string for saving to note
let dataString: string;
@ -158,7 +233,6 @@ export default class DataFetcherPlugin extends Plugin {
// Get the data as a string for display and copying
let dataString: string;
console.log("Rendering data:", result.data);
if (result.data === null || result.data === undefined) {
content.setText("No data returned");
@ -197,13 +271,10 @@ export default class DataFetcherPlugin extends Plugin {
saveResultToNote(dataString: string, container: HTMLElement): void {
try {
console.log("Save to Note clicked - attempting to save data");
// Get the active view
const activeView = this.app.workspace.getActiveViewOfType(MarkdownView);
if (!activeView) {
console.log("No active markdown view found");
new Notice('No active markdown view - please open a markdown file first');
return;
}
@ -229,9 +300,6 @@ export default class DataFetcherPlugin extends Plugin {
if (container.dataset.sourcePath &&
container.dataset.lineStart &&
container.dataset.lineEnd) {
console.log("Found source info in container dataset");
// Check if the source path matches the current file
const currentPath = activeView.file?.path;
if (currentPath === container.dataset.sourcePath) {
@ -244,8 +312,6 @@ export default class DataFetcherPlugin extends Plugin {
ch: editor.getLine(parseInt(container.dataset.lineEnd)).length
};
console.log(`Replacing content from line ${start.line} to ${end.line}`);
// Use editor transaction for safer text replacement
editor.transaction({
changes: [
@ -259,11 +325,7 @@ export default class DataFetcherPlugin extends Plugin {
new Notice('Data block replaced with static content');
return;
} else {
console.log("Source path doesn't match current file");
}
} else {
console.log("No source info found in container dataset");
}
// Fallback: Try to use the Obsidian view to locate the code block
@ -286,7 +348,6 @@ export default class DataFetcherPlugin extends Plugin {
}
if (endLine > i) {
console.log(`Found code block from line ${i} to ${endLine}`);
// Replace the entire code block
const start = { line: i, ch: 0 };
const end = { line: endLine, ch: lines[endLine].length };
@ -309,7 +370,6 @@ export default class DataFetcherPlugin extends Plugin {
}
// If all attempts to find the code block failed, insert at cursor position
console.log("Falling back to cursor position insertion");
const cursor = editor.getCursor();
editor.transaction({
@ -331,7 +391,6 @@ export default class DataFetcherPlugin extends Plugin {
}
onunload() {
console.log('Unloading Data Fetcher plugin');
// WeakMap will be garbage collected automatically when plugin is unloaded
}
@ -611,4 +670,4 @@ class HeadersModal extends Modal {
const {contentEl} = this;
contentEl.empty();
}
}
}

4
manifest.json
View file

@ -1,7 +1,7 @@
{
"id": "data-fetcher",
"name": "Data Fetcher",
"version": "1.0.5",
"version": "1.0.6",
"minAppVersion": "1.8.3",
"description": "Fetch data from multiple sources (REST APIs, RPC, gRPC, GraphQL) and insert results into notes",
"author": "qf3l3k",
@ -10,4 +10,4 @@
"Buy me a Coffee": "https://buymeacoffee.com/qf3l3k"
},
"isDesktopOnly": false
}
}

4
package-lock.json generated
View file

@ -1,12 +1,12 @@
{
"name": "obsidian-api-fetcher",
"version": "1.0.0",
"version": "1.0.6",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "obsidian-api-fetcher",
"version": "1.0.0",
"version": "1.0.6",
"license": "MIT",
"devDependencies": {
"@types/node": "^16.11.6",

2
package.json
View file

@ -1,6 +1,6 @@
{
"name": "obsidian-api-fetcher",
"version": "1.0.4",
"version": "1.0.6",
"description": "Fetch data from multiple sources (REST APIs, RPC, gRPC, GraphQL) and insert results into notes",
"main": "main.js",
"scripts": {

21
src/cacheManager.ts
View file

@ -1,6 +1,5 @@
import { App, TFile, TFolder } from 'obsidian';
import { QueryParams, QueryResult } from './queryEngine';
import * as crypto from 'crypto';
export class CacheManager {
private app: App;
@ -42,7 +41,19 @@ export class CacheManager {
variables: params.variables
});
return crypto.createHash('md5').update(stringToHash).digest('hex');
return this.hashString(stringToHash);
}
/**
* Lightweight deterministic hash to keep cache keys stable across platforms.
*/
private hashString(input: string): string {
let hash = 2166136261;
for (let i = 0; i < input.length; i++) {
hash ^= input.charCodeAt(i);
hash = Math.imul(hash, 16777619);
}
return (hash >>> 0).toString(16).padStart(8, '0');
}
/**
@ -106,17 +117,13 @@ export class CacheManager {
const files = await this.app.vault.adapter.list(this.cacheFolder);
// Remove each file from the cache folder
let deletedCount = 0;
for (const file of files.files) {
try {
await this.app.vault.adapter.remove(file);
deletedCount++;
} catch (err) {
console.error(`Failed to delete cache file ${file}:`, err);
}
}
console.log(`Cleared ${deletedCount} cache files`);
} catch (error) {
console.error("Error clearing cache:", error);
throw error; // Re-throw to be handled by the caller
@ -151,4 +158,4 @@ export class CacheManager {
return { count: 0, size: 0 };
}
}
}
}

8
styles.css
View file

@ -49,8 +49,10 @@
.data-fetcher-error {
padding: 10px;
color: var(--text-error);
background-color: var(--background-modifier-error);
color: var(--text-normal);
background-color: var(--background-secondary);
border: 1px solid var(--text-error);
border-left: 4px solid var(--text-error);
border-radius: 5px;
}
@ -133,4 +135,4 @@
margin-bottom: 10px;
font-size: 0.9em;
color: var(--text-muted);
}
}

3
versions.json
View file

@ -1,3 +1,4 @@
{
"1.0.5": "1.8.3"
"1.0.5": "1.8.3",
"1.0.6": "1.8.3"
}