Merge branch 'main' into chore/scss-split

This commit is contained in:
Kacper Kula 2025-05-22 14:53:29 +01:00
commit 672e0fdba6
27 changed files with 784 additions and 538 deletions

View file

@ -0,0 +1,5 @@
---
"sqlseal": minor
---
added ability to manually define column types

View file

@ -36,9 +36,8 @@ jobs:
with:
publish: pnpm run ci:publish
version: pnpm run ci:version
title: "Release: $VERSION"
commit: "release: Release $VERSION"
title: "[Release] Next Release"
commit: "automated release"
createGithubReleases: false # this is handled inside ci:publish script
setupGitUser: false # this is handled inside ci:publish script
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View file

@ -1,22 +1,24 @@
# 0.36.0 (2025-05-06)
# Releases
## 0.36.0 (2025-05-06)
- fix: deleting columns and renaming columns works again
- chore: updated dependencies to the latest versions
# 0.35.0 (2025-05-02)
## 0.35.0 (2025-05-02)
- fix: fixed issue with CSV files edits not being saved
# 0.34.0 (2025-04-30)
## 0.34.0 (2025-04-30)
- feat: you can now navigate to linked CSV and JSON file sources
- feat: improved support for minimal theme custom classes
# 0.33.0 (2025-04-09)
## 0.33.0 (2025-04-09)
- fix: better text links for images (by @satkowski)
- fix: lists are now rendered properly (by @satkowski)
- chore: exposing filename in the renderer (by @satkowski)
- fix: numbers are now properly handled (by @satkowski)
- fix: SQLSeal results are now properly rendered on canvas
# 0.32.0 (2025-04-02)
## 0.32.0 (2025-04-02)
- feat: added text rendering for links, images and checkboxes (for MARKDOWN renderer)
- fix: fixed issue with rendering numbers
- fix: fixed how grid renders columns - now they automatically match content and don't truncate the text
@ -24,35 +26,35 @@
- fix: columns with dots in the name render properly in grid view
- chore: now external plugins can use cellRenderer from the main plugin
# 0.31.0 (2025-03-31)
## 0.31.0 (2025-03-31)
- feat: TEMPLATE renderer can now use checkboxes, links and images like other views.
# 0.30.1 (2025-03-27)
## 0.30.1 (2025-03-27)
- feat: added basename and parent SQL variables
# 0.30.0 (2025-03-26)
## 0.30.0 (2025-03-26)
- feat: added ability to reference tables from the other files in `table()` functions
- feat: added ability to reference tables by their header name
- feat: new heading and heading_level columns in tasks table (by @kDCYorke)
# 0.29.0 (2025-03-19)
## 0.29.0 (2025-03-19)
- feat: LIST views with single column are rendered as a regular list (rather than nested one) (by @kDCYorke)
- feat: files table now contains tags column with tags defined directly in properties (by @sksizer)
- feat: frontmatter links are now exposed in links table (by @kDCYorke)
- feat: added checkbox field to tasks which adds **interactive** checkbox so you can mark tasks as completed directly from the SQLSeal
# 0.28.4 (2025-03-15)
## 0.28.4 (2025-03-15)
- Feat: Template view can now access file properties
# 0.28.3 (2025-03-15)
## 0.28.3 (2025-03-15)
- Fix: Filename field in files column no longer truncates name after dot
# 0.28.2 (2025-03-14)
## 0.28.2 (2025-03-14)
- Added ability to define default view (Grid, HTML or Markdown)
- Added ability to configure default page size for the GRID view
# 0.28.1 (2025-03-14)
## 0.28.1 (2025-03-14)
- Adding syntax highlighting for javascript (to be used with GRAPH view)
# 0.28.0 (2025-03-14)
## 0.28.0 (2025-03-14)
- Added TEMPLATE view that allow to render your template with custom Handlebars template.
- Improved syntax highlighting - now lines with errors will get highlighted with appropriate colour to indicate the issue
- Added @path, @fileName and @extension variables you can use inside your SQL alongside other Frontmatter properties
@ -63,16 +65,16 @@ Fixes:
Technical:
- Other plugins can now register flags to allow for extra configuration
# 0.27.0 (2025-02-17)
## 0.27.0 (2025-02-17)
- Better syntax highlighting! Now it highlights SQL query parts
- Support for comments. You can now add comments like `--` and `/* */` to you queries
# 0.26.0 (2025-02-16)
## 0.26.0 (2025-02-16)
- Better language parser! Reworked from the ground up, now SQLSeal uses Ohm.js parser which works much better and opens much more possibilities (like syntax highlighting also introduced in this version).
- Syntax highlighting! Your SQLSeal text is now being highlighted helping you spot potential syntax issues quickly. The highlighting will be iterated on so please join discussion on our Discord!
- fix: database did not load properly when vault name consisted spaces or was in the subfolder (mostly causing issue on mobile)
# 0.25.0 (2025-02-06)
## 0.25.0 (2025-02-06)
- feat: you can now generate list of links
- feat: lists are now properly rendered
- feat: ability to edit column names in CSV files
@ -80,125 +82,125 @@ Technical:
- fix: empty links no longer generate errors
- fix: fixed how lists behave when you refresh the file
# 0.24.2 (2025-02-05)
## 0.24.2 (2025-02-05)
- Fixed how links are displayed. Now you can use links as `a(href)` or `a(href, name)`.
# 0.24.1 (2025-02-05)
## 0.24.1 (2025-02-05)
- Added `path` columns to `tags` and `tasks` tables. Thanks to that you can use `NATURAL JOIN`s now to connect join them.
# 0.24.0 (2025-02-05)
## 0.24.0 (2025-02-05)
- Better column names! Now non-latin characters are being romanised so you can access your properties (and other column names) easier: geändert becomes geandert, 类别 becomes lei_bie, ノートタイプ becomes nototaipu, etc.
# 0.23.2 (2025-02-04)
## 0.23.2 (2025-02-04)
- No longer captalising renderer options (it was breaking SQLSeal Charts integration)
# 0.23.1 (2025-02-04)
## 0.23.1 (2025-02-04)
- SQLSeal now works with multiple vaults open. It registers database with unique filename to avoid conflicts.
# 0.23.0 (2025-02-04)
## 0.23.0 (2025-02-04)
- Added support for wikilinks in the `a` function. You can now use markdown links in your queries.
- Improved how links are being rendered. Now using native links with the preview on hover
# 0.22.3 (2025-02-04)
## 0.22.3 (2025-02-04)
- Ability to enable / disable query refreshing individually for each query (REFRESH / NO REFRESH statement)
- (Advanced) Added ability to show SQL execution plan for the query
# 0.22.2 (2025-02-01)
## 0.22.2 (2025-02-01)
- Added index to tags.tag which was missed in the last release
# 0.22.1 (2025-02-01)
## 0.22.1 (2025-02-01)
- Added indexes to the `files`, `tags`, `links` and `tasks` tables. This should speed up a lot of common queries performed agains these tables.
# 0.22.0 (2025-01-31)
## 0.22.0 (2025-01-31)
Breaking change: `created_at` and `modified_at` has been changed from JS unix epoch to ISO 8601 dates (human readable). Thanks to that many of date operations are now easier to perform.
Example, extracting year from the creation date:
Before: `strftime("%Y", datetime(ROUND(created_at / 1000), 'unixepoch'))`
Now: `strftime("%Y", created_at)`
# 0.21.3 (2025-01-31)
## 0.21.3 (2025-01-31)
- fix: fixed issue with inline queries breaking rendering
# 0.21.2 (2025-01-30)
## 0.21.2 (2025-01-30)
- feat: now dynamic refreshing of the views can be disabled in the settings
# 0.21.1 (2025-01-30)
## 0.21.1 (2025-01-30)
- feat: ability to register new tables from other plugins
# 0.21.0 (2025-01-30)
## 0.21.0 (2025-01-30)
- feat: other plugins can now register custom database functions
- feat: improved API for other plugins
- chore: added subproject to release types for other plugins to use. See [`@hypersphere/sqlseal` package in NPM](https://www.npmjs.com/package/@hypersphere/sqlseal) for more info.
# 0.20.0 (2025-01-28)
## 0.20.0 (2025-01-28)
Added `links` table containing all the references between files. [Read more about the file structure in our documentation](https://hypersphere.blog/sql-seal/data-sources/vault-data.html#links-table)
# 0.19.2 (2025-01-27)
## 0.19.2 (2025-01-27)
Fixed issue with files and folders containing dots in them.
# 0.19.1 (2025-01-27)
## 0.19.1 (2025-01-27)
Added logging for unprocessable data to help reporting.
# 0.19.0 (2025-01-24)
## 0.19.0 (2025-01-24)
Adding support for JSON and JSON5 files. You can now query these data types and create tables based on them. You can use JSONPath to traverse the JSON files and extract data from it. [See more in the documentation](http://hypersphere.blog/sql-seal/data-sources/json-and-json5.html).
Added support for viewing JSON files. For now it is very simple preview with plans to expand in the future. You can disable this preview in the settings.
# 0.18.1 (2025-01-23)
## 0.18.1 (2025-01-23)
Adding created_at, modified_at and file_size field to files table.
Also added documentation and fixed issue with files not refreshing already visible views.
# 0.18.0 (2025-01-23)
## 0.18.0 (2025-01-23)
Synchronisation code has been greatly refactored. Thanks to that many common issue with files not being refreshed or pointing to the wrong file occasionally should be fixed.
Also fixed many instances of potential memory leaks. Now plugin should be more stable and reliable.
# 0.17.0 (2025-01-20)
## 0.17.0 (2025-01-20)
Adding support for `LIST` view. You can now set custom classes for `HTML` (table) view.
# 0.16.1 (2025-01-17)
## 0.16.1 (2025-01-17)
Fixed issue with variables causing error. Now you can pass variables from your frontmatter with no problem.
# 0.16.0 (2025-01-16)
## 0.16.0 (2025-01-16)
Now you can query tables in your note and use them as data source! [Check more information in the documentation](http://hypersphere.blog/sql-seal/query-markdown-tables.html).
# 0.15.0 (2025-01-15)
## 0.15.0 (2025-01-15)
This version brings plenty of the bugfixes
- fix: fix issue with local links not being clickable
- fix: query is now less case sensitive. You can use keywords like `TABLE`, `HTML`, `MARKDOWN` in any casing you want
- fix: improved SQL parser - now more complex syntax like recursive CTE, window functions, etc. should work properly. Migrated from `node-sql-parser` to `sql-parser-cst`
- fix: codeblocks now observe only the tables that are relevant to them instead of all. This should fix some tables refresh too often
# 0.14.1 (2025-01-13)
## 0.14.1 (2025-01-13)
- fix: fixed the issue where rows with extra data in them (rows with more columns that a header) were not synchronised correctly
- fix: fixed the issue where queries with lowercase SELECT would not work in certain cases.
# 0.14.0 (2025-01-10)
## 0.14.0 (2025-01-10)
Added support for inline code blocks. Create a codeblock (backtick `) and use prefix S> to indicate it's SQLSeal query.
# 0.13.0 (2025-01-09)
## 0.13.0 (2025-01-09)
Huge upgrade to the code codebase. SQLSeal should be now much faster and more reliable thanks to the following:
- Rewritten how files are synched - now each CSV file creates AT MOST one table in the database (synchronisations are being reused accross files)
- Rewritten SQL parser - this enables using more advanced SQLite functionality like recursive CTEs, UNIONS and `json_each`!
- (minor, technical): code got restructured and fixed so it's easier to contribute.
# 0.12.4 (2025-01-09)
## 0.12.4 (2025-01-09)
- added "Create CSV file" option in context menu in file explorer.
# 0.12.3 (2024-12-20)
## 0.12.3 (2024-12-20)
- fix: Fixing completed flag in tasks being reversed
# 0.12.2 (2024-12-20)
## 0.12.2 (2024-12-20)
fix: Fixed issue with CTE table names being incorrectly processed causing error
# 0.12.1 (2024-12-20)
## 0.12.1 (2024-12-20)
fix: Fixed issue with the library on mobile - now it should load properly.
# 0.12.0 (2024-12-19)
## 0.12.0 (2024-12-19)
This update does not bring any functionality changes yet but it fixes some problems with underlying architecture, helping to make SQLSeal more efficient in the future. If you work with many datasets, you might see slightly smaller memory usage as the database is now persisted in the IndexedDb (tech comment: as block storage).
Technical update:
Database has been moved to WebWorker, making data loading and retrieval slightly more efficient. Also integrated with [Absurd-SQL](https://github.com/jlongster/absurd-sql) which allows to store SQLite inside IndexedDb block storage, meaning it can be persisted and offloaded from the memory. Due to Obsidian API limitations, we still need to load huge files CSV to the memory instead of streaming their content but once the content is loaded, the memory usage should significantly go down now. More changes using this update to come soon!
# 0.11.0 (2024-11-24)
## 0.11.0 (2024-11-24)
The biggest update yet with plenty of exciting features:
Added CSV Viewer! Now you can see all your CSV files in your vault in the file explorer and open it to preview the data. You can also edit the data in place (remember always to backup your files!)
@ -208,7 +210,7 @@ Now the files in the queries are resolved relatively to the file they are in. Yo
Added more lax parser implementation for now
Minor: updated dependencies to the latest versions
# 0.10.1 (2024-11-07)
## 0.10.1 (2024-11-07)
Many small fixes:
- Changed how parsing is done to simplify code greatly
- Grid now supports dark mode
@ -216,48 +218,48 @@ Many small fixes:
- Restored ability to have `TABLE` syntax without select.
# 0.10.0 (2024-11-06)
## 0.10.0 (2024-11-06)
SQLSeal is now compatible with mobile! Plugin now uses SQL.JS instead of better-sqlite3 which is written in Web Assembly so no longer native binaries are needed. This makes plugin portable.
Also reworked implementation of the parser from Antlr4TS into Antlr4.
# 0.9.2 (2024-11-03)
## 0.9.2 (2024-11-03)
Adding checkbox method to display boolean values as checkboxes
Fixing how updates are parsed
# 0.9.1 (2024-11-03)
## 0.9.1 (2024-11-03)
Turning off verbose mode
# 0.9.0 (2024-11-03)
## 0.9.0 (2024-11-03)
We now use proper grid library to render data. This allow for many great features like pagination, sorting and more visally pleasing UI out of the box.
Rewritten internal communication to use Signals.
# 0.8.0 (2024-09-05)
## 0.8.0 (2024-09-05)
Now you can embed links and images (both local and external). Introduced `a` and `img` custom SQL functions.
# 0.7.0 (2024-09-02)
## 0.7.0 (2024-09-02)
A lot of changes packed in this one! Now SQLSeal is parsing the input using proper language parser (thanks to ANTLR4) rather than relying on RegExes. This finally allows for proper support of CTE statements (WITH) and fixes a lot of other minor problems with the syntax.
Also improved the way files are being observed and updated which should lead to performance improvements.
# 0.6.0 (2024-09-01)
## 0.6.0 (2024-09-01)
- fixing CSV loading when names has been converted to camelCase
- Fixing constantly reloading files when modifying sql inside the same block that has table declaration in
- Auto-parsing JSON values with JSON5
- Better errors when error occurs in select statement
# 0.5.0 (2024-08-25)
## 0.5.0 (2024-08-25)
- Adding support for JSON objects! Automatically detecting JSON in the frontmatter and converting it to JSON type in SQLite. You can query fields using built-in SQLite functions like `json_extract`, `json_array_length`, etc. More about SQlite functionality [can be found here.](https://www.sqlite.org/json1.html).
- Fixed issue with bookean types not being saved properly.
# 0.4.1 (2024-08-25)
## 0.4.1 (2024-08-25)
- Fixed issue with potential duplicate column sql error when two different fields resolve to the same uppercase structure
# 0.4.0 (2024-08-23)
## 0.4.0 (2024-08-23)
- Fixed issue with better_sqlite3 not loading on different systems and architectures
# 0.3.0 (2024-08-23)
## 0.3.0 (2024-08-23)
- Added ability to query files in the fault directly.
- Added observability - when CSV or file in the vault is changed, all SELECTS that uses it should update too
- Added custom class to sqlseal tables and ability to scroll vertically when the data is overflowing horizontally.
# 0.1.0 (2024-06-23)
## 0.1.0 (2024-06-23)
- Initial release. Allows to create tables based on CSV files in your vault and query them using SQL.

View file

@ -37,6 +37,7 @@ export default defineConfig({
{ text: 'Links and Images', link: '/links-and-images' },
{ text: 'CSV Viewer', link: '/csv-viewer' },
{ text: 'Query Configuration', link: '/query-configuration' },
{ text: 'Define Column Types', link: '/define-column-types' },
{ text: 'Troubleshooting', link: '/troubleshooting' },
{ text: 'Future Plans', link: '/future-plans' },
{ text: 'Changelog', link: '/changelog' }

View file

@ -0,0 +1,7 @@
# Define Column Types (CSV files)
SQLSeal sets types of your data automatically. This is the mechanism which for most of the cases works perfectly, but for more advanced use-cases, you might want to force specific type of the column (for example you might have ids which you want to treat like text, even if they consist of just the numbers). To enforce a specific type, you can set individual types inside the CSV Viewer:
![Setting Column Type](./setting-column-type.png)
Changing data type enables corresponding controls.

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

View file

@ -31,11 +31,11 @@
"@types/estraverse": "^5.1.7",
"@types/jest": "^29.5.14",
"@types/lodash": "^4.17.16",
"@types/node": "^22.15.12",
"@types/papaparse": "^5.3.15",
"@types/node": "^22.15.18",
"@types/papaparse": "^5.3.16",
"@types/sql.js": "^1.4.9",
"@typescript-eslint/eslint-plugin": "8.32.0",
"@typescript-eslint/parser": "8.32.0",
"@typescript-eslint/eslint-plugin": "8.32.1",
"@typescript-eslint/parser": "8.32.1",
"builtin-modules": "5.0.0",
"electron-rebuild": "^3.2.9",
"esbuild": "0.25.4",
@ -44,23 +44,23 @@
"jest": "^29.7.0",
"obsidian": "^1.8.7",
"prettier": "3.5.3",
"ts-jest": "^29.3.2",
"ts-jest": "^29.3.4",
"tslib": "2.8.1",
"typescript": "5.8.3",
"vitepress": "^1.6.3",
"vue": "^3.5.13"
"vue": "^3.5.14"
},
"dependencies": {
"@ag-grid-community/theming": "^32.3.5",
"@codemirror/language": "^6.11.0",
"@codemirror/state": "^6.5.2",
"@codemirror/view": "^6.36.7",
"@codemirror/view": "^6.36.8",
"@hypersphere/omnibus": "^0.1.6",
"@jlongster/sql.js": "^1.6.7",
"@types/jsonpath": "^0.2.4",
"@vanakat/plugin-api": "^0.2.1",
"absurd-sql": "^0.0.54",
"ag-grid-community": "^33.2.4",
"ag-grid-community": "^33.3.0",
"comlink": "^4.4.2",
"esbuild-plugin-polyfill-node": "^0.3.0",
"esprima": "^4.0.1",

File diff suppressed because it is too large Load diff

View file

@ -4,6 +4,7 @@ import * as Comlink from 'comlink'
import workerCode from 'virtual:worker-code'
import { WorkerDatabase } from "./worker/database";
import { sanitise } from "../utils/sanitiseColumn";
import { ColumnDefinition } from "../utils/types";
export class SqlSealDatabase {
db: Comlink.Remote<WorkerDatabase>
@ -83,6 +84,10 @@ export class SqlSealDatabase {
await this.db.createTableNoTypes(name, columns, noDrop)
}
async createTable(name: string, columns: ColumnDefinition[], noDrop?: boolean) {
this.db.createTable(name, columns, noDrop)
}
async createIndex(indexName: string, tableName: string, columns: string[]) {
await this.db.createIndex(indexName, tableName, columns)
}

View file

@ -9,7 +9,8 @@ import { SQLiteFS } from 'absurd-sql';
import IndexedDBBackend from '../../../node_modules/absurd-sql/dist/indexeddb-backend.js';
import type { BindParams, Database, Statement } from "sql.js";
import { sanitise } from "../../utils/sanitiseColumn";
import { uniq } from "lodash";
import { uniq, uniqBy } from "lodash";
import { ColumnDefinition } from "../../utils/types";
function toObjectArray(stmt: Statement) {
@ -131,6 +132,22 @@ export class WorkerDatabase {
this.db.run(createStmt)
}
async createTable(tableName: string, columns: ColumnDefinition[], noDrop: boolean = false) {
const fields = uniqBy(columns.map(c => ({
...c,
name: sanitise(c.name)
})), 'name')
if (!noDrop) {
await this.dropTable(tableName)
}
const fieldDefinitions = fields.map(c => c.name) // Setting type inside engine changes nothing for SQLite
const createStmt = `CREATE TABLE IF NOT EXISTS ${tableName}(${fieldDefinitions.join(', ')})`
this.db.run(createStmt)
}
async clearTable(tableName: string) {
this.db.run(`DELETE FROM ${tableName}`)

View file

@ -92,7 +92,7 @@ export class Sync {
const { data, columns } = await syncObject.returnData()
const tableName = entry.table_name
await this.db.createTableNoTypes(tableName, columns)
await this.db.createTable(tableName, columns)
await this.db.insertData(tableName, data)
await this.tableDefinitionsRepo.update(entry.id, { file_hash: file.stat.mtime.toString() })
this.bus.trigger('change::' + tableName)

View file

@ -5,6 +5,7 @@ import { FilepathHasher } from "../../utils/hasher";
import { TableDefinitionExternal } from "../repository/tableDefinitions";
import { ParserTableDefinition } from "./types";
import { App } from "obsidian";
import { loadConfig } from "../../utils/csvConfig";
const DEFAULT_FILE_HASH = ''
@ -60,12 +61,12 @@ export class CsvFileSyncStrategy extends ISyncStrategy {
skipEmptyLines: true,
transformHeader: sanitise
})
// const typeStatements = toTypeStatements(parsed.meta.fields ?? [], parsed.data)
// const columns = Object.entries(typeStatements.types).map(([key, value]) => ({
// name: key,
// type: value as FieldTypes
// }));
return { data: parsed.data, columns: parsed.meta.fields ?? [] }
const config = await loadConfig(file, this.app.vault)
return { data: parsed.data, columns: parsed.meta.fields?.map(f => ({
name: f,
type: config.columnDefinitions[f]?.type ?? 'auto' as const
})) ?? [] }
}
}

View file

@ -53,7 +53,7 @@ export class JsonFileSyncStrategy extends ISyncStrategy {
throw new Error('Resulting data is not an array')
}
const columns = uniq(data.map(d => Object.keys(d)).flat())
const columns = uniq(data.map(d => Object.keys(d)).flat()).map(c => ({ name: c, type: 'auto' as const }))
return { columns, data: data }
}

View file

@ -171,7 +171,7 @@ export class MarkdownTableSyncStrategy extends ISyncStrategy {
return {
data,
columns: headers ?? []
columns: headers.map(c => ({ name: c, type: 'auto' as const })) ?? []
};
}
}

View file

@ -1,6 +1,7 @@
import { App } from "obsidian";
import { TableDefinitionExternal } from "../repository/tableDefinitions";
import { ParserTableDefinition } from "./types";
import { ColumnDefinition } from "../../utils/types";
export abstract class ISyncStrategy {
constructor(protected def: TableDefinitionExternal, protected app: App) {
@ -11,7 +12,7 @@ export abstract class ISyncStrategy {
}
abstract returnData(): Promise<{
data: Record<string, unknown>[],
columns: string[]
columns: ColumnDefinition[]
}>;
static async fromParser(def: ParserTableDefinition, app: App): Promise<ISyncStrategy> {

View file

@ -26,13 +26,13 @@ const getAgGridTheme = (theme: 'dark' | 'light') => {
} as const
}
class GridRendererCommunicator {
export class GridRendererCommunicator {
constructor(
private el: HTMLElement,
private config: Partial<GridOptions>,
private plugin: SqlSealPlugin | null,
private app: App,
private cellParser: ModernCellParser
private cellParser?: ModernCellParser
) {
this.initialize()
this.setupLayoutObservers()
@ -105,11 +105,11 @@ class GridRendererCommunicator {
theme: myTheme,
defaultColDef: {
resizable: false,
cellRendererSelector: () => {
cellRendererSelector: this.cellParser ? () => {
return {
component: ({ value }: { value: string }) => this.cellParser.render(value)
component: ({ value }: { value: string }) => this.cellParser!.render(value)
}
},
} : undefined,
autoHeight: true
},
autoSizeStrategy: {
@ -137,10 +137,6 @@ class GridRendererCommunicator {
if (!this.gridApi) {
throw new Error('Grid has not been initiated')
}
this.gridApi.setGridOption('columnDefs', columns.map((c: any) => ({
headerName: c,
valueGetter: (params) => params.data[c]
})))
this.gridApi.setGridOption('rowData', data)
this.gridApi.setGridOption('loading', false)
}
@ -194,7 +190,8 @@ export class GridRenderer implements RendererConfig {
cleanup: () => {
communicator.cleanup()
communicator.gridApi.destroy()
}
},
communicator
}
}
}

View file

@ -57,7 +57,7 @@ export class ListRenderer implements RendererConfig {
text: createEl('span', { text: c, cls: 'sqlseal-column-name' }) as any, // FIXME: this should be properly typed
cls: singleCol ? ['sqlseal-list-element', 'sqlseal-list-element-single'] : ['sqlseal-list-element-single']
})
const val: any = cellParser.render(d[c])
const val: any = cellParser!.render(d[c])
el.append(val)
el.dataset.sqlsealColumn = c
})

View file

@ -5,7 +5,6 @@ import { RendererConfig, RendererContext } from "../renderer/rendererRegistry";
import { displayError } from "../utils/ui";
import { ViewDefinition } from "../grammar/parser";
import { ParseResults } from "../cellParser/parseResults";
import { ModernCellParser } from "../cellParser/ModernCellParser";
const mapDataFromHeaders = (columns: string[], data: Record<string, any>[]) => {
return data.map(d => columns.map(c => String(d[c])))
@ -31,7 +30,7 @@ export class MarkdownRenderer implements RendererConfig {
}
render(config: ReturnType<typeof this.validateConfig>, el: HTMLElement, { cellParser } : RendererContext) {
const parseResult = new ParseResults(cellParser)
const parseResult = new ParseResults(cellParser!)
return {
render: ({ columns, data }: any) => {
const tab = getMarkdownTable({

View file

@ -3,7 +3,6 @@ import { App } from "obsidian";
import { RendererConfig, RendererContext } from "../renderer/rendererRegistry";
import { displayError } from "../utils/ui";
import { ViewDefinition } from "../grammar/parser";
import { ModernCellParser } from "../cellParser/ModernCellParser";
interface HTMLRendererConfig {
classNames: string[]
@ -73,7 +72,7 @@ export class TableRenderer implements RendererConfig {
data.forEach((d: any) => {
const row = body.createEl("tr")
columns.forEach((c: any) => {
const parsed = cellParser.render(d[c]) as string
const parsed = cellParser!.render(d[c]) as string
if (adjustLayout) {
const td = row.createEl("td")
td.createSpan({ text: parsed })

View file

@ -42,7 +42,7 @@ export class TemplateRenderer implements RendererConfig {
render: ({ columns, data, frontmatter }: any) => {
el.empty()
const parser = new ParseResults(cellParser, (el) => new Handlebars.SafeString(el.outerHTML))
const parser = new ParseResults(cellParser!, (el) => new Handlebars.SafeString(el.outerHTML))
// Seems to be the only way to render handlebars into DOM. Don't like it but what can we do.
el.innerHTML = config.template({

View file

@ -7,7 +7,7 @@ export interface DataFormat {
}
export interface RendererContext {
cellParser: ModernCellParser,
cellParser?: ModernCellParser,
sourcePath: string
}

4
src/styles/agGrid.scss Normal file
View file

@ -0,0 +1,4 @@
/* AG GRID CUSTOMS */
.ag-checkbox-input {
opacity: 1 !important;
}

View file

@ -3,4 +3,5 @@
@use 'inlineQueries';
@use 'listView';
@use 'syntaxHighlight';
@use 'obsidianMinimal';
@use 'obsidianMinimal';
@use 'agGrid';

46
src/utils/csvConfig.ts Normal file
View file

@ -0,0 +1,46 @@
import { TFile, Vault } from "obsidian"
import { parse as jsonParse, stringify as jsonStringify } from 'json5'
// export type ColumnType = 'auto' | 'number' | 'text'
export type ColumnType = string
interface ColumnDefinition {
type: ColumnType
}
export interface ConfigObject {
columnDefinitions: {[key: string]: ColumnDefinition }
}
export const loadConfig = async (file: TFile, vault: Vault): Promise<ConfigObject> => {
const config = {
columnDefinitions: {}
}
// Now loading saved one
const configPath = file!.path + '.sqlsealconfig'
const configFile = vault.getFileByPath(configPath)
if (!configFile) {
return config
}
const loadedConfig = await vault.read(configFile)
// decode
const decoded = jsonParse(loadedConfig)
return decoded
}
export const saveConfig = async (file: TFile, content: Object, vault: Vault) => {
const serialised = jsonStringify(content, null, 2)
// Check if exists
const configPath = file!.path + '.sqlsealconfig'
const configFile = vault.getFileByPath(configPath)
if (!configFile) {
// CREATE NEW
await vault.create(configPath, serialised)
return
}
await vault.modify(configFile, serialised)
}

7
src/utils/types.ts Normal file
View file

@ -0,0 +1,7 @@
// export type ColumnType = 'auto' | 'text' | 'number'
export type ColumnType = string
export interface ColumnDefinition {
name: string;
type: ColumnType
}

View file

@ -1,11 +1,15 @@
import { WorkspaceLeaf, TextFileView, Menu, Notice } from 'obsidian';
import { WorkspaceLeaf, TextFileView, Menu, Notice, MenuItem } from 'obsidian';
import { parse, unparse } from 'papaparse';
import { DeleteConfirmationModal } from '../modal/deleteConfirmationModal';
import { RenameColumnModal } from '../modal/renameColumnModal';
import { CodeSampleModal } from '../modal/showCodeSample';
import { GridRenderer } from '../renderer/GridRenderer';
import { GridRenderer, GridRendererCommunicator } from '../renderer/GridRenderer';
import { errorNotice } from '../utils/notice';
import { ModernCellParser } from '../cellParser/ModernCellParser';
import { ConfigObject, loadConfig, saveConfig } from 'src/utils/csvConfig';
import { ColumnType } from '../utils/types';
const delay = (n: number) => new Promise(resolve => setTimeout(resolve, n))
export const CSV_VIEW_TYPE = "csv-viewer" as const;
export const CSV_VIEW_EXTENSIONS = ['csv'];
@ -13,6 +17,7 @@ export const CSV_VIEW_EXTENSIONS = ['csv'];
export class CSVView extends TextFileView {
private content: string;
private table: HTMLTableElement;
private config: ConfigObject;
constructor(
leaf: WorkspaceLeaf,
@ -60,7 +65,7 @@ export class CSVView extends TextFileView {
private result: any;
updateRow(newRowData: Record<string, any>) {
this.result.data[parseInt(newRowData.__index, 10)] = newRowData;
this.saveData()
this.saveData(true)
}
deleteRow(idx: number) {
@ -102,6 +107,7 @@ export class CSVView extends TextFileView {
return d
})
this.saveData()
this.loadDataIntoGrid()
}
setIsEditable(newValue: boolean) {
@ -109,9 +115,25 @@ export class CSVView extends TextFileView {
// FIXME: if there already rendered view, use this to rerender it?
}
saveData() {
const output = unparse(this.result)
saveData(noRefresh: boolean = false) {
if (noRefresh) {
this.refreshSkip = Date.now() + 500
}
// Map results
const res = [...this.result.data].map(r => {
const r2 = {...r}
Object.keys(r).forEach(k => {
if (typeof r[k] === 'boolean') {
r2[k] = r[k] ? 1 : 0
}
})
return r2
})
const output = unparse({ ...this.result, data: res })
this.app.vault.modify(this.file!, output)
this.refreshTypes()
}
createRow() {
@ -120,6 +142,66 @@ export class CSVView extends TextFileView {
}
getColumnType(columnName: string) {
if (this.config.columnDefinitions[columnName]) {
return this.config.columnDefinitions[columnName].type
}
return 'auto'
}
async changeColumnType(columnName: string, type: ColumnType) {
const prev = this.config.columnDefinitions[columnName] ?? {}
this.config.columnDefinitions[columnName] = {
...prev,
type: type
}
await this.saveConfig()
}
async loadConfig() {
while (!this.file) {
await delay(100)
}
this.config = await loadConfig(this.file, this.app.vault)
}
async saveConfig() {
await saveConfig(this.file!, this.config, this.app.vault)
}
private getColumnConfigurations(columns: string[]) {
return columns.map(f => {
if (!this.config) {
return {
field: f
}
}
const def = this.config.columnDefinitions[f]?.type
if (!def || def === 'auto') {
return { field: f }
}
if (def === 'date') {
return {
field: f,
cellDataType: 'dateString'
}
}
return {
field: f,
cellDataType: def
}
})
}
refreshTypes() {
if (this.gridCommunicator) {
const columns = this.result.fields
if (columns && columns.length) {
this.gridCommunicator.gridApi.setGridOption('columnDefs', this.getColumnConfigurations(columns))
}
}
}
moveColumn(name: string, toIndex: number) {
let fields = this.result.fields as Array<string>
fields = fields.filter(f => f !== name)
@ -133,42 +215,82 @@ export class CSVView extends TextFileView {
}
api: any = null;
gridCommunicator: GridRendererCommunicator | null = null
refreshSkip: number = 0
formatWithTypes(d: Record<string, string | boolean | number>) {
Object.entries(this.config.columnDefinitions).forEach(([key, value]) => {
if (!d[key]) {
return
}
if (value.type === 'boolean') {
if (d[key] === 'false' || d[key] === '0') {
d[key] = false
}
d[key] = !!d[key]
}
if (value.type === 'number') {
if (d[key] === null || d[key] === '') {
d[key] = ''
} else {
d[key] = parseFloat(d[key] as string)
}
}
if (value.type === 'date') {
// try parsing
const val = d[key] as string
const date = new Date(val)
d[key] = date.toISOString().split('T')[0]
}
})
return d
}
prepareData() {
const result = parse(this.content, {
header: true,
skipEmptyLines: true,
});
const data = result.data.map((d: any, i) => ({
...this.formatWithTypes(d),
__index: i.toString()
}))
return {
data: data,
fields: result.meta.fields
}
}
loadDataIntoGrid() {
if (this.refreshSkip > Date.now()) {
return
}
requestAnimationFrame(() => {
const result = parse(this.content, {
header: true,
skipEmptyLines: true,
});
const data = result.data.map((d: any, i) => ({
...d,
__index: i.toString()
}))
this.result = {
data: data,
fields: result.meta.fields
}
this.api!.render({
data: data,
columns: result.meta.fields
})
const result = this.prepareData()
this.result = result
this.refreshTypes()
this.api!.render(result)
})
}
isLoading: boolean = false
private async renderCSV() {
if (this.api) {
this.loadDataIntoGrid()
if (this.isLoading) {
if (this.api) {
this.loadDataIntoGrid()
}
return
}
this.isLoading = true
this.contentEl.empty()
const csvEditorDiv = this.contentEl.createDiv({ cls: 'sql-seal-csv-editor' })
const buttonsRow = csvEditorDiv.createDiv({ cls: 'sql-seal-csv-viewer-buttons' })
await this.loadConfig()
if (this.enableEditing) {
const createColumn = buttonsRow.createEl('button', { text: 'Add Column' })
const createRow = buttonsRow.createEl('button', { text: 'Add Row' })
@ -199,16 +321,14 @@ export class CSVView extends TextFileView {
modal.open()
})
const grid = new GridRenderer(this.app, null)
const csvView = this;
const data = this.prepareData()
this.result = data
const api = grid.render({
columnDefs: this.getColumnConfigurations(data.fields ?? []),
defaultColDef: {
editable: this.enableEditing,
valueSetter: (e) => {
e.data[e.column.getUserProvidedColDef()?.headerName!] = e.newValue
return e.newValue
},
headerComponentParams: {
enableMenu: this.enableEditing,
showColumnMenu: function (e: any) {
@ -219,7 +339,7 @@ export class CSVView extends TextFileView {
item.onClick(() => {
const modal = new RenameColumnModal(csvView.app, (res) => {
csvView.renameColumn(
this.column.userProvidedColDef.headerName, res)
this.column.userProvidedColDef.field, res)
})
modal.open()
})
@ -228,7 +348,7 @@ export class CSVView extends TextFileView {
menu.addItem(item => {
item.setTitle('Delete Column')
item.onClick(() => {
const colName = this.column.userProvidedColDef.headerName
const colName = this.column.userProvidedColDef.field
const modal = new DeleteConfirmationModal(csvView.app, `column ${colName}`, () => {
csvView.deleteColumn(colName)
})
@ -236,7 +356,34 @@ export class CSVView extends TextFileView {
})
})
// FIXME: rework it to submenus.
menu.addSeparator()
menu.addItem(item => {
// item.setDisabled(true)
item.setTitle('Data Type')
// item.setIsLabel(true)
const ipfSubmenu = (item as any).setSubmenu();
const types = ['auto', 'text', 'number', 'boolean', 'date'] as ColumnType[]
const colName = this.column.userProvidedColDef.field
const current = csvView.getColumnType(colName)
types.forEach(type => {
ipfSubmenu.addItem((subItem: MenuItem) => {
const checkbox = type === current ? '✓ ' : ''
subItem.setTitle(checkbox + type)
subItem.onClick(() => {
csvView.changeColumnType(colName, type)
csvView.refreshTypes()
csvView.loadDataIntoGrid()
})
})
})
})
const pos = e.getBoundingClientRect();
menu.showAtPosition({ x: pos.x, y: pos.y + 20 })
}
@ -254,7 +401,7 @@ export class CSVView extends TextFileView {
if (!columnName) {
return
}
csvView.moveColumn(columnName?.headerName!, e.toIndex!)
csvView.moveColumn(columnName?.field!, e.toIndex!)
},
domLayout: 'normal',
getRowId: (p) => p.data.__index,
@ -280,9 +427,9 @@ export class CSVView extends TextFileView {
})
menu.showAtMouseEvent(e.event as any)
}
}, gridEl, { cellParser: this.cellParser, sourcePath: this.file?.path || '' })
}, gridEl, { sourcePath: this.file?.path || '' })
this.api = api;
this.loadDataIntoGrid()
this.gridCommunicator = api.communicator
api.render(data)
}
}

2
styles.css Normal file
View file

@ -0,0 +1,2 @@
.sqlseal-table-container{overflow-y:scroll}.sqlseal-table-container table{width:100%;border-collapse:collapse}.sqlseal-grid-wrapper{height:auto;position:relative}.block-language-sqlseal{overflow-y:auto}.sqlseal-error{padding:1em;background:#b80f0f;color:#fff}.sqlseal-notice{padding:1em;background:#1f3f98;color:#fff}.sqlseal-grid-error-message{box-sizing:border-box;width:80%;position:absolute;top:50%;left:50%;transform:translate(-50%,-50%);text-align:center;background:#b80f0f;font-size:.8em;color:#fff;padding:.5em 2em}.sqlseal-grid-error-message-overlay{content:"";position:absolute;inset:0;background:#fffc;z-index:5}.sqlseal-grid-error-message-overlay.hidden{display:none}.edit-block-button{z-index:1000}.sqlseal-markdown-table{font-family:monospace;white-space:pre-wrap}.sqlseal-markdown-table.no-wrap{display:inline-block;overflow-x:scroll;width:fit-content}.sqlseal-parse-error{background:#b80f0f;color:#fff;padding:.2em;border-radius:2px}.sqlseal-notice-error{color:#f26464}.sqlseal-notice-error:before{content:"";width:16px;height:16px;display:inline-block;margin-right:8px;transform:translateY(3px);background-image:url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24'%3E%3Cpath fill='%23ff0000' d='M12 2C6.48 2 2 6.48 2 12s4.48 10 10 10 10-4.48 10-10S17.52 2 12 2zm1 15h-2v-2h2v2zm0-4h-2V7h2v6z'/%3E%3C/svg%3E")}.sql-seal-csv-editor{display:flex;flex-direction:column;height:100%;gap:var(--size-4-4)}.sql-seal-csv-viewer{flex:1;display:flex;flex-direction:column}.sqlseal-grid-wrapper{height:100%;flex:1;display:flex;flex-direction:column}.ag-theme-quartz{flex:1}.sql-seal-csv-viewer-buttons{display:flex;gap:var(--size-4-4)}.sql-seal-modal-code{width:100%;font-family:monospace;padding:8px}.sqlseal-inline-result{display:inline-block;padding:0 4px;margin:0 2px;background-color:var(--background-secondary);border-radius:4px;font-size:.9em;cursor:pointer}.sqlseal-inline-result .error{color:var(--text-error);font-style:italic}.sqlseal-list-element-single .sqlseal-column-name{display:none}.sqlseal-list-element-single .sqlseal-column-name:after{content:": "}.sqlseal-list.show-column-names .sqlseal-list-element-single .sqlseal-column-name{display:inline}.sqlseal-element-inline-list{display:inline-block}.cm-sqlseal-block-query{--color: #297f30}.cm-sqlseal-block-view{--color: #c834dd}.cm-sqlseal-block-flag{--color: #d08306;font-weight:700;color:var(--color)}.cm-sqlseal-block-table{--color: #4985af}.cm-sqlseal-identifier{color:#2d7b33}.cm-sqlseal-function,.cm-sqlseal-parameter{color:#2c0e9b}.cm-sqlseal-comment{color:#7a7a7a;font-style:italic}.cm-sqlseal-keyword{color:var(--color);font-weight:700}.cm-sqlseal-literal{color:#03f}.cm-sqlseal-error{color:#eb2f2f;font-style:italic}.cm-sqlseal-template-keyword{color:#e3ab53}.theme-dark .cm-sqlseal-template-keyword{color:#dd8e10}.theme-dark .cm-sqlseal-block-error{color:red}.theme-dark .cm-sqlseal-block-query{--color: #5eff6a}.theme-dark .cm-sqlseal-comment{color:#898787}.theme-dark .cm-sqlseal-parameter{color:#95ed22}.theme-dark .cm-sqlseal-literal{color:#ff0}.theme-dark .cm-sqlseal-block-view{--color: #e945ff}.theme-dark .cm-sqlseal-block-flag{--color: #e3ab53}.theme-dark .cm-sqlseal-block-table{--color: #56b5fa}.theme-dark .cm-sqlseal-function{color:#c4b8ef}.theme-dark .cm-sqlseal-identifier{color:#6ff77a}:is(.cm-sqlseal-block-query,.cm-sqlseal-block-view,.cm-sqlseal-block-flag,.cm-sqlseal-block-table):before{background-color:var(--color);width:5px;top:-3px;bottom:-3px;left:0;content:"";display:block;position:absolute}.cm-indent+:is(.cm-sqlseal-block-query,.cm-sqlseal-block-view,.cm-sqlseal-block-flag,.cm-sqlseal-block-table):before{display:none}.markdown-source-view.mod-cm6.is-readable-line-width .cm-contentContainer.cm-contentContainer>.cm-content>:has(>.block-language-sqlseal table.dataview){width:var(--container-dataview-table-width);max-width:var(--container-table-max-width)}.ag-checkbox-input{opacity:1!important}
/*# sourceMappingURL=styles.css.map */