mirror of
https://github.com/core-hn/pseudobsidian-ization.git
synced 2026-07-22 06:57:06 +00:00
release: v0.1.5 — noScribe import, scan candidats, exceptions
This commit is contained in:
commit
561b43f8a9
38 changed files with 4064 additions and 363 deletions
|
|
@ -1,5 +1,40 @@
|
|||
# Changelog
|
||||
|
||||
## [prod] v0.1.5 — 17 mai 2026
|
||||
|
||||
### Nouvelles fonctionnalités
|
||||
|
||||
**Intégration noScribe**
|
||||
- **`VttParser`** — WebVTT standard avec word timestamps Whisper (`<00:00:01.240><c>mot</c>`), tags `<v Nom>`, round-trip garanti
|
||||
- **`NoScribeHtmlParser`** — HTML Qt Rich Text noScribe : ancres `ts_DEBUT_FIN_SXX` (ms), labels locuteurs (`S00:`, `S01:`), timestamps d'affichage filtrés, entités HTML décodées ; `extractAudioSource()` lit `<meta name="audio_source">`
|
||||
- **`NoScribeVttParser`** — VTT noScribe v0.7 : cues splitées (label / timestamp affichage / texte), détection locuteur par préfixe `SXX:`, entités HTML, source audio depuis `NOTE media:`
|
||||
- **Format Markdown noScribe** — `**S00** [HH:MM:SS] : texte` par tour ; `pseudobs-format: html|vtt` et `pseudobs-audio` optionnel en frontmatter ; timestamps word-level dans `<basename>.words.json`
|
||||
- **Import audio automatique** — depuis la meta tag HTML ou le dossier source (VTT, un seul fichier audio)
|
||||
- **`Redaction.ts`** — caviardage par syllabes avec `🀫` ; `generateRedaction()` et `isRedaction()`
|
||||
- **Re-export VTT** (commande "Exporter en VTT") — WebVTT propre depuis `.md` pseudonymisé + `.words.json`
|
||||
|
||||
**Modale de scan — candidats par occurrence**
|
||||
- **`MappingScanReviewModal`** — colonne occurrences cliquable (compteur mis à jour en direct : `N / total` si exceptions)
|
||||
- **`OccurrencesContextModal`** — cartes par occurrence (✓ / ✗ / ⚠) avec contexte, callback "Confirmer la sélection", sans application immédiate
|
||||
- **Bouton "Enregistrer les exceptions"** — sauvegarde les décisions dans le `mapping.json` sans pseudonymiser
|
||||
- `MappingRuleResult` porte maintenant `occurrences: Occurrence[]`
|
||||
|
||||
**Exceptions**
|
||||
- Type **`IgnoredOccurrence`** (`{text, contextBefore, contextAfter}`) sur `MappingRule.ignoredOccurrences`
|
||||
- Persistance dans `mapping.json`, chargement au démarrage — surlignage rouge sans re-scan
|
||||
- **Surlignage rouge** (`pseudobs-exception`, sensible à la casse, priorité 0)
|
||||
- **Section "Exceptions" dans l'onglet Mappings** — cartes contextuelles avec bouton supprimer
|
||||
|
||||
### Modifications
|
||||
- Labels de statut clarifiés : `validated` → "Actif", `ignored` → "Ignoré", `partial` → "Partiel", `suggested` → "Suggéré" ; colonne renommée "État"
|
||||
- `getValidatedFor()` inclut maintenant `partial` comme état actif (avec `validated`)
|
||||
- Bouton "Scanner" de `PseudonymizationView` corrigé (`occurrences` manquant dans `MappingRuleResult`)
|
||||
|
||||
### Tests
|
||||
149 tests · 10 suites · `VttParser`, `NoScribeHtmlParser`, `NoScribeVttParser`, `markdownToVtt`, `extractWordData` couverts ; fixtures : `fight_club.vtt`, `juste-leblanc.html`
|
||||
|
||||
---
|
||||
|
||||
## [prod] v0.1.3 — 14 mai 2026
|
||||
|
||||
**Branche :** `main` | **Tag :** `0.1.3`
|
||||
|
|
|
|||
35
CHANGELOG.md
35
CHANGELOG.md
|
|
@ -2,6 +2,41 @@
|
|||
|
||||
> Previous entries in French: [CHANGELOG.fr.md](CHANGELOG.fr.md)
|
||||
|
||||
## [prod] v0.1.5 — 17 May 2026
|
||||
|
||||
### New features
|
||||
|
||||
**noScribe integration**
|
||||
- **`VttParser`** — standard WebVTT with Whisper word-level timestamps (`<00:00:01.240><c>word</c>`), speaker tags `<v Name>`, round-trip guarantee
|
||||
- **`NoScribeHtmlParser`** — Qt Rich Text HTML produced by noScribe: `ts_START_END_SXX` anchors (ms), speaker labels (`S00:`, `S01:`), display timestamps filtered, HTML entities decoded; `extractAudioSource()` reads `<meta name="audio_source">`
|
||||
- **`NoScribeVttParser`** — noScribe VTT v0.7: split cues (label / display timestamp / text), speaker detection from `SXX:` prefix, HTML entity decoding, audio source from `NOTE media:`
|
||||
- **noScribe Markdown format** — `**S00** [HH:MM:SS] : text` per turn; `pseudobs-format: html|vtt` and optional `pseudobs-audio` in frontmatter; word timestamps in `<basename>.words.json`
|
||||
- **Audio import** — automatic on HTML import (via `audio_source` meta) and on VTT import via file picker (single audio file in source folder)
|
||||
- **`Redaction.ts`** — syllable-count redaction with `🀫`; `generateRedaction()` and `isRedaction()`
|
||||
- **VTT re-export** (`Export as VTT` command) — clean WebVTT from pseudonymized `.md` + `.words.json`
|
||||
|
||||
**Scan modal — per-occurrence candidates**
|
||||
- **`MappingScanReviewModal`** — occurrences column is a clickable button (count updated live to `N / total` when some are ignored)
|
||||
- **`OccurrencesContextModal`** — per-occurrence cards (✓ / ✗ / ⚠) with context, "Confirm selection" callback, no immediate application
|
||||
- **"Save exceptions" button** — saves ignored decisions to `mapping.json` without pseudonymizing
|
||||
- `MappingRuleResult` now carries `occurrences: Occurrence[]`
|
||||
|
||||
**Exceptions**
|
||||
- **`IgnoredOccurrence`** type (`{text, contextBefore, contextAfter}`) on `MappingRule.ignoredOccurrences`
|
||||
- Persisted in `mapping.json`, loaded at startup — red highlighting without re-scanning
|
||||
- **Red highlighting** (`pseudobs-exception`, case-sensitive, priority 0)
|
||||
- **Exceptions section in Mappings tab** — context cards with delete button
|
||||
|
||||
### Changes
|
||||
- Status labels: `validated` → "Active", `ignored` → "Ignored", `partial` → "Partial", `suggested` → "Suggested"; column renamed "State"
|
||||
- `getValidatedFor()` now includes `partial` as active (alongside `validated`)
|
||||
- `PseudonymizationView` scan button fixed (missing `occurrences` in `MappingRuleResult`)
|
||||
|
||||
### Tests
|
||||
149 tests · 10 suites · `VttParser`, `NoScribeHtmlParser`, `NoScribeVttParser`, `markdownToVtt`, `extractWordData` covered; fixtures: `fight_club.vtt`, `juste-leblanc.html`
|
||||
|
||||
---
|
||||
|
||||
## [prod] v0.1.4 — 16 May 2026
|
||||
|
||||
**Branch:** `main` | **Tag:** upcoming
|
||||
|
|
|
|||
19
README.fr.md
19
README.fr.md
|
|
@ -10,13 +10,13 @@ Plugin Obsidian de **pseudonymisation et de correction de transcriptions** pour
|
|||
## Workflow
|
||||
|
||||
```
|
||||
Transcription brute (.srt, .cha, .md, .txt)
|
||||
↓ import automatique
|
||||
Obsidian — édition native Markdown
|
||||
↓ pseudonymisation (manuelle ou NER)
|
||||
Fichier source annoté + table de correspondance (séparée)
|
||||
noScribe (.html, .vtt) ou transcription brute (.srt, .cha, .md, .txt)
|
||||
↓ import automatique (fichier audio importé en même temps)
|
||||
Obsidian — édition native Markdown (**S00** [HH:MM:SS] : texte)
|
||||
↓ pseudonymisation (manuelle · NER · scan dictionnaires)
|
||||
Fichier source annoté + table de correspondance + timestamps (.words.json)
|
||||
↓ export
|
||||
Transcription pseudonymisée (.pseudonymized.*)
|
||||
Transcription pseudonymisée (.pseudonymized.md / .pseudonymized.vtt)
|
||||
```
|
||||
|
||||
Deux approches, combinables librement :
|
||||
|
|
@ -44,12 +44,17 @@ L'assistant est relançable à tout moment : Paramètres → Pseudonymizer tool
|
|||
|
||||
| Format | Extension | Notes |
|
||||
|---|---|---|
|
||||
| **HTML noScribe** | `.html` | Qt Rich Text produit par noScribe — locuteurs, timestamps word-level, chemin audio |
|
||||
| **VTT noScribe** | `.vtt` | Sortie noScribe v0.7 — aussi WebVTT Whisper standard avec timestamps mot par mot |
|
||||
| Sous-titres horodatés | `.srt` | Sortie Whisper / IA — horodatages et structure préservés |
|
||||
| CHAT / CLAN | `.cha`, `.chat` | Lignes `@`, `*`, `%` préservées |
|
||||
| Markdown annoté | `.md` | Conventions Jefferson ou ICOR |
|
||||
| Texte brut | `.txt` | Sans marqueurs de convention |
|
||||
|
||||
Les fichiers `.srt` et `.cha` sont automatiquement convertis en Markdown à l'import. Un fichier de mapping JSON vide est créé en même temps.
|
||||
Tous les formats sont automatiquement convertis en Markdown à l'import. En parallèle, le plugin crée :
|
||||
- `<basename>.mapping.json` — règles de pseudonymisation
|
||||
- `<basename>.words.json` — timestamps word-level (noScribe / Whisper uniquement), utilisé pour le re-export VTT
|
||||
- Si un fichier audio est référencé dans la transcription, il est importé dans le vault automatiquement.
|
||||
|
||||
> Installez le plugin [Data Files Editor](https://github.com/zuktol/obsidian-data-files-editor) pour visionner les fichiers de mapping JSON directement depuis votre vault.
|
||||
|
||||
|
|
|
|||
39
README.md
39
README.md
|
|
@ -9,13 +9,13 @@ An Obsidian plugin for **pseudonymizing and correcting interactional transcripts
|
|||
## Workflow
|
||||
|
||||
```
|
||||
Raw transcript (.srt, .cha, .md, .txt)
|
||||
↓ automatic import
|
||||
Obsidian — native Markdown editing
|
||||
↓ pseudonymization (manual or NER)
|
||||
Annotated source file + correspondence table (separate)
|
||||
noScribe (.html, .vtt) or raw transcript (.srt, .cha, .md, .txt)
|
||||
↓ automatic import (audio file imported alongside)
|
||||
Obsidian — native Markdown editing (**S00** [HH:MM:SS] : text)
|
||||
↓ pseudonymization (manual · NER · dictionary scan)
|
||||
Annotated source file + correspondence table + word timestamps (.words.json)
|
||||
↓ export
|
||||
Pseudonymized transcript (.pseudonymized.*)
|
||||
Pseudonymized transcript (.pseudonymized.md / .pseudonymized.vtt)
|
||||
```
|
||||
|
||||
Two approaches, freely combined:
|
||||
|
|
@ -45,12 +45,17 @@ The wizard can be relaunched at any time: Settings → Pseudonymizer Tool → Se
|
|||
|
||||
| Format | Extension | Notes |
|
||||
|---|---|---|
|
||||
| **noScribe HTML** | `.html` | Qt Rich Text from noScribe — speaker labels, word timestamps, audio path |
|
||||
| **noScribe VTT** | `.vtt` | noScribe v0.7 output — also standard Whisper WebVTT with word timestamps |
|
||||
| Timestamped subtitles | `.srt` | Whisper / AI output — timestamps and structure preserved |
|
||||
| CHAT / CLAN | `.cha`, `.chat` | `@`, `*`, `%` lines preserved |
|
||||
| Annotated Markdown | `.md` | Jefferson or ICOR conventions |
|
||||
| Plain text | `.txt` | No convention markers |
|
||||
|
||||
`.srt` and `.cha` files are automatically converted to Markdown on import. An empty mapping JSON file is created at the same time.
|
||||
All formats are automatically converted to Markdown on import. Alongside the `.md`, the plugin creates:
|
||||
- `<basename>.mapping.json` — pseudonymization rules
|
||||
- `<basename>.words.json` — word-level timestamps (noScribe / Whisper only), used for VTT re-export
|
||||
- If an audio file is referenced in the transcript, it is imported to the vault automatically.
|
||||
|
||||
> Install the [Data Files Editor](https://github.com/zuktol/obsidian-data-files-editor) plugin to view mapping JSON files directly in your vault.
|
||||
|
||||
|
|
@ -103,9 +108,9 @@ Accessible via the ribbon icon or `Ctrl+P → Pseudonymization: open panel`.
|
|||
|
||||
| Tab | Content |
|
||||
|---|---|
|
||||
| **Mappings** | Active rules · Edit · Delete · Add · Scan file |
|
||||
| **Mappings** | Active rules · Edit · Delete · Add · Scan file · **Exceptions section** |
|
||||
| **Dictionaries** | Mini cards · Dictionary scan · Local import |
|
||||
| **Exports** | Pseudonymize and export · Export correspondence table |
|
||||
| **Exports** | Pseudonymize and export · Export correspondence table · **Export as VTT** |
|
||||
| **NER** | Visible if NER enabled · Identify candidates · Confidence threshold · Function words |
|
||||
|
||||
### Highlighting and markers
|
||||
|
|
@ -117,13 +122,15 @@ Highlighting is active in all open files, including `.pseudonymized.*` export fi
|
|||
| 🟠 Orange + outline | Source term still present — to be pseudonymized |
|
||||
| 🟢 Green + underline | Pseudonym applied directly in the file |
|
||||
| 🔵 Blue + outline | NER candidate — no rule yet |
|
||||
| 🔴 Red + underline | **Exception** — occurrence explicitly ignored during scan (case-sensitive; persisted in mapping) |
|
||||
|
||||
In exported files, pseudonyms are wrapped in `{{Pierre}}` markers to distinguish them from raw data (enabled by default, configurable in settings).
|
||||
|
||||
### Correspondence tables
|
||||
|
||||
- Three scope levels: `file.mapping.json` · `folder.mapping.json` · `vault.mapping.json`
|
||||
- Per-occurrence statuses: `validated`, `ignored`, `partial`, `conflict`, `needs_review`
|
||||
- Per-rule statuses: **Active** (validated), **Partial**, **Ignored**, **Suggested**
|
||||
- Per-occurrence ignored exceptions: `IgnoredOccurrence {text, contextBefore, contextAfter}` stored in the rule
|
||||
- **Z-index priority**: free integer — longer entities take precedence (`Saint-Jean-de-Luz` > `Jean`)
|
||||
- JSON format documented in `SPECS.md §5`
|
||||
|
||||
|
|
@ -183,14 +190,16 @@ Repository structure:
|
|||
src/
|
||||
├── main.ts # Obsidian entry point
|
||||
├── settings.ts # Persistent settings
|
||||
├── types.ts # Shared types
|
||||
├── types.ts # Shared types (MappingRule, IgnoredOccurrence, …)
|
||||
├── i18n/ # Internationalization (en, fr)
|
||||
├── parsers/ # SrtParser, ChatParser, TranscriptConverter
|
||||
├── parsers/ # SrtParser, ChatParser, VttParser,
|
||||
│ # NoScribeHtmlParser, NoScribeVttParser, TranscriptConverter
|
||||
├── mappings/ # MappingStore, ScopeResolver
|
||||
├── pseudonymizer/ # Engine, ReplacementPlanner, SpanProtector
|
||||
├── pseudonymizer/ # Engine, ReplacementPlanner, SpanProtector, Redaction
|
||||
├── scanner/ # OccurrenceScanner, OnnxNerScanner
|
||||
├── dictionaries/ # DictionaryLoader
|
||||
└── ui/ # PseudonymizationView, modals, CM6 highlighting
|
||||
└── ui/ # PseudonymizationView, modals (incl. OccurrencesContextModal),
|
||||
# CM6 highlighting (PseudonymHighlighter)
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -203,7 +212,7 @@ src/
|
|||
| 7 — Coulmont | ✅ | Equivalent first name suggestions · JSON/CSV import |
|
||||
| 8 — Side panel | ✅ | 3 tabs · Embedded NER · Wizard · Cancellation · Export highlighting |
|
||||
| 9 — Structured dictionaries | ✅ | Format v1.1 · DictionaryLoader · Dictionary scan · Review modal · French communes |
|
||||
| 10 — Refinement | 🔄 | i18n · Corpus organization · Settings redesign · Phase in progress |
|
||||
| 10 — Refinement & noScribe | 🔄 | i18n · Corpus org · noScribe HTML/VTT import · per-occurrence scan · exceptions · VTT re-export |
|
||||
| 11 — EMCA functions | ⏳ | Turn navigation · Jefferson/ICOR correction · ELAN export |
|
||||
|
||||
See [ROADMAP.md](ROADMAP.md) for the full phase breakdown and planned features.
|
||||
|
|
|
|||
26
ROADMAP.md
26
ROADMAP.md
|
|
@ -15,7 +15,9 @@ Architectural decision (May 2026): **identifying entity detection** is handled b
|
|||
```
|
||||
✅ Phases 0–8 Parsers · Engine · UI · Scopes · Highlighting · Validation · Coulmont · Panel · NER · Wizard
|
||||
✅ Phase 9 Structured dictionaries · DictionaryLoader · Review modals · French communes
|
||||
🔄 Phase 10 Refinement & EMCA specialization (v0.1.x → v0.2.0)
|
||||
🔄 Phase 10 Refinement & noScribe integration (v0.1.x → v0.2.0)
|
||||
✅ i18n · Corpus org · noScribe HTML/VTT/audio · Scan candidates · Exceptions · VTT re-export
|
||||
⏳ Test coverage · Jefferson/ICOR checker · Audio redaction · Timestamp UI
|
||||
⏳ Phase 11 Interactional analysis functions (v1.0.0)
|
||||
```
|
||||
|
||||
|
|
@ -219,9 +221,11 @@ Goal: consolidate all existing features and add the EMCA-specific functions that
|
|||
- [x] **Settings redesign** — 6 sections ordered by frequency of use
|
||||
- [x] **Broad-scope warning** — callout in RuleModal / EditRuleModal for name rules with folder/vault scope
|
||||
- [x] **Mappings tab grouped by scope** — File / Folder / Vault sections with active-file filter
|
||||
- [x] **Status labels clarified** — "Active / Partial / Ignored / Suggested" instead of ✓/◑/✗/?; `getValidatedFor()` includes `partial`
|
||||
- [x] **Scan modal — per-occurrence candidates** — `MappingScanReviewModal` count column opens `OccurrencesContextModal` (✓/✗/⚠ per occurrence, "Save exceptions" button)
|
||||
- [x] **Exceptions** — `IgnoredOccurrence` on `MappingRule`, persisted in mapping.json, red highlighting (`pseudobs-exception`, case-sensitive, priority 0), Exceptions section in Mappings tab
|
||||
- [ ] Unit test coverage ≥ 80% for parsers, engine, NER scanner, DictionaryLoader
|
||||
- [ ] Jefferson / ICOR convention checker: hover suggestions, editor highlighting
|
||||
- [ ] Exports in original format for corpus re-opening
|
||||
- [ ] EMCA publication exports (PNG)
|
||||
- [ ] NER performance: measure and optimize on a 500-turn file
|
||||
- [ ] [Meld Encrypt](https://github.com/meld-cp/obsidian-encrypt) integration in Exports tab for encrypting correspondence tables and pseudonymized exports
|
||||
|
|
@ -229,14 +233,20 @@ Goal: consolidate all existing features and add the EMCA-specific functions that
|
|||
|
||||
### noScribe integration & audio pseudonymization
|
||||
|
||||
[noScribe](https://github.com/kaixxx/noScribe) is a local transcription tool (Whisper + pyannote) widely used in qualitative research. It produces VTT files with **word-level timestamps** and **speaker diarization**.
|
||||
[noScribe](https://github.com/kaixxx/noScribe) is a local transcription tool (Whisper + pyannote) widely used in qualitative research. It produces HTML and VTT files with **word-level timestamps** and **speaker diarization**.
|
||||
|
||||
- [ ] **`VttParser.ts`** — import noScribe VTT: word-level timestamps, speaker labels, round-trip guarantee; auto-conversion to `.md` on import alongside `.mapping.json`
|
||||
- [ ] **Timestamp adjustment UI** — when a word timestamp from Whisper is imprecise, the researcher can fine-tune it: playback of ±1 s around the word (Web Audio API, no external dependency), editable start/end fields, saved back to the mapping metadata
|
||||
- [ ] **Audio bleep export** — once terms are pseudonymized and timestamps are validated, generate an audio export where each pseudonymized occurrence is replaced by a sine tone at the exact word position; export as WAV via Web Audio API
|
||||
- [ ] **Speaker-aware pseudonymization** — rules can be scoped to a specific speaker label (e.g. `SPEAKER_1`); when a name is pseudonymized in one turn, offer to apply the rule to all turns by the same speaker
|
||||
- [x] **`VttParser.ts`** — standard WebVTT with Whisper word-level timestamps, speaker tags, round-trip guarantee; auto-conversion to `.md` + `.words.json`
|
||||
- [x] **`NoScribeHtmlParser.ts`** — Qt Rich Text HTML: `ts_START_END_SXX` anchors, speaker extraction, audio source meta, HTML entities
|
||||
- [x] **`NoScribeVttParser.ts`** — noScribe VTT v0.7: split-cue format, `SXX:` speaker detection, `NOTE media:` audio source
|
||||
- [x] **noScribe Markdown format** — `**S00** [HH:MM:SS] : text`, `pseudobs-format: html|vtt`, `pseudobs-audio`; word timestamps in `.words.json`
|
||||
- [x] **Audio import** — automatic from `audio_source` meta (HTML) or single audio in source folder (VTT)
|
||||
- [x] **`Redaction.ts`** — syllable-count redaction with `🀫`
|
||||
- [x] **VTT re-export** — `Export as VTT` command: clean WebVTT from pseudonymized `.md` + `.words.json`
|
||||
- [ ] **Timestamp adjustment UI** — playback of ±1 s around a word (Web Audio API), editable start/end, saved to mapping metadata
|
||||
- [ ] **Audio redaction export** — WAV export with bleep / silence / white noise at pseudonymized positions
|
||||
- [ ] **Speaker-aware pseudonymization** — rules scoped to a speaker label (e.g. `S01`)
|
||||
|
||||
**Testable (v0.2.0):** stable end-to-end workflow on a real corpus of 10 interviews, including noScribe VTT import, audio bleep export, and timestamp adjustment.
|
||||
**Testable (v0.2.0):** stable end-to-end workflow on a real corpus of 10 interviews, including noScribe import, audio bleep export, and timestamp adjustment.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"id": "pseudonymizer-tool",
|
||||
"name": "Pseudonymizer Tool",
|
||||
"version": "0.1.4",
|
||||
"version": "0.1.5",
|
||||
"minAppVersion": "1.7.2",
|
||||
"description": "Pseudonymize and correct interactional transcripts (Jefferson, ICOR, SRT, CHAT/CHA). Designed for qualitative researchers in linguistics and conversation analysis.",
|
||||
"author": "Axelle Abbadie",
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
{
|
||||
"name": "pseudonymizer-tool",
|
||||
"version": "0.1.4",
|
||||
"version": "0.1.5",
|
||||
"description": "Obsidian plugin for pseudonymizing and correcting interactional transcripts (Jefferson, ICOR, SRT, CHAT/CHA).",
|
||||
"author": "Axelle Abbadie",
|
||||
"homepage": "https://cv.hal.science/axelle-abbadie/",
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@
|
|||
"notice.noDictDetection": "No detection dictionary loaded.\nInstall a dictionary from the Dictionaries panel.",
|
||||
"notice.noDictEntities": "No entities found in the detection dictionaries.",
|
||||
"notice.noDictReplacements": "No replacements available for the found entities.",
|
||||
"notice.ruleDeleted": "✓ Rule deleted — \"{0}\" restored in active file",
|
||||
"notice.ruleCreated": "✓ Rule created: \"{0}\" → \"{1}\"",
|
||||
"notice.rulesCreated": "✓ {0} {1} created",
|
||||
"notice.rulesCreated.rule": "rule",
|
||||
|
|
@ -66,6 +67,14 @@
|
|||
"notice.nerModelLoading": "Loading NER model (first use — ~66 MB)…",
|
||||
"notice.ruleNotFound": "Rule not found in mappings.",
|
||||
"notice.noCheckedDicts": "No dictionary checked.",
|
||||
"mappingScanModal.saveExceptions": "Save exceptions",
|
||||
"mappingScanModal.exceptionsSaved": "Exceptions saved to mapping.",
|
||||
"panel.mappings.exceptions": "Exceptions",
|
||||
"panel.mappings.exceptions.hint": "Occurrences explicitly ignored for this rule.",
|
||||
"notice.notNoScribeFormat": "This file is not a noScribe transcript (pseudobs-format: vtt or html expected).",
|
||||
"notice.wordsJsonMissing": "Timestamp file not found: {0}.words.json",
|
||||
"notice.vttMismatch": "Warning: Markdown cue count and words.json cue count differ — exported VTT may be incomplete.",
|
||||
"notice.vttExported": "VTT exported: {0}",
|
||||
|
||||
"command.organizeCorpus": "Organize corpus",
|
||||
"command.addTranscription": "Add a transcription",
|
||||
|
|
@ -75,6 +84,7 @@
|
|||
"command.scanNer": "Scan file with NER detection",
|
||||
"command.scanDictionaries": "Scan file with dictionaries",
|
||||
"command.pseudonymizeSelection": "Pseudonymize selection",
|
||||
"command.exportAsVtt": "Export as VTT",
|
||||
"command.openPanel": "Pseudonymization: open panel",
|
||||
|
||||
"contextMenu.cancelPseudonymization": "Cancel pseudonymization of \"{0}\"",
|
||||
|
|
@ -84,6 +94,9 @@
|
|||
"ruleModal.scopeWarnTitle": "Longitudinal scope",
|
||||
"ruleModal.scopeOkTitle": "File scope",
|
||||
"ruleModal.scopeOk": "Good practice. This pseudonym will only apply to this transcription — each file remains independent.",
|
||||
"contextMenu.redact": "Redact \"{0}\"",
|
||||
"redaction.checkbox": "Redact (🀫)",
|
||||
"redaction.checkboxDesc": "Replace with 🀫 symbols (one per syllable) — for non-essential identifying content",
|
||||
"contextMenu.coulmont": "Pseudonymize with Prof. Baptiste Coulmont",
|
||||
"contextMenu.createRule": "Create a replacement rule…",
|
||||
|
||||
|
|
@ -121,13 +134,13 @@
|
|||
"scope.folder": "Folder",
|
||||
"scope.vault": "Vault",
|
||||
|
||||
"status.validated": "✓",
|
||||
"status.ignored": "✗",
|
||||
"status.partial": "◑",
|
||||
"status.suggested": "?",
|
||||
"status.conflict": "⚠",
|
||||
"status.disabled": "–",
|
||||
"status.needs_review": "👁",
|
||||
"status.validated": "Active",
|
||||
"status.ignored": "Ignored",
|
||||
"status.partial": "Partial",
|
||||
"status.suggested": "Suggested",
|
||||
"status.conflict": "Conflict",
|
||||
"status.disabled": "Inactive",
|
||||
"status.needs_review": "Review",
|
||||
|
||||
"panel.tab.mappings": "Mappings",
|
||||
"panel.tab.dictionaries": "Dictionaries",
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@
|
|||
"notice.noDictDetection": "Aucun dictionnaire de détection chargé.\nInstallez un dictionnaire depuis le panneau Dictionnaires.",
|
||||
"notice.noDictEntities": "Aucune entité trouvée dans les dictionnaires de détection.",
|
||||
"notice.noDictReplacements": "Aucun remplacement disponible pour les entités trouvées.",
|
||||
"notice.ruleDeleted": "✓ Règle supprimée — \"{0}\" rétabli dans le fichier actif",
|
||||
"notice.ruleCreated": "✓ Règle créée : \"{0}\" → \"{1}\"",
|
||||
"notice.rulesCreated": "✓ {0} règle{1} créée{1}",
|
||||
"notice.rulesCreated.rule": "",
|
||||
|
|
@ -66,6 +67,14 @@
|
|||
"notice.nerModelLoading": "Chargement du modèle NER (première utilisation — ~66 Mo)…",
|
||||
"notice.ruleNotFound": "Règle introuvable dans les mappings.",
|
||||
"notice.noCheckedDicts": "Aucun dictionnaire coché.",
|
||||
"mappingScanModal.saveExceptions": "Enregistrer les exceptions",
|
||||
"mappingScanModal.exceptionsSaved": "Exceptions enregistrées dans le mapping.",
|
||||
"panel.mappings.exceptions": "Exceptions",
|
||||
"panel.mappings.exceptions.hint": "Occurrences explicitement ignorées pour cette règle.",
|
||||
"notice.notNoScribeFormat": "Ce fichier n'est pas une transcription noScribe (pseudobs-format: vtt ou html attendu).",
|
||||
"notice.wordsJsonMissing": "Fichier de timestamps introuvable : {0}.words.json",
|
||||
"notice.vttMismatch": "Attention : le nombre de cues du Markdown et du words.json diffère — le VTT exporté peut être incomplet.",
|
||||
"notice.vttExported": "VTT exporté : {0}",
|
||||
|
||||
"command.organizeCorpus": "Organiser le corpus",
|
||||
"command.addTranscription": "Ajouter une transcription",
|
||||
|
|
@ -75,6 +84,7 @@
|
|||
"command.scanNer": "Scanner le fichier avec détection NER",
|
||||
"command.scanDictionaries": "Scanner le fichier avec les dictionnaires",
|
||||
"command.pseudonymizeSelection": "Pseudonymiser la sélection",
|
||||
"command.exportAsVtt": "Exporter en VTT",
|
||||
"command.openPanel": "Pseudonymisation : ouvrir le panneau",
|
||||
|
||||
"contextMenu.cancelPseudonymization": "Annuler la pseudonymisation de \"{0}\"",
|
||||
|
|
@ -84,6 +94,9 @@
|
|||
"ruleModal.scopeWarnTitle": "Portée longitudinale",
|
||||
"ruleModal.scopeOkTitle": "Portée fichier",
|
||||
"ruleModal.scopeOk": "Bonne pratique. Ce pseudonyme ne s'appliquera qu'à cette transcription — chaque fichier reste indépendant.",
|
||||
"contextMenu.redact": "Caviarder \"{0}\"",
|
||||
"redaction.checkbox": "Caviardage (🀫)",
|
||||
"redaction.checkboxDesc": "Remplace par des 🀫 (1 par syllabe) — pour les informations identifiantes non essentielles",
|
||||
"contextMenu.coulmont": "Pseudonymiser avec Pr Baptiste Coulmont",
|
||||
"contextMenu.createRule": "Créer une règle de remplacement…",
|
||||
|
||||
|
|
@ -121,13 +134,13 @@
|
|||
"scope.folder": "Dossier",
|
||||
"scope.vault": "Vault",
|
||||
|
||||
"status.validated": "✓",
|
||||
"status.ignored": "✗",
|
||||
"status.partial": "◑",
|
||||
"status.suggested": "?",
|
||||
"status.conflict": "⚠",
|
||||
"status.disabled": "–",
|
||||
"status.needs_review": "👁",
|
||||
"status.validated": "Actif",
|
||||
"status.ignored": "Ignoré",
|
||||
"status.partial": "Partiel",
|
||||
"status.suggested": "Suggéré",
|
||||
"status.conflict": "Conflit",
|
||||
"status.disabled": "Inactif",
|
||||
"status.needs_review": "À réviser",
|
||||
|
||||
"panel.tab.mappings": "Mappings",
|
||||
"panel.tab.dictionaries": "Dictionnaires",
|
||||
|
|
@ -147,7 +160,7 @@
|
|||
"panel.mappings.col.category": "Catégorie",
|
||||
"panel.mappings.col.scope": "Portée",
|
||||
"panel.mappings.col.priority": "P.",
|
||||
"panel.mappings.col.status": "Statut",
|
||||
"panel.mappings.col.status": "État",
|
||||
|
||||
"panel.dict.noneInstalled": "Aucun dictionnaire installé. Installez-en un depuis le wizard (Paramètres → Reconfigurer) ou importez un fichier local.",
|
||||
"panel.dict.checkbox": "Inclure dans le scan groupé",
|
||||
|
|
|
|||
381
src/main.ts
381
src/main.ts
|
|
@ -1,4 +1,4 @@
|
|||
import { Plugin, Notice, TFile, TAbstractFile, Editor, Menu, MarkdownView, requestUrl, WorkspaceLeaf } from 'obsidian';
|
||||
import { Plugin, Notice, TFile, TFolder, TAbstractFile, Editor, Menu, MarkdownView, requestUrl, WorkspaceLeaf } from 'obsidian';
|
||||
import { t, setLocale } from './i18n';
|
||||
import { EditorView } from '@codemirror/view';
|
||||
import { PseudObsSettings, DEFAULT_SETTINGS, PseudObsSettingTab } from './settings';
|
||||
|
|
@ -11,6 +11,7 @@ import { OnboardingModal } from './ui/OnboardingModal';
|
|||
import { OnnxNerScanner } from './scanner/OnnxNerScanner';
|
||||
import { DictionaryLoader } from './dictionaries/DictionaryLoader';
|
||||
import { DictScanReviewModal } from './ui/DictScanReviewModal';
|
||||
import { generateRedaction } from './pseudonymizer/Redaction';
|
||||
import { CorpusModal, ClassSelectModal, getCorpusClasses } from './ui/CorpusModal';
|
||||
import type { DictScanResultItem } from './ui/DictScanReviewModal';
|
||||
import { MappingScanReviewModal } from './ui/MappingScanReviewModal';
|
||||
|
|
@ -18,7 +19,10 @@ import type { MappingRuleResult } from './ui/MappingScanReviewModal';
|
|||
import { scanOccurrences } from './scanner/OccurrenceScanner';
|
||||
import { SrtParser } from './parsers/SrtParser';
|
||||
import { ChatParser } from './parsers/ChatParser';
|
||||
import { srtToMarkdown, chatToMarkdown } from './parsers/TranscriptConverter';
|
||||
import { VttParser } from './parsers/VttParser';
|
||||
import { NoScribeHtmlParser } from './parsers/NoScribeHtmlParser';
|
||||
import { NoScribeVttParser } from './parsers/NoScribeVttParser';
|
||||
import { srtToMarkdown, chatToMarkdown, vttToMarkdown, noScribeHtmlToMarkdown, extractWordData, markdownToVtt, type VttCueData } from './parsers/TranscriptConverter';
|
||||
import { MappingStore } from './mappings/MappingStore';
|
||||
import { ScopeResolver } from './mappings/ScopeResolver';
|
||||
import { PseudonymizationEngine } from './pseudonymizer/PseudonymizationEngine';
|
||||
|
|
@ -26,7 +30,7 @@ import { findSpansForRule } from './pseudonymizer/ReplacementPlanner';
|
|||
import { applySpans } from './pseudonymizer/SpanProtector';
|
||||
import type { MappingRule, MappingStatus, Occurrence } from './types';
|
||||
|
||||
const CONVERTIBLE_EXTS = ['srt', 'cha', 'chat'];
|
||||
const CONVERTIBLE_EXTS = ['srt', 'cha', 'chat', 'vtt', 'html'];
|
||||
|
||||
export default class PseudObsPlugin extends Plugin {
|
||||
settings!: PseudObsSettings;
|
||||
|
|
@ -34,10 +38,14 @@ export default class PseudObsPlugin extends Plugin {
|
|||
nerScanner!: OnnxNerScanner;
|
||||
dictionaryLoader!: DictionaryLoader;
|
||||
// Cache synchrone pour le surlignage CM6 (mis à jour de façon asynchrone)
|
||||
private highlightData: HighlightData = { sources: [], replacements: [], nerCandidates: [] };
|
||||
private highlightData: HighlightData = { sources: [], replacements: [], nerCandidates: [], ignoredTerms: [] };
|
||||
// Candidats NER par fichier (effacés au changement de fichier ou à un nouveau scan)
|
||||
private nerCandidateFile: TFile | null = null;
|
||||
private nerCandidates: string[] = [];
|
||||
// Dernière MarkdownView connue — pour mettre à jour le surlignage même
|
||||
// quand le panneau latéral a le focus (getActiveViewOfType retourne null dans ce cas)
|
||||
private lastMarkdownView: MarkdownView | null = null;
|
||||
private _viewRefreshTimer: number | null = null;
|
||||
|
||||
async onload(): Promise<void> {
|
||||
await this.loadSettings();
|
||||
|
|
@ -57,12 +65,31 @@ export default class PseudObsPlugin extends Plugin {
|
|||
createPseudonymHighlighter(() => this.highlightData)
|
||||
);
|
||||
|
||||
// Rafraîchir le cache de surlignage à chaque changement de fichier actif
|
||||
// Tracker la dernière MarkdownView pour garder le surlignage actif
|
||||
// même quand le panneau latéral prend le focus
|
||||
this.registerEvent(
|
||||
this.app.workspace.on('active-leaf-change', () => { void this.refreshHighlightData(); })
|
||||
this.app.workspace.on('active-leaf-change', (leaf) => {
|
||||
const v = leaf?.view;
|
||||
if (v instanceof MarkdownView) this.lastMarkdownView = v;
|
||||
void this.refresh();
|
||||
})
|
||||
);
|
||||
|
||||
// Rafraîchir panneau + surlignage quand le FICHIER TRANSCRIPT actif est modifié.
|
||||
// Les fichiers .mapping.json sont exclus — leurs changements sont déjà gérés
|
||||
// par les appels explicites à refresh() dans les modales, évitant le double rendu.
|
||||
this.registerEvent(
|
||||
this.app.vault.on('modify', (file) => {
|
||||
if (file instanceof TFile
|
||||
&& !file.name.endsWith('.mapping.json')
|
||||
&& file === this.app.workspace.getActiveFile()) {
|
||||
void this.refresh();
|
||||
}
|
||||
})
|
||||
);
|
||||
|
||||
// Premier chargement au démarrage
|
||||
void this.refreshHighlightData();
|
||||
void this.refresh();
|
||||
|
||||
// Onboarding au premier lancement
|
||||
if (!this.settings.onboardingCompleted) {
|
||||
|
|
@ -126,6 +153,12 @@ export default class PseudObsPlugin extends Plugin {
|
|||
callback: () => void this.scanCurrentFileWithDictionaries(),
|
||||
});
|
||||
|
||||
this.addCommand({
|
||||
id: 'export-as-vtt',
|
||||
name: t('command.exportAsVtt'),
|
||||
callback: () => void this.exportCurrentFileAsVtt(),
|
||||
});
|
||||
|
||||
this.addCommand({
|
||||
id: 'pseudonymize-selection',
|
||||
name: t('command.pseudonymizeSelection'),
|
||||
|
|
@ -164,7 +197,7 @@ export default class PseudObsPlugin extends Plugin {
|
|||
const location = await this.scopeResolver.findRuleByTerm(bare);
|
||||
if (!location) { new Notice(t('notice.ruleNotFound')); return; }
|
||||
editor.replaceSelection(location.rule.source);
|
||||
void this.refreshHighlightData();
|
||||
void this.refresh();
|
||||
})
|
||||
);
|
||||
}
|
||||
|
|
@ -192,6 +225,13 @@ export default class PseudObsPlugin extends Plugin {
|
|||
.onClick(() => new QuickPseudonymizeModal(this.app, this, editor).open())
|
||||
);
|
||||
|
||||
menu.addItem((item) =>
|
||||
item
|
||||
.setTitle(t('contextMenu.redact', truncate(selection)))
|
||||
.setIcon('square')
|
||||
.onClick(() => new QuickPseudonymizeModal(this.app, this, editor, generateRedaction(selection), [], true).open())
|
||||
);
|
||||
|
||||
menu.addItem((item) =>
|
||||
item
|
||||
.setTitle(t('contextMenu.coulmont'))
|
||||
|
|
@ -262,9 +302,8 @@ export default class PseudObsPlugin extends Plugin {
|
|||
async refreshHighlightData(): Promise<void> {
|
||||
const file = this.app.workspace.getActiveFile();
|
||||
if (!file) {
|
||||
this.highlightData = { sources: [], replacements: [], nerCandidates: [] };
|
||||
this.highlightData = { sources: [], replacements: [], nerCandidates: [], ignoredTerms: [] };
|
||||
} else {
|
||||
// Candidats NER : uniquement si le fichier actif est celui du dernier scan
|
||||
const nerCandidates = file === this.nerCandidateFile ? this.nerCandidates : [];
|
||||
|
||||
try {
|
||||
|
|
@ -287,24 +326,57 @@ export default class PseudObsPlugin extends Plugin {
|
|||
} else {
|
||||
rules = await this.scopeResolver.getRulesFor(file.path);
|
||||
}
|
||||
// Termes ignorés : extraits des ignoredOccurrences de chaque règle
|
||||
const ignoredTerms = rules.flatMap((r) =>
|
||||
(r.ignoredOccurrences ?? []).map((o) => o.text)
|
||||
);
|
||||
this.highlightData = {
|
||||
sources: rules.map((r) => r.source).filter(Boolean),
|
||||
replacements: rules.map((r) => r.replacement).filter(Boolean),
|
||||
nerCandidates,
|
||||
ignoredTerms,
|
||||
};
|
||||
} catch {
|
||||
this.highlightData = { sources: [], replacements: [], nerCandidates };
|
||||
this.highlightData = { sources: [], replacements: [], nerCandidates, ignoredTerms: [] };
|
||||
}
|
||||
}
|
||||
|
||||
// Dispatcher le StateEffect sur l'éditeur actif pour déclencher
|
||||
// la reconstruction des décorations CM6 (le ViewPlugin ne se déclenche
|
||||
// pas sur un changement de données externe sans ce signal)
|
||||
const view = this.app.workspace.getActiveViewOfType(MarkdownView);
|
||||
// Dispatcher le StateEffect — utiliser lastMarkdownView si le panneau a le focus
|
||||
const view = this.app.workspace.getActiveViewOfType(MarkdownView) ?? this.lastMarkdownView;
|
||||
const cm = view?.editor && ((view.editor as unknown as { cm?: EditorView }).cm);
|
||||
cm?.dispatch({ effects: highlightDataChanged.of(undefined) });
|
||||
}
|
||||
|
||||
/**
|
||||
* Rafraîchit à la fois le surlignage CM6 ET le panneau latéral.
|
||||
* À appeler après toute action qui crée, modifie ou supprime une règle.
|
||||
*/
|
||||
async refresh(): Promise<void> {
|
||||
await this.refreshHighlightData();
|
||||
this.refreshView();
|
||||
}
|
||||
|
||||
/**
|
||||
* Demande au panneau latéral ouvert de re-rendre son onglet actif.
|
||||
* Debounce 80 ms pour coalescer les appels multiples rapides
|
||||
* (vault watcher + appel explicite dans la même action).
|
||||
*/
|
||||
refreshView(): void {
|
||||
if (this._viewRefreshTimer !== null) {
|
||||
window.clearTimeout(this._viewRefreshTimer);
|
||||
}
|
||||
this._viewRefreshTimer = window.setTimeout(() => {
|
||||
this._viewRefreshTimer = null;
|
||||
const leaves = this.app.workspace.getLeavesOfType(VIEW_TYPE_PSEUDOBS);
|
||||
for (const leaf of leaves) {
|
||||
const view = leaf.view;
|
||||
if (view instanceof PseudonymizationView) {
|
||||
void view.refreshActiveTab();
|
||||
}
|
||||
}
|
||||
}, 80);
|
||||
}
|
||||
|
||||
// --- Conversion automatique ---
|
||||
|
||||
private async autoConvert(file: TFile): Promise<void> {
|
||||
|
|
@ -315,28 +387,76 @@ export default class PseudObsPlugin extends Plugin {
|
|||
const folder = file.parent?.path ?? '';
|
||||
const mdPath = folder ? `${folder}/${basename}.md` : `${basename}.md`;
|
||||
|
||||
let mdContent: string;
|
||||
if (ext === 'srt') {
|
||||
mdContent = srtToMarkdown(new SrtParser().parse(raw), file.name);
|
||||
} else {
|
||||
mdContent = chatToMarkdown(new ChatParser().parse(raw), file.name);
|
||||
}
|
||||
|
||||
// Si un .md du même nom existe déjà, ne pas écraser
|
||||
// Ne pas écraser un .md existant
|
||||
if (this.app.vault.getAbstractFileByPath(mdPath) instanceof TFile) {
|
||||
new Notice(t('notice.conversionSkipped', basename, file.name));
|
||||
return;
|
||||
}
|
||||
|
||||
// Chercher un fichier audio déjà présent dans le même dossier du vault
|
||||
let audioFilename = this.findAudioInVaultFolder(folder);
|
||||
|
||||
// Conversion
|
||||
let mdContent: string;
|
||||
let wordData: ReturnType<typeof extractWordData> | null = null;
|
||||
|
||||
if (ext === 'srt') {
|
||||
mdContent = srtToMarkdown(new SrtParser().parse(raw), file.name);
|
||||
} else if (ext === 'vtt') {
|
||||
const doc = NoScribeVttParser.isNoScribeVtt(raw)
|
||||
? new NoScribeVttParser().parse(raw)
|
||||
: new VttParser().parse(raw);
|
||||
wordData = extractWordData(doc);
|
||||
// Pour les VTT noScribe, chercher aussi l'audio via NOTE media
|
||||
if (!audioFilename && NoScribeVttParser.isNoScribeVtt(raw)) {
|
||||
const audioSource = NoScribeVttParser.extractAudioSource(raw);
|
||||
if (audioSource) audioFilename = await this.importAudioFromPath(audioSource, folder || this.settings.transcriptionsFolder);
|
||||
}
|
||||
const toMd = NoScribeVttParser.isNoScribeVtt(raw) ? noScribeHtmlToMarkdown : vttToMarkdown;
|
||||
mdContent = toMd(doc, file.name, audioFilename ?? undefined);
|
||||
} else if (ext === 'html') {
|
||||
if (!NoScribeHtmlParser.isNoScribeHtml(raw)) return;
|
||||
const doc = new NoScribeHtmlParser().parse(raw);
|
||||
wordData = extractWordData(doc);
|
||||
// Importer l'audio depuis le chemin absolu dans la meta tag si pas encore dans le vault
|
||||
if (!audioFilename) {
|
||||
const audioSource = NoScribeHtmlParser.extractAudioSource(raw);
|
||||
if (audioSource) {
|
||||
audioFilename = await this.importAudioFromPath(audioSource, folder || this.settings.transcriptionsFolder);
|
||||
}
|
||||
}
|
||||
mdContent = noScribeHtmlToMarkdown(doc, file.name, audioFilename ?? undefined);
|
||||
} else {
|
||||
mdContent = chatToMarkdown(new ChatParser().parse(raw), file.name);
|
||||
}
|
||||
|
||||
await this.app.vault.create(mdPath, mdContent);
|
||||
|
||||
const mappingPath = `${this.settings.mappingFolder}/${basename}.mapping.json`;
|
||||
// Structure miroir pour les mappings
|
||||
const transcRoot = this.settings.transcriptionsFolder;
|
||||
const fileFolder = file.parent?.path ?? '';
|
||||
const relSubFolder = fileFolder.startsWith(transcRoot)
|
||||
? fileFolder.slice(transcRoot.length).replace(/^\//, '')
|
||||
: '';
|
||||
const mappingDir = relSubFolder
|
||||
? `${this.settings.mappingFolder}/${relSubFolder}`
|
||||
: this.settings.mappingFolder;
|
||||
await this.ensureFolder(mappingDir);
|
||||
|
||||
const mappingPath = `${mappingDir}/${basename}.mapping.json`;
|
||||
if (!this.app.vault.getAbstractFileByPath(mappingPath)) {
|
||||
await this.ensureFolder(this.settings.mappingFolder);
|
||||
const store = new MappingStore({ type: 'file', path: mdPath });
|
||||
await this.app.vault.create(mappingPath, JSON.stringify(store.toJSON(), null, 2));
|
||||
}
|
||||
|
||||
// Écrire les timestamps word-level dans un fichier auxiliaire
|
||||
if (wordData && wordData.length > 0) {
|
||||
const wordsPath = `${mappingDir}/${basename}.words.json`;
|
||||
if (!this.app.vault.getAbstractFileByPath(wordsPath)) {
|
||||
await this.app.vault.create(wordsPath, JSON.stringify(wordData, null, 2));
|
||||
}
|
||||
}
|
||||
|
||||
await this.app.fileManager.trashFile(file);
|
||||
|
||||
const mdFile = this.app.vault.getAbstractFileByPath(mdPath);
|
||||
|
|
@ -350,12 +470,52 @@ export default class PseudObsPlugin extends Plugin {
|
|||
}
|
||||
}
|
||||
|
||||
/** Retourne le nom du premier fichier audio trouvé dans un dossier du vault. */
|
||||
private findAudioInVaultFolder(folderPath: string): string | null {
|
||||
const AUDIO_EXTS = new Set(['m4a', 'mp3', 'wav', 'ogg', 'flac', 'mp4', 'aac', 'aiff']);
|
||||
const folder = this.app.vault.getAbstractFileByPath(folderPath || '/');
|
||||
if (!(folder instanceof TFolder)) return null;
|
||||
const audioFile = folder.children.find(
|
||||
(f) => f instanceof TFile && AUDIO_EXTS.has((f as TFile).extension.toLowerCase())
|
||||
) as TFile | undefined;
|
||||
return audioFile?.name ?? null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Copie un fichier audio externe (chemin absolu sur disque) dans le vault.
|
||||
* Utilise l'API Node.js fs — desktop uniquement.
|
||||
* Retourne le nom du fichier importé, ou null en cas d'échec.
|
||||
*/
|
||||
private async importAudioFromPath(sourcePath: string, targetFolder: string): Promise<string | null> {
|
||||
try {
|
||||
// eslint-disable-next-line @typescript-eslint/no-var-requires
|
||||
const nodeFs = require('fs') as typeof import('fs');
|
||||
if (!nodeFs.existsSync(sourcePath)) return null;
|
||||
|
||||
const audioFilename = sourcePath.replace(/\\/g, '/').split('/').pop()!;
|
||||
const destPath = targetFolder ? `${targetFolder}/${audioFilename}` : audioFilename;
|
||||
|
||||
if (this.app.vault.getAbstractFileByPath(destPath) instanceof TFile) {
|
||||
return audioFilename; // déjà présent
|
||||
}
|
||||
|
||||
const buffer: Buffer = await nodeFs.promises.readFile(sourcePath);
|
||||
await this.ensureFolder(targetFolder);
|
||||
const arrayBuf = buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) as ArrayBuffer;
|
||||
await this.app.vault.createBinary(destPath, arrayBuf);
|
||||
new Notice(`Audio importé : ${audioFilename}`);
|
||||
return audioFilename;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// --- Commande "Ajouter une transcription" ---
|
||||
|
||||
private openFilePicker(): void {
|
||||
const input = activeDocument.createElement('input');
|
||||
input.type = 'file';
|
||||
input.accept = '.srt,.cha,.chat,.txt,.md';
|
||||
input.accept = '.srt,.vtt,.cha,.chat,.html,.txt,.md';
|
||||
input.multiple = true;
|
||||
// Pas de display:none — bloque le change event dans certaines versions d'Electron
|
||||
input.classList.add('pseudobs-hidden-input');
|
||||
|
|
@ -376,6 +536,7 @@ export default class PseudObsPlugin extends Plugin {
|
|||
|
||||
private async copyToVault(browserFile: File): Promise<void> {
|
||||
const raw = await browserFile.text();
|
||||
const ext = browserFile.name.split('.').pop()?.toLowerCase() ?? '';
|
||||
|
||||
// Sélection de classe si le corpus est organisé en sous-dossiers
|
||||
const classes = getCorpusClasses(this.app, this.settings.transcriptionsFolder);
|
||||
|
|
@ -397,6 +558,39 @@ export default class PseudObsPlugin extends Plugin {
|
|||
}
|
||||
|
||||
await this.app.vault.create(destPath, raw);
|
||||
|
||||
// Pour VTT : chercher un fichier audio dans le dossier source (Electron expose le chemin complet)
|
||||
if (ext === 'vtt') {
|
||||
const sourcePath = (browserFile as unknown as { path?: string }).path;
|
||||
if (sourcePath) {
|
||||
const sourceDir = sourcePath.replace(/\\/g, '/').replace(/\/[^/]+$/, '');
|
||||
const audioPath = await this.findAudioInSourceFolder(sourceDir);
|
||||
if (audioPath) await this.importAudioFromPath(audioPath, targetFolder);
|
||||
}
|
||||
}
|
||||
// Pour HTML : l'audio sera importé par autoConvert via la meta tag audio_source
|
||||
}
|
||||
|
||||
/**
|
||||
* Cherche un fichier audio dans un dossier sur le disque (hors vault).
|
||||
* Retourne le chemin absolu du seul fichier audio trouvé, ou null si 0 ou >1.
|
||||
*/
|
||||
private async findAudioInSourceFolder(folderPath: string): Promise<string | null> {
|
||||
try {
|
||||
// eslint-disable-next-line @typescript-eslint/no-var-requires
|
||||
const nodeFs = require('fs') as typeof import('fs');
|
||||
const AUDIO_EXTS = new Set(['m4a', 'mp3', 'wav', 'ogg', 'flac', 'mp4', 'aac', 'aiff']);
|
||||
const entries = await nodeFs.promises.readdir(folderPath);
|
||||
const audioFiles = entries.filter((f) => {
|
||||
const ext = f.split('.').pop()?.toLowerCase() ?? '';
|
||||
return AUDIO_EXTS.has(ext);
|
||||
});
|
||||
// Import uniquement s'il y a exactement un fichier audio (évite l'ambiguïté)
|
||||
if (audioFiles.length === 1) return `${folderPath}/${audioFiles[0]}`;
|
||||
return null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// --- Pseudonymisation ---
|
||||
|
|
@ -490,7 +684,7 @@ export default class PseudObsPlugin extends Plugin {
|
|||
const unique = [...new Set(occurrences.map((o) => o.text).filter(Boolean))];
|
||||
this.nerCandidateFile = file;
|
||||
this.nerCandidates = unique;
|
||||
void this.refreshHighlightData();
|
||||
void this.refresh();
|
||||
|
||||
new Notice(t('notice.nerEntitiesFound', String(unique.length), unique.length > 1 ? t('notice.nerEntitiesFound.entities') : t('notice.nerEntitiesFound.entity')), 6000);
|
||||
} catch (e) {
|
||||
|
|
@ -576,7 +770,7 @@ export default class PseudObsPlugin extends Plugin {
|
|||
// Surlignage bleu préventif (aperçu pendant que la modale est ouverte)
|
||||
this.nerCandidateFile = file;
|
||||
this.nerCandidates = results.map((r) => r.term);
|
||||
void this.refreshHighlightData();
|
||||
void this.refresh();
|
||||
|
||||
new DictScanReviewModal(this.app, this, file, results, existingReplacements).open();
|
||||
}
|
||||
|
|
@ -585,9 +779,10 @@ export default class PseudObsPlugin extends Plugin {
|
|||
clearNerCandidates(): void {
|
||||
this.nerCandidates = [];
|
||||
this.nerCandidateFile = null;
|
||||
void this.refreshHighlightData();
|
||||
void this.refresh();
|
||||
}
|
||||
|
||||
|
||||
async exportMappingForFile(file: TFile): Promise<void> {
|
||||
const mappingPath = `${this.settings.mappingFolder}/${file.basename}.mapping.json`;
|
||||
const mappingFile = this.app.vault.getAbstractFileByPath(mappingPath);
|
||||
|
|
@ -611,6 +806,77 @@ export default class PseudObsPlugin extends Plugin {
|
|||
new Notice(t('notice.mappingExported', destPath));
|
||||
}
|
||||
|
||||
/**
|
||||
* Exporte le fichier Markdown noScribe actif en WebVTT pseudonymisé.
|
||||
* Lit le .words.json correspondant pour les timestamps précis.
|
||||
*/
|
||||
async exportCurrentFileAsVtt(): Promise<void> {
|
||||
const file = this.app.workspace.getActiveFile();
|
||||
if (!file || file.extension !== 'md') {
|
||||
new Notice(t('notice.noActiveFile'));
|
||||
return;
|
||||
}
|
||||
|
||||
const content = await this.app.vault.read(file);
|
||||
|
||||
// Vérifier que c'est bien un fichier noScribe converti
|
||||
const formatMatch = /^pseudobs-format:\s*(\w+)/m.exec(content);
|
||||
const format = formatMatch?.[1];
|
||||
if (format !== 'vtt' && format !== 'html') {
|
||||
new Notice(t('notice.notNoScribeFormat'));
|
||||
return;
|
||||
}
|
||||
|
||||
// Trouver le .words.json : même basename, dans le dossier mappings
|
||||
// Le basename peut être "juste-leblanc" ou "juste-leblanc.pseudonymized"
|
||||
const rawBasename = file.basename.replace(/\.pseudonymized$/, '');
|
||||
const wordsJson = await this.findWordsJson(rawBasename);
|
||||
if (!wordsJson) {
|
||||
new Notice(t('notice.wordsJsonMissing', rawBasename));
|
||||
return;
|
||||
}
|
||||
|
||||
const wordData = JSON.parse(wordsJson) as VttCueData[];
|
||||
const { vtt, mismatch } = markdownToVtt(content, wordData);
|
||||
|
||||
if (mismatch) {
|
||||
new Notice(t('notice.vttMismatch'));
|
||||
}
|
||||
|
||||
await this.ensureFolder(this.settings.exportsFolder);
|
||||
const outputPath = `${this.settings.exportsFolder}/${rawBasename}.pseudonymized.vtt`;
|
||||
const existing = this.app.vault.getAbstractFileByPath(outputPath);
|
||||
if (existing instanceof TFile) {
|
||||
await this.app.vault.modify(existing, vtt);
|
||||
} else {
|
||||
await this.app.vault.create(outputPath, vtt);
|
||||
}
|
||||
|
||||
new Notice(t('notice.vttExported', outputPath));
|
||||
}
|
||||
|
||||
/** Cherche <basename>.words.json dans le dossier mappings et ses sous-dossiers. */
|
||||
private async findWordsJson(basename: string): Promise<string | null> {
|
||||
const filename = `${basename}.words.json`;
|
||||
// Chercher dans l'ensemble du dossier mappings
|
||||
const search = (folder: TFolder): TFile | null => {
|
||||
for (const child of folder.children) {
|
||||
if (child instanceof TFile && child.name === filename) return child;
|
||||
if (child instanceof TFolder) {
|
||||
const found = search(child);
|
||||
if (found) return found;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
const mappingRoot = this.app.vault.getAbstractFileByPath(this.settings.mappingFolder);
|
||||
if (!(mappingRoot instanceof TFolder)) return null;
|
||||
|
||||
const wordsFile = search(mappingRoot);
|
||||
return wordsFile ? this.app.vault.read(wordsFile) : null;
|
||||
}
|
||||
|
||||
private async scanCurrentFile(): Promise<void> {
|
||||
const file = this.app.workspace.getActiveFile();
|
||||
if (!file) { new Notice(t('notice.noActiveFile')); return; }
|
||||
|
|
@ -632,14 +898,19 @@ export default class PseudObsPlugin extends Plugin {
|
|||
return;
|
||||
}
|
||||
|
||||
const countByRule = new Map<string, number>();
|
||||
const occsByRule = new Map<string, Occurrence[]>();
|
||||
for (const occ of occurrences) {
|
||||
const id = occ.mappingId ?? '';
|
||||
countByRule.set(id, (countByRule.get(id) ?? 0) + 1);
|
||||
if (!occsByRule.has(id)) occsByRule.set(id, []);
|
||||
occsByRule.get(id)!.push(occ);
|
||||
}
|
||||
const ruleResults: MappingRuleResult[] = rules
|
||||
.filter((r) => countByRule.has(r.id))
|
||||
.map((r) => ({ rule: r, matchCount: countByRule.get(r.id)! }));
|
||||
.filter((r) => occsByRule.has(r.id))
|
||||
.map((r) => ({
|
||||
rule: r,
|
||||
matchCount: occsByRule.get(r.id)!.length,
|
||||
occurrences: occsByRule.get(r.id)!,
|
||||
}));
|
||||
|
||||
new MappingScanReviewModal(this.app, this, file, content, ruleResults).open();
|
||||
}
|
||||
|
|
@ -694,6 +965,50 @@ export default class PseudObsPlugin extends Plugin {
|
|||
return spans.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Annule l'application d'une règle dans le fichier actif :
|
||||
* cherche le remplacement (avec ou sans marqueurs) et le réécrit avec la source.
|
||||
* Appelé automatiquement à la suppression d'une règle dans EditRuleModal.
|
||||
*/
|
||||
async revertRuleInFile(source: string, replacement: string): Promise<void> {
|
||||
const file = this.app.workspace.getActiveFile();
|
||||
if (!file) return;
|
||||
|
||||
const s = this.settings;
|
||||
// Chercher la version avec marqueurs EN PREMIER — sinon on trouve le texte
|
||||
// à l'intérieur des marqueurs et on obtient {{source}} au lieu de source.
|
||||
const variants: string[] = [];
|
||||
if (s.useMarkerInExport) {
|
||||
variants.push(`${s.markerOpen}${replacement}${s.markerClose}`);
|
||||
}
|
||||
variants.push(replacement); // version sans marqueurs en dernier
|
||||
|
||||
let content = await this.app.vault.read(file);
|
||||
let changed = false;
|
||||
|
||||
for (const variant of variants) {
|
||||
const fakeRule: MappingRule = {
|
||||
id: '_revert', source: variant, replacement: source, category: 'custom',
|
||||
scope: { type: 'file', path: file.path }, status: 'validated',
|
||||
priority: 0, createdBy: 'user', createdAt: new Date().toISOString(),
|
||||
};
|
||||
const spans = findSpansForRule(content, fakeRule, {
|
||||
caseSensitive: false,
|
||||
wholeWordOnly: false, // le remplacement peut contenir des 🀫 ou marqueurs
|
||||
});
|
||||
if (spans.length > 0) {
|
||||
spans.sort((a, b) => b.start - a.start);
|
||||
content = applySpans(content, spans);
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (changed) {
|
||||
await this.app.vault.modify(file, content);
|
||||
void this.refresh();
|
||||
}
|
||||
}
|
||||
|
||||
// --- Utilitaires ---
|
||||
|
||||
async ensureFolder(folderPath: string): Promise<void> {
|
||||
|
|
|
|||
|
|
@ -64,8 +64,13 @@ export class MappingStore {
|
|||
|
||||
// Règles validées applicables à un fichier donné (cascade file → folder → vault)
|
||||
getValidatedFor(filePath: string): MappingRule[] {
|
||||
// 'validated' et 'partial' sont des règles actives.
|
||||
// 'ignored' = l'utilisateur a ignoré toutes les occurrences au dernier scan
|
||||
// → la règle reste définie mais désactivée.
|
||||
// 'suggested' = non encore confirmée → inactive.
|
||||
const ACTIVE: Set<string> = new Set(['validated', 'partial']);
|
||||
return this.getAll().filter((r) => {
|
||||
if (r.status !== 'validated') return false;
|
||||
if (!ACTIVE.has(r.status)) return false;
|
||||
if (r.scope.type === 'vault') return true;
|
||||
if (r.scope.type === 'folder') return filePath.startsWith(r.scope.path ?? '');
|
||||
return r.scope.path === filePath;
|
||||
|
|
|
|||
|
|
@ -17,16 +17,26 @@ export class ScopeResolver {
|
|||
private mappingFolder: string
|
||||
) {}
|
||||
|
||||
/** Collecte récursivement tous les fichiers .mapping.json dans un dossier. */
|
||||
private collectMappingFiles(folder: TFolder): TFile[] {
|
||||
const files: TFile[] = [];
|
||||
for (const child of folder.children) {
|
||||
if (child instanceof TFile && child.name.endsWith('.mapping.json')) {
|
||||
files.push(child);
|
||||
} else if (child instanceof TFolder) {
|
||||
files.push(...this.collectMappingFiles(child));
|
||||
}
|
||||
}
|
||||
return files;
|
||||
}
|
||||
|
||||
async getRulesFor(filePath: string): Promise<MappingRule[]> {
|
||||
const folder = this.vault.getAbstractFileByPath(this.mappingFolder);
|
||||
if (!(folder instanceof TFolder)) return [];
|
||||
|
||||
const allRules: MappingRule[] = [];
|
||||
|
||||
for (const child of folder.children) {
|
||||
if (!(child instanceof TFile)) continue;
|
||||
if (!child.name.endsWith('.mapping.json')) continue;
|
||||
|
||||
for (const child of this.collectMappingFiles(folder)) {
|
||||
try {
|
||||
const raw = await this.vault.read(child);
|
||||
const data = JSON.parse(raw) as MappingFile;
|
||||
|
|
@ -56,8 +66,7 @@ export class ScopeResolver {
|
|||
|
||||
const needle = term.toLowerCase();
|
||||
|
||||
for (const child of folder.children) {
|
||||
if (!(child instanceof TFile) || !child.name.endsWith('.mapping.json')) continue;
|
||||
for (const child of this.collectMappingFiles(folder)) {
|
||||
try {
|
||||
const data = JSON.parse(await this.vault.read(child)) as MappingFile;
|
||||
const store = MappingStore.fromJSON(data);
|
||||
|
|
@ -82,8 +91,7 @@ export class ScopeResolver {
|
|||
|
||||
const result: RuleLocation[] = [];
|
||||
|
||||
for (const child of folder.children) {
|
||||
if (!(child instanceof TFile) || !child.name.endsWith('.mapping.json')) continue;
|
||||
for (const child of this.collectMappingFiles(folder)) {
|
||||
try {
|
||||
const data = JSON.parse(await this.vault.read(child)) as MappingFile;
|
||||
const store = MappingStore.fromJSON(data);
|
||||
|
|
@ -111,8 +119,7 @@ export class ScopeResolver {
|
|||
|
||||
const result: RuleLocation[] = [];
|
||||
|
||||
for (const child of folder.children) {
|
||||
if (!(child instanceof TFile) || !child.name.endsWith('.mapping.json')) continue;
|
||||
for (const child of this.collectMappingFiles(folder)) {
|
||||
try {
|
||||
const data = JSON.parse(await this.vault.read(child)) as MappingFile;
|
||||
const store = MappingStore.fromJSON(data);
|
||||
|
|
|
|||
150
src/parsers/NoScribeHtmlParser.ts
Normal file
150
src/parsers/NoScribeHtmlParser.ts
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
/**
|
||||
* Parser HTML noScribe — produit un VttDocument compatible avec VttParser/TranscriptConverter.
|
||||
*
|
||||
* Format noScribe HTML (Qt Rich Text) :
|
||||
* - <meta name="audio_source"> : chemin vers le fichier audio source
|
||||
* - Chaque <p> = un tour de parole ou une pause
|
||||
* - Chaque <a name="ts_START_END_SPEAKER"> = un segment avec timestamps
|
||||
* START et END sont en millisecondes depuis le début de l'audio
|
||||
* - Le nom du locuteur précède " :" en début de tour
|
||||
* - Les timestamps d'affichage [HH:MM:SS] (couleur #78909c) sont à ignorer
|
||||
*/
|
||||
|
||||
import type { VttDocument, VttCue, VttWord } from './VttParser';
|
||||
|
||||
// ---- Regex ------------------------------------------------------------------
|
||||
|
||||
const PARA_RE = /<p(?:\s[^>]*)?>(?<content>.*?)<\/p>/gs;
|
||||
const ANCHOR_RE = /<a\s+name="(ts_[^"]+)"[^>]*>(.*?)<\/a>/gs;
|
||||
const TS_NAME_RE = /^ts_(\d+)_(\d+)_(\w*)$/;
|
||||
// Timestamps d'affichage [HH:MM:SS] générés par noScribe pour l'interface — ignorer
|
||||
const DISPLAY_TS_RE = /^\[\d{2}:\d{2}:\d{2}\]$/;
|
||||
// Locuteur suivi de " :" en début de texte
|
||||
const SPEAKER_RE = /^(.+?)\s*:\s*/;
|
||||
const ALL_TAGS_RE = /<[^>]+>/g;
|
||||
|
||||
const HTML_ENTITIES: Record<string, string> = {
|
||||
'&': '&', '<': '<', '>': '>', '"': '"',
|
||||
''': "'", ''': "'", ' ': ' ',
|
||||
};
|
||||
|
||||
// ---- Helpers ----------------------------------------------------------------
|
||||
|
||||
function decodeEntities(text: string): string {
|
||||
return text.replace(/&[^;]+;/g, (m) => HTML_ENTITIES[m] ?? m);
|
||||
}
|
||||
|
||||
function stripTags(html: string): string {
|
||||
return decodeEntities(html.replace(ALL_TAGS_RE, ''));
|
||||
}
|
||||
|
||||
function pad2(n: number): string { return String(n).padStart(2, '0'); }
|
||||
function pad3(n: number): string { return String(n).padStart(3, '0'); }
|
||||
|
||||
/** Convertit des millisecondes en timestamp HH:MM:SS.mmm. */
|
||||
function msToTimestamp(ms: number): string {
|
||||
const h = Math.floor(ms / 3600000);
|
||||
const m = Math.floor((ms % 3600000) / 60000);
|
||||
const s = Math.floor((ms % 60000) / 1000);
|
||||
const mss = ms % 1000;
|
||||
return `${pad2(h)}:${pad2(m)}:${pad2(s)}.${pad3(mss)}`;
|
||||
}
|
||||
|
||||
// ---- Parser -----------------------------------------------------------------
|
||||
|
||||
export class NoScribeHtmlParser {
|
||||
/** Vérifie qu'un contenu HTML est une sortie noScribe. */
|
||||
static isNoScribeHtml(content: string): boolean {
|
||||
return content.includes('qrichtext') && content.includes('<a name="ts_');
|
||||
}
|
||||
|
||||
/** Extrait le chemin absolu du fichier audio depuis la meta tag noScribe. */
|
||||
static extractAudioSource(content: string): string | null {
|
||||
const m = /meta\s+name="audio_source"\s+content="([^"]+)"/.exec(content);
|
||||
return m ? m[1] : null;
|
||||
}
|
||||
|
||||
parse(content: string): VttDocument {
|
||||
const cues: VttCue[] = [];
|
||||
|
||||
PARA_RE.lastIndex = 0;
|
||||
let paraMatch: RegExpExecArray | null;
|
||||
|
||||
while ((paraMatch = PARA_RE.exec(content)) !== null) {
|
||||
const paraHtml = paraMatch[1];
|
||||
if (!paraHtml.includes('ts_')) continue;
|
||||
|
||||
// Extraire et grouper les ancres par nom de timestamp (plusieurs ancres peuvent
|
||||
// partager le même ts_ pour des raisons de mise en forme Qt)
|
||||
const grouped = new Map<string, {
|
||||
startMs: number; endMs: number; speakerId: string; texts: string[];
|
||||
}>();
|
||||
const groupOrder: string[] = [];
|
||||
|
||||
ANCHOR_RE.lastIndex = 0;
|
||||
let anchorMatch: RegExpExecArray | null;
|
||||
|
||||
while ((anchorMatch = ANCHOR_RE.exec(paraHtml)) !== null) {
|
||||
const tsName = anchorMatch[1];
|
||||
const innerHtml = anchorMatch[2];
|
||||
const tsMatch = TS_NAME_RE.exec(tsName);
|
||||
if (!tsMatch) continue;
|
||||
|
||||
const text = stripTags(innerHtml);
|
||||
if (DISPLAY_TS_RE.test(text.trim())) continue;
|
||||
|
||||
if (!grouped.has(tsName)) {
|
||||
grouped.set(tsName, {
|
||||
startMs: parseInt(tsMatch[1], 10),
|
||||
endMs: parseInt(tsMatch[2], 10),
|
||||
speakerId: tsMatch[3],
|
||||
texts: [],
|
||||
});
|
||||
groupOrder.push(tsName);
|
||||
}
|
||||
if (text) grouped.get(tsName)!.texts.push(text);
|
||||
}
|
||||
|
||||
if (groupOrder.length === 0) continue;
|
||||
|
||||
// Construire les VttWords à partir des groupes
|
||||
const rawWords: VttWord[] = [];
|
||||
for (const tsName of groupOrder) {
|
||||
const g = grouped.get(tsName)!;
|
||||
const text = g.texts.join('');
|
||||
if (text.trim()) rawWords.push({ text, time: msToTimestamp(g.startMs) });
|
||||
}
|
||||
if (rawWords.length === 0) continue;
|
||||
|
||||
// Extraire le locuteur depuis le début du premier mot
|
||||
let speaker: string | undefined;
|
||||
const firstText = rawWords[0].text;
|
||||
const speakerMatch = SPEAKER_RE.exec(firstText);
|
||||
if (speakerMatch) {
|
||||
const candidate = speakerMatch[1].trim();
|
||||
// Pas de locuteur si c'est juste une marque de pause
|
||||
if (candidate && !/^\(\.+\)$/.test(candidate)) {
|
||||
speaker = candidate;
|
||||
rawWords[0] = { ...rawWords[0], text: firstText.slice(speakerMatch[0].length) };
|
||||
}
|
||||
}
|
||||
|
||||
const words = rawWords.filter((w) => w.text.trim().length > 0);
|
||||
if (words.length === 0) continue;
|
||||
|
||||
const startMs = grouped.get(groupOrder[0])!.startMs;
|
||||
const endMs = grouped.get(groupOrder[groupOrder.length - 1])!.endMs;
|
||||
|
||||
cues.push({
|
||||
startTime: msToTimestamp(startMs),
|
||||
endTime: msToTimestamp(endMs),
|
||||
speaker,
|
||||
text: words.map((w) => w.text).join(''),
|
||||
words,
|
||||
rawLines: [],
|
||||
});
|
||||
}
|
||||
|
||||
return { cues, trailingNewline: false };
|
||||
}
|
||||
}
|
||||
209
src/parsers/NoScribeVttParser.ts
Normal file
209
src/parsers/NoScribeVttParser.ts
Normal file
|
|
@ -0,0 +1,209 @@
|
|||
/**
|
||||
* Parser VTT noScribe — format produit par noScribe ≥ 0.7.
|
||||
*
|
||||
* Le VTT noScribe n'est PAS du WebVTT standard : chaque tour de parole est
|
||||
* décomposé en plusieurs cues successives ayant souvent le même intervalle :
|
||||
* - cue "label" : "<v SXX>SXX: " (détection du locuteur)
|
||||
* - cue "display" : "<v SXX>[HH:MM:SS]" (timestamp lisible — ignorer)
|
||||
* - cue "texte" : "<v SXX>contenu transcrit" (texte réel)
|
||||
*
|
||||
* Parfois le label et le texte sont fusionnés :
|
||||
* "S01: [00:00:09] Vous avez fini ces rapports ?"
|
||||
*
|
||||
* Contraintes :
|
||||
* - Le tag <v SXX> n'est pas fiable pour l'attribution des locuteurs.
|
||||
* - Le timestamp des cues "label" est erroné ; utiliser celui de la cue texte.
|
||||
* - Pas de word-level timestamps (contrairement au VTT Whisper).
|
||||
*
|
||||
* Produit un VttDocument compatible avec vttToMarkdown / extractWordData.
|
||||
*/
|
||||
|
||||
import type { VttDocument, VttCue } from './VttParser';
|
||||
|
||||
// ---- Regex ------------------------------------------------------------------
|
||||
|
||||
// Ligne de timestamp VTT
|
||||
const TIMESTAMP_LINE_RE = /^(\d{2}:\d{2}:\d{2}\.\d{3})\s+-->\s+(\d{2}:\d{2}:\d{2}\.\d{3})/;
|
||||
|
||||
// Tag <v SXX> noScribe
|
||||
const SPEAKER_V_TAG_RE = /^<v\s+\w+>/;
|
||||
|
||||
// Label locuteur en début de texte : "S00: " ou "S00 : "
|
||||
const SPEAKER_LABEL_RE = /^(S\d+)\s*:\s*/;
|
||||
|
||||
// Timestamp d'affichage : [HH:MM:SS]
|
||||
const DISPLAY_TS_RE = /^\[\d{2}:\d{2}:\d{2}\]\s*/;
|
||||
|
||||
// Entités HTML basiques
|
||||
const HTML_ENTITIES: Record<string, string> = {
|
||||
''': "'", ''': "'", '&': '&',
|
||||
'<': '<', '>': '>', '"': '"', ' ': ' ',
|
||||
};
|
||||
|
||||
function decodeEntities(text: string): string {
|
||||
return text.replace(/&#?x?[0-9a-zA-Z]+;/g, (m) => HTML_ENTITIES[m] ?? m);
|
||||
}
|
||||
|
||||
function stripSpeakerVTag(line: string): string {
|
||||
return SPEAKER_V_TAG_RE.test(line) ? line.replace(SPEAKER_V_TAG_RE, '') : line;
|
||||
}
|
||||
|
||||
// ---- Types intermédiaires ---------------------------------------------------
|
||||
|
||||
interface RawCue {
|
||||
startTime: string;
|
||||
endTime: string;
|
||||
text: string; // texte nettoyé (sans <v>), entités décodées
|
||||
}
|
||||
|
||||
interface Fragment {
|
||||
startTime: string;
|
||||
endTime: string;
|
||||
speaker: string | undefined;
|
||||
text: string;
|
||||
}
|
||||
|
||||
// ---- Parser -----------------------------------------------------------------
|
||||
|
||||
export class NoScribeVttParser {
|
||||
/** Détecte un VTT produit par noScribe (présence de NOTE noScribe). */
|
||||
static isNoScribeVtt(content: string): boolean {
|
||||
return content.startsWith('WEBVTT') && content.includes('noScribe');
|
||||
}
|
||||
|
||||
/** Extrait le chemin audio depuis la ligne NOTE media. */
|
||||
static extractAudioSource(content: string): string | null {
|
||||
const m = /^NOTE\s+media:\s*(.+)$/m.exec(content);
|
||||
return m ? m[1].trim() : null;
|
||||
}
|
||||
|
||||
parse(content: string): VttDocument {
|
||||
const rawCues = this.parseRawCues(content);
|
||||
const fragments = this.buildFragments(rawCues);
|
||||
const cues = this.mergeIntoCues(fragments);
|
||||
return { cues, trailingNewline: content.endsWith('\n') };
|
||||
}
|
||||
|
||||
// --- Étape 1 : parser les cues brutes ------------------------------------
|
||||
|
||||
private parseRawCues(content: string): RawCue[] {
|
||||
const normalized = content.replace(/\r\n/g, '\n').replace(/\r/g, '\n');
|
||||
const lines = normalized.split('\n');
|
||||
const cues: RawCue[] = [];
|
||||
|
||||
let i = 0;
|
||||
// Sauter jusqu'à la première ligne de timestamp (ignore header + NOTE)
|
||||
while (i < lines.length) {
|
||||
const tsMatch = TIMESTAMP_LINE_RE.exec(lines[i]);
|
||||
if (tsMatch) {
|
||||
const startTime = tsMatch[1];
|
||||
const endTime = tsMatch[2];
|
||||
i++;
|
||||
|
||||
// Collecter les lignes de texte jusqu'à la ligne vide
|
||||
const textLines: string[] = [];
|
||||
while (i < lines.length && lines[i].trim() !== '') {
|
||||
const cleaned = decodeEntities(stripSpeakerVTag(lines[i]));
|
||||
textLines.push(cleaned);
|
||||
i++;
|
||||
}
|
||||
|
||||
const text = textLines.join(' ').trim();
|
||||
cues.push({ startTime, endTime, text });
|
||||
} else {
|
||||
i++;
|
||||
}
|
||||
}
|
||||
return cues;
|
||||
}
|
||||
|
||||
// --- Étape 2 : convertir en fragments (speaker + text + timestamp) --------
|
||||
|
||||
private buildFragments(cues: RawCue[]): Fragment[] {
|
||||
const fragments: Fragment[] = [];
|
||||
let currentSpeaker: string | undefined;
|
||||
|
||||
for (const cue of cues) {
|
||||
let text = cue.text;
|
||||
|
||||
// Extraire le label locuteur s'il est présent
|
||||
const labelMatch = SPEAKER_LABEL_RE.exec(text);
|
||||
if (labelMatch) {
|
||||
currentSpeaker = labelMatch[1]; // "S00", "S01", etc.
|
||||
text = text.slice(labelMatch[0].length);
|
||||
}
|
||||
|
||||
// Supprimer le timestamp d'affichage [HH:MM:SS]
|
||||
text = text.replace(DISPLAY_TS_RE, '').trim();
|
||||
|
||||
// Ignorer les cues vides, les cues label seuls, et les timestamps seuls
|
||||
if (!text || /^\[[\d:]+\]$/.test(text)) continue;
|
||||
|
||||
fragments.push({
|
||||
startTime: cue.startTime,
|
||||
endTime: cue.endTime,
|
||||
speaker: currentSpeaker,
|
||||
text,
|
||||
});
|
||||
}
|
||||
|
||||
return fragments;
|
||||
}
|
||||
|
||||
// --- Étape 3 : fusionner les fragments en VttCues -------------------------
|
||||
//
|
||||
// Un "tour" commence à chaque nouveau label de locuteur dans l'étape 2.
|
||||
// Les fragments sans changement de locuteur sont rattachés au tour courant.
|
||||
// Heuristique de fusion : on regroupe les fragments temporellement proches
|
||||
// (gap < 2 s) appartenant au même locuteur.
|
||||
|
||||
private mergeIntoCues(fragments: Fragment[]): VttCue[] {
|
||||
if (fragments.length === 0) return [];
|
||||
|
||||
const cues: VttCue[] = [];
|
||||
let batch: Fragment[] = [fragments[0]];
|
||||
|
||||
const flushBatch = () => {
|
||||
if (batch.length === 0) return;
|
||||
const text = batch.map((f) => f.text).join(' ').replace(/\s+/g, ' ').trim();
|
||||
if (text) {
|
||||
cues.push({
|
||||
startTime: batch[0].startTime,
|
||||
endTime: batch[batch.length - 1].endTime,
|
||||
speaker: batch[0].speaker,
|
||||
text,
|
||||
words: [{ text, time: batch[0].startTime }],
|
||||
rawLines: [],
|
||||
});
|
||||
}
|
||||
batch = [];
|
||||
};
|
||||
|
||||
for (let i = 1; i < fragments.length; i++) {
|
||||
const prev = batch[batch.length - 1];
|
||||
const curr = fragments[i];
|
||||
|
||||
// Nouveau tour si le locuteur change
|
||||
const speakerChanged = curr.speaker !== prev.speaker;
|
||||
// Ou si le gap temporel est significatif (> 2 s) entre deux fragments du même locuteur
|
||||
const prevEndSec = timeToSeconds(prev.endTime);
|
||||
const currStartSec = timeToSeconds(curr.startTime);
|
||||
const bigGap = !speakerChanged && (currStartSec - prevEndSec) > 2;
|
||||
|
||||
if (speakerChanged || bigGap) {
|
||||
flushBatch();
|
||||
batch = [curr];
|
||||
} else {
|
||||
batch.push(curr);
|
||||
}
|
||||
}
|
||||
|
||||
flushBatch();
|
||||
return cues;
|
||||
}
|
||||
}
|
||||
|
||||
function timeToSeconds(ts: string): number {
|
||||
const [h, m, s] = ts.split(':');
|
||||
return parseInt(h) * 3600 + parseInt(m) * 60 + parseFloat(s);
|
||||
}
|
||||
|
|
@ -1,5 +1,6 @@
|
|||
import type { SrtDocument } from './SrtParser';
|
||||
import type { ChatDocument, ChatLine } from './ChatParser';
|
||||
import type { VttDocument, VttWord } from './VttParser';
|
||||
|
||||
// Converts a SrtDocument to a structured, Obsidian-readable Markdown string.
|
||||
// Timestamps are preserved as italic headers; text lines are the editable content.
|
||||
|
|
@ -19,7 +20,6 @@ export function srtToMarkdown(doc: SrtDocument, sourceName: string): string {
|
|||
lines.push('');
|
||||
}
|
||||
|
||||
// Retirer la dernière ligne vide superflue, puis ajouter \n final
|
||||
while (lines[lines.length - 1] === '') lines.pop();
|
||||
lines.push('');
|
||||
|
||||
|
|
@ -43,7 +43,6 @@ export function chatToMarkdown(doc: ChatDocument, sourceName: string): string {
|
|||
for (const chatLine of doc.lines) {
|
||||
const group = lineGroup(chatLine);
|
||||
|
||||
// Ligne vide entre groupes structurels et tours de parole
|
||||
if (prevGroup !== null && prevGroup !== group) {
|
||||
lines.push('');
|
||||
}
|
||||
|
|
@ -57,13 +56,11 @@ export function chatToMarkdown(doc: ChatDocument, sourceName: string): string {
|
|||
lines.push(`**${chatLine.speaker}** : ${chatLine.content}`);
|
||||
break;
|
||||
case 'continuation':
|
||||
// Rattacher au tour précédent (rare en pratique)
|
||||
if (lines.length > 0) {
|
||||
lines[lines.length - 1] += ` ${chatLine.raw.trim()}`;
|
||||
}
|
||||
break;
|
||||
case 'blank':
|
||||
// Les blancs du source sont absorbés par la logique de groupe ci-dessus
|
||||
break;
|
||||
}
|
||||
|
||||
|
|
@ -76,6 +73,136 @@ export function chatToMarkdown(doc: ChatDocument, sourceName: string): string {
|
|||
return lines.join('\n');
|
||||
}
|
||||
|
||||
// ---- VTT / noScribe HTML ---------------------------------------------------
|
||||
|
||||
/** Données word-level d'une cue — stockées dans <basename>.words.json. */
|
||||
export interface VttCueData {
|
||||
index: number;
|
||||
startTime: string;
|
||||
endTime: string;
|
||||
speaker?: string;
|
||||
words: VttWord[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Extrait les données word-level d'un VttDocument pour le fichier .words.json.
|
||||
* Seules les cues avec au moins un word timestamp sont incluses.
|
||||
*/
|
||||
export function extractWordData(doc: VttDocument): VttCueData[] {
|
||||
return doc.cues
|
||||
.map((cue, index) => ({
|
||||
index,
|
||||
startTime: cue.startTime,
|
||||
endTime: cue.endTime,
|
||||
speaker: cue.speaker,
|
||||
words: cue.words,
|
||||
}))
|
||||
.filter((c) => c.words.some((w) => w.time !== ''));
|
||||
}
|
||||
|
||||
// ---- Re-export VTT ----------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Regex pour une ligne de corps noScribe dans le Markdown.
|
||||
* Capture optionnellement le locuteur (groupe 1) et le texte (groupe 2).
|
||||
* **S00** [00:01:28] : texte → speaker=S00, text=texte
|
||||
* [00:00:00] (..) → speaker=undefined, text=(..)
|
||||
*/
|
||||
const MD_CUE_RE = /^(?:\*\*([^*]+)\*\*\s+)?\[[\d:]+\]\s*(?::\s*)?(.*)$/;
|
||||
|
||||
/**
|
||||
* Reconstruit un fichier WebVTT pseudonymisé depuis :
|
||||
* - `mdContent` : le Markdown noScribe (pseudonymisé)
|
||||
* - `wordData` : le contenu de <basename>.words.json (timestamps précis)
|
||||
*
|
||||
* Alignement par index : ligne de corps N = cue N du words.json.
|
||||
* Le texte vient du Markdown ; les timestamps et le speaker de secours
|
||||
* viennent du words.json.
|
||||
*
|
||||
* Retourne null si le nombre de lignes et de cues ne correspondent pas
|
||||
* (le caller affiche un avertissement dans ce cas).
|
||||
*/
|
||||
export function markdownToVtt(
|
||||
mdContent: string,
|
||||
wordData: VttCueData[],
|
||||
): { vtt: string; mismatch: boolean } {
|
||||
// Extraire le corps (après le frontmatter --- ... ---)
|
||||
const bodyMatch = /^---\n[\s\S]*?\n---\n+([\s\S]*)$/.exec(mdContent);
|
||||
const body = bodyMatch ? bodyMatch[1] : mdContent;
|
||||
|
||||
const cueLines = body.split('\n').filter((l) => MD_CUE_RE.test(l.trim()));
|
||||
const mismatch = cueLines.length !== wordData.length;
|
||||
|
||||
const parts = ['WEBVTT', ''];
|
||||
const count = Math.min(cueLines.length, wordData.length);
|
||||
|
||||
for (let i = 0; i < count; i++) {
|
||||
const m = MD_CUE_RE.exec(cueLines[i].trim())!;
|
||||
// Locuteur : priorité au Markdown, fallback words.json
|
||||
const speaker = m[1]?.trim() || wordData[i].speaker;
|
||||
const text = m[2]?.trim() ?? '';
|
||||
|
||||
if (!text) continue;
|
||||
|
||||
parts.push(String(i + 1));
|
||||
parts.push(`${wordData[i].startTime} --> ${wordData[i].endTime}`);
|
||||
parts.push(speaker ? `<v ${speaker}>${text}` : text);
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
return { vtt: parts.join('\n'), mismatch };
|
||||
}
|
||||
|
||||
/** Convertit HH:MM:SS.mmm en [HH:MM:SS] — format d'affichage noScribe. */
|
||||
function displayTime(ts: string): string {
|
||||
return `[${ts.slice(0, 8)}]`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convertit un VttDocument en Markdown structuré au format noScribe.
|
||||
*
|
||||
* Format par cue :
|
||||
* **Locuteur** [HH:MM:SS] : texte
|
||||
* [HH:MM:SS] texte (sans locuteur)
|
||||
*
|
||||
* Les timestamps précis (avec ms) et les word timestamps sont dans .words.json,
|
||||
* pas dans le corps Markdown. Le frontmatter peut inclure pseudobs-audio si
|
||||
* un fichier audio a été importé avec la transcription.
|
||||
*/
|
||||
function vttDocToMarkdown(
|
||||
doc: VttDocument,
|
||||
sourceName: string,
|
||||
format: 'vtt' | 'html',
|
||||
audioFilename?: string,
|
||||
): string {
|
||||
const lines: string[] = ['---', `pseudobs-format: ${format}`, `pseudobs-source: "${sourceName}"`];
|
||||
if (audioFilename) lines.push(`pseudobs-audio: "${audioFilename}"`);
|
||||
lines.push('---', '');
|
||||
|
||||
for (const cue of doc.cues) {
|
||||
const ts = displayTime(cue.startTime);
|
||||
if (cue.speaker) {
|
||||
lines.push(`**${cue.speaker}** ${ts} : ${cue.text}`);
|
||||
} else {
|
||||
lines.push(`${ts} ${cue.text}`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
while (lines[lines.length - 1] === '') lines.pop();
|
||||
lines.push('');
|
||||
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
export function vttToMarkdown(doc: VttDocument, sourceName: string, audioFilename?: string): string {
|
||||
return vttDocToMarkdown(doc, sourceName, 'vtt', audioFilename);
|
||||
}
|
||||
|
||||
export function noScribeHtmlToMarkdown(doc: VttDocument, sourceName: string, audioFilename?: string): string {
|
||||
return vttDocToMarkdown(doc, sourceName, 'html', audioFilename);
|
||||
}
|
||||
|
||||
function lineGroup(line: ChatLine): 'structural' | 'turn' | null {
|
||||
if (line.type === 'meta' || line.type === 'dependent') return 'structural';
|
||||
if (line.type === 'turn' || line.type === 'continuation') return 'turn';
|
||||
|
|
|
|||
250
src/parsers/VttParser.ts
Normal file
250
src/parsers/VttParser.ts
Normal file
|
|
@ -0,0 +1,250 @@
|
|||
/**
|
||||
* Parser VTT (WebVTT) — optimisé pour les sorties noScribe (Whisper + pyannote).
|
||||
*
|
||||
* Formats pris en charge :
|
||||
* - VTT standard (blocs sans word timestamps)
|
||||
* - Whisper word timestamps : <HH:MM:SS.mmm><c>mot</c>
|
||||
* - Speaker tags : <v SPEAKER_NAME> ou [SPEAKER_NAME]
|
||||
*
|
||||
* Round-trip garanti : les word timestamps sont stockés séparément du texte
|
||||
* et réinsérés à la reconstruction autour du texte (éventuellement pseudonymisé).
|
||||
*/
|
||||
|
||||
import type { BaseTranscriptParser } from './BaseTranscriptParser';
|
||||
|
||||
/** Un mot avec sa position temporelle dans l'audio. */
|
||||
export interface VttWord {
|
||||
text: string; // texte du mot (tel qu'il apparaît dans le fichier, espaces inclus)
|
||||
time: string; // timestamp Whisper "HH:MM:SS.mmm" — vide si absent
|
||||
}
|
||||
|
||||
export interface VttCue {
|
||||
id?: string; // identifiant optionnel de la cue (nombre ou chaîne)
|
||||
startTime: string; // "HH:MM:SS.mmm"
|
||||
endTime: string; // "HH:MM:SS.mmm"
|
||||
speaker?: string; // locuteur extrait du tag <v ...> ou [...]
|
||||
text: string; // texte nettoyé (pseudonymisable)
|
||||
words: VttWord[]; // mots avec timestamps — vide si pas de word timestamps
|
||||
// Ligne(s) de texte brut original (pour le round-trip si pas de word timestamps)
|
||||
rawLines: string[];
|
||||
}
|
||||
|
||||
export interface VttDocument {
|
||||
cues: VttCue[];
|
||||
trailingNewline: boolean;
|
||||
}
|
||||
|
||||
// ---- Regex ------------------------------------------------------------------
|
||||
|
||||
// Timestamp VTT : HH:MM:SS.mmm ou MM:SS.mmm
|
||||
const TIME_RE = /(\d{1,2}:\d{2}:\d{2}\.\d{3}|\d{2}:\d{2}\.\d{3})/;
|
||||
const TIMESTAMP_LINE_RE = new RegExp(
|
||||
`^${TIME_RE.source}\\s+-->\\s+${TIME_RE.source}\\s*(.*)$`
|
||||
);
|
||||
|
||||
// Word timestamp Whisper : <HH:MM:SS.mmm>
|
||||
const WORD_TIME_RE = /<(\d{1,2}:\d{2}:\d{2}\.\d{3}|\d{2}:\d{2}\.\d{3})>/g;
|
||||
|
||||
// Tag de classe Whisper : <c>...</c>
|
||||
const CLASS_TAG_RE = /<\/?c>/g;
|
||||
|
||||
// Speaker tag : <v Nom> ou <v.classname Nom>
|
||||
const SPEAKER_V_RE = /^<v(?:\.[^\s>]+)?\s+([^>]+)>/;
|
||||
|
||||
// Speaker bracket : [Nom]
|
||||
const SPEAKER_BRACKET_RE = /^\[([^\]]+)\]/;
|
||||
|
||||
// Tout tag HTML/VTT générique
|
||||
const ALL_TAGS_RE = /<[^>]+>/g;
|
||||
|
||||
// ---- Helpers ----------------------------------------------------------------
|
||||
|
||||
/** Normalise un timestamp "MM:SS.mmm" en "HH:MM:SS.mmm". */
|
||||
function normalizeTime(t: string): string {
|
||||
return t.includes(':') && t.split(':').length === 2 ? `00:${t}` : t;
|
||||
}
|
||||
|
||||
/** Retire tous les tags VTT du texte pour obtenir le texte brut lisible. */
|
||||
function stripTags(text: string): string {
|
||||
return text.replace(ALL_TAGS_RE, '').trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extrait les mots avec leurs timestamps Whisper depuis une ligne de texte cue.
|
||||
* Si aucun timestamp n'est présent, retourne un tableau avec le texte complet.
|
||||
*/
|
||||
function extractWords(rawText: string): VttWord[] {
|
||||
// Vérifier si des word timestamps sont présents
|
||||
const hasWordTimes = WORD_TIME_RE.test(rawText);
|
||||
WORD_TIME_RE.lastIndex = 0; // reset après test
|
||||
|
||||
if (!hasWordTimes) {
|
||||
const clean = stripTags(rawText);
|
||||
return clean ? [{ text: clean, time: '' }] : [];
|
||||
}
|
||||
|
||||
// Découper en segments : [timestamp, texte, timestamp, texte, ...]
|
||||
const words: VttWord[] = [];
|
||||
const remaining = rawText;
|
||||
let currentTime = '';
|
||||
|
||||
// Traiter segment par segment
|
||||
const parts = remaining.split(WORD_TIME_RE);
|
||||
// parts alterne : [texte_avant_premier_ts, ts1, texte1, ts2, texte2, ...]
|
||||
|
||||
for (let i = 0; i < parts.length; i++) {
|
||||
const part = parts[i];
|
||||
if (TIME_RE.test(part) && part.match(/^\d/)) {
|
||||
// C'est un timestamp
|
||||
currentTime = normalizeTime(part);
|
||||
} else {
|
||||
// C'est du texte — nettoyer les tags de classe
|
||||
const clean = part.replace(CLASS_TAG_RE, '').replace(ALL_TAGS_RE, '');
|
||||
if (clean.trim()) {
|
||||
words.push({ text: clean, time: currentTime });
|
||||
currentTime = '';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return words;
|
||||
}
|
||||
|
||||
/**
|
||||
* Extrait le locuteur et le texte brut (sans speaker tag) d'une ligne cue.
|
||||
*/
|
||||
function extractSpeakerAndText(line: string): { speaker: string | undefined; text: string } {
|
||||
const vMatch = SPEAKER_V_RE.exec(line);
|
||||
if (vMatch) {
|
||||
return { speaker: vMatch[1].trim(), text: line.slice(vMatch[0].length) };
|
||||
}
|
||||
const bMatch = SPEAKER_BRACKET_RE.exec(line);
|
||||
if (bMatch) {
|
||||
return { speaker: bMatch[1].trim(), text: line.slice(bMatch[0].length).trim() };
|
||||
}
|
||||
return { speaker: undefined, text: line };
|
||||
}
|
||||
|
||||
// ---- Parser -----------------------------------------------------------------
|
||||
|
||||
export class VttParser implements BaseTranscriptParser<VttDocument> {
|
||||
|
||||
parse(content: string): VttDocument {
|
||||
const trailingNewline = content.endsWith('\n');
|
||||
const normalized = content.replace(/\r\n/g, '\n').replace(/\r/g, '\n').trimEnd();
|
||||
|
||||
const lines = normalized.split('\n');
|
||||
let i = 0;
|
||||
|
||||
// Sauter le header WEBVTT (et les métadonnées éventuelles)
|
||||
while (i < lines.length && !lines[i].startsWith('WEBVTT')) i++;
|
||||
i++; // sauter la ligne WEBVTT elle-même
|
||||
|
||||
const cues: VttCue[] = [];
|
||||
|
||||
while (i < lines.length) {
|
||||
// Sauter les lignes vides
|
||||
while (i < lines.length && lines[i].trim() === '') i++;
|
||||
if (i >= lines.length) break;
|
||||
|
||||
// ID optionnel (ligne non-timestamp, non-vide avant le timestamp)
|
||||
let cueId: string | undefined;
|
||||
if (i < lines.length && !TIMESTAMP_LINE_RE.test(lines[i])) {
|
||||
cueId = lines[i].trim();
|
||||
i++;
|
||||
}
|
||||
|
||||
// Ligne timestamp
|
||||
if (i >= lines.length) break;
|
||||
const tsMatch = TIMESTAMP_LINE_RE.exec(lines[i]);
|
||||
if (!tsMatch) { i++; continue; } // ligne inattendue
|
||||
|
||||
const startTime = normalizeTime(tsMatch[1]);
|
||||
const endTime = normalizeTime(tsMatch[2]);
|
||||
i++;
|
||||
|
||||
// Lignes de texte de la cue
|
||||
const rawLines: string[] = [];
|
||||
while (i < lines.length && lines[i].trim() !== '') {
|
||||
rawLines.push(lines[i]);
|
||||
i++;
|
||||
}
|
||||
|
||||
if (rawLines.length === 0) continue;
|
||||
|
||||
// Extraire locuteur depuis la première ligne
|
||||
const firstLineResult = extractSpeakerAndText(rawLines[0]);
|
||||
const speaker = firstLineResult.speaker;
|
||||
|
||||
// Reconstruire le texte complet de la cue (sans speaker tag sur la 1ère ligne)
|
||||
const textLines = [firstLineResult.text, ...rawLines.slice(1)];
|
||||
const fullRaw = textLines.join('\n');
|
||||
|
||||
// Extraire les word timestamps
|
||||
const words = extractWords(fullRaw);
|
||||
|
||||
// Texte nettoyé pour pseudonymisation
|
||||
const text = words.map((w) => w.text).join('');
|
||||
|
||||
cues.push({ id: cueId, startTime, endTime, speaker, text, words, rawLines });
|
||||
}
|
||||
|
||||
return { cues, trailingNewline };
|
||||
}
|
||||
|
||||
reconstruct(doc: VttDocument): string {
|
||||
const parts: string[] = ['WEBVTT', ''];
|
||||
|
||||
for (const cue of doc.cues) {
|
||||
if (cue.id !== undefined) parts.push(cue.id);
|
||||
|
||||
parts.push(`${cue.startTime} --> ${cue.endTime}`);
|
||||
|
||||
if (cue.words.length > 0 && cue.words.some((w) => w.time !== '')) {
|
||||
// Reconstruire avec word timestamps depuis le tableau words
|
||||
const line = this.reconstructWithWordTimestamps(cue);
|
||||
parts.push(line);
|
||||
} else {
|
||||
// Pas de word timestamps — reconstruire depuis le texte pseudonymisé
|
||||
// en préservant le speaker tag si présent
|
||||
const speaker = cue.speaker;
|
||||
const textLine = speaker ? `<v ${speaker}>${cue.text}` : cue.text;
|
||||
parts.push(textLine);
|
||||
}
|
||||
|
||||
parts.push('');
|
||||
}
|
||||
|
||||
const body = parts.join('\n');
|
||||
return doc.trailingNewline ? body : body.trimEnd();
|
||||
}
|
||||
|
||||
private reconstructWithWordTimestamps(cue: VttCue): string {
|
||||
let line = cue.speaker ? `<v ${cue.speaker}>` : '';
|
||||
for (const word of cue.words) {
|
||||
if (word.time) line += `<${word.time}>`;
|
||||
line += `<c>${word.text}</c>`;
|
||||
}
|
||||
return line;
|
||||
}
|
||||
|
||||
/**
|
||||
* Met à jour le texte d'une cue après pseudonymisation.
|
||||
* Propage le remplacement dans le tableau words pour maintenir le round-trip.
|
||||
*/
|
||||
static applyTextToWords(cue: VttCue, newText: string): void {
|
||||
cue.text = newText;
|
||||
// Si pas de word timestamps, on met tout dans le premier mot
|
||||
if (cue.words.length === 0 || cue.words.every((w) => w.time === '')) {
|
||||
cue.words = [{ text: newText, time: '' }];
|
||||
return;
|
||||
}
|
||||
// Avec word timestamps : le remplacement affecte potentiellement plusieurs mots.
|
||||
// Stratégie simple : on remplace le texte du premier mot et on vide les suivants.
|
||||
// Une future version pourrait aligner mot à mot via diff.
|
||||
cue.words[0].text = newText;
|
||||
for (let i = 1; i < cue.words.length; i++) {
|
||||
cue.words[i].text = '';
|
||||
}
|
||||
}
|
||||
}
|
||||
31
src/pseudonymizer/Redaction.ts
Normal file
31
src/pseudonymizer/Redaction.ts
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
export const REDACTION_CHAR = '🀫';
|
||||
|
||||
/**
|
||||
* Compte les syllabes d'un texte par heuristique de groupes de voyelles.
|
||||
* Fonctionne pour le français et les principales langues romanes/germaniques.
|
||||
* Minimum 1 syllabe retourné.
|
||||
*/
|
||||
export function countSyllables(text: string): number {
|
||||
const groups = text.match(/[aeiouyàâäéèêëîïôùûüœæAEIOUYÀÂÄÉÈÊËÎÏÔÙÛÜŒÆ]+/g);
|
||||
return Math.max(1, groups?.length ?? 1);
|
||||
}
|
||||
|
||||
/**
|
||||
* Génère un remplacement caviardé : 1 🀫 par syllabe de chaque mot.
|
||||
* Les espaces entre mots sont préservés.
|
||||
* Ex : "Marie Dupont" → "🀫🀫 🀫🀫🀫" · "Saint-Jean" → "🀫🀫🀫"
|
||||
*/
|
||||
export function generateRedaction(text: string): string {
|
||||
return text
|
||||
.split(/( +)/) // séparer sur les espaces en les conservant
|
||||
.map((part) => {
|
||||
if (/^ +$/.test(part)) return part; // préserver les espaces tels quels
|
||||
return REDACTION_CHAR.repeat(countSyllables(part));
|
||||
})
|
||||
.join('');
|
||||
}
|
||||
|
||||
/** Retourne true si un remplacement est un caviardage (commence par 🀫). */
|
||||
export function isRedaction(replacement: string): boolean {
|
||||
return replacement.startsWith(REDACTION_CHAR);
|
||||
}
|
||||
|
|
@ -18,6 +18,39 @@ const TAG_TO_CATEGORY: Record<string, EntityCategory> = {
|
|||
|
||||
const MIN_ENTITY_LENGTH = 2;
|
||||
|
||||
// Préambules Markdown à stripper avant envoi au modèle NER.
|
||||
// Chaque regex matche le début d'une ligne (format produit par les parsers du plugin).
|
||||
// Le résultat : { cleanText, preambleLength } pour recaler les positions.
|
||||
const PREAMBLE_PATTERNS: RegExp[] = [
|
||||
// VTT noScribe : *HH:MM:SS.mmm → HH:MM:SS.mmm* **SPEAKER**
|
||||
/^\*[\d:.]+\s*→\s*[\d:.]+\*(?:\s*\*\*[^*]+\*\*)?\s*/,
|
||||
// SRT bloc de texte : **[N]** *HH:MM:SS,mmm → HH:MM:SS,mmm* (ligne à ignorer entièrement)
|
||||
/^\*\*\[\d+\]\*\*\s*\*[^*]+\*/,
|
||||
// CHAT tour de parole : **SPEAKER** :
|
||||
/^\*\*[A-Z0-9_]+\*\*\s*:\s*/,
|
||||
];
|
||||
|
||||
// Lignes à ignorer entièrement (pas de texte à analyser)
|
||||
const SKIP_PATTERNS: RegExp[] = [
|
||||
/^---/, // frontmatter YAML
|
||||
/^pseudobs-/, // clés frontmatter du plugin
|
||||
/^> /, // métadonnées CHAT (@, %)
|
||||
/^\*\*\[\d+\]\*\*/, // en-tête de bloc SRT (index + timestamp sur la même ligne)
|
||||
];
|
||||
|
||||
function stripMarkdownPreamble(line: string): { cleanText: string; preambleLength: number } {
|
||||
// Lignes à ignorer complètement
|
||||
for (const skip of SKIP_PATTERNS) {
|
||||
if (skip.test(line)) return { cleanText: '', preambleLength: line.length };
|
||||
}
|
||||
// Stripper le préambule
|
||||
for (const pat of PREAMBLE_PATTERNS) {
|
||||
const m = pat.exec(line);
|
||||
if (m) return { cleanText: line.slice(m[0].length), preambleLength: m[0].length };
|
||||
}
|
||||
return { cleanText: line, preambleLength: 0 };
|
||||
}
|
||||
|
||||
type NerResult = {
|
||||
entity_group: string;
|
||||
score: number;
|
||||
|
|
@ -103,6 +136,9 @@ export class OnnxNerScanner {
|
|||
// numThreads=1 : SharedArrayBuffer non disponible dans Electron renderer
|
||||
// sans COOP/COEP → pas de WASM threadé.
|
||||
env.backends.onnx.wasm.numThreads = 1;
|
||||
// proxy=true : délègue la compilation WASM à un worker thread Node.js
|
||||
// → libère le thread renderer pendant l'initialisation du modèle.
|
||||
env.backends.onnx.wasm.proxy = true;
|
||||
}
|
||||
|
||||
// Autoriser le téléchargement depuis HuggingFace Hub
|
||||
|
|
@ -147,18 +183,20 @@ export class OnnxNerScanner {
|
|||
let offset = 0;
|
||||
|
||||
for (const line of lines) {
|
||||
if (line.trim().length > 2) {
|
||||
const { cleanText, preambleLength } = stripMarkdownPreamble(line);
|
||||
|
||||
if (cleanText.trim().length > 2) {
|
||||
try {
|
||||
const entities: NerResult[] = await _pipeline(line, { aggregation_strategy: 'simple' });
|
||||
const entities: NerResult[] = await _pipeline(cleanText, { aggregation_strategy: 'simple' });
|
||||
for (const ent of entities) {
|
||||
if (ent.score < minScore) continue;
|
||||
const word = ent.word.trim().replace(/^#+/, ''); // supprimer les ## de sous-mots
|
||||
// Filtrer les artefacts de tokenisation BERT (mots fonctionnels, tokens trop courts)
|
||||
const word = ent.word.trim().replace(/^#+/, '');
|
||||
if (word.length < MIN_ENTITY_LENGTH) continue;
|
||||
if (functionWords.has(word.toLowerCase())) continue;
|
||||
const category = TAG_TO_CATEGORY[ent.entity_group] ?? 'custom';
|
||||
const start = offset + ent.start;
|
||||
const end = offset + ent.end;
|
||||
// Positions recalées : offset de ligne + préambule strippé + position dans cleanText
|
||||
const start = offset + preambleLength + ent.start;
|
||||
const end = offset + preambleLength + ent.end;
|
||||
const ctxLen = 45;
|
||||
results.push({
|
||||
id: `ner_${Date.now()}_${++_counter}`,
|
||||
|
|
|
|||
|
|
@ -35,6 +35,13 @@ export interface OccurrenceRef {
|
|||
status: MappingStatus;
|
||||
}
|
||||
|
||||
/** Occurrence d'une règle explicitement ignorée par l'utilisateur. */
|
||||
export interface IgnoredOccurrence {
|
||||
text: string; // texte exact trouvé dans le document (sensible à la casse)
|
||||
contextBefore: string;
|
||||
contextAfter: string;
|
||||
}
|
||||
|
||||
export interface MappingRule {
|
||||
id: string;
|
||||
source: string;
|
||||
|
|
@ -50,6 +57,7 @@ export interface MappingRule {
|
|||
updatedAt?: string;
|
||||
notes?: string;
|
||||
occurrences?: OccurrenceRef[];
|
||||
ignoredOccurrences?: IgnoredOccurrence[];
|
||||
}
|
||||
|
||||
export interface Occurrence {
|
||||
|
|
|
|||
|
|
@ -248,7 +248,7 @@ export class DictScanReviewModal extends Modal {
|
|||
|
||||
const n = toCreate.length;
|
||||
new Notice(t('notice.rulesCreated', String(n), n > 1 ? t('notice.rulesCreated.rules') : t('notice.rulesCreated.rule')));
|
||||
void this.plugin.refreshHighlightData();
|
||||
void this.plugin.refresh();
|
||||
this.close();
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -128,7 +128,7 @@ export class EditRuleModal extends Modal {
|
|||
|
||||
await this.plugin.scopeResolver.saveStore(store, filePath);
|
||||
new Notice(t('notice.ruleCreated', rule.source, this.replacement.trim()));
|
||||
void this.plugin.refreshHighlightData();
|
||||
void this.plugin.refresh();
|
||||
this.close();
|
||||
}
|
||||
|
||||
|
|
@ -136,8 +136,9 @@ export class EditRuleModal extends Modal {
|
|||
const { store, filePath, rule } = this.location;
|
||||
store.remove(rule.id);
|
||||
await this.plugin.scopeResolver.saveStore(store, filePath);
|
||||
new Notice(`✓ "${rule.source}"`);
|
||||
void this.plugin.refreshHighlightData();
|
||||
// Rétablir le texte original dans le fichier actif
|
||||
await this.plugin.revertRuleInFile(rule.source, rule.replacement);
|
||||
new Notice(t('notice.ruleDeleted', rule.source));
|
||||
this.close();
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +1,16 @@
|
|||
import { App, Modal, Notice, TFile } from 'obsidian';
|
||||
import { t } from '../i18n';
|
||||
import type PseudObsPlugin from '../main';
|
||||
import type { MappingRule } from '../types';
|
||||
import { findSpansForRule } from '../pseudonymizer/ReplacementPlanner';
|
||||
import type { MappingRule, Occurrence } from '../types';
|
||||
import { resolveSpans, applySpans } from '../pseudonymizer/SpanProtector';
|
||||
import type { ReplacementSpan } from '../types';
|
||||
import { OccurrencesContextModal, type OccurrenceDecision } from './OccurrencesContextModal';
|
||||
import type { IgnoredOccurrence } from '../types';
|
||||
|
||||
export interface MappingRuleResult {
|
||||
rule: MappingRule;
|
||||
matchCount: number;
|
||||
occurrences: Occurrence[];
|
||||
}
|
||||
|
||||
export class MappingScanReviewModal extends Modal {
|
||||
|
|
@ -16,7 +19,11 @@ export class MappingScanReviewModal extends Modal {
|
|||
private content: string;
|
||||
private ruleResults: MappingRuleResult[];
|
||||
private checked: boolean[];
|
||||
// Décisions par règle : ruleId → occId → decision
|
||||
private decisionsMap = new Map<string, Map<string, OccurrenceDecision>>();
|
||||
private applyBtn!: HTMLButtonElement;
|
||||
// Cellules de comptage — pour mise à jour live
|
||||
private countCells: HTMLElement[] = [];
|
||||
|
||||
constructor(
|
||||
app: App,
|
||||
|
|
@ -31,6 +38,13 @@ export class MappingScanReviewModal extends Modal {
|
|||
this.content = content;
|
||||
this.ruleResults = ruleResults;
|
||||
this.checked = ruleResults.map(() => true);
|
||||
|
||||
// Décisions initiales : toutes les occurrences validées
|
||||
for (const { rule, occurrences } of ruleResults) {
|
||||
const map = new Map<string, OccurrenceDecision>();
|
||||
for (const occ of occurrences) map.set(occ.id, 'validated');
|
||||
this.decisionsMap.set(rule.id, map);
|
||||
}
|
||||
}
|
||||
|
||||
onOpen(): void {
|
||||
|
|
@ -41,7 +55,9 @@ export class MappingScanReviewModal extends Modal {
|
|||
contentEl.createEl('h2', { text: t('mappingScanModal.title') });
|
||||
const nr = this.ruleResults.length;
|
||||
contentEl.createEl('p', {
|
||||
text: t('mappingScanModal.summary', String(nr), nr > 1 ? t('mappingScanModal.summary.rules') : t('mappingScanModal.summary.rule'), this.file.name),
|
||||
text: t('mappingScanModal.summary', String(nr),
|
||||
nr > 1 ? t('mappingScanModal.summary.rules') : t('mappingScanModal.summary.rule'),
|
||||
this.file.name),
|
||||
cls: 'pseudobs-scan-summary',
|
||||
});
|
||||
contentEl.createEl('p', { text: t('mappingScanModal.hint'), cls: 'pseudobs-view-hint' });
|
||||
|
|
@ -56,7 +72,7 @@ export class MappingScanReviewModal extends Modal {
|
|||
);
|
||||
|
||||
const tbody = table.createEl('tbody');
|
||||
this.ruleResults.forEach(({ rule, matchCount }, i) => {
|
||||
this.ruleResults.forEach(({ rule, occurrences }, i) => {
|
||||
const tr = tbody.createEl('tr', { cls: 'pseudobs-dict-review-row' });
|
||||
|
||||
// Checkbox
|
||||
|
|
@ -72,11 +88,9 @@ export class MappingScanReviewModal extends Modal {
|
|||
|
||||
// Source
|
||||
tr.createEl('td', { text: rule.source, cls: 'pseudobs-dict-review-term' });
|
||||
|
||||
// Flèche
|
||||
tr.createEl('td', { text: '→', cls: 'pseudobs-dict-review-arrow' });
|
||||
|
||||
// Remplacement (lecture seule — déjà défini dans la règle)
|
||||
// Remplacement
|
||||
const repCell = tr.createEl('td');
|
||||
const s = this.plugin.settings;
|
||||
const displayRep = s.useMarkerInExport
|
||||
|
|
@ -84,68 +98,156 @@ export class MappingScanReviewModal extends Modal {
|
|||
: rule.replacement;
|
||||
repCell.createEl('span', { text: displayRep, cls: 'pseudobs-dict-review-rep-static' });
|
||||
|
||||
// Occurrences
|
||||
tr.createEl('td', { text: String(matchCount), cls: 'pseudobs-dict-review-count' });
|
||||
// Occurrences — bouton cliquable qui ouvre la modale de contexte
|
||||
const countCell = tr.createEl('td', { cls: 'pseudobs-dict-review-count' });
|
||||
this.countCells.push(countCell);
|
||||
this.renderCountCell(countCell, rule, occurrences, i);
|
||||
});
|
||||
|
||||
const footer = contentEl.createDiv('pseudobs-dict-review-footer');
|
||||
footer.createEl('button', { text: t('mappingScanModal.cancel') })
|
||||
.addEventListener('click', () => this.close());
|
||||
|
||||
footer.createEl('button', { text: t('mappingScanModal.saveExceptions'), cls: 'pseudobs-save-exceptions-btn' })
|
||||
.addEventListener('click', () => void this.saveExceptions());
|
||||
|
||||
this.applyBtn = footer.createEl('button', { cls: 'mod-cta' });
|
||||
this.applyBtn.addEventListener('click', () => void this.apply());
|
||||
this.updateApplyLabel();
|
||||
}
|
||||
|
||||
private renderCountCell(
|
||||
cell: HTMLElement,
|
||||
rule: MappingRule,
|
||||
occurrences: Occurrence[],
|
||||
ruleIndex: number
|
||||
): void {
|
||||
cell.empty();
|
||||
const decisions = this.decisionsMap.get(rule.id)!;
|
||||
const validated = occurrences.filter((o) => decisions.get(o.id) === 'validated').length;
|
||||
const total = occurrences.length;
|
||||
|
||||
if (total === 0) {
|
||||
cell.createSpan({ text: '0' });
|
||||
return;
|
||||
}
|
||||
|
||||
const btn = cell.createEl('button', { cls: 'pseudobs-count-btn' });
|
||||
// Afficher "N / total" si des occurrences ont été ignorées, "N" sinon
|
||||
btn.setText(validated < total ? `${validated} / ${total}` : String(total));
|
||||
btn.title = 'Voir et sélectionner les candidats';
|
||||
|
||||
btn.addEventListener('click', () => {
|
||||
new OccurrencesContextModal(
|
||||
this.app,
|
||||
rule,
|
||||
occurrences,
|
||||
decisions,
|
||||
(newDecisions) => {
|
||||
this.decisionsMap.set(rule.id, newDecisions);
|
||||
this.renderCountCell(cell, rule, occurrences, ruleIndex);
|
||||
this.updateApplyLabel();
|
||||
}
|
||||
).open();
|
||||
});
|
||||
}
|
||||
|
||||
private countValidated(): number {
|
||||
return this.ruleResults.reduce((sum, { rule, occurrences }, i) => {
|
||||
if (!this.checked[i]) return sum;
|
||||
const decisions = this.decisionsMap.get(rule.id)!;
|
||||
return sum + occurrences.filter((o) => decisions.get(o.id) === 'validated').length;
|
||||
}, 0);
|
||||
}
|
||||
|
||||
private updateApplyLabel(): void {
|
||||
const n = this.checked.filter(Boolean).length;
|
||||
const total = this.ruleResults
|
||||
.filter((_, i) => this.checked[i])
|
||||
.reduce((sum, r) => sum + r.matchCount, 0);
|
||||
this.applyBtn.textContent = n === 0
|
||||
const rules = this.checked.filter(Boolean).length;
|
||||
const total = this.countValidated();
|
||||
this.applyBtn.textContent = rules === 0
|
||||
? t('mappingScanModal.noRules')
|
||||
: t('mappingScanModal.apply',
|
||||
String(n), n > 1 ? t('mappingScanModal.apply.rules') : t('mappingScanModal.apply.rule'),
|
||||
String(rules), rules > 1 ? t('mappingScanModal.apply.rules') : t('mappingScanModal.apply.rule'),
|
||||
String(total), total > 1 ? t('mappingScanModal.apply.occurrences') : t('mappingScanModal.apply.occurrence'));
|
||||
this.applyBtn.toggleClass('pseudobs-dict-review-btn-empty', n === 0);
|
||||
this.applyBtn.toggleClass('pseudobs-dict-review-btn-empty', rules === 0);
|
||||
}
|
||||
|
||||
/** Enregistre les exceptions dans le mapping sans appliquer de remplacements. */
|
||||
private async saveExceptions(): Promise<void> {
|
||||
await this.persistIgnoredOccurrences();
|
||||
new Notice(t('mappingScanModal.exceptionsSaved'));
|
||||
this.close();
|
||||
}
|
||||
|
||||
/**
|
||||
* Persiste les occurrences ignorées (✗ et ⚠) dans le mapping.json de chaque règle.
|
||||
* S'appuie sur findRuleByTerm pour localiser le bon mapping file.
|
||||
* Les nouvelles exceptions sont fusionnées avec les existantes (déduplication par texte).
|
||||
*/
|
||||
private async persistIgnoredOccurrences(): Promise<void> {
|
||||
for (let i = 0; i < this.ruleResults.length; i++) {
|
||||
const { rule, occurrences } = this.ruleResults[i];
|
||||
const decisions = this.decisionsMap.get(rule.id)!;
|
||||
|
||||
const newIgnored: IgnoredOccurrence[] = occurrences
|
||||
.filter((occ) => {
|
||||
const d = decisions.get(occ.id) ?? 'validated';
|
||||
return d === 'ignored' || d === 'false_positive';
|
||||
})
|
||||
.map((occ) => ({ text: occ.text, contextBefore: occ.contextBefore, contextAfter: occ.contextAfter }));
|
||||
|
||||
if (newIgnored.length === 0) continue;
|
||||
|
||||
const location = await this.plugin.scopeResolver.findRuleByTerm(rule.source);
|
||||
if (!location) continue;
|
||||
|
||||
const existing = location.rule.ignoredOccurrences ?? [];
|
||||
const existingTexts = new Set(existing.map((o) => o.text));
|
||||
const merged = [...existing, ...newIgnored.filter((o) => !existingTexts.has(o.text))];
|
||||
|
||||
location.store.update(rule.id, { ignoredOccurrences: merged });
|
||||
await this.plugin.scopeResolver.saveStore(location.store, location.filePath);
|
||||
}
|
||||
void this.plugin.refresh();
|
||||
}
|
||||
|
||||
private async apply(): Promise<void> {
|
||||
const checkedRules = this.ruleResults
|
||||
.filter((_, i) => this.checked[i])
|
||||
.map((r) => r.rule);
|
||||
if (checkedRules.length === 0) { this.close(); return; }
|
||||
|
||||
this.applyBtn.setAttr('disabled', 'true');
|
||||
|
||||
const s = this.plugin.settings;
|
||||
const marker = s.useMarkerInExport
|
||||
? { open: s.markerOpen, close: s.markerClose }
|
||||
: undefined;
|
||||
const wrap = (r: string) => s.useMarkerInExport
|
||||
? `${s.markerOpen}${r}${s.markerClose}` : r;
|
||||
|
||||
// Collecter tous les spans (toutes règles cochées), résoudre les chevauchements,
|
||||
// appliquer de droite à gauche comme le moteur principal
|
||||
const allSpans = checkedRules.flatMap((rule) =>
|
||||
findSpansForRule(this.content, rule, {
|
||||
caseSensitive: s.caseSensitive,
|
||||
wholeWordOnly: s.wholeWordOnly,
|
||||
}).map((span) =>
|
||||
marker
|
||||
? { ...span, replacement: `${marker.open}${span.replacement}${marker.close}` }
|
||||
: span
|
||||
)
|
||||
);
|
||||
const spans: ReplacementSpan[] = [];
|
||||
for (let i = 0; i < this.ruleResults.length; i++) {
|
||||
if (!this.checked[i]) continue;
|
||||
const { rule, occurrences } = this.ruleResults[i];
|
||||
const decisions = this.decisionsMap.get(rule.id)!;
|
||||
|
||||
const resolved = resolveSpans(allSpans);
|
||||
if (resolved.length === 0) {
|
||||
for (const occ of occurrences) {
|
||||
if ((decisions.get(occ.id) ?? 'validated') !== 'validated') continue;
|
||||
spans.push({
|
||||
start: occ.start,
|
||||
end: occ.end,
|
||||
source: occ.text,
|
||||
replacement: wrap(rule.replacement),
|
||||
mappingId: rule.id,
|
||||
priority: rule.priority,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (spans.length === 0) {
|
||||
new Notice(t('notice.noOccurrences'));
|
||||
this.close();
|
||||
return;
|
||||
}
|
||||
|
||||
this.applyBtn.setAttr('disabled', 'true');
|
||||
|
||||
const resolved = resolveSpans(spans);
|
||||
const modified = applySpans(this.content, resolved);
|
||||
await this.app.vault.modify(this.file, modified);
|
||||
void this.plugin.refreshHighlightData();
|
||||
|
||||
// Persister les exceptions ignorées dans le mapping
|
||||
await this.persistIgnoredOccurrences();
|
||||
|
||||
const total = resolved.length;
|
||||
new Notice(t('notice.occurrencesPseudonymized',
|
||||
|
|
|
|||
163
src/ui/OccurrencesContextModal.ts
Normal file
163
src/ui/OccurrencesContextModal.ts
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
/**
|
||||
* Modale de contexte légère — affiche les candidats (occurrences) d'une règle
|
||||
* et retourne les décisions sans appliquer immédiatement.
|
||||
*
|
||||
* Utilisée depuis MappingScanReviewModal pour permettre une sélection fine
|
||||
* occurrence par occurrence (ex : valider "Juste" nom propre, ignorer "juste" adjectif).
|
||||
*/
|
||||
|
||||
import { App, Modal, Setting } from 'obsidian';
|
||||
import type { MappingRule, Occurrence } from '../types';
|
||||
|
||||
export type OccurrenceDecision = 'validated' | 'ignored' | 'false_positive';
|
||||
|
||||
interface CardRef {
|
||||
card: HTMLElement;
|
||||
buttons: Map<OccurrenceDecision, HTMLElement>;
|
||||
arrow: HTMLElement;
|
||||
resLine: HTMLElement;
|
||||
statusLabel: HTMLElement;
|
||||
}
|
||||
|
||||
export class OccurrencesContextModal extends Modal {
|
||||
private rule: MappingRule;
|
||||
private occurrences: Occurrence[];
|
||||
private decisions: Map<string, OccurrenceDecision>;
|
||||
private onConfirm: (decisions: Map<string, OccurrenceDecision>) => void;
|
||||
private cardRefs = new Map<string, CardRef>();
|
||||
|
||||
constructor(
|
||||
app: App,
|
||||
rule: MappingRule,
|
||||
occurrences: Occurrence[],
|
||||
existingDecisions: Map<string, OccurrenceDecision>,
|
||||
onConfirm: (decisions: Map<string, OccurrenceDecision>) => void
|
||||
) {
|
||||
super(app);
|
||||
this.rule = rule;
|
||||
this.occurrences = occurrences;
|
||||
this.decisions = new Map(existingDecisions);
|
||||
this.onConfirm = onConfirm;
|
||||
}
|
||||
|
||||
onOpen(): void {
|
||||
const { contentEl } = this;
|
||||
contentEl.addClass('pseudobs-ctx-modal');
|
||||
|
||||
contentEl.createEl('h3', {
|
||||
text: `${this.rule.source} → ${this.rule.replacement}`,
|
||||
cls: 'pseudobs-ctx-modal-title',
|
||||
});
|
||||
contentEl.createEl('p', {
|
||||
text: `${this.occurrences.length} occurrence${this.occurrences.length > 1 ? 's' : ''} — sélectionnez celles à remplacer.`,
|
||||
cls: 'pseudobs-view-hint',
|
||||
});
|
||||
|
||||
// Boutons globaux
|
||||
new Setting(contentEl)
|
||||
.addButton((b) =>
|
||||
b.setButtonText('Tout valider').onClick(() => {
|
||||
for (const occ of this.occurrences) this.decisions.set(occ.id, 'validated');
|
||||
this.updateAllCards();
|
||||
})
|
||||
)
|
||||
.addButton((b) =>
|
||||
b.setButtonText('Tout ignorer').onClick(() => {
|
||||
for (const occ of this.occurrences) this.decisions.set(occ.id, 'ignored');
|
||||
this.updateAllCards();
|
||||
})
|
||||
);
|
||||
|
||||
// Cartes
|
||||
const scroll = contentEl.createDiv('pseudobs-ctx-modal-scroll');
|
||||
for (const occ of this.occurrences) {
|
||||
this.buildCard(scroll, occ);
|
||||
}
|
||||
|
||||
contentEl.createEl('hr');
|
||||
|
||||
new Setting(contentEl)
|
||||
.addButton((b) =>
|
||||
b.setButtonText('Annuler').onClick(() => this.close())
|
||||
)
|
||||
.addButton((b) =>
|
||||
b.setButtonText('Confirmer la sélection').setCta().onClick(() => {
|
||||
this.onConfirm(new Map(this.decisions));
|
||||
this.close();
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
private buildCard(container: HTMLElement, occ: Occurrence): void {
|
||||
const card = container.createDiv('pseudobs-occ-card');
|
||||
|
||||
// Ligne source
|
||||
const srcLine = card.createDiv('pseudobs-occ-line');
|
||||
srcLine.createSpan({ text: occ.contextBefore, cls: 'pseudobs-ctx-side' });
|
||||
srcLine.createSpan({ text: occ.text, cls: 'pseudobs-occ-term' });
|
||||
srcLine.createSpan({ text: occ.contextAfter, cls: 'pseudobs-ctx-side' });
|
||||
|
||||
// Flèche + ligne résultat
|
||||
const arrow = card.createDiv('pseudobs-occ-arrow');
|
||||
arrow.setText('↓');
|
||||
|
||||
const resLine = card.createDiv('pseudobs-occ-line pseudobs-occ-result-line');
|
||||
resLine.createSpan({ text: occ.contextBefore, cls: 'pseudobs-ctx-side' });
|
||||
resLine.createSpan({ text: this.rule.replacement, cls: 'pseudobs-occ-replacement' });
|
||||
resLine.createSpan({ text: occ.contextAfter, cls: 'pseudobs-ctx-side' });
|
||||
|
||||
const statusLabel = card.createDiv('pseudobs-occ-status-label');
|
||||
card.createEl('small', { text: `ligne ${occ.line}`, cls: 'pseudobs-occ-meta' });
|
||||
|
||||
// Boutons décision
|
||||
const actions = card.createDiv('pseudobs-occ-actions');
|
||||
const btnRefs = new Map<OccurrenceDecision, HTMLElement>();
|
||||
|
||||
for (const [label, value, title] of [
|
||||
['✓', 'validated', 'Valider'],
|
||||
['✗', 'ignored', 'Ignorer'],
|
||||
['⚠', 'false_positive', 'Faux positif'],
|
||||
] as [string, OccurrenceDecision, string][]) {
|
||||
const btn = actions.createEl('button', { text: label });
|
||||
btn.title = title;
|
||||
btn.addClass('pseudobs-occ-btn');
|
||||
btn.addEventListener('click', () => {
|
||||
this.decisions.set(occ.id, value);
|
||||
this.updateCard(occ.id);
|
||||
});
|
||||
btnRefs.set(value, btn);
|
||||
}
|
||||
|
||||
this.cardRefs.set(occ.id, { card, buttons: btnRefs, arrow, resLine, statusLabel });
|
||||
this.updateCard(occ.id);
|
||||
}
|
||||
|
||||
private updateCard(occId: string): void {
|
||||
const ref = this.cardRefs.get(occId);
|
||||
if (!ref) return;
|
||||
const decision = this.decisions.get(occId) ?? 'validated';
|
||||
|
||||
ref.card.removeClass('pseudobs-occ-validated', 'pseudobs-occ-ignored', 'pseudobs-occ-false_positive');
|
||||
ref.card.addClass(`pseudobs-occ-${decision}`);
|
||||
|
||||
for (const [value, btn] of ref.buttons) {
|
||||
btn.toggleClass('pseudobs-occ-btn-active', value === decision);
|
||||
}
|
||||
|
||||
const show = decision === 'validated';
|
||||
ref.arrow.toggle(show);
|
||||
ref.resLine.toggle(show);
|
||||
ref.statusLabel.toggle(!show);
|
||||
ref.statusLabel.setText(
|
||||
decision === 'ignored' ? 'Conservé tel quel' : decision === 'false_positive' ? 'Faux positif — exclu' : ''
|
||||
);
|
||||
}
|
||||
|
||||
private updateAllCards(): void {
|
||||
for (const occId of this.cardRefs.keys()) this.updateCard(occId);
|
||||
}
|
||||
|
||||
onClose(): void {
|
||||
this.contentEl.empty();
|
||||
}
|
||||
}
|
||||
|
|
@ -238,7 +238,7 @@ export class OccurrencesModal extends Modal {
|
|||
const nv = validated.length, ni = ignored.length;
|
||||
new Notice(`✓ ${nv} remplacement${nv > 1 ? 's' : ''} appliqué${nv > 1 ? 's' : ''}` + (ni > 0 ? `, ${ni} ignoré${ni > 1 ? 's' : ''}` : ''));
|
||||
|
||||
void this.plugin.refreshHighlightData();
|
||||
void this.plugin.refresh();
|
||||
this.close();
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -6,9 +6,10 @@ import { Decoration, DecorationSet, EditorView, ViewPlugin, ViewUpdate } from '@
|
|||
export const highlightDataChanged = StateEffect.define<void>();
|
||||
|
||||
export interface HighlightData {
|
||||
sources: string[]; // termes originaux encore présents → orange (à pseudonymiser)
|
||||
replacements: string[]; // pseudonymes déjà appliqués → vert + souligné
|
||||
nerCandidates: string[]; // entités détectées par NER → bleu (candidats à pseudonymiser)
|
||||
sources: string[]; // termes originaux encore présents → orange
|
||||
replacements: string[]; // pseudonymes déjà appliqués → vert + souligné
|
||||
nerCandidates: string[]; // entités NER → bleu
|
||||
ignoredTerms: string[]; // textes exacts des occurrences ignorées → rouge (sensible à la casse)
|
||||
}
|
||||
|
||||
// Extension CodeMirror 6 qui surligne dans l'éditeur :
|
||||
|
|
@ -34,17 +35,20 @@ export function createPseudonymHighlighter(getData: () => HighlightData): Extens
|
|||
}
|
||||
|
||||
private build(view: EditorView): DecorationSet {
|
||||
const { sources, replacements, nerCandidates } = getData();
|
||||
if (sources.length === 0 && replacements.length === 0 && nerCandidates.length === 0)
|
||||
const { sources, replacements, nerCandidates, ignoredTerms } = getData();
|
||||
if (sources.length === 0 && replacements.length === 0
|
||||
&& nerCandidates.length === 0 && ignoredTerms.length === 0)
|
||||
return Decoration.none;
|
||||
|
||||
const text = view.state.doc.toString();
|
||||
const lower = text.toLowerCase();
|
||||
|
||||
type Span = { from: number; to: number; cls: string };
|
||||
// prio : 0 = plus haute priorité (gagne en cas de chevauchement)
|
||||
type Span = { from: number; to: number; cls: string; prio: number };
|
||||
const spans: Span[] = [];
|
||||
|
||||
const collect = (terms: string[], cls: string) => {
|
||||
// Matching insensible à la casse (sources, remplacements, NER)
|
||||
const collect = (terms: string[], cls: string, prio: number) => {
|
||||
for (const term of terms) {
|
||||
if (!term) continue;
|
||||
const needle = term.toLowerCase();
|
||||
|
|
@ -52,7 +56,21 @@ export function createPseudonymHighlighter(getData: () => HighlightData): Extens
|
|||
while (pos < lower.length) {
|
||||
const idx = lower.indexOf(needle, pos);
|
||||
if (idx === -1) break;
|
||||
spans.push({ from: idx, to: idx + term.length, cls });
|
||||
spans.push({ from: idx, to: idx + term.length, cls, prio });
|
||||
pos = idx + term.length;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Matching SENSIBLE à la casse (exceptions — "juste" ≠ "Juste")
|
||||
const collectExact = (terms: string[], cls: string, prio: number) => {
|
||||
for (const term of terms) {
|
||||
if (!term) continue;
|
||||
let pos = 0;
|
||||
while (pos < text.length) {
|
||||
const idx = text.indexOf(term, pos);
|
||||
if (idx === -1) break;
|
||||
spans.push({ from: idx, to: idx + term.length, cls, prio });
|
||||
pos = idx + term.length;
|
||||
}
|
||||
}
|
||||
|
|
@ -73,12 +91,14 @@ export function createPseudonymHighlighter(getData: () => HighlightData): Extens
|
|||
return !sourcesLower.some((src) => src !== cl && src.includes(cl));
|
||||
});
|
||||
|
||||
collect(freshCandidates, 'pseudobs-ner-candidate');
|
||||
collect(sources, 'pseudobs-source');
|
||||
collect(replacements, 'pseudobs-replaced');
|
||||
// Priorités explicites : 0 = gagne en cas de chevauchement
|
||||
collectExact(ignoredTerms, 'pseudobs-exception', 0);
|
||||
collect(replacements, 'pseudobs-replaced', 1);
|
||||
collect(sources, 'pseudobs-source', 2);
|
||||
collect(freshCandidates, 'pseudobs-ner-candidate', 3);
|
||||
|
||||
// Trier par position (RangeSetBuilder l'exige) et éliminer les chevauchements
|
||||
spans.sort((a, b) => a.from - b.from || a.to - b.to);
|
||||
// Trier par position, puis par priorité (0 = gagne) en cas d'égalité de position
|
||||
spans.sort((a, b) => a.from - b.from || a.prio - b.prio || a.to - b.to);
|
||||
|
||||
const builder = new RangeSetBuilder<Decoration>();
|
||||
let lastTo = -1;
|
||||
|
|
|
|||
|
|
@ -94,6 +94,11 @@ export class PseudonymizationView extends ItemView {
|
|||
await this.renderTab(tab);
|
||||
}
|
||||
|
||||
/** Appelé par le plugin pour forcer un re-rendu de l'onglet actif. */
|
||||
async refreshActiveTab(): Promise<void> {
|
||||
if (!this._renderingTab) await this.renderTab(this.activeTab);
|
||||
}
|
||||
|
||||
private async renderTab(tab: Tab): Promise<void> {
|
||||
const pane = this.panes[tab];
|
||||
pane.empty();
|
||||
|
|
@ -153,14 +158,19 @@ export class PseudonymizationView extends ItemView {
|
|||
caseSensitive: this.plugin.settings.caseSensitive,
|
||||
wholeWordOnly: this.plugin.settings.wholeWordOnly,
|
||||
});
|
||||
const countByRule = new Map<string, number>();
|
||||
const occsByRule = new Map<string, typeof occs>();
|
||||
for (const occ of occs) {
|
||||
const id = occ.mappingId ?? '';
|
||||
countByRule.set(id, (countByRule.get(id) ?? 0) + 1);
|
||||
if (!occsByRule.has(id)) occsByRule.set(id, []);
|
||||
occsByRule.get(id)!.push(occ);
|
||||
}
|
||||
const ruleResults: MappingRuleResult[] = rules
|
||||
.filter((r) => countByRule.has(r.id))
|
||||
.map((r) => ({ rule: r, matchCount: countByRule.get(r.id)! }));
|
||||
.filter((r) => occsByRule.has(r.id))
|
||||
.map((r) => ({
|
||||
rule: r,
|
||||
matchCount: occsByRule.get(r.id)!.length,
|
||||
occurrences: occsByRule.get(r.id)!,
|
||||
}));
|
||||
if (ruleResults.length === 0) { new Notice(t('notice.noOccurrences')); return; }
|
||||
new MappingScanReviewModal(this.app, this.plugin, file, content, ruleResults).open();
|
||||
} finally {
|
||||
|
|
@ -240,6 +250,46 @@ export class PseudonymizationView extends ItemView {
|
|||
editBtn.addEventListener('click', () => new EditRuleModal(this.app, this.plugin, loc).open());
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Section Exceptions ----
|
||||
const allIgnored = locations.flatMap(({ rule, store, filePath }) =>
|
||||
(rule.ignoredOccurrences ?? []).map((occ) => ({ occ, rule, store, filePath }))
|
||||
);
|
||||
|
||||
if (allIgnored.length > 0) {
|
||||
el.createEl('h3', { text: t('panel.mappings.exceptions'), cls: 'pseudobs-mappings-scope-heading' });
|
||||
el.createEl('p', { text: t('panel.mappings.exceptions.hint'), cls: 'pseudobs-view-hint' });
|
||||
|
||||
const exceptionsGrid = el.createDiv('pseudobs-exceptions-grid');
|
||||
|
||||
for (const { occ, rule, store, filePath } of allIgnored) {
|
||||
const card = exceptionsGrid.createDiv('pseudobs-exception-card');
|
||||
|
||||
// En-tête : règle concernée
|
||||
card.createEl('div', {
|
||||
text: `${rule.source} → ${rule.replacement}`,
|
||||
cls: 'pseudobs-exception-card-rule',
|
||||
});
|
||||
|
||||
// Contexte
|
||||
const ctx = card.createDiv('pseudobs-exception-card-ctx');
|
||||
ctx.createSpan({ text: occ.contextBefore, cls: 'pseudobs-ctx-side' });
|
||||
ctx.createSpan({ text: occ.text, cls: 'pseudobs-exception-card-term' });
|
||||
ctx.createSpan({ text: occ.contextAfter, cls: 'pseudobs-ctx-side' });
|
||||
|
||||
// Bouton supprimer
|
||||
const delBtn = card.createEl('button', { cls: 'pseudobs-exception-card-del' });
|
||||
setIcon(delBtn, 'x');
|
||||
delBtn.title = 'Supprimer cette exception';
|
||||
delBtn.addEventListener('click', async () => {
|
||||
const updated = (rule.ignoredOccurrences ?? []).filter((o) => o.text !== occ.text);
|
||||
store.update(rule.id, { ignoredOccurrences: updated });
|
||||
await this.plugin.scopeResolver.saveStore(store, filePath);
|
||||
void this.plugin.refresh();
|
||||
void this.renderTab('mappings');
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Onglet Dictionnaires --------------------------------------
|
||||
|
|
@ -386,10 +436,22 @@ export class PseudonymizationView extends ItemView {
|
|||
const s = this.plugin.settings;
|
||||
|
||||
const nerScanBtn = el.createEl('button', { cls: 'pseudobs-view-action-btn mod-cta' });
|
||||
setIcon(nerScanBtn, 'scan-search');
|
||||
nerScanBtn.createSpan({ text: t('panel.ner.scanBtn') });
|
||||
const nerScanIcon = nerScanBtn.createSpan();
|
||||
setIcon(nerScanIcon, 'scan-search');
|
||||
nerScanBtn.createSpan({ text: ` ${t('panel.ner.scanBtn')}` });
|
||||
nerScanBtn.title = t('panel.ner.scanBtn');
|
||||
nerScanBtn.addEventListener('click', () => void this.plugin.scanCurrentFileNer());
|
||||
nerScanBtn.addEventListener('click', () => { void (async () => {
|
||||
nerScanBtn.setAttr('disabled', 'true');
|
||||
setIcon(nerScanIcon, 'loader-circle');
|
||||
nerScanIcon.addClass('pseudobs-spin');
|
||||
try {
|
||||
await this.plugin.scanCurrentFileNer();
|
||||
} finally {
|
||||
nerScanBtn.removeAttribute('disabled');
|
||||
setIcon(nerScanIcon, 'scan-search');
|
||||
nerScanIcon.removeClass('pseudobs-spin');
|
||||
}
|
||||
})(); });
|
||||
|
||||
el.createEl('hr');
|
||||
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import type PseudObsPlugin from '../main';
|
|||
import { MappingStore } from '../mappings/MappingStore';
|
||||
import type { EntityCategory, MappingFile } from '../types';
|
||||
import { t } from '../i18n';
|
||||
import { generateRedaction } from '../pseudonymizer/Redaction';
|
||||
|
||||
type ApplyScope = 'occurrence' | 'file';
|
||||
|
||||
|
|
@ -18,19 +19,23 @@ export class QuickPseudonymizeModal extends Modal {
|
|||
private applyScope: ApplyScope = 'file';
|
||||
private suggestions: string[];
|
||||
|
||||
private isRedactionMode: boolean;
|
||||
|
||||
constructor(
|
||||
app: App,
|
||||
plugin: PseudObsPlugin,
|
||||
editor: Editor,
|
||||
prefillReplacement = '',
|
||||
suggestions: string[] = []
|
||||
suggestions: string[] = [],
|
||||
isRedactionMode = false,
|
||||
) {
|
||||
super(app);
|
||||
this.plugin = plugin;
|
||||
this.editor = editor;
|
||||
this.source = editor.getSelection();
|
||||
this.replacement = prefillReplacement;
|
||||
this.replacement = prefillReplacement || (isRedactionMode ? generateRedaction(this.source) : '');
|
||||
this.suggestions = suggestions;
|
||||
this.isRedactionMode = isRedactionMode;
|
||||
if (suggestions.length > 0) this.category = 'first_name';
|
||||
this.from = editor.getCursor('from');
|
||||
this.to = editor.getCursor('to');
|
||||
|
|
@ -48,7 +53,7 @@ export class QuickPseudonymizeModal extends Modal {
|
|||
tx.inputEl.addClass('pseudobs-disabled-input');
|
||||
});
|
||||
|
||||
let replacementInput: HTMLInputElement;
|
||||
let replacementInput: HTMLInputElement | undefined;
|
||||
|
||||
if (this.suggestions.length > 0) {
|
||||
const suggBox = contentEl.createDiv('pseudobs-suggestions-box');
|
||||
|
|
@ -98,6 +103,30 @@ export class QuickPseudonymizeModal extends Modal {
|
|||
d.onChange((v) => (this.applyScope = v as ApplyScope));
|
||||
});
|
||||
|
||||
// Checkbox caviardage
|
||||
const redactRow = contentEl.createDiv('pseudobs-redact-row');
|
||||
const redactCb = redactRow.createEl('input');
|
||||
redactCb.type = 'checkbox';
|
||||
redactCb.checked = this.isRedactionMode;
|
||||
redactCb.addClass('pseudobs-dict-review-cb');
|
||||
const redactLabel = redactRow.createSpan({ text: ` ${t('redaction.checkbox')}` });
|
||||
redactLabel.title = t('redaction.checkboxDesc');
|
||||
redactCb.addEventListener('change', () => {
|
||||
this.isRedactionMode = redactCb.checked;
|
||||
if (this.isRedactionMode) {
|
||||
this.replacement = generateRedaction(this.source);
|
||||
if (replacementInput) replacementInput.value = this.replacement;
|
||||
replacementInput?.setAttr('disabled', 'true');
|
||||
} else {
|
||||
this.replacement = '';
|
||||
if (replacementInput) replacementInput.value = '';
|
||||
replacementInput?.removeAttribute('disabled');
|
||||
}
|
||||
});
|
||||
if (this.isRedactionMode && replacementInput) {
|
||||
replacementInput.setAttr('disabled', 'true');
|
||||
}
|
||||
|
||||
new Setting(contentEl).addButton((btn) =>
|
||||
btn.setButtonText(t('quickModal.submit')).setCta().onClick(() => void this.apply())
|
||||
);
|
||||
|
|
@ -133,7 +162,7 @@ export class QuickPseudonymizeModal extends Modal {
|
|||
new Notice(t('notice.appliedFile', this.source, marked, String(count), count > 1 ? 's' : ''));
|
||||
}
|
||||
|
||||
void this.plugin.refreshHighlightData();
|
||||
void this.plugin.refresh();
|
||||
this.close();
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
import { App, Modal, Setting, TFile, Notice } from 'obsidian';
|
||||
import { t } from '../i18n';
|
||||
import { generateRedaction, REDACTION_CHAR } from '../pseudonymizer/Redaction';
|
||||
import type PseudObsPlugin from '../main';
|
||||
import { MappingStore } from '../mappings/MappingStore';
|
||||
import type { EntityCategory, MappingFile, ScopeType } from '../types';
|
||||
|
|
@ -211,6 +212,23 @@ export class RuleModal extends Modal {
|
|||
tx.setValue('0').onChange((v) => { this.priority = parseInt(v, 10) || 0; })
|
||||
);
|
||||
|
||||
// Checkbox caviardage
|
||||
const redactRow = contentEl.createDiv('pseudobs-redact-row');
|
||||
const redactCb = redactRow.createEl('input');
|
||||
redactCb.type = 'checkbox';
|
||||
redactCb.addClass('pseudobs-dict-review-cb');
|
||||
redactRow.createSpan({ text: ` ${t('redaction.checkbox')}` }).title = t('redaction.checkboxDesc');
|
||||
redactCb.addEventListener('change', () => {
|
||||
if (redactCb.checked) {
|
||||
this.replacement = generateRedaction(this.source || REDACTION_CHAR);
|
||||
this.useClass = false;
|
||||
if (replacementInput) { replacementInput.value = this.replacement; replacementInput.setAttr('disabled', 'true'); }
|
||||
} else {
|
||||
this.replacement = '';
|
||||
if (replacementInput) { replacementInput.value = ''; replacementInput.removeAttribute('disabled'); }
|
||||
}
|
||||
});
|
||||
|
||||
new Setting(contentEl).addButton((btn) =>
|
||||
btn.setButtonText(t('ruleModal.submit')).setCta().onClick(() => void this.createRule())
|
||||
);
|
||||
|
|
@ -283,7 +301,7 @@ export class RuleModal extends Modal {
|
|||
}
|
||||
|
||||
new Notice(t('notice.ruleCreated', this.source.trim(), this.replacement.trim()));
|
||||
void this.plugin.refreshHighlightData();
|
||||
void this.plugin.refresh();
|
||||
this.close();
|
||||
}
|
||||
|
||||
|
|
|
|||
77
styles.css
77
styles.css
|
|
@ -7,6 +7,14 @@
|
|||
outline: 1px solid rgba(80, 160, 255, 0.45);
|
||||
}
|
||||
|
||||
/* Occurrences explicitement ignorées — exceptions (sensibles à la casse) */
|
||||
.pseudobs-exception {
|
||||
background-color: rgba(220, 50, 50, 0.15);
|
||||
border-radius: 2px;
|
||||
outline: 1px solid rgba(220, 50, 50, 0.4);
|
||||
border-bottom: 2px solid rgba(220, 50, 50, 0.7);
|
||||
}
|
||||
|
||||
/* Termes sources encore présents dans le texte (à pseudonymiser) */
|
||||
.pseudobs-source {
|
||||
background-color: rgba(255, 160, 0, 0.22);
|
||||
|
|
@ -955,6 +963,7 @@
|
|||
}
|
||||
.pseudobs-onboarding-icon-btn-loading { cursor: wait; }
|
||||
.pseudobs-onboarding-icon-btn-loading svg { animation: pseudobs-spin 0.9s linear infinite; }
|
||||
.pseudobs-spin svg { animation: pseudobs-spin 0.9s linear infinite; }
|
||||
|
||||
/* ---- CorpusModal ---- */
|
||||
|
||||
|
|
@ -1048,3 +1057,71 @@
|
|||
font-size: 0.95em;
|
||||
line-height: 1.55;
|
||||
}
|
||||
|
||||
/* ---- Section Exceptions (onglet Mappings) ---- */
|
||||
.pseudobs-exceptions-grid {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
|
||||
.pseudobs-exception-card {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 4px;
|
||||
padding: 8px 10px;
|
||||
border-radius: 6px;
|
||||
border: 1px solid rgba(220, 50, 50, 0.35);
|
||||
background: rgba(220, 50, 50, 0.06);
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.pseudobs-exception-card-rule {
|
||||
font-size: 0.8em;
|
||||
color: var(--text-muted);
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.pseudobs-exception-card-ctx {
|
||||
font-size: 0.9em;
|
||||
}
|
||||
|
||||
.pseudobs-exception-card-term {
|
||||
background: rgba(220, 50, 50, 0.2);
|
||||
border-radius: 2px;
|
||||
outline: 1px solid rgba(220, 50, 50, 0.45);
|
||||
padding: 0 2px;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.pseudobs-exception-card-del {
|
||||
position: absolute;
|
||||
top: 6px;
|
||||
right: 6px;
|
||||
background: none;
|
||||
border: none;
|
||||
cursor: pointer;
|
||||
color: var(--text-muted);
|
||||
padding: 2px;
|
||||
border-radius: 3px;
|
||||
line-height: 1;
|
||||
}
|
||||
.pseudobs-exception-card-del:hover {
|
||||
color: var(--text-error);
|
||||
background: var(--background-modifier-error-hover);
|
||||
}
|
||||
|
||||
.pseudobs-save-exceptions-btn {
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
/* Ligne checkbox caviardage (QuickPseudonymizeModal, RuleModal) */
|
||||
.pseudobs-redact-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
margin: 6px 0 10px;
|
||||
font-size: 0.88em;
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
|
|
|||
298
tests/fixtures/fight_club.vtt
vendored
Normal file
298
tests/fixtures/fight_club.vtt
vendored
Normal file
|
|
@ -0,0 +1,298 @@
|
|||
WEBVTT fight_club
|
||||
|
||||
NOTE
|
||||
Transcrit avec noScribe vers. 0.7
|
||||
Fichier audio : /Users/axelleabbadie/SynologyDrive/Scientifique/Général/Tools/Pseudobsidianization/tests/fixtures/fight_club.mp3
|
||||
(Langue : French (fr) | Détection des locuteurs : 3 | Chevauchements de parole 1 | Horodatage : 1 | Mots de remplissage : 1 | Marquer les pauses : 1)
|
||||
|
||||
|
||||
NOTE media: /Users/axelleabbadie/SynologyDrive/Scientifique/Général/Tools/Pseudobsidianization/tests/fixtures/fight_club.mp3
|
||||
|
||||
1
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>S00:
|
||||
|
||||
2
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>[00:00:00]
|
||||
|
||||
3
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>Après un combat, tout le reste de la vie prenait beaucoup moins d'importance.
|
||||
|
||||
4
|
||||
00:00:04.160 --> 00:00:07.584
|
||||
<v S00>
|
||||
|
||||
5
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>S01:
|
||||
|
||||
6
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>[00:00:04]
|
||||
|
||||
7
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>
|
||||
|
||||
8
|
||||
00:00:04.160 --> 00:00:07.584
|
||||
<v S00>Qu'est-ce vous avez encore fichu hier soir ?
|
||||
|
||||
9
|
||||
00:00:07.584 --> 00:00:07.990
|
||||
<v S00>
|
||||
|
||||
10
|
||||
00:00:07.584 --> 00:00:07.990
|
||||
<v S00>S00:
|
||||
|
||||
11
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>[00:00:07]
|
||||
|
||||
12
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>
|
||||
|
||||
13
|
||||
00:00:07.584 --> 00:00:07.990
|
||||
<v S00>Quoi ?
|
||||
|
||||
14
|
||||
00:00:07.584 --> 00:00:07.990
|
||||
<v S00>S02:
|
||||
|
||||
15
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>[00:00:04]
|
||||
|
||||
16
|
||||
00:00:00.530 --> 00:00:04.160
|
||||
<v S00>
|
||||
|
||||
17
|
||||
00:00:07.990 --> 00:00:09.312
|
||||
<v S00>On pouvait endurer n'importe quoi.
|
||||
|
||||
18
|
||||
00:00:09.312 --> 00:00:16.416
|
||||
<v S00>
|
||||
|
||||
19
|
||||
00:00:09.312 --> 00:00:16.416
|
||||
<v S00>S01: [00:00:09] Vous avez fini ces rapports ?
|
||||
|
||||
20
|
||||
00:00:16.416 --> 00:00:18.080
|
||||
<v S01>S02:
|
||||
|
||||
21
|
||||
00:00:16.416 --> 00:00:18.080
|
||||
<v S01>[00:00:16]
|
||||
|
||||
22
|
||||
00:00:16.416 --> 00:00:18.080
|
||||
<v S01>Tu pouvais choisir ton adversaire, qui tu prendrais ?
|
||||
|
||||
23
|
||||
00:00:18.080 --> 00:00:19.808
|
||||
<v S01>(..)
|
||||
|
||||
24
|
||||
00:00:19.808 --> 00:00:20.960
|
||||
<v S01>
|
||||
|
||||
25
|
||||
00:00:19.808 --> 00:00:20.960
|
||||
<v S01>S00: Je prendrais sans doute mon patron.
|
||||
|
||||
26
|
||||
00:00:21.824 --> 00:00:22.208
|
||||
<v S01>
|
||||
|
||||
27
|
||||
00:00:21.824 --> 00:00:22.208
|
||||
<v S01>S02: Sérieux ?
|
||||
|
||||
28
|
||||
00:00:22.208 --> 00:00:23.936
|
||||
<v S01>(..)
|
||||
|
||||
29
|
||||
00:00:23.936 --> 00:00:24.992
|
||||
<v S00>S00:
|
||||
|
||||
30
|
||||
00:00:23.936 --> 00:00:24.992
|
||||
<v S00>[00:00:23]
|
||||
|
||||
31
|
||||
00:00:23.936 --> 00:00:24.992
|
||||
<v S00>Bah oui, pourquoi tu prendrais qui toi ?
|
||||
|
||||
32
|
||||
00:00:25.696 --> 00:00:26.336
|
||||
<v S00>
|
||||
|
||||
33
|
||||
00:00:25.696 --> 00:00:26.336
|
||||
<v S00>S02: Je prendrais mon père.
|
||||
|
||||
34
|
||||
00:00:26.336 --> 00:00:27.616
|
||||
<v S00>(.)
|
||||
|
||||
35
|
||||
00:00:27.616 --> 00:00:28.224
|
||||
<v S00>
|
||||
|
||||
36
|
||||
00:00:27.616 --> 00:00:28.224
|
||||
<v S00>S00: Je connais pas le mien.
|
||||
|
||||
37
|
||||
00:00:28.224 --> 00:00:29.408
|
||||
<v S00>(.)
|
||||
|
||||
38
|
||||
00:00:29.408 --> 00:00:32.832
|
||||
<v S00>Si, je le connais, mais il est parti quand je devais avoir 6 ans.
|
||||
|
||||
39
|
||||
00:00:33.408 --> 00:00:34.816
|
||||
<v S00>Il s'est remarié, il a eu d'autres gamins.
|
||||
|
||||
40
|
||||
00:00:34.816 --> 00:00:36.160
|
||||
<v S00>(.)
|
||||
|
||||
41
|
||||
00:00:36.160 --> 00:00:36.960
|
||||
<v S00>Il fait ça tous les 6 ans.
|
||||
|
||||
42
|
||||
00:00:36.960 --> 00:00:38.016
|
||||
<v S00>(.)
|
||||
|
||||
43
|
||||
00:00:38.016 --> 00:00:39.740
|
||||
<v S00>Il va dans une autre ville et il fonde une nouvelle famille.
|
||||
|
||||
44
|
||||
00:00:39.960 --> 00:00:41.340
|
||||
<v S01>S01:
|
||||
|
||||
45
|
||||
00:00:39.960 --> 00:00:41.340
|
||||
<v S01>[00:00:39]
|
||||
|
||||
46
|
||||
00:00:39.960 --> 00:00:41.340
|
||||
<v S01>Ce salaud, il monte des franchises.
|
||||
|
||||
47
|
||||
00:00:41.340 --> 00:00:44.288
|
||||
<v S01>(...)
|
||||
|
||||
48
|
||||
00:00:44.288 --> 00:00:45.520
|
||||
<v S01>Mon père, il est jamais allé en fac.
|
||||
|
||||
49
|
||||
00:00:45.900 --> 00:00:47.552
|
||||
<v S01>Alors pour lui, c'était important que j'y aille.
|
||||
|
||||
50
|
||||
00:00:48.320 --> 00:00:48.990
|
||||
<v S01>
|
||||
|
||||
51
|
||||
00:00:48.320 --> 00:00:48.990
|
||||
<v S01>S00: J'connais ce genre d'histoire.
|
||||
|
||||
52
|
||||
00:00:48.990 --> 00:00:51.880
|
||||
<v S01>
|
||||
|
||||
53
|
||||
00:00:48.990 --> 00:00:51.880
|
||||
<v S01>S01: Alors je passe ma licence, je l'appelle à l'autre bout du pays.
|
||||
|
||||
54
|
||||
00:00:52.000 --> 00:00:52.992
|
||||
<v S01>Je lui dis, papa, qu'est-ce que je fais ?
|
||||
|
||||
55
|
||||
00:00:53.824 --> 00:00:54.650
|
||||
<v S01>Il me dit, trouve-toi un boulot.
|
||||
|
||||
56
|
||||
00:00:54.950 --> 00:00:55.750
|
||||
<v S01>
|
||||
|
||||
57
|
||||
00:00:54.950 --> 00:00:55.750
|
||||
<v S01>S00: J'ai connu ça aussi.
|
||||
|
||||
58
|
||||
00:00:55.990 --> 00:00:58.130
|
||||
<v S01>
|
||||
|
||||
59
|
||||
00:00:55.990 --> 00:00:58.130
|
||||
<v S01>S01: Ensuite, j'ai 25 ans, je lui passe mon coup de fil annuel.
|
||||
|
||||
60
|
||||
00:00:58.310 --> 00:01:00.670
|
||||
<v S01>Je lui dis, papa, qu'est-ce que je fais ?
|
||||
|
||||
61
|
||||
00:01:00.670 --> 00:01:02.770
|
||||
<v S01>Il dit, je sais pas, tiens, marie-toi.
|
||||
|
||||
62
|
||||
00:01:03.290 --> 00:01:04.192
|
||||
<v S00>S00:
|
||||
|
||||
63
|
||||
00:01:03.290 --> 00:01:04.192
|
||||
<v S00>[00:01:03]
|
||||
|
||||
64
|
||||
00:01:03.290 --> 00:01:04.192
|
||||
<v S00>Ouais, pareil pour moi.
|
||||
|
||||
65
|
||||
00:01:04.192 --> 00:01:05.440
|
||||
<v S00>(.)
|
||||
|
||||
66
|
||||
00:01:05.440 --> 00:01:08.928
|
||||
<v S00>On peut pas se marier, je suis un gamin de 30 ans.
|
||||
|
||||
67
|
||||
00:01:08.928 --> 00:01:10.176
|
||||
<v S00>(.)
|
||||
|
||||
68
|
||||
00:01:10.176 --> 00:01:12.032
|
||||
<v S01>S01:
|
||||
|
||||
69
|
||||
00:01:10.176 --> 00:01:12.032
|
||||
<v S01>[00:01:10]
|
||||
|
||||
70
|
||||
00:01:10.176 --> 00:01:12.032
|
||||
<v S01>On a une génération d'hommes élevés par des femmes.
|
||||
|
||||
71
|
||||
00:01:12.032 --> 00:01:13.376
|
||||
<v S01>(.)
|
||||
|
||||
72
|
||||
00:01:13.376 --> 00:01:15.840
|
||||
<v S01>J'suis pas sûr qu'une autre femme soit la solution à nos problèmes.
|
||||
|
||||
13
tests/fixtures/juste-leblanc.html
vendored
Normal file
13
tests/fixtures/juste-leblanc.html
vendored
Normal file
File diff suppressed because one or more lines are too long
BIN
tests/fixtures/juste-leblanc.mp3
vendored
Normal file
BIN
tests/fixtures/juste-leblanc.mp3
vendored
Normal file
Binary file not shown.
104
tests/unit/NoScribeHtmlParser.test.ts
Normal file
104
tests/unit/NoScribeHtmlParser.test.ts
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
import { NoScribeHtmlParser } from '../../src/parsers/NoScribeHtmlParser';
|
||||
|
||||
const parser = new NoScribeHtmlParser();
|
||||
|
||||
// ---- Fixture ----------------------------------------------------------------
|
||||
// Structure calquée sur une vraie sortie noScribe (Qt Rich Text HTML).
|
||||
// Les timestamps sont en millisecondes depuis le début de l'audio.
|
||||
|
||||
const NOSCRIBE_HTML = `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
|
||||
<html><head><meta name="qrichtext" content="1" />
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="audio_source" content="/Corpus/entretien_01.m4a" />
|
||||
</head><body>
|
||||
<div class="WordSection1">
|
||||
|
||||
<p><span style="font-weight: 600">entretien_01</span></p>
|
||||
|
||||
<p><span style="font-size: 0.8em; color: #909090">Transcrit avec noScribe vers. 0.5<br />
|
||||
(Détection des locuteurs : 2)</span></p>
|
||||
|
||||
<p><a name="ts_0_2130_" ><span style="color: #000000">(..) </span></a></p>
|
||||
|
||||
<p><a name="ts_2130_7730_S00" ><span style="color: #000000">Expérimentateur </span></a><a name="ts_2130_7730_S00" ><span style="color: #78909c">[00:00:02]</span></a><a name="ts_2130_7730_S00" ><span style="color: #000000">: Ok. D'abord, merci d'avoir accepté.</span></a><a name="ts_8330_9730_S00" ><span style="color: #000000"> On s'était déjà parlé.</span></a><a name="ts_9730_12230_S00" ><span style="color: #000000"> (..)</span></a></p>
|
||||
|
||||
<p><a name="ts_17900_18660_S01" ><span style="color: #000000">SC01 : Si je me présente ?</span></a></p>
|
||||
|
||||
<p><a name="ts_1668980_1673180_S00" ><span style="color: #000000">Expérimentateur : si jamais demain j'arrive à être en contact avec quelqu'un.</span></a></p>
|
||||
|
||||
</div>
|
||||
</body></html>`;
|
||||
|
||||
// ---- Tests ------------------------------------------------------------------
|
||||
|
||||
describe('NoScribeHtmlParser — isNoScribeHtml', () => {
|
||||
test('reconnaît un HTML noScribe', () => {
|
||||
expect(NoScribeHtmlParser.isNoScribeHtml(NOSCRIBE_HTML)).toBe(true);
|
||||
});
|
||||
|
||||
test('rejette un HTML ordinaire', () => {
|
||||
expect(NoScribeHtmlParser.isNoScribeHtml('<html><body><p>Bonjour</p></body></html>')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('NoScribeHtmlParser — parse', () => {
|
||||
test('ignore les paragraphes sans ancre ts_', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
// Les 2 paragraphes d'en-tête (titre + métadonnées) ne doivent pas produire de cue
|
||||
// Les cues sont : pause, tour S00, tour S01, tour S00 long = 4 cues
|
||||
expect(doc.cues.length).toBe(4);
|
||||
});
|
||||
|
||||
test('extrait la pause sans locuteur', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const pause = doc.cues[0];
|
||||
expect(pause.speaker).toBeUndefined();
|
||||
expect(pause.text).toContain('..');
|
||||
expect(pause.startTime).toBe('00:00:00.000');
|
||||
expect(pause.endTime).toBe('00:00:02.130');
|
||||
});
|
||||
|
||||
test('extrait le locuteur depuis "Nom : texte" multi-ancres', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const turn = doc.cues[1];
|
||||
expect(turn.speaker).toBe('Expérimentateur');
|
||||
expect(turn.text).not.toContain('Expérimentateur');
|
||||
expect(turn.text).toContain('Ok. D\'abord');
|
||||
});
|
||||
|
||||
test('extrait le locuteur depuis "Nom : texte" mono-ancre', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const turn = doc.cues[2];
|
||||
expect(turn.speaker).toBe('SC01');
|
||||
expect(turn.text).toContain('Si je me présente');
|
||||
expect(turn.text).not.toContain('SC01');
|
||||
});
|
||||
|
||||
test('convertit les timestamps en HH:MM:SS.mmm', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const turn = doc.cues[1];
|
||||
expect(turn.startTime).toBe('00:00:02.130');
|
||||
expect(turn.endTime).toBe('00:00:12.230');
|
||||
});
|
||||
|
||||
test('timestamp long (> 27 min) correctement converti', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const turn = doc.cues[3];
|
||||
// 1668980ms = 27min 48.980s
|
||||
expect(turn.startTime).toBe('00:27:48.980');
|
||||
expect(turn.endTime).toBe('00:27:53.180');
|
||||
});
|
||||
|
||||
test('word timestamps produits pour chaque ancre distincte', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const turn = doc.cues[1]; // Expérimentateur avec 3 ancres ts_ distinctes
|
||||
expect(turn.words.length).toBeGreaterThanOrEqual(2);
|
||||
expect(turn.words[0].time).toBe('00:00:02.130');
|
||||
});
|
||||
|
||||
test('ignore les timestamps d\'affichage [HH:MM:SS]', () => {
|
||||
const doc = parser.parse(NOSCRIBE_HTML);
|
||||
const turn = doc.cues[1];
|
||||
expect(turn.text).not.toMatch(/\[\d{2}:\d{2}:\d{2}\]/);
|
||||
});
|
||||
});
|
||||
138
tests/unit/NoScribeVttParser.test.ts
Normal file
138
tests/unit/NoScribeVttParser.test.ts
Normal file
|
|
@ -0,0 +1,138 @@
|
|||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import { NoScribeVttParser } from '../../src/parsers/NoScribeVttParser';
|
||||
import { NoScribeHtmlParser } from '../../src/parsers/NoScribeHtmlParser';
|
||||
import { noScribeHtmlToMarkdown } from '../../src/parsers/TranscriptConverter';
|
||||
|
||||
const VTT_FIXTURE = path.join(__dirname, '../fixtures/fight_club.vtt');
|
||||
const HTML_FIXTURE = path.join(__dirname, '../fixtures/juste-leblanc.html');
|
||||
|
||||
// ---- Détection --------------------------------------------------------------
|
||||
|
||||
describe('NoScribeVttParser — détection', () => {
|
||||
test('reconnaît un VTT noScribe', () => {
|
||||
const content = fs.readFileSync(VTT_FIXTURE, 'utf-8');
|
||||
expect(NoScribeVttParser.isNoScribeVtt(content)).toBe(true);
|
||||
});
|
||||
|
||||
test('rejette un VTT standard', () => {
|
||||
expect(NoScribeVttParser.isNoScribeVtt('WEBVTT\n\n1\n00:00:01.000 --> 00:00:02.000\nBonjour\n')).toBe(false);
|
||||
});
|
||||
|
||||
test('extrait le chemin audio depuis NOTE media', () => {
|
||||
const content = fs.readFileSync(VTT_FIXTURE, 'utf-8');
|
||||
const src = NoScribeVttParser.extractAudioSource(content);
|
||||
expect(src).toMatch(/fight_club\.mp3$/);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- Parsing VTT fight_club -------------------------------------------------
|
||||
|
||||
describe('NoScribeVttParser — fight_club.vtt', () => {
|
||||
let doc: ReturnType<NoScribeVttParser['parse']>;
|
||||
|
||||
beforeAll(() => {
|
||||
const content = fs.readFileSync(VTT_FIXTURE, 'utf-8');
|
||||
doc = new NoScribeVttParser().parse(content);
|
||||
});
|
||||
|
||||
test('produit au moins 5 cues', () => {
|
||||
expect(doc.cues.length).toBeGreaterThanOrEqual(5);
|
||||
});
|
||||
|
||||
test('détecte les locuteurs S00, S01, S02', () => {
|
||||
const speakers = new Set(doc.cues.map((c) => c.speaker).filter(Boolean));
|
||||
expect(speakers.has('S00')).toBe(true);
|
||||
expect(speakers.has('S01')).toBe(true);
|
||||
expect(speakers.has('S02')).toBe(true);
|
||||
});
|
||||
|
||||
test('premier cue : S00, timestamp correct, texte non vide', () => {
|
||||
const first = doc.cues[0];
|
||||
expect(first.speaker).toBe('S00');
|
||||
expect(first.startTime).toBe('00:00:00.530');
|
||||
expect(first.text).toContain('combat');
|
||||
});
|
||||
|
||||
test('les timestamps d\'affichage [HH:MM:SS] sont absents du texte', () => {
|
||||
for (const cue of doc.cues) {
|
||||
expect(cue.text).not.toMatch(/\[\d{2}:\d{2}:\d{2}\]/);
|
||||
}
|
||||
});
|
||||
|
||||
test('les entités HTML sont décodées', () => {
|
||||
const allText = doc.cues.map((c) => c.text).join(' ');
|
||||
expect(allText).not.toContain(''');
|
||||
expect(allText).toContain("'"); // apostrophes décodées
|
||||
});
|
||||
|
||||
test('les labels SXX: sont absents du texte', () => {
|
||||
for (const cue of doc.cues) {
|
||||
expect(cue.text).not.toMatch(/^S\d+\s*:/);
|
||||
}
|
||||
});
|
||||
|
||||
test('chaque cue a un mot pour le timestamps word-level', () => {
|
||||
for (const cue of doc.cues) {
|
||||
expect(cue.words.length).toBeGreaterThan(0);
|
||||
expect(cue.words[0].time).toBe(cue.startTime);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// ---- Parsing HTML juste-leblanc --------------------------------------------
|
||||
|
||||
describe('NoScribeHtmlParser — juste-leblanc.html', () => {
|
||||
let doc: ReturnType<NoScribeHtmlParser['parse']>;
|
||||
|
||||
beforeAll(() => {
|
||||
const content = fs.readFileSync(HTML_FIXTURE, 'utf-8');
|
||||
doc = new NoScribeHtmlParser().parse(content);
|
||||
});
|
||||
|
||||
test('produit au moins 8 cues', () => {
|
||||
expect(doc.cues.length).toBeGreaterThanOrEqual(8);
|
||||
});
|
||||
|
||||
test('détecte S00 et S01', () => {
|
||||
const speakers = new Set(doc.cues.map((c) => c.speaker).filter(Boolean));
|
||||
expect(speakers.has('S00')).toBe(true);
|
||||
expect(speakers.has('S01')).toBe(true);
|
||||
});
|
||||
|
||||
test('premier cue : S00, contient "Juste Leblanc"', () => {
|
||||
const first = doc.cues.find((c) => c.speaker === 'S00');
|
||||
expect(first?.text).toContain('Juste Leblanc');
|
||||
});
|
||||
|
||||
test('les labels SXX: sont absents du texte', () => {
|
||||
for (const cue of doc.cues) {
|
||||
expect(cue.text).not.toMatch(/^S\d+\s*:/);
|
||||
}
|
||||
});
|
||||
|
||||
test('les timestamps [HH:MM:SS] sont absents du texte', () => {
|
||||
for (const cue of doc.cues) {
|
||||
expect(cue.text).not.toMatch(/\[\d{2}:\d{2}:\d{2}\]/);
|
||||
}
|
||||
});
|
||||
|
||||
test('audio source extrait correctement', () => {
|
||||
const content = fs.readFileSync(HTML_FIXTURE, 'utf-8');
|
||||
expect(NoScribeHtmlParser.extractAudioSource(content)).toMatch(/juste-leblanc\.mp3$/);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- Markdown produit -------------------------------------------------------
|
||||
|
||||
describe('noScribeHtmlToMarkdown — juste-leblanc', () => {
|
||||
test('format clean : pas de commentaires HTML', () => {
|
||||
const content = fs.readFileSync(HTML_FIXTURE, 'utf-8');
|
||||
const doc = new NoScribeHtmlParser().parse(content);
|
||||
const md = noScribeHtmlToMarkdown(doc, 'juste-leblanc.html');
|
||||
expect(md).toContain('pseudobs-format: html');
|
||||
expect(md).not.toContain('<!--');
|
||||
expect(md).toContain('**S00**');
|
||||
expect(md).toContain('**S01**');
|
||||
});
|
||||
});
|
||||
|
|
@ -2,7 +2,9 @@ import * as fs from 'fs';
|
|||
import * as path from 'path';
|
||||
import { SrtParser } from '../../src/parsers/SrtParser';
|
||||
import { ChatParser } from '../../src/parsers/ChatParser';
|
||||
import { srtToMarkdown, chatToMarkdown } from '../../src/parsers/TranscriptConverter';
|
||||
import { VttParser } from '../../src/parsers/VttParser';
|
||||
import { NoScribeHtmlParser } from '../../src/parsers/NoScribeHtmlParser';
|
||||
import { srtToMarkdown, chatToMarkdown, vttToMarkdown, noScribeHtmlToMarkdown, extractWordData, markdownToVtt } from '../../src/parsers/TranscriptConverter';
|
||||
|
||||
const SRT_FIXTURE = path.join(__dirname, '../fixtures/entretien_01.srt');
|
||||
const CHA_FIXTURE = path.join(__dirname, '../fixtures/entretien_02.cha');
|
||||
|
|
@ -91,3 +93,185 @@ describe('chatToMarkdown', () => {
|
|||
expect(md.endsWith('\n')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ---- vttToMarkdown / noScribeHtmlToMarkdown --------------------------------
|
||||
|
||||
const VTT_FIXTURE = `WEBVTT
|
||||
|
||||
1
|
||||
00:00:01.240 --> 00:00:05.800
|
||||
<v Expérimentateur>Bonjour, je m'appelle Marie.
|
||||
|
||||
2
|
||||
00:00:06.100 --> 00:00:09.300
|
||||
<v SC01>Si je me présente ?
|
||||
|
||||
3
|
||||
00:00:09.800 --> 00:00:11.200
|
||||
(..)
|
||||
`;
|
||||
|
||||
const NOSCRIBE_HTML_FIXTURE = `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
|
||||
<html><head><meta name="qrichtext" content="1" /><meta charset="UTF-8" /></head><body>
|
||||
<div class="WordSection1">
|
||||
<p><a name="ts_0_2130_" ><span>(..) </span></a></p>
|
||||
<p><a name="ts_2130_5800_S00" ><span>Expérimentateur </span></a><a name="ts_2130_5800_S00" ><span style="color: #78909c">[00:00:02]</span></a><a name="ts_2130_5800_S00" ><span>: Bonjour, je m'appelle Marie.</span></a></p>
|
||||
<p><a name="ts_6100_9300_S01" ><span>SC01 : Si je me présente ?</span></a></p>
|
||||
</div></body></html>`;
|
||||
|
||||
describe('vttToMarkdown — format noScribe', () => {
|
||||
let md: string;
|
||||
beforeAll(() => { md = vttToMarkdown(new VttParser().parse(VTT_FIXTURE), 'entretien.vtt'); });
|
||||
|
||||
it('frontmatter pseudobs-format: vtt', () => {
|
||||
expect(md).toContain('pseudobs-format: vtt');
|
||||
});
|
||||
|
||||
it('locuteur en gras suivi du timestamp [HH:MM:SS]', () => {
|
||||
expect(md).toContain('**Expérimentateur** [00:00:01]');
|
||||
expect(md).toContain('**SC01** [00:00:06]');
|
||||
});
|
||||
|
||||
it('texte précédé de " : " quand il y a un locuteur', () => {
|
||||
expect(md).toContain('**Expérimentateur** [00:00:01] : Bonjour');
|
||||
});
|
||||
|
||||
it('cue sans locuteur : timestamp seul', () => {
|
||||
expect(md).toContain('[00:00:09] (..)');
|
||||
expect(md).not.toMatch(/\*\*\*\* \[/);
|
||||
});
|
||||
|
||||
it('aucun commentaire HTML dans le corps', () => {
|
||||
const body = md.split('---').slice(2).join('---');
|
||||
expect(body).not.toContain('<!--');
|
||||
});
|
||||
|
||||
it('termine par un saut de ligne', () => {
|
||||
expect(md.endsWith('\n')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('noScribeHtmlToMarkdown — format html', () => {
|
||||
let md: string;
|
||||
beforeAll(() => {
|
||||
const doc = new NoScribeHtmlParser().parse(NOSCRIBE_HTML_FIXTURE);
|
||||
md = noScribeHtmlToMarkdown(doc, 'entretien.html');
|
||||
});
|
||||
|
||||
it('frontmatter pseudobs-format: html', () => {
|
||||
expect(md).toContain('pseudobs-format: html');
|
||||
expect(md).not.toContain('pseudobs-format: vtt');
|
||||
});
|
||||
|
||||
it('locuteur en gras avec timestamp [HH:MM:SS]', () => {
|
||||
expect(md).toContain('**Expérimentateur** [00:00:02]');
|
||||
expect(md).toContain('**SC01** [00:00:06]');
|
||||
});
|
||||
|
||||
it('texte présent après " : "', () => {
|
||||
expect(md).toContain(': Bonjour, je m\'appelle Marie.');
|
||||
expect(md).toContain(': Si je me présente ?');
|
||||
});
|
||||
|
||||
it('aucun commentaire HTML dans le corps', () => {
|
||||
const body = md.split('---').slice(2).join('---');
|
||||
expect(body).not.toContain('<!--');
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractWordData', () => {
|
||||
const VTT_WITH_WORDS = `WEBVTT
|
||||
|
||||
1
|
||||
00:00:01.240 --> 00:00:05.800
|
||||
<v Expérimentateur><00:00:01.240><c> Bonjour,</c><00:00:02.100><c> Marie.</c>
|
||||
|
||||
2
|
||||
00:00:06.100 --> 00:00:09.300
|
||||
<v SC01>Si je me présente ?
|
||||
`;
|
||||
|
||||
it('extrait les cues avec word timestamps', () => {
|
||||
const doc = new VttParser().parse(VTT_WITH_WORDS);
|
||||
const data = extractWordData(doc);
|
||||
expect(data).toHaveLength(1); // cue 2 sans word timestamps exclue
|
||||
expect(data[0].index).toBe(0);
|
||||
expect(data[0].speaker).toBe('Expérimentateur');
|
||||
expect(data[0].startTime).toBe('00:00:01.240');
|
||||
expect(data[0].endTime).toBe('00:00:05.800');
|
||||
expect(data[0].words.some((w) => w.time !== '')).toBe(true);
|
||||
});
|
||||
|
||||
it('exclut les cues sans word timestamps', () => {
|
||||
const doc = new VttParser().parse(VTT_WITH_WORDS);
|
||||
const data = extractWordData(doc);
|
||||
expect(data.every((c) => c.words.some((w) => w.time !== ''))).toBe(true);
|
||||
});
|
||||
|
||||
it('produit un JSON sérialisable', () => {
|
||||
const doc = new VttParser().parse(VTT_WITH_WORDS);
|
||||
const data = extractWordData(doc);
|
||||
expect(() => JSON.stringify(data)).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
// ---- markdownToVtt ----------------------------------------------------------
|
||||
|
||||
describe('markdownToVtt — re-export VTT', () => {
|
||||
const WORDS_DATA = [
|
||||
{ index: 0, startTime: '00:01:28.080', endTime: '00:01:33.536', speaker: 'S00',
|
||||
words: [{ text: 'original', time: '00:01:28.080' }] },
|
||||
{ index: 1, startTime: '00:01:32.672', endTime: '00:01:34.816', speaker: 'S01',
|
||||
words: [{ text: 'original', time: '00:01:32.672' }] },
|
||||
{ index: 2, startTime: '00:01:34.816', endTime: '00:01:45.920', speaker: 'S00',
|
||||
words: [{ text: 'original', time: '00:01:34.816' }] },
|
||||
];
|
||||
|
||||
const MD_PSEUDONYMIZED = `---
|
||||
pseudobs-format: html
|
||||
pseudobs-source: "juste-leblanc.html"
|
||||
---
|
||||
|
||||
**S00** [00:01:28] : Je vais le faire moi-même. Il s'appelle Pierre Martin.
|
||||
|
||||
**S01** [00:01:32] : Ah bon, il n'a pas de prénom ?
|
||||
|
||||
**S00** [00:01:34] : Je viens de vous le dire, Pierre Martin.
|
||||
`;
|
||||
|
||||
it('produit un WebVTT valide', () => {
|
||||
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
|
||||
expect(vtt).toMatch(/^WEBVTT/);
|
||||
expect(vtt).toContain('00:01:28.080 --> 00:01:33.536');
|
||||
expect(vtt).toContain('00:01:32.672 --> 00:01:34.816');
|
||||
});
|
||||
|
||||
it('intègre le texte pseudonymisé', () => {
|
||||
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
|
||||
expect(vtt).toContain('Pierre Martin');
|
||||
expect(vtt).not.toContain('Juste Leblanc');
|
||||
});
|
||||
|
||||
it('préserve les locuteurs', () => {
|
||||
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
|
||||
expect(vtt).toContain('<v S00>');
|
||||
expect(vtt).toContain('<v S01>');
|
||||
});
|
||||
|
||||
it('utilise les timestamps du words.json, pas du Markdown', () => {
|
||||
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
|
||||
// Les ms précises viennent du words.json
|
||||
expect(vtt).toContain('00:01:28.080');
|
||||
expect(vtt).toContain('00:01:33.536');
|
||||
});
|
||||
|
||||
it('signale une incohérence si le nombre de cues diffère', () => {
|
||||
const { mismatch } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA.slice(0, 2));
|
||||
expect(mismatch).toBe(true);
|
||||
});
|
||||
|
||||
it('pas de mismatch si les comptes sont égaux', () => {
|
||||
const { mismatch } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
|
||||
expect(mismatch).toBe(false);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
124
tests/unit/VttParser.test.ts
Normal file
124
tests/unit/VttParser.test.ts
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
import { VttParser } from '../../src/parsers/VttParser';
|
||||
|
||||
const parser = new VttParser();
|
||||
|
||||
// ---- Helpers ----------------------------------------------------------------
|
||||
|
||||
function roundTrip(input: string): string {
|
||||
return parser.reconstruct(parser.parse(input));
|
||||
}
|
||||
|
||||
// ---- Fixtures ---------------------------------------------------------------
|
||||
|
||||
const VTT_SIMPLE = `WEBVTT
|
||||
|
||||
1
|
||||
00:00:01.240 --> 00:00:05.800
|
||||
Bonjour, je m'appelle Marie Dupont.
|
||||
|
||||
2
|
||||
00:00:06.100 --> 00:00:09.300
|
||||
Et tu habites à Lyon ?
|
||||
`;
|
||||
|
||||
const VTT_SPEAKER = `WEBVTT
|
||||
|
||||
1
|
||||
00:00:01.240 --> 00:00:05.800
|
||||
<v SPEAKER_1>Bonjour, je m'appelle Marie Dupont.
|
||||
|
||||
2
|
||||
00:00:06.100 --> 00:00:09.300
|
||||
<v SPEAKER_2>Et tu habites à Lyon ?
|
||||
`;
|
||||
|
||||
const VTT_WORD_TIMESTAMPS = `WEBVTT
|
||||
|
||||
1
|
||||
00:00:01.240 --> 00:00:05.800
|
||||
<v SPEAKER_1><00:00:01.240><c> Bonjour,</c><00:00:02.100><c> je</c><00:00:02.300><c> m'appelle</c><00:00:03.400><c> Marie</c><00:00:04.200><c> Dupont.</c>
|
||||
|
||||
2
|
||||
00:00:06.100 --> 00:00:09.300
|
||||
<v SPEAKER_2><00:00:06.100><c> Et</c><00:00:06.500><c> tu</c><00:00:06.800><c> habites</c><00:00:07.200><c> à</c><00:00:07.800><c> Lyon</c><00:00:08.200><c> ?</c>
|
||||
`;
|
||||
|
||||
// ---- Tests ------------------------------------------------------------------
|
||||
|
||||
describe('VttParser — parse', () => {
|
||||
test('VTT simple : 2 cues sans speaker', () => {
|
||||
const doc = parser.parse(VTT_SIMPLE);
|
||||
expect(doc.cues).toHaveLength(2);
|
||||
expect(doc.cues[0].startTime).toBe('00:00:01.240');
|
||||
expect(doc.cues[0].endTime).toBe('00:00:05.800');
|
||||
expect(doc.cues[0].text).toContain('Marie Dupont');
|
||||
expect(doc.cues[0].speaker).toBeUndefined();
|
||||
});
|
||||
|
||||
test('VTT avec speakers : extraction correcte', () => {
|
||||
const doc = parser.parse(VTT_SPEAKER);
|
||||
expect(doc.cues[0].speaker).toBe('SPEAKER_1');
|
||||
expect(doc.cues[1].speaker).toBe('SPEAKER_2');
|
||||
expect(doc.cues[0].text).toContain('Marie Dupont');
|
||||
expect(doc.cues[0].text).not.toContain('<v');
|
||||
});
|
||||
|
||||
test('VTT avec word timestamps : extraction des mots', () => {
|
||||
const doc = parser.parse(VTT_WORD_TIMESTAMPS);
|
||||
const cue = doc.cues[0];
|
||||
expect(cue.speaker).toBe('SPEAKER_1');
|
||||
expect(cue.words.length).toBeGreaterThan(1);
|
||||
expect(cue.words.some(w => w.time === '00:00:03.400')).toBe(true);
|
||||
// Le mot "Marie" doit être associé au timestamp 00:00:03.400
|
||||
const marie = cue.words.find(w => w.time === '00:00:03.400');
|
||||
expect(marie?.text).toContain('Marie');
|
||||
});
|
||||
|
||||
test('Header WEBVTT reconnu', () => {
|
||||
const doc = parser.parse(VTT_SIMPLE);
|
||||
expect(doc.cues.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
test('Timestamps normalisés en HH:MM:SS.mmm', () => {
|
||||
const vtt = `WEBVTT\n\n01:23.456 --> 01:27.800\nTest\n`;
|
||||
const doc = parser.parse(vtt);
|
||||
expect(doc.cues[0].startTime).toBe('00:01:23.456');
|
||||
expect(doc.cues[0].endTime).toBe('00:01:27.800');
|
||||
});
|
||||
});
|
||||
|
||||
describe('VttParser — round-trip', () => {
|
||||
test('VTT simple : round-trip exact', () => {
|
||||
const doc = parser.parse(VTT_SIMPLE);
|
||||
const out = parser.reconstruct(doc);
|
||||
expect(out).toContain('WEBVTT');
|
||||
expect(out).toContain('00:00:01.240 --> 00:00:05.800');
|
||||
expect(out).toContain('Marie Dupont');
|
||||
});
|
||||
|
||||
test('VTT avec speakers : speaker préservé', () => {
|
||||
const doc = parser.parse(VTT_SPEAKER);
|
||||
const out = parser.reconstruct(doc);
|
||||
expect(out).toContain('<v SPEAKER_1>');
|
||||
expect(out).toContain('<v SPEAKER_2>');
|
||||
});
|
||||
|
||||
test('VTT word timestamps : reconstruction avec timestamps', () => {
|
||||
const doc = parser.parse(VTT_WORD_TIMESTAMPS);
|
||||
const out = parser.reconstruct(doc);
|
||||
expect(out).toContain('<v SPEAKER_1>');
|
||||
expect(out).toContain('00:00:03.400');
|
||||
expect(out).toContain('Marie');
|
||||
});
|
||||
});
|
||||
|
||||
describe('VttParser — pseudonymisation', () => {
|
||||
test('Remplacement dans le texte reflété dans la reconstruction', () => {
|
||||
const doc = parser.parse(VTT_SPEAKER);
|
||||
doc.cues[0].text = doc.cues[0].text.replace('Marie Dupont', 'Sophie Martin');
|
||||
VttParser.applyTextToWords(doc.cues[0], doc.cues[0].text);
|
||||
const out = parser.reconstruct(doc);
|
||||
expect(out).toContain('Sophie Martin');
|
||||
expect(out).not.toContain('Marie Dupont');
|
||||
});
|
||||
});
|
||||
|
|
@ -9,5 +9,6 @@
|
|||
"0.1.1": "1.7.2",
|
||||
"0.1.2": "1.7.2",
|
||||
"0.1.3": "1.7.2",
|
||||
"0.1.4": "1.7.2"
|
||||
"0.1.4": "1.7.2",
|
||||
"0.1.5": "1.7.2"
|
||||
}
|
||||
Loading…
Reference in a new issue