release: v0.1.5 — noScribe import, scan candidats, exceptions

This commit is contained in:
Axelle Abbadie 2026-05-17 02:04:10 +02:00
commit 561b43f8a9
38 changed files with 4064 additions and 363 deletions

View file

@ -1,5 +1,40 @@
# Changelog
## [prod] v0.1.5 — 17 mai 2026
### Nouvelles fonctionnalités
**Intégration noScribe**
- **`VttParser`** — WebVTT standard avec word timestamps Whisper (`<00:00:01.240><c>mot</c>`), tags `<v Nom>`, round-trip garanti
- **`NoScribeHtmlParser`** — HTML Qt Rich Text noScribe : ancres `ts_DEBUT_FIN_SXX` (ms), labels locuteurs (`S00:`, `S01:`), timestamps d'affichage filtrés, entités HTML décodées ; `extractAudioSource()` lit `<meta name="audio_source">`
- **`NoScribeVttParser`** — VTT noScribe v0.7 : cues splitées (label / timestamp affichage / texte), détection locuteur par préfixe `SXX:`, entités HTML, source audio depuis `NOTE media:`
- **Format Markdown noScribe**`**S00** [HH:MM:SS] : texte` par tour ; `pseudobs-format: html|vtt` et `pseudobs-audio` optionnel en frontmatter ; timestamps word-level dans `<basename>.words.json`
- **Import audio automatique** — depuis la meta tag HTML ou le dossier source (VTT, un seul fichier audio)
- **`Redaction.ts`** — caviardage par syllabes avec `🀫` ; `generateRedaction()` et `isRedaction()`
- **Re-export VTT** (commande "Exporter en VTT") — WebVTT propre depuis `.md` pseudonymisé + `.words.json`
**Modale de scan — candidats par occurrence**
- **`MappingScanReviewModal`** — colonne occurrences cliquable (compteur mis à jour en direct : `N / total` si exceptions)
- **`OccurrencesContextModal`** — cartes par occurrence (✓ / ✗ / ⚠) avec contexte, callback "Confirmer la sélection", sans application immédiate
- **Bouton "Enregistrer les exceptions"** — sauvegarde les décisions dans le `mapping.json` sans pseudonymiser
- `MappingRuleResult` porte maintenant `occurrences: Occurrence[]`
**Exceptions**
- Type **`IgnoredOccurrence`** (`{text, contextBefore, contextAfter}`) sur `MappingRule.ignoredOccurrences`
- Persistance dans `mapping.json`, chargement au démarrage — surlignage rouge sans re-scan
- **Surlignage rouge** (`pseudobs-exception`, sensible à la casse, priorité 0)
- **Section "Exceptions" dans l'onglet Mappings** — cartes contextuelles avec bouton supprimer
### Modifications
- Labels de statut clarifiés : `validated` → "Actif", `ignored` → "Ignoré", `partial` → "Partiel", `suggested` → "Suggéré" ; colonne renommée "État"
- `getValidatedFor()` inclut maintenant `partial` comme état actif (avec `validated`)
- Bouton "Scanner" de `PseudonymizationView` corrigé (`occurrences` manquant dans `MappingRuleResult`)
### Tests
149 tests · 10 suites · `VttParser`, `NoScribeHtmlParser`, `NoScribeVttParser`, `markdownToVtt`, `extractWordData` couverts ; fixtures : `fight_club.vtt`, `juste-leblanc.html`
---
## [prod] v0.1.3 — 14 mai 2026
**Branche :** `main` | **Tag :** `0.1.3`

View file

@ -2,6 +2,41 @@
> Previous entries in French: [CHANGELOG.fr.md](CHANGELOG.fr.md)
## [prod] v0.1.5 — 17 May 2026
### New features
**noScribe integration**
- **`VttParser`** — standard WebVTT with Whisper word-level timestamps (`<00:00:01.240><c>word</c>`), speaker tags `<v Name>`, round-trip guarantee
- **`NoScribeHtmlParser`** — Qt Rich Text HTML produced by noScribe: `ts_START_END_SXX` anchors (ms), speaker labels (`S00:`, `S01:`), display timestamps filtered, HTML entities decoded; `extractAudioSource()` reads `<meta name="audio_source">`
- **`NoScribeVttParser`** — noScribe VTT v0.7: split cues (label / display timestamp / text), speaker detection from `SXX:` prefix, HTML entity decoding, audio source from `NOTE media:`
- **noScribe Markdown format**`**S00** [HH:MM:SS] : text` per turn; `pseudobs-format: html|vtt` and optional `pseudobs-audio` in frontmatter; word timestamps in `<basename>.words.json`
- **Audio import** — automatic on HTML import (via `audio_source` meta) and on VTT import via file picker (single audio file in source folder)
- **`Redaction.ts`** — syllable-count redaction with `🀫`; `generateRedaction()` and `isRedaction()`
- **VTT re-export** (`Export as VTT` command) — clean WebVTT from pseudonymized `.md` + `.words.json`
**Scan modal — per-occurrence candidates**
- **`MappingScanReviewModal`** — occurrences column is a clickable button (count updated live to `N / total` when some are ignored)
- **`OccurrencesContextModal`** — per-occurrence cards (✓ / ✗ / ⚠) with context, "Confirm selection" callback, no immediate application
- **"Save exceptions" button** — saves ignored decisions to `mapping.json` without pseudonymizing
- `MappingRuleResult` now carries `occurrences: Occurrence[]`
**Exceptions**
- **`IgnoredOccurrence`** type (`{text, contextBefore, contextAfter}`) on `MappingRule.ignoredOccurrences`
- Persisted in `mapping.json`, loaded at startup — red highlighting without re-scanning
- **Red highlighting** (`pseudobs-exception`, case-sensitive, priority 0)
- **Exceptions section in Mappings tab** — context cards with delete button
### Changes
- Status labels: `validated` → "Active", `ignored` → "Ignored", `partial` → "Partial", `suggested` → "Suggested"; column renamed "State"
- `getValidatedFor()` now includes `partial` as active (alongside `validated`)
- `PseudonymizationView` scan button fixed (missing `occurrences` in `MappingRuleResult`)
### Tests
149 tests · 10 suites · `VttParser`, `NoScribeHtmlParser`, `NoScribeVttParser`, `markdownToVtt`, `extractWordData` covered; fixtures: `fight_club.vtt`, `juste-leblanc.html`
---
## [prod] v0.1.4 — 16 May 2026
**Branch:** `main` | **Tag:** upcoming

View file

@ -10,13 +10,13 @@ Plugin Obsidian de **pseudonymisation et de correction de transcriptions** pour
## Workflow
```
Transcription brute (.srt, .cha, .md, .txt)
↓ import automatique
Obsidian — édition native Markdown
↓ pseudonymisation (manuelle ou NER)
Fichier source annoté + table de correspondance (séparée)
noScribe (.html, .vtt) ou transcription brute (.srt, .cha, .md, .txt)
↓ import automatique (fichier audio importé en même temps)
Obsidian — édition native Markdown (**S00** [HH:MM:SS] : texte)
↓ pseudonymisation (manuelle · NER · scan dictionnaires)
Fichier source annoté + table de correspondance + timestamps (.words.json)
↓ export
Transcription pseudonymisée (.pseudonymized.*)
Transcription pseudonymisée (.pseudonymized.md / .pseudonymized.vtt)
```
Deux approches, combinables librement :
@ -44,12 +44,17 @@ L'assistant est relançable à tout moment : Paramètres → Pseudonymizer tool
| Format | Extension | Notes |
|---|---|---|
| **HTML noScribe** | `.html` | Qt Rich Text produit par noScribe — locuteurs, timestamps word-level, chemin audio |
| **VTT noScribe** | `.vtt` | Sortie noScribe v0.7 — aussi WebVTT Whisper standard avec timestamps mot par mot |
| Sous-titres horodatés | `.srt` | Sortie Whisper / IA — horodatages et structure préservés |
| CHAT / CLAN | `.cha`, `.chat` | Lignes `@`, `*`, `%` préservées |
| Markdown annoté | `.md` | Conventions Jefferson ou ICOR |
| Texte brut | `.txt` | Sans marqueurs de convention |
Les fichiers `.srt` et `.cha` sont automatiquement convertis en Markdown à l'import. Un fichier de mapping JSON vide est créé en même temps.
Tous les formats sont automatiquement convertis en Markdown à l'import. En parallèle, le plugin crée :
- `<basename>.mapping.json` — règles de pseudonymisation
- `<basename>.words.json` — timestamps word-level (noScribe / Whisper uniquement), utilisé pour le re-export VTT
- Si un fichier audio est référencé dans la transcription, il est importé dans le vault automatiquement.
> Installez le plugin [Data Files Editor](https://github.com/zuktol/obsidian-data-files-editor) pour visionner les fichiers de mapping JSON directement depuis votre vault.

View file

@ -9,13 +9,13 @@ An Obsidian plugin for **pseudonymizing and correcting interactional transcripts
## Workflow
```
Raw transcript (.srt, .cha, .md, .txt)
↓ automatic import
Obsidian — native Markdown editing
↓ pseudonymization (manual or NER)
Annotated source file + correspondence table (separate)
noScribe (.html, .vtt) or raw transcript (.srt, .cha, .md, .txt)
↓ automatic import (audio file imported alongside)
Obsidian — native Markdown editing (**S00** [HH:MM:SS] : text)
↓ pseudonymization (manual · NER · dictionary scan)
Annotated source file + correspondence table + word timestamps (.words.json)
↓ export
Pseudonymized transcript (.pseudonymized.*)
Pseudonymized transcript (.pseudonymized.md / .pseudonymized.vtt)
```
Two approaches, freely combined:
@ -45,12 +45,17 @@ The wizard can be relaunched at any time: Settings → Pseudonymizer Tool → Se
| Format | Extension | Notes |
|---|---|---|
| **noScribe HTML** | `.html` | Qt Rich Text from noScribe — speaker labels, word timestamps, audio path |
| **noScribe VTT** | `.vtt` | noScribe v0.7 output — also standard Whisper WebVTT with word timestamps |
| Timestamped subtitles | `.srt` | Whisper / AI output — timestamps and structure preserved |
| CHAT / CLAN | `.cha`, `.chat` | `@`, `*`, `%` lines preserved |
| Annotated Markdown | `.md` | Jefferson or ICOR conventions |
| Plain text | `.txt` | No convention markers |
`.srt` and `.cha` files are automatically converted to Markdown on import. An empty mapping JSON file is created at the same time.
All formats are automatically converted to Markdown on import. Alongside the `.md`, the plugin creates:
- `<basename>.mapping.json` — pseudonymization rules
- `<basename>.words.json` — word-level timestamps (noScribe / Whisper only), used for VTT re-export
- If an audio file is referenced in the transcript, it is imported to the vault automatically.
> Install the [Data Files Editor](https://github.com/zuktol/obsidian-data-files-editor) plugin to view mapping JSON files directly in your vault.
@ -103,9 +108,9 @@ Accessible via the ribbon icon or `Ctrl+P → Pseudonymization: open panel`.
| Tab | Content |
|---|---|
| **Mappings** | Active rules · Edit · Delete · Add · Scan file |
| **Mappings** | Active rules · Edit · Delete · Add · Scan file · **Exceptions section** |
| **Dictionaries** | Mini cards · Dictionary scan · Local import |
| **Exports** | Pseudonymize and export · Export correspondence table |
| **Exports** | Pseudonymize and export · Export correspondence table · **Export as VTT** |
| **NER** | Visible if NER enabled · Identify candidates · Confidence threshold · Function words |
### Highlighting and markers
@ -117,13 +122,15 @@ Highlighting is active in all open files, including `.pseudonymized.*` export fi
| 🟠 Orange + outline | Source term still present — to be pseudonymized |
| 🟢 Green + underline | Pseudonym applied directly in the file |
| 🔵 Blue + outline | NER candidate — no rule yet |
| 🔴 Red + underline | **Exception** — occurrence explicitly ignored during scan (case-sensitive; persisted in mapping) |
In exported files, pseudonyms are wrapped in `{{Pierre}}` markers to distinguish them from raw data (enabled by default, configurable in settings).
### Correspondence tables
- Three scope levels: `file.mapping.json` · `folder.mapping.json` · `vault.mapping.json`
- Per-occurrence statuses: `validated`, `ignored`, `partial`, `conflict`, `needs_review`
- Per-rule statuses: **Active** (validated), **Partial**, **Ignored**, **Suggested**
- Per-occurrence ignored exceptions: `IgnoredOccurrence {text, contextBefore, contextAfter}` stored in the rule
- **Z-index priority**: free integer — longer entities take precedence (`Saint-Jean-de-Luz` > `Jean`)
- JSON format documented in `SPECS.md §5`
@ -183,14 +190,16 @@ Repository structure:
src/
├── main.ts # Obsidian entry point
├── settings.ts # Persistent settings
├── types.ts # Shared types
├── types.ts # Shared types (MappingRule, IgnoredOccurrence, …)
├── i18n/ # Internationalization (en, fr)
├── parsers/ # SrtParser, ChatParser, TranscriptConverter
├── parsers/ # SrtParser, ChatParser, VttParser,
│ # NoScribeHtmlParser, NoScribeVttParser, TranscriptConverter
├── mappings/ # MappingStore, ScopeResolver
├── pseudonymizer/ # Engine, ReplacementPlanner, SpanProtector
├── pseudonymizer/ # Engine, ReplacementPlanner, SpanProtector, Redaction
├── scanner/ # OccurrenceScanner, OnnxNerScanner
├── dictionaries/ # DictionaryLoader
└── ui/ # PseudonymizationView, modals, CM6 highlighting
└── ui/ # PseudonymizationView, modals (incl. OccurrencesContextModal),
# CM6 highlighting (PseudonymHighlighter)
```
---
@ -203,7 +212,7 @@ src/
| 7 — Coulmont | ✅ | Equivalent first name suggestions · JSON/CSV import |
| 8 — Side panel | ✅ | 3 tabs · Embedded NER · Wizard · Cancellation · Export highlighting |
| 9 — Structured dictionaries | ✅ | Format v1.1 · DictionaryLoader · Dictionary scan · Review modal · French communes |
| 10 — Refinement | 🔄 | i18n · Corpus organization · Settings redesign · Phase in progress |
| 10 — Refinement & noScribe | 🔄 | i18n · Corpus org · noScribe HTML/VTT import · per-occurrence scan · exceptions · VTT re-export |
| 11 — EMCA functions | ⏳ | Turn navigation · Jefferson/ICOR correction · ELAN export |
See [ROADMAP.md](ROADMAP.md) for the full phase breakdown and planned features.

View file

@ -15,7 +15,9 @@ Architectural decision (May 2026): **identifying entity detection** is handled b
```
✅ Phases 08 Parsers · Engine · UI · Scopes · Highlighting · Validation · Coulmont · Panel · NER · Wizard
✅ Phase 9 Structured dictionaries · DictionaryLoader · Review modals · French communes
🔄 Phase 10 Refinement & EMCA specialization (v0.1.x → v0.2.0)
🔄 Phase 10 Refinement & noScribe integration (v0.1.x → v0.2.0)
✅ i18n · Corpus org · noScribe HTML/VTT/audio · Scan candidates · Exceptions · VTT re-export
⏳ Test coverage · Jefferson/ICOR checker · Audio redaction · Timestamp UI
⏳ Phase 11 Interactional analysis functions (v1.0.0)
```
@ -219,9 +221,11 @@ Goal: consolidate all existing features and add the EMCA-specific functions that
- [x] **Settings redesign** — 6 sections ordered by frequency of use
- [x] **Broad-scope warning** — callout in RuleModal / EditRuleModal for name rules with folder/vault scope
- [x] **Mappings tab grouped by scope** — File / Folder / Vault sections with active-file filter
- [x] **Status labels clarified** — "Active / Partial / Ignored / Suggested" instead of ✓/◑/✗/?; `getValidatedFor()` includes `partial`
- [x] **Scan modal — per-occurrence candidates**`MappingScanReviewModal` count column opens `OccurrencesContextModal` (✓/✗/⚠ per occurrence, "Save exceptions" button)
- [x] **Exceptions**`IgnoredOccurrence` on `MappingRule`, persisted in mapping.json, red highlighting (`pseudobs-exception`, case-sensitive, priority 0), Exceptions section in Mappings tab
- [ ] Unit test coverage ≥ 80% for parsers, engine, NER scanner, DictionaryLoader
- [ ] Jefferson / ICOR convention checker: hover suggestions, editor highlighting
- [ ] Exports in original format for corpus re-opening
- [ ] EMCA publication exports (PNG)
- [ ] NER performance: measure and optimize on a 500-turn file
- [ ] [Meld Encrypt](https://github.com/meld-cp/obsidian-encrypt) integration in Exports tab for encrypting correspondence tables and pseudonymized exports
@ -229,14 +233,20 @@ Goal: consolidate all existing features and add the EMCA-specific functions that
### noScribe integration & audio pseudonymization
[noScribe](https://github.com/kaixxx/noScribe) is a local transcription tool (Whisper + pyannote) widely used in qualitative research. It produces VTT files with **word-level timestamps** and **speaker diarization**.
[noScribe](https://github.com/kaixxx/noScribe) is a local transcription tool (Whisper + pyannote) widely used in qualitative research. It produces HTML and VTT files with **word-level timestamps** and **speaker diarization**.
- [ ] **`VttParser.ts`** — import noScribe VTT: word-level timestamps, speaker labels, round-trip guarantee; auto-conversion to `.md` on import alongside `.mapping.json`
- [ ] **Timestamp adjustment UI** — when a word timestamp from Whisper is imprecise, the researcher can fine-tune it: playback of ±1 s around the word (Web Audio API, no external dependency), editable start/end fields, saved back to the mapping metadata
- [ ] **Audio bleep export** — once terms are pseudonymized and timestamps are validated, generate an audio export where each pseudonymized occurrence is replaced by a sine tone at the exact word position; export as WAV via Web Audio API
- [ ] **Speaker-aware pseudonymization** — rules can be scoped to a specific speaker label (e.g. `SPEAKER_1`); when a name is pseudonymized in one turn, offer to apply the rule to all turns by the same speaker
- [x] **`VttParser.ts`** — standard WebVTT with Whisper word-level timestamps, speaker tags, round-trip guarantee; auto-conversion to `.md` + `.words.json`
- [x] **`NoScribeHtmlParser.ts`** — Qt Rich Text HTML: `ts_START_END_SXX` anchors, speaker extraction, audio source meta, HTML entities
- [x] **`NoScribeVttParser.ts`** — noScribe VTT v0.7: split-cue format, `SXX:` speaker detection, `NOTE media:` audio source
- [x] **noScribe Markdown format**`**S00** [HH:MM:SS] : text`, `pseudobs-format: html|vtt`, `pseudobs-audio`; word timestamps in `.words.json`
- [x] **Audio import** — automatic from `audio_source` meta (HTML) or single audio in source folder (VTT)
- [x] **`Redaction.ts`** — syllable-count redaction with `🀫`
- [x] **VTT re-export**`Export as VTT` command: clean WebVTT from pseudonymized `.md` + `.words.json`
- [ ] **Timestamp adjustment UI** — playback of ±1 s around a word (Web Audio API), editable start/end, saved to mapping metadata
- [ ] **Audio redaction export** — WAV export with bleep / silence / white noise at pseudonymized positions
- [ ] **Speaker-aware pseudonymization** — rules scoped to a speaker label (e.g. `S01`)
**Testable (v0.2.0):** stable end-to-end workflow on a real corpus of 10 interviews, including noScribe VTT import, audio bleep export, and timestamp adjustment.
**Testable (v0.2.0):** stable end-to-end workflow on a real corpus of 10 interviews, including noScribe import, audio bleep export, and timestamp adjustment.
---

1485
main.js

File diff suppressed because it is too large Load diff

View file

@ -1,7 +1,7 @@
{
"id": "pseudonymizer-tool",
"name": "Pseudonymizer Tool",
"version": "0.1.4",
"version": "0.1.5",
"minAppVersion": "1.7.2",
"description": "Pseudonymize and correct interactional transcripts (Jefferson, ICOR, SRT, CHAT/CHA). Designed for qualitative researchers in linguistics and conversation analysis.",
"author": "Axelle Abbadie",

View file

@ -1,6 +1,6 @@
{
"name": "pseudonymizer-tool",
"version": "0.1.4",
"version": "0.1.5",
"description": "Obsidian plugin for pseudonymizing and correcting interactional transcripts (Jefferson, ICOR, SRT, CHAT/CHA).",
"author": "Axelle Abbadie",
"homepage": "https://cv.hal.science/axelle-abbadie/",

View file

@ -41,6 +41,7 @@
"notice.noDictDetection": "No detection dictionary loaded.\nInstall a dictionary from the Dictionaries panel.",
"notice.noDictEntities": "No entities found in the detection dictionaries.",
"notice.noDictReplacements": "No replacements available for the found entities.",
"notice.ruleDeleted": "✓ Rule deleted — \"{0}\" restored in active file",
"notice.ruleCreated": "✓ Rule created: \"{0}\" → \"{1}\"",
"notice.rulesCreated": "✓ {0} {1} created",
"notice.rulesCreated.rule": "rule",
@ -66,6 +67,14 @@
"notice.nerModelLoading": "Loading NER model (first use — ~66 MB)…",
"notice.ruleNotFound": "Rule not found in mappings.",
"notice.noCheckedDicts": "No dictionary checked.",
"mappingScanModal.saveExceptions": "Save exceptions",
"mappingScanModal.exceptionsSaved": "Exceptions saved to mapping.",
"panel.mappings.exceptions": "Exceptions",
"panel.mappings.exceptions.hint": "Occurrences explicitly ignored for this rule.",
"notice.notNoScribeFormat": "This file is not a noScribe transcript (pseudobs-format: vtt or html expected).",
"notice.wordsJsonMissing": "Timestamp file not found: {0}.words.json",
"notice.vttMismatch": "Warning: Markdown cue count and words.json cue count differ — exported VTT may be incomplete.",
"notice.vttExported": "VTT exported: {0}",
"command.organizeCorpus": "Organize corpus",
"command.addTranscription": "Add a transcription",
@ -75,6 +84,7 @@
"command.scanNer": "Scan file with NER detection",
"command.scanDictionaries": "Scan file with dictionaries",
"command.pseudonymizeSelection": "Pseudonymize selection",
"command.exportAsVtt": "Export as VTT",
"command.openPanel": "Pseudonymization: open panel",
"contextMenu.cancelPseudonymization": "Cancel pseudonymization of \"{0}\"",
@ -84,6 +94,9 @@
"ruleModal.scopeWarnTitle": "Longitudinal scope",
"ruleModal.scopeOkTitle": "File scope",
"ruleModal.scopeOk": "Good practice. This pseudonym will only apply to this transcription — each file remains independent.",
"contextMenu.redact": "Redact \"{0}\"",
"redaction.checkbox": "Redact (🀫)",
"redaction.checkboxDesc": "Replace with 🀫 symbols (one per syllable) — for non-essential identifying content",
"contextMenu.coulmont": "Pseudonymize with Prof. Baptiste Coulmont",
"contextMenu.createRule": "Create a replacement rule…",
@ -121,13 +134,13 @@
"scope.folder": "Folder",
"scope.vault": "Vault",
"status.validated": "",
"status.ignored": "",
"status.partial": "",
"status.suggested": "?",
"status.conflict": "",
"status.disabled": "",
"status.needs_review": "👁",
"status.validated": "Active",
"status.ignored": "Ignored",
"status.partial": "Partial",
"status.suggested": "Suggested",
"status.conflict": "Conflict",
"status.disabled": "Inactive",
"status.needs_review": "Review",
"panel.tab.mappings": "Mappings",
"panel.tab.dictionaries": "Dictionaries",

View file

@ -41,6 +41,7 @@
"notice.noDictDetection": "Aucun dictionnaire de détection chargé.\nInstallez un dictionnaire depuis le panneau Dictionnaires.",
"notice.noDictEntities": "Aucune entité trouvée dans les dictionnaires de détection.",
"notice.noDictReplacements": "Aucun remplacement disponible pour les entités trouvées.",
"notice.ruleDeleted": "✓ Règle supprimée — \"{0}\" rétabli dans le fichier actif",
"notice.ruleCreated": "✓ Règle créée : \"{0}\" → \"{1}\"",
"notice.rulesCreated": "✓ {0} règle{1} créée{1}",
"notice.rulesCreated.rule": "",
@ -66,6 +67,14 @@
"notice.nerModelLoading": "Chargement du modèle NER (première utilisation — ~66 Mo)…",
"notice.ruleNotFound": "Règle introuvable dans les mappings.",
"notice.noCheckedDicts": "Aucun dictionnaire coché.",
"mappingScanModal.saveExceptions": "Enregistrer les exceptions",
"mappingScanModal.exceptionsSaved": "Exceptions enregistrées dans le mapping.",
"panel.mappings.exceptions": "Exceptions",
"panel.mappings.exceptions.hint": "Occurrences explicitement ignorées pour cette règle.",
"notice.notNoScribeFormat": "Ce fichier n'est pas une transcription noScribe (pseudobs-format: vtt ou html attendu).",
"notice.wordsJsonMissing": "Fichier de timestamps introuvable : {0}.words.json",
"notice.vttMismatch": "Attention : le nombre de cues du Markdown et du words.json diffère — le VTT exporté peut être incomplet.",
"notice.vttExported": "VTT exporté : {0}",
"command.organizeCorpus": "Organiser le corpus",
"command.addTranscription": "Ajouter une transcription",
@ -75,6 +84,7 @@
"command.scanNer": "Scanner le fichier avec détection NER",
"command.scanDictionaries": "Scanner le fichier avec les dictionnaires",
"command.pseudonymizeSelection": "Pseudonymiser la sélection",
"command.exportAsVtt": "Exporter en VTT",
"command.openPanel": "Pseudonymisation : ouvrir le panneau",
"contextMenu.cancelPseudonymization": "Annuler la pseudonymisation de \"{0}\"",
@ -84,6 +94,9 @@
"ruleModal.scopeWarnTitle": "Portée longitudinale",
"ruleModal.scopeOkTitle": "Portée fichier",
"ruleModal.scopeOk": "Bonne pratique. Ce pseudonyme ne s'appliquera qu'à cette transcription — chaque fichier reste indépendant.",
"contextMenu.redact": "Caviarder \"{0}\"",
"redaction.checkbox": "Caviardage (🀫)",
"redaction.checkboxDesc": "Remplace par des 🀫 (1 par syllabe) — pour les informations identifiantes non essentielles",
"contextMenu.coulmont": "Pseudonymiser avec Pr Baptiste Coulmont",
"contextMenu.createRule": "Créer une règle de remplacement…",
@ -121,13 +134,13 @@
"scope.folder": "Dossier",
"scope.vault": "Vault",
"status.validated": "",
"status.ignored": "",
"status.partial": "",
"status.suggested": "?",
"status.conflict": "",
"status.disabled": "",
"status.needs_review": "👁",
"status.validated": "Actif",
"status.ignored": "Ignoré",
"status.partial": "Partiel",
"status.suggested": "Suggéré",
"status.conflict": "Conflit",
"status.disabled": "Inactif",
"status.needs_review": "À réviser",
"panel.tab.mappings": "Mappings",
"panel.tab.dictionaries": "Dictionnaires",
@ -147,7 +160,7 @@
"panel.mappings.col.category": "Catégorie",
"panel.mappings.col.scope": "Portée",
"panel.mappings.col.priority": "P.",
"panel.mappings.col.status": "Statut",
"panel.mappings.col.status": "État",
"panel.dict.noneInstalled": "Aucun dictionnaire installé. Installez-en un depuis le wizard (Paramètres → Reconfigurer) ou importez un fichier local.",
"panel.dict.checkbox": "Inclure dans le scan groupé",

View file

@ -1,4 +1,4 @@
import { Plugin, Notice, TFile, TAbstractFile, Editor, Menu, MarkdownView, requestUrl, WorkspaceLeaf } from 'obsidian';
import { Plugin, Notice, TFile, TFolder, TAbstractFile, Editor, Menu, MarkdownView, requestUrl, WorkspaceLeaf } from 'obsidian';
import { t, setLocale } from './i18n';
import { EditorView } from '@codemirror/view';
import { PseudObsSettings, DEFAULT_SETTINGS, PseudObsSettingTab } from './settings';
@ -11,6 +11,7 @@ import { OnboardingModal } from './ui/OnboardingModal';
import { OnnxNerScanner } from './scanner/OnnxNerScanner';
import { DictionaryLoader } from './dictionaries/DictionaryLoader';
import { DictScanReviewModal } from './ui/DictScanReviewModal';
import { generateRedaction } from './pseudonymizer/Redaction';
import { CorpusModal, ClassSelectModal, getCorpusClasses } from './ui/CorpusModal';
import type { DictScanResultItem } from './ui/DictScanReviewModal';
import { MappingScanReviewModal } from './ui/MappingScanReviewModal';
@ -18,7 +19,10 @@ import type { MappingRuleResult } from './ui/MappingScanReviewModal';
import { scanOccurrences } from './scanner/OccurrenceScanner';
import { SrtParser } from './parsers/SrtParser';
import { ChatParser } from './parsers/ChatParser';
import { srtToMarkdown, chatToMarkdown } from './parsers/TranscriptConverter';
import { VttParser } from './parsers/VttParser';
import { NoScribeHtmlParser } from './parsers/NoScribeHtmlParser';
import { NoScribeVttParser } from './parsers/NoScribeVttParser';
import { srtToMarkdown, chatToMarkdown, vttToMarkdown, noScribeHtmlToMarkdown, extractWordData, markdownToVtt, type VttCueData } from './parsers/TranscriptConverter';
import { MappingStore } from './mappings/MappingStore';
import { ScopeResolver } from './mappings/ScopeResolver';
import { PseudonymizationEngine } from './pseudonymizer/PseudonymizationEngine';
@ -26,7 +30,7 @@ import { findSpansForRule } from './pseudonymizer/ReplacementPlanner';
import { applySpans } from './pseudonymizer/SpanProtector';
import type { MappingRule, MappingStatus, Occurrence } from './types';
const CONVERTIBLE_EXTS = ['srt', 'cha', 'chat'];
const CONVERTIBLE_EXTS = ['srt', 'cha', 'chat', 'vtt', 'html'];
export default class PseudObsPlugin extends Plugin {
settings!: PseudObsSettings;
@ -34,10 +38,14 @@ export default class PseudObsPlugin extends Plugin {
nerScanner!: OnnxNerScanner;
dictionaryLoader!: DictionaryLoader;
// Cache synchrone pour le surlignage CM6 (mis à jour de façon asynchrone)
private highlightData: HighlightData = { sources: [], replacements: [], nerCandidates: [] };
private highlightData: HighlightData = { sources: [], replacements: [], nerCandidates: [], ignoredTerms: [] };
// Candidats NER par fichier (effacés au changement de fichier ou à un nouveau scan)
private nerCandidateFile: TFile | null = null;
private nerCandidates: string[] = [];
// Dernière MarkdownView connue — pour mettre à jour le surlignage même
// quand le panneau latéral a le focus (getActiveViewOfType retourne null dans ce cas)
private lastMarkdownView: MarkdownView | null = null;
private _viewRefreshTimer: number | null = null;
async onload(): Promise<void> {
await this.loadSettings();
@ -57,12 +65,31 @@ export default class PseudObsPlugin extends Plugin {
createPseudonymHighlighter(() => this.highlightData)
);
// Rafraîchir le cache de surlignage à chaque changement de fichier actif
// Tracker la dernière MarkdownView pour garder le surlignage actif
// même quand le panneau latéral prend le focus
this.registerEvent(
this.app.workspace.on('active-leaf-change', () => { void this.refreshHighlightData(); })
this.app.workspace.on('active-leaf-change', (leaf) => {
const v = leaf?.view;
if (v instanceof MarkdownView) this.lastMarkdownView = v;
void this.refresh();
})
);
// Rafraîchir panneau + surlignage quand le FICHIER TRANSCRIPT actif est modifié.
// Les fichiers .mapping.json sont exclus — leurs changements sont déjà gérés
// par les appels explicites à refresh() dans les modales, évitant le double rendu.
this.registerEvent(
this.app.vault.on('modify', (file) => {
if (file instanceof TFile
&& !file.name.endsWith('.mapping.json')
&& file === this.app.workspace.getActiveFile()) {
void this.refresh();
}
})
);
// Premier chargement au démarrage
void this.refreshHighlightData();
void this.refresh();
// Onboarding au premier lancement
if (!this.settings.onboardingCompleted) {
@ -126,6 +153,12 @@ export default class PseudObsPlugin extends Plugin {
callback: () => void this.scanCurrentFileWithDictionaries(),
});
this.addCommand({
id: 'export-as-vtt',
name: t('command.exportAsVtt'),
callback: () => void this.exportCurrentFileAsVtt(),
});
this.addCommand({
id: 'pseudonymize-selection',
name: t('command.pseudonymizeSelection'),
@ -164,7 +197,7 @@ export default class PseudObsPlugin extends Plugin {
const location = await this.scopeResolver.findRuleByTerm(bare);
if (!location) { new Notice(t('notice.ruleNotFound')); return; }
editor.replaceSelection(location.rule.source);
void this.refreshHighlightData();
void this.refresh();
})
);
}
@ -192,6 +225,13 @@ export default class PseudObsPlugin extends Plugin {
.onClick(() => new QuickPseudonymizeModal(this.app, this, editor).open())
);
menu.addItem((item) =>
item
.setTitle(t('contextMenu.redact', truncate(selection)))
.setIcon('square')
.onClick(() => new QuickPseudonymizeModal(this.app, this, editor, generateRedaction(selection), [], true).open())
);
menu.addItem((item) =>
item
.setTitle(t('contextMenu.coulmont'))
@ -262,9 +302,8 @@ export default class PseudObsPlugin extends Plugin {
async refreshHighlightData(): Promise<void> {
const file = this.app.workspace.getActiveFile();
if (!file) {
this.highlightData = { sources: [], replacements: [], nerCandidates: [] };
this.highlightData = { sources: [], replacements: [], nerCandidates: [], ignoredTerms: [] };
} else {
// Candidats NER : uniquement si le fichier actif est celui du dernier scan
const nerCandidates = file === this.nerCandidateFile ? this.nerCandidates : [];
try {
@ -287,24 +326,57 @@ export default class PseudObsPlugin extends Plugin {
} else {
rules = await this.scopeResolver.getRulesFor(file.path);
}
// Termes ignorés : extraits des ignoredOccurrences de chaque règle
const ignoredTerms = rules.flatMap((r) =>
(r.ignoredOccurrences ?? []).map((o) => o.text)
);
this.highlightData = {
sources: rules.map((r) => r.source).filter(Boolean),
replacements: rules.map((r) => r.replacement).filter(Boolean),
nerCandidates,
ignoredTerms,
};
} catch {
this.highlightData = { sources: [], replacements: [], nerCandidates };
this.highlightData = { sources: [], replacements: [], nerCandidates, ignoredTerms: [] };
}
}
// Dispatcher le StateEffect sur l'éditeur actif pour déclencher
// la reconstruction des décorations CM6 (le ViewPlugin ne se déclenche
// pas sur un changement de données externe sans ce signal)
const view = this.app.workspace.getActiveViewOfType(MarkdownView);
// Dispatcher le StateEffect — utiliser lastMarkdownView si le panneau a le focus
const view = this.app.workspace.getActiveViewOfType(MarkdownView) ?? this.lastMarkdownView;
const cm = view?.editor && ((view.editor as unknown as { cm?: EditorView }).cm);
cm?.dispatch({ effects: highlightDataChanged.of(undefined) });
}
/**
* Rafraîchit à la fois le surlignage CM6 ET le panneau latéral.
* À appeler après toute action qui crée, modifie ou supprime une règle.
*/
async refresh(): Promise<void> {
await this.refreshHighlightData();
this.refreshView();
}
/**
* Demande au panneau latéral ouvert de re-rendre son onglet actif.
* Debounce 80 ms pour coalescer les appels multiples rapides
* (vault watcher + appel explicite dans la même action).
*/
refreshView(): void {
if (this._viewRefreshTimer !== null) {
window.clearTimeout(this._viewRefreshTimer);
}
this._viewRefreshTimer = window.setTimeout(() => {
this._viewRefreshTimer = null;
const leaves = this.app.workspace.getLeavesOfType(VIEW_TYPE_PSEUDOBS);
for (const leaf of leaves) {
const view = leaf.view;
if (view instanceof PseudonymizationView) {
void view.refreshActiveTab();
}
}
}, 80);
}
// --- Conversion automatique ---
private async autoConvert(file: TFile): Promise<void> {
@ -315,28 +387,76 @@ export default class PseudObsPlugin extends Plugin {
const folder = file.parent?.path ?? '';
const mdPath = folder ? `${folder}/${basename}.md` : `${basename}.md`;
let mdContent: string;
if (ext === 'srt') {
mdContent = srtToMarkdown(new SrtParser().parse(raw), file.name);
} else {
mdContent = chatToMarkdown(new ChatParser().parse(raw), file.name);
}
// Si un .md du même nom existe déjà, ne pas écraser
// Ne pas écraser un .md existant
if (this.app.vault.getAbstractFileByPath(mdPath) instanceof TFile) {
new Notice(t('notice.conversionSkipped', basename, file.name));
return;
}
// Chercher un fichier audio déjà présent dans le même dossier du vault
let audioFilename = this.findAudioInVaultFolder(folder);
// Conversion
let mdContent: string;
let wordData: ReturnType<typeof extractWordData> | null = null;
if (ext === 'srt') {
mdContent = srtToMarkdown(new SrtParser().parse(raw), file.name);
} else if (ext === 'vtt') {
const doc = NoScribeVttParser.isNoScribeVtt(raw)
? new NoScribeVttParser().parse(raw)
: new VttParser().parse(raw);
wordData = extractWordData(doc);
// Pour les VTT noScribe, chercher aussi l'audio via NOTE media
if (!audioFilename && NoScribeVttParser.isNoScribeVtt(raw)) {
const audioSource = NoScribeVttParser.extractAudioSource(raw);
if (audioSource) audioFilename = await this.importAudioFromPath(audioSource, folder || this.settings.transcriptionsFolder);
}
const toMd = NoScribeVttParser.isNoScribeVtt(raw) ? noScribeHtmlToMarkdown : vttToMarkdown;
mdContent = toMd(doc, file.name, audioFilename ?? undefined);
} else if (ext === 'html') {
if (!NoScribeHtmlParser.isNoScribeHtml(raw)) return;
const doc = new NoScribeHtmlParser().parse(raw);
wordData = extractWordData(doc);
// Importer l'audio depuis le chemin absolu dans la meta tag si pas encore dans le vault
if (!audioFilename) {
const audioSource = NoScribeHtmlParser.extractAudioSource(raw);
if (audioSource) {
audioFilename = await this.importAudioFromPath(audioSource, folder || this.settings.transcriptionsFolder);
}
}
mdContent = noScribeHtmlToMarkdown(doc, file.name, audioFilename ?? undefined);
} else {
mdContent = chatToMarkdown(new ChatParser().parse(raw), file.name);
}
await this.app.vault.create(mdPath, mdContent);
const mappingPath = `${this.settings.mappingFolder}/${basename}.mapping.json`;
// Structure miroir pour les mappings
const transcRoot = this.settings.transcriptionsFolder;
const fileFolder = file.parent?.path ?? '';
const relSubFolder = fileFolder.startsWith(transcRoot)
? fileFolder.slice(transcRoot.length).replace(/^\//, '')
: '';
const mappingDir = relSubFolder
? `${this.settings.mappingFolder}/${relSubFolder}`
: this.settings.mappingFolder;
await this.ensureFolder(mappingDir);
const mappingPath = `${mappingDir}/${basename}.mapping.json`;
if (!this.app.vault.getAbstractFileByPath(mappingPath)) {
await this.ensureFolder(this.settings.mappingFolder);
const store = new MappingStore({ type: 'file', path: mdPath });
await this.app.vault.create(mappingPath, JSON.stringify(store.toJSON(), null, 2));
}
// Écrire les timestamps word-level dans un fichier auxiliaire
if (wordData && wordData.length > 0) {
const wordsPath = `${mappingDir}/${basename}.words.json`;
if (!this.app.vault.getAbstractFileByPath(wordsPath)) {
await this.app.vault.create(wordsPath, JSON.stringify(wordData, null, 2));
}
}
await this.app.fileManager.trashFile(file);
const mdFile = this.app.vault.getAbstractFileByPath(mdPath);
@ -350,12 +470,52 @@ export default class PseudObsPlugin extends Plugin {
}
}
/** Retourne le nom du premier fichier audio trouvé dans un dossier du vault. */
private findAudioInVaultFolder(folderPath: string): string | null {
const AUDIO_EXTS = new Set(['m4a', 'mp3', 'wav', 'ogg', 'flac', 'mp4', 'aac', 'aiff']);
const folder = this.app.vault.getAbstractFileByPath(folderPath || '/');
if (!(folder instanceof TFolder)) return null;
const audioFile = folder.children.find(
(f) => f instanceof TFile && AUDIO_EXTS.has((f as TFile).extension.toLowerCase())
) as TFile | undefined;
return audioFile?.name ?? null;
}
/**
* Copie un fichier audio externe (chemin absolu sur disque) dans le vault.
* Utilise l'API Node.js fs desktop uniquement.
* Retourne le nom du fichier importé, ou null en cas d'échec.
*/
private async importAudioFromPath(sourcePath: string, targetFolder: string): Promise<string | null> {
try {
// eslint-disable-next-line @typescript-eslint/no-var-requires
const nodeFs = require('fs') as typeof import('fs');
if (!nodeFs.existsSync(sourcePath)) return null;
const audioFilename = sourcePath.replace(/\\/g, '/').split('/').pop()!;
const destPath = targetFolder ? `${targetFolder}/${audioFilename}` : audioFilename;
if (this.app.vault.getAbstractFileByPath(destPath) instanceof TFile) {
return audioFilename; // déjà présent
}
const buffer: Buffer = await nodeFs.promises.readFile(sourcePath);
await this.ensureFolder(targetFolder);
const arrayBuf = buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength) as ArrayBuffer;
await this.app.vault.createBinary(destPath, arrayBuf);
new Notice(`Audio importé : ${audioFilename}`);
return audioFilename;
} catch {
return null;
}
}
// --- Commande "Ajouter une transcription" ---
private openFilePicker(): void {
const input = activeDocument.createElement('input');
input.type = 'file';
input.accept = '.srt,.cha,.chat,.txt,.md';
input.accept = '.srt,.vtt,.cha,.chat,.html,.txt,.md';
input.multiple = true;
// Pas de display:none — bloque le change event dans certaines versions d'Electron
input.classList.add('pseudobs-hidden-input');
@ -376,6 +536,7 @@ export default class PseudObsPlugin extends Plugin {
private async copyToVault(browserFile: File): Promise<void> {
const raw = await browserFile.text();
const ext = browserFile.name.split('.').pop()?.toLowerCase() ?? '';
// Sélection de classe si le corpus est organisé en sous-dossiers
const classes = getCorpusClasses(this.app, this.settings.transcriptionsFolder);
@ -397,6 +558,39 @@ export default class PseudObsPlugin extends Plugin {
}
await this.app.vault.create(destPath, raw);
// Pour VTT : chercher un fichier audio dans le dossier source (Electron expose le chemin complet)
if (ext === 'vtt') {
const sourcePath = (browserFile as unknown as { path?: string }).path;
if (sourcePath) {
const sourceDir = sourcePath.replace(/\\/g, '/').replace(/\/[^/]+$/, '');
const audioPath = await this.findAudioInSourceFolder(sourceDir);
if (audioPath) await this.importAudioFromPath(audioPath, targetFolder);
}
}
// Pour HTML : l'audio sera importé par autoConvert via la meta tag audio_source
}
/**
* Cherche un fichier audio dans un dossier sur le disque (hors vault).
* Retourne le chemin absolu du seul fichier audio trouvé, ou null si 0 ou >1.
*/
private async findAudioInSourceFolder(folderPath: string): Promise<string | null> {
try {
// eslint-disable-next-line @typescript-eslint/no-var-requires
const nodeFs = require('fs') as typeof import('fs');
const AUDIO_EXTS = new Set(['m4a', 'mp3', 'wav', 'ogg', 'flac', 'mp4', 'aac', 'aiff']);
const entries = await nodeFs.promises.readdir(folderPath);
const audioFiles = entries.filter((f) => {
const ext = f.split('.').pop()?.toLowerCase() ?? '';
return AUDIO_EXTS.has(ext);
});
// Import uniquement s'il y a exactement un fichier audio (évite l'ambiguïté)
if (audioFiles.length === 1) return `${folderPath}/${audioFiles[0]}`;
return null;
} catch {
return null;
}
}
// --- Pseudonymisation ---
@ -490,7 +684,7 @@ export default class PseudObsPlugin extends Plugin {
const unique = [...new Set(occurrences.map((o) => o.text).filter(Boolean))];
this.nerCandidateFile = file;
this.nerCandidates = unique;
void this.refreshHighlightData();
void this.refresh();
new Notice(t('notice.nerEntitiesFound', String(unique.length), unique.length > 1 ? t('notice.nerEntitiesFound.entities') : t('notice.nerEntitiesFound.entity')), 6000);
} catch (e) {
@ -576,7 +770,7 @@ export default class PseudObsPlugin extends Plugin {
// Surlignage bleu préventif (aperçu pendant que la modale est ouverte)
this.nerCandidateFile = file;
this.nerCandidates = results.map((r) => r.term);
void this.refreshHighlightData();
void this.refresh();
new DictScanReviewModal(this.app, this, file, results, existingReplacements).open();
}
@ -585,9 +779,10 @@ export default class PseudObsPlugin extends Plugin {
clearNerCandidates(): void {
this.nerCandidates = [];
this.nerCandidateFile = null;
void this.refreshHighlightData();
void this.refresh();
}
async exportMappingForFile(file: TFile): Promise<void> {
const mappingPath = `${this.settings.mappingFolder}/${file.basename}.mapping.json`;
const mappingFile = this.app.vault.getAbstractFileByPath(mappingPath);
@ -611,6 +806,77 @@ export default class PseudObsPlugin extends Plugin {
new Notice(t('notice.mappingExported', destPath));
}
/**
* Exporte le fichier Markdown noScribe actif en WebVTT pseudonymisé.
* Lit le .words.json correspondant pour les timestamps précis.
*/
async exportCurrentFileAsVtt(): Promise<void> {
const file = this.app.workspace.getActiveFile();
if (!file || file.extension !== 'md') {
new Notice(t('notice.noActiveFile'));
return;
}
const content = await this.app.vault.read(file);
// Vérifier que c'est bien un fichier noScribe converti
const formatMatch = /^pseudobs-format:\s*(\w+)/m.exec(content);
const format = formatMatch?.[1];
if (format !== 'vtt' && format !== 'html') {
new Notice(t('notice.notNoScribeFormat'));
return;
}
// Trouver le .words.json : même basename, dans le dossier mappings
// Le basename peut être "juste-leblanc" ou "juste-leblanc.pseudonymized"
const rawBasename = file.basename.replace(/\.pseudonymized$/, '');
const wordsJson = await this.findWordsJson(rawBasename);
if (!wordsJson) {
new Notice(t('notice.wordsJsonMissing', rawBasename));
return;
}
const wordData = JSON.parse(wordsJson) as VttCueData[];
const { vtt, mismatch } = markdownToVtt(content, wordData);
if (mismatch) {
new Notice(t('notice.vttMismatch'));
}
await this.ensureFolder(this.settings.exportsFolder);
const outputPath = `${this.settings.exportsFolder}/${rawBasename}.pseudonymized.vtt`;
const existing = this.app.vault.getAbstractFileByPath(outputPath);
if (existing instanceof TFile) {
await this.app.vault.modify(existing, vtt);
} else {
await this.app.vault.create(outputPath, vtt);
}
new Notice(t('notice.vttExported', outputPath));
}
/** Cherche <basename>.words.json dans le dossier mappings et ses sous-dossiers. */
private async findWordsJson(basename: string): Promise<string | null> {
const filename = `${basename}.words.json`;
// Chercher dans l'ensemble du dossier mappings
const search = (folder: TFolder): TFile | null => {
for (const child of folder.children) {
if (child instanceof TFile && child.name === filename) return child;
if (child instanceof TFolder) {
const found = search(child);
if (found) return found;
}
}
return null;
};
const mappingRoot = this.app.vault.getAbstractFileByPath(this.settings.mappingFolder);
if (!(mappingRoot instanceof TFolder)) return null;
const wordsFile = search(mappingRoot);
return wordsFile ? this.app.vault.read(wordsFile) : null;
}
private async scanCurrentFile(): Promise<void> {
const file = this.app.workspace.getActiveFile();
if (!file) { new Notice(t('notice.noActiveFile')); return; }
@ -632,14 +898,19 @@ export default class PseudObsPlugin extends Plugin {
return;
}
const countByRule = new Map<string, number>();
const occsByRule = new Map<string, Occurrence[]>();
for (const occ of occurrences) {
const id = occ.mappingId ?? '';
countByRule.set(id, (countByRule.get(id) ?? 0) + 1);
if (!occsByRule.has(id)) occsByRule.set(id, []);
occsByRule.get(id)!.push(occ);
}
const ruleResults: MappingRuleResult[] = rules
.filter((r) => countByRule.has(r.id))
.map((r) => ({ rule: r, matchCount: countByRule.get(r.id)! }));
.filter((r) => occsByRule.has(r.id))
.map((r) => ({
rule: r,
matchCount: occsByRule.get(r.id)!.length,
occurrences: occsByRule.get(r.id)!,
}));
new MappingScanReviewModal(this.app, this, file, content, ruleResults).open();
}
@ -694,6 +965,50 @@ export default class PseudObsPlugin extends Plugin {
return spans.length;
}
/**
* Annule l'application d'une règle dans le fichier actif :
* cherche le remplacement (avec ou sans marqueurs) et le réécrit avec la source.
* Appelé automatiquement à la suppression d'une règle dans EditRuleModal.
*/
async revertRuleInFile(source: string, replacement: string): Promise<void> {
const file = this.app.workspace.getActiveFile();
if (!file) return;
const s = this.settings;
// Chercher la version avec marqueurs EN PREMIER — sinon on trouve le texte
// à l'intérieur des marqueurs et on obtient {{source}} au lieu de source.
const variants: string[] = [];
if (s.useMarkerInExport) {
variants.push(`${s.markerOpen}${replacement}${s.markerClose}`);
}
variants.push(replacement); // version sans marqueurs en dernier
let content = await this.app.vault.read(file);
let changed = false;
for (const variant of variants) {
const fakeRule: MappingRule = {
id: '_revert', source: variant, replacement: source, category: 'custom',
scope: { type: 'file', path: file.path }, status: 'validated',
priority: 0, createdBy: 'user', createdAt: new Date().toISOString(),
};
const spans = findSpansForRule(content, fakeRule, {
caseSensitive: false,
wholeWordOnly: false, // le remplacement peut contenir des 🀫 ou marqueurs
});
if (spans.length > 0) {
spans.sort((a, b) => b.start - a.start);
content = applySpans(content, spans);
changed = true;
}
}
if (changed) {
await this.app.vault.modify(file, content);
void this.refresh();
}
}
// --- Utilitaires ---
async ensureFolder(folderPath: string): Promise<void> {

View file

@ -64,8 +64,13 @@ export class MappingStore {
// Règles validées applicables à un fichier donné (cascade file → folder → vault)
getValidatedFor(filePath: string): MappingRule[] {
// 'validated' et 'partial' sont des règles actives.
// 'ignored' = l'utilisateur a ignoré toutes les occurrences au dernier scan
// → la règle reste définie mais désactivée.
// 'suggested' = non encore confirmée → inactive.
const ACTIVE: Set<string> = new Set(['validated', 'partial']);
return this.getAll().filter((r) => {
if (r.status !== 'validated') return false;
if (!ACTIVE.has(r.status)) return false;
if (r.scope.type === 'vault') return true;
if (r.scope.type === 'folder') return filePath.startsWith(r.scope.path ?? '');
return r.scope.path === filePath;

View file

@ -17,16 +17,26 @@ export class ScopeResolver {
private mappingFolder: string
) {}
/** Collecte récursivement tous les fichiers .mapping.json dans un dossier. */
private collectMappingFiles(folder: TFolder): TFile[] {
const files: TFile[] = [];
for (const child of folder.children) {
if (child instanceof TFile && child.name.endsWith('.mapping.json')) {
files.push(child);
} else if (child instanceof TFolder) {
files.push(...this.collectMappingFiles(child));
}
}
return files;
}
async getRulesFor(filePath: string): Promise<MappingRule[]> {
const folder = this.vault.getAbstractFileByPath(this.mappingFolder);
if (!(folder instanceof TFolder)) return [];
const allRules: MappingRule[] = [];
for (const child of folder.children) {
if (!(child instanceof TFile)) continue;
if (!child.name.endsWith('.mapping.json')) continue;
for (const child of this.collectMappingFiles(folder)) {
try {
const raw = await this.vault.read(child);
const data = JSON.parse(raw) as MappingFile;
@ -56,8 +66,7 @@ export class ScopeResolver {
const needle = term.toLowerCase();
for (const child of folder.children) {
if (!(child instanceof TFile) || !child.name.endsWith('.mapping.json')) continue;
for (const child of this.collectMappingFiles(folder)) {
try {
const data = JSON.parse(await this.vault.read(child)) as MappingFile;
const store = MappingStore.fromJSON(data);
@ -82,8 +91,7 @@ export class ScopeResolver {
const result: RuleLocation[] = [];
for (const child of folder.children) {
if (!(child instanceof TFile) || !child.name.endsWith('.mapping.json')) continue;
for (const child of this.collectMappingFiles(folder)) {
try {
const data = JSON.parse(await this.vault.read(child)) as MappingFile;
const store = MappingStore.fromJSON(data);
@ -111,8 +119,7 @@ export class ScopeResolver {
const result: RuleLocation[] = [];
for (const child of folder.children) {
if (!(child instanceof TFile) || !child.name.endsWith('.mapping.json')) continue;
for (const child of this.collectMappingFiles(folder)) {
try {
const data = JSON.parse(await this.vault.read(child)) as MappingFile;
const store = MappingStore.fromJSON(data);

View file

@ -0,0 +1,150 @@
/**
* Parser HTML noScribe produit un VttDocument compatible avec VttParser/TranscriptConverter.
*
* Format noScribe HTML (Qt Rich Text) :
* - <meta name="audio_source"> : chemin vers le fichier audio source
* - Chaque <p> = un tour de parole ou une pause
* - Chaque <a name="ts_START_END_SPEAKER"> = un segment avec timestamps
* START et END sont en millisecondes depuis le début de l'audio
* - Le nom du locuteur précède " :" en début de tour
* - Les timestamps d'affichage [HH:MM:SS] (couleur #78909c) sont à ignorer
*/
import type { VttDocument, VttCue, VttWord } from './VttParser';
// ---- Regex ------------------------------------------------------------------
const PARA_RE = /<p(?:\s[^>]*)?>(?<content>.*?)<\/p>/gs;
const ANCHOR_RE = /<a\s+name="(ts_[^"]+)"[^>]*>(.*?)<\/a>/gs;
const TS_NAME_RE = /^ts_(\d+)_(\d+)_(\w*)$/;
// Timestamps d'affichage [HH:MM:SS] générés par noScribe pour l'interface — ignorer
const DISPLAY_TS_RE = /^\[\d{2}:\d{2}:\d{2}\]$/;
// Locuteur suivi de " :" en début de texte
const SPEAKER_RE = /^(.+?)\s*:\s*/;
const ALL_TAGS_RE = /<[^>]+>/g;
const HTML_ENTITIES: Record<string, string> = {
'&amp;': '&', '&lt;': '<', '&gt;': '>', '&quot;': '"',
'&#039;': "'", '&apos;': "'", '&nbsp;': ' ',
};
// ---- Helpers ----------------------------------------------------------------
function decodeEntities(text: string): string {
return text.replace(/&[^;]+;/g, (m) => HTML_ENTITIES[m] ?? m);
}
function stripTags(html: string): string {
return decodeEntities(html.replace(ALL_TAGS_RE, ''));
}
function pad2(n: number): string { return String(n).padStart(2, '0'); }
function pad3(n: number): string { return String(n).padStart(3, '0'); }
/** Convertit des millisecondes en timestamp HH:MM:SS.mmm. */
function msToTimestamp(ms: number): string {
const h = Math.floor(ms / 3600000);
const m = Math.floor((ms % 3600000) / 60000);
const s = Math.floor((ms % 60000) / 1000);
const mss = ms % 1000;
return `${pad2(h)}:${pad2(m)}:${pad2(s)}.${pad3(mss)}`;
}
// ---- Parser -----------------------------------------------------------------
export class NoScribeHtmlParser {
/** Vérifie qu'un contenu HTML est une sortie noScribe. */
static isNoScribeHtml(content: string): boolean {
return content.includes('qrichtext') && content.includes('<a name="ts_');
}
/** Extrait le chemin absolu du fichier audio depuis la meta tag noScribe. */
static extractAudioSource(content: string): string | null {
const m = /meta\s+name="audio_source"\s+content="([^"]+)"/.exec(content);
return m ? m[1] : null;
}
parse(content: string): VttDocument {
const cues: VttCue[] = [];
PARA_RE.lastIndex = 0;
let paraMatch: RegExpExecArray | null;
while ((paraMatch = PARA_RE.exec(content)) !== null) {
const paraHtml = paraMatch[1];
if (!paraHtml.includes('ts_')) continue;
// Extraire et grouper les ancres par nom de timestamp (plusieurs ancres peuvent
// partager le même ts_ pour des raisons de mise en forme Qt)
const grouped = new Map<string, {
startMs: number; endMs: number; speakerId: string; texts: string[];
}>();
const groupOrder: string[] = [];
ANCHOR_RE.lastIndex = 0;
let anchorMatch: RegExpExecArray | null;
while ((anchorMatch = ANCHOR_RE.exec(paraHtml)) !== null) {
const tsName = anchorMatch[1];
const innerHtml = anchorMatch[2];
const tsMatch = TS_NAME_RE.exec(tsName);
if (!tsMatch) continue;
const text = stripTags(innerHtml);
if (DISPLAY_TS_RE.test(text.trim())) continue;
if (!grouped.has(tsName)) {
grouped.set(tsName, {
startMs: parseInt(tsMatch[1], 10),
endMs: parseInt(tsMatch[2], 10),
speakerId: tsMatch[3],
texts: [],
});
groupOrder.push(tsName);
}
if (text) grouped.get(tsName)!.texts.push(text);
}
if (groupOrder.length === 0) continue;
// Construire les VttWords à partir des groupes
const rawWords: VttWord[] = [];
for (const tsName of groupOrder) {
const g = grouped.get(tsName)!;
const text = g.texts.join('');
if (text.trim()) rawWords.push({ text, time: msToTimestamp(g.startMs) });
}
if (rawWords.length === 0) continue;
// Extraire le locuteur depuis le début du premier mot
let speaker: string | undefined;
const firstText = rawWords[0].text;
const speakerMatch = SPEAKER_RE.exec(firstText);
if (speakerMatch) {
const candidate = speakerMatch[1].trim();
// Pas de locuteur si c'est juste une marque de pause
if (candidate && !/^\(\.+\)$/.test(candidate)) {
speaker = candidate;
rawWords[0] = { ...rawWords[0], text: firstText.slice(speakerMatch[0].length) };
}
}
const words = rawWords.filter((w) => w.text.trim().length > 0);
if (words.length === 0) continue;
const startMs = grouped.get(groupOrder[0])!.startMs;
const endMs = grouped.get(groupOrder[groupOrder.length - 1])!.endMs;
cues.push({
startTime: msToTimestamp(startMs),
endTime: msToTimestamp(endMs),
speaker,
text: words.map((w) => w.text).join(''),
words,
rawLines: [],
});
}
return { cues, trailingNewline: false };
}
}

View file

@ -0,0 +1,209 @@
/**
* Parser VTT noScribe format produit par noScribe 0.7.
*
* Le VTT noScribe n'est PAS du WebVTT standard : chaque tour de parole est
* décomposé en plusieurs cues successives ayant souvent le même intervalle :
* - cue "label" : "<v SXX>SXX: " (détection du locuteur)
* - cue "display" : "<v SXX>[HH:MM:SS]" (timestamp lisible ignorer)
* - cue "texte" : "<v SXX>contenu transcrit" (texte réel)
*
* Parfois le label et le texte sont fusionnés :
* "S01: [00:00:09] Vous avez fini ces rapports ?"
*
* Contraintes :
* - Le tag <v SXX> n'est pas fiable pour l'attribution des locuteurs.
* - Le timestamp des cues "label" est erroné ; utiliser celui de la cue texte.
* - Pas de word-level timestamps (contrairement au VTT Whisper).
*
* Produit un VttDocument compatible avec vttToMarkdown / extractWordData.
*/
import type { VttDocument, VttCue } from './VttParser';
// ---- Regex ------------------------------------------------------------------
// Ligne de timestamp VTT
const TIMESTAMP_LINE_RE = /^(\d{2}:\d{2}:\d{2}\.\d{3})\s+-->\s+(\d{2}:\d{2}:\d{2}\.\d{3})/;
// Tag <v SXX> noScribe
const SPEAKER_V_TAG_RE = /^<v\s+\w+>/;
// Label locuteur en début de texte : "S00: " ou "S00 : "
const SPEAKER_LABEL_RE = /^(S\d+)\s*:\s*/;
// Timestamp d'affichage : [HH:MM:SS]
const DISPLAY_TS_RE = /^\[\d{2}:\d{2}:\d{2}\]\s*/;
// Entités HTML basiques
const HTML_ENTITIES: Record<string, string> = {
'&#x27;': "'", '&#39;': "'", '&amp;': '&',
'&lt;': '<', '&gt;': '>', '&quot;': '"', '&nbsp;': ' ',
};
function decodeEntities(text: string): string {
return text.replace(/&#?x?[0-9a-zA-Z]+;/g, (m) => HTML_ENTITIES[m] ?? m);
}
function stripSpeakerVTag(line: string): string {
return SPEAKER_V_TAG_RE.test(line) ? line.replace(SPEAKER_V_TAG_RE, '') : line;
}
// ---- Types intermédiaires ---------------------------------------------------
interface RawCue {
startTime: string;
endTime: string;
text: string; // texte nettoyé (sans <v>), entités décodées
}
interface Fragment {
startTime: string;
endTime: string;
speaker: string | undefined;
text: string;
}
// ---- Parser -----------------------------------------------------------------
export class NoScribeVttParser {
/** Détecte un VTT produit par noScribe (présence de NOTE noScribe). */
static isNoScribeVtt(content: string): boolean {
return content.startsWith('WEBVTT') && content.includes('noScribe');
}
/** Extrait le chemin audio depuis la ligne NOTE media. */
static extractAudioSource(content: string): string | null {
const m = /^NOTE\s+media:\s*(.+)$/m.exec(content);
return m ? m[1].trim() : null;
}
parse(content: string): VttDocument {
const rawCues = this.parseRawCues(content);
const fragments = this.buildFragments(rawCues);
const cues = this.mergeIntoCues(fragments);
return { cues, trailingNewline: content.endsWith('\n') };
}
// --- Étape 1 : parser les cues brutes ------------------------------------
private parseRawCues(content: string): RawCue[] {
const normalized = content.replace(/\r\n/g, '\n').replace(/\r/g, '\n');
const lines = normalized.split('\n');
const cues: RawCue[] = [];
let i = 0;
// Sauter jusqu'à la première ligne de timestamp (ignore header + NOTE)
while (i < lines.length) {
const tsMatch = TIMESTAMP_LINE_RE.exec(lines[i]);
if (tsMatch) {
const startTime = tsMatch[1];
const endTime = tsMatch[2];
i++;
// Collecter les lignes de texte jusqu'à la ligne vide
const textLines: string[] = [];
while (i < lines.length && lines[i].trim() !== '') {
const cleaned = decodeEntities(stripSpeakerVTag(lines[i]));
textLines.push(cleaned);
i++;
}
const text = textLines.join(' ').trim();
cues.push({ startTime, endTime, text });
} else {
i++;
}
}
return cues;
}
// --- Étape 2 : convertir en fragments (speaker + text + timestamp) --------
private buildFragments(cues: RawCue[]): Fragment[] {
const fragments: Fragment[] = [];
let currentSpeaker: string | undefined;
for (const cue of cues) {
let text = cue.text;
// Extraire le label locuteur s'il est présent
const labelMatch = SPEAKER_LABEL_RE.exec(text);
if (labelMatch) {
currentSpeaker = labelMatch[1]; // "S00", "S01", etc.
text = text.slice(labelMatch[0].length);
}
// Supprimer le timestamp d'affichage [HH:MM:SS]
text = text.replace(DISPLAY_TS_RE, '').trim();
// Ignorer les cues vides, les cues label seuls, et les timestamps seuls
if (!text || /^\[[\d:]+\]$/.test(text)) continue;
fragments.push({
startTime: cue.startTime,
endTime: cue.endTime,
speaker: currentSpeaker,
text,
});
}
return fragments;
}
// --- Étape 3 : fusionner les fragments en VttCues -------------------------
//
// Un "tour" commence à chaque nouveau label de locuteur dans l'étape 2.
// Les fragments sans changement de locuteur sont rattachés au tour courant.
// Heuristique de fusion : on regroupe les fragments temporellement proches
// (gap < 2 s) appartenant au même locuteur.
private mergeIntoCues(fragments: Fragment[]): VttCue[] {
if (fragments.length === 0) return [];
const cues: VttCue[] = [];
let batch: Fragment[] = [fragments[0]];
const flushBatch = () => {
if (batch.length === 0) return;
const text = batch.map((f) => f.text).join(' ').replace(/\s+/g, ' ').trim();
if (text) {
cues.push({
startTime: batch[0].startTime,
endTime: batch[batch.length - 1].endTime,
speaker: batch[0].speaker,
text,
words: [{ text, time: batch[0].startTime }],
rawLines: [],
});
}
batch = [];
};
for (let i = 1; i < fragments.length; i++) {
const prev = batch[batch.length - 1];
const curr = fragments[i];
// Nouveau tour si le locuteur change
const speakerChanged = curr.speaker !== prev.speaker;
// Ou si le gap temporel est significatif (> 2 s) entre deux fragments du même locuteur
const prevEndSec = timeToSeconds(prev.endTime);
const currStartSec = timeToSeconds(curr.startTime);
const bigGap = !speakerChanged && (currStartSec - prevEndSec) > 2;
if (speakerChanged || bigGap) {
flushBatch();
batch = [curr];
} else {
batch.push(curr);
}
}
flushBatch();
return cues;
}
}
function timeToSeconds(ts: string): number {
const [h, m, s] = ts.split(':');
return parseInt(h) * 3600 + parseInt(m) * 60 + parseFloat(s);
}

View file

@ -1,5 +1,6 @@
import type { SrtDocument } from './SrtParser';
import type { ChatDocument, ChatLine } from './ChatParser';
import type { VttDocument, VttWord } from './VttParser';
// Converts a SrtDocument to a structured, Obsidian-readable Markdown string.
// Timestamps are preserved as italic headers; text lines are the editable content.
@ -19,7 +20,6 @@ export function srtToMarkdown(doc: SrtDocument, sourceName: string): string {
lines.push('');
}
// Retirer la dernière ligne vide superflue, puis ajouter \n final
while (lines[lines.length - 1] === '') lines.pop();
lines.push('');
@ -43,7 +43,6 @@ export function chatToMarkdown(doc: ChatDocument, sourceName: string): string {
for (const chatLine of doc.lines) {
const group = lineGroup(chatLine);
// Ligne vide entre groupes structurels et tours de parole
if (prevGroup !== null && prevGroup !== group) {
lines.push('');
}
@ -57,13 +56,11 @@ export function chatToMarkdown(doc: ChatDocument, sourceName: string): string {
lines.push(`**${chatLine.speaker}** : ${chatLine.content}`);
break;
case 'continuation':
// Rattacher au tour précédent (rare en pratique)
if (lines.length > 0) {
lines[lines.length - 1] += ` ${chatLine.raw.trim()}`;
}
break;
case 'blank':
// Les blancs du source sont absorbés par la logique de groupe ci-dessus
break;
}
@ -76,6 +73,136 @@ export function chatToMarkdown(doc: ChatDocument, sourceName: string): string {
return lines.join('\n');
}
// ---- VTT / noScribe HTML ---------------------------------------------------
/** Données word-level d'une cue — stockées dans <basename>.words.json. */
export interface VttCueData {
index: number;
startTime: string;
endTime: string;
speaker?: string;
words: VttWord[];
}
/**
* Extrait les données word-level d'un VttDocument pour le fichier .words.json.
* Seules les cues avec au moins un word timestamp sont incluses.
*/
export function extractWordData(doc: VttDocument): VttCueData[] {
return doc.cues
.map((cue, index) => ({
index,
startTime: cue.startTime,
endTime: cue.endTime,
speaker: cue.speaker,
words: cue.words,
}))
.filter((c) => c.words.some((w) => w.time !== ''));
}
// ---- Re-export VTT ----------------------------------------------------------
/**
* Regex pour une ligne de corps noScribe dans le Markdown.
* Capture optionnellement le locuteur (groupe 1) et le texte (groupe 2).
* **S00** [00:01:28] : texte speaker=S00, text=texte
* [00:00:00] (..) speaker=undefined, text=(..)
*/
const MD_CUE_RE = /^(?:\*\*([^*]+)\*\*\s+)?\[[\d:]+\]\s*(?::\s*)?(.*)$/;
/**
* Reconstruit un fichier WebVTT pseudonymisé depuis :
* - `mdContent` : le Markdown noScribe (pseudonymisé)
* - `wordData` : le contenu de <basename>.words.json (timestamps précis)
*
* Alignement par index : ligne de corps N = cue N du words.json.
* Le texte vient du Markdown ; les timestamps et le speaker de secours
* viennent du words.json.
*
* Retourne null si le nombre de lignes et de cues ne correspondent pas
* (le caller affiche un avertissement dans ce cas).
*/
export function markdownToVtt(
mdContent: string,
wordData: VttCueData[],
): { vtt: string; mismatch: boolean } {
// Extraire le corps (après le frontmatter --- ... ---)
const bodyMatch = /^---\n[\s\S]*?\n---\n+([\s\S]*)$/.exec(mdContent);
const body = bodyMatch ? bodyMatch[1] : mdContent;
const cueLines = body.split('\n').filter((l) => MD_CUE_RE.test(l.trim()));
const mismatch = cueLines.length !== wordData.length;
const parts = ['WEBVTT', ''];
const count = Math.min(cueLines.length, wordData.length);
for (let i = 0; i < count; i++) {
const m = MD_CUE_RE.exec(cueLines[i].trim())!;
// Locuteur : priorité au Markdown, fallback words.json
const speaker = m[1]?.trim() || wordData[i].speaker;
const text = m[2]?.trim() ?? '';
if (!text) continue;
parts.push(String(i + 1));
parts.push(`${wordData[i].startTime} --> ${wordData[i].endTime}`);
parts.push(speaker ? `<v ${speaker}>${text}` : text);
parts.push('');
}
return { vtt: parts.join('\n'), mismatch };
}
/** Convertit HH:MM:SS.mmm en [HH:MM:SS] — format d'affichage noScribe. */
function displayTime(ts: string): string {
return `[${ts.slice(0, 8)}]`;
}
/**
* Convertit un VttDocument en Markdown structuré au format noScribe.
*
* Format par cue :
* **Locuteur** [HH:MM:SS] : texte
* [HH:MM:SS] texte (sans locuteur)
*
* Les timestamps précis (avec ms) et les word timestamps sont dans .words.json,
* pas dans le corps Markdown. Le frontmatter peut inclure pseudobs-audio si
* un fichier audio a é importé avec la transcription.
*/
function vttDocToMarkdown(
doc: VttDocument,
sourceName: string,
format: 'vtt' | 'html',
audioFilename?: string,
): string {
const lines: string[] = ['---', `pseudobs-format: ${format}`, `pseudobs-source: "${sourceName}"`];
if (audioFilename) lines.push(`pseudobs-audio: "${audioFilename}"`);
lines.push('---', '');
for (const cue of doc.cues) {
const ts = displayTime(cue.startTime);
if (cue.speaker) {
lines.push(`**${cue.speaker}** ${ts} : ${cue.text}`);
} else {
lines.push(`${ts} ${cue.text}`);
}
lines.push('');
}
while (lines[lines.length - 1] === '') lines.pop();
lines.push('');
return lines.join('\n');
}
export function vttToMarkdown(doc: VttDocument, sourceName: string, audioFilename?: string): string {
return vttDocToMarkdown(doc, sourceName, 'vtt', audioFilename);
}
export function noScribeHtmlToMarkdown(doc: VttDocument, sourceName: string, audioFilename?: string): string {
return vttDocToMarkdown(doc, sourceName, 'html', audioFilename);
}
function lineGroup(line: ChatLine): 'structural' | 'turn' | null {
if (line.type === 'meta' || line.type === 'dependent') return 'structural';
if (line.type === 'turn' || line.type === 'continuation') return 'turn';

250
src/parsers/VttParser.ts Normal file
View file

@ -0,0 +1,250 @@
/**
* Parser VTT (WebVTT) optimisé pour les sorties noScribe (Whisper + pyannote).
*
* Formats pris en charge :
* - VTT standard (blocs sans word timestamps)
* - Whisper word timestamps : <HH:MM:SS.mmm><c>mot</c>
* - Speaker tags : <v SPEAKER_NAME> ou [SPEAKER_NAME]
*
* Round-trip garanti : les word timestamps sont stockés séparément du texte
* et réinsérés à la reconstruction autour du texte (éventuellement pseudonymisé).
*/
import type { BaseTranscriptParser } from './BaseTranscriptParser';
/** Un mot avec sa position temporelle dans l'audio. */
export interface VttWord {
text: string; // texte du mot (tel qu'il apparaît dans le fichier, espaces inclus)
time: string; // timestamp Whisper "HH:MM:SS.mmm" — vide si absent
}
export interface VttCue {
id?: string; // identifiant optionnel de la cue (nombre ou chaîne)
startTime: string; // "HH:MM:SS.mmm"
endTime: string; // "HH:MM:SS.mmm"
speaker?: string; // locuteur extrait du tag <v ...> ou [...]
text: string; // texte nettoyé (pseudonymisable)
words: VttWord[]; // mots avec timestamps — vide si pas de word timestamps
// Ligne(s) de texte brut original (pour le round-trip si pas de word timestamps)
rawLines: string[];
}
export interface VttDocument {
cues: VttCue[];
trailingNewline: boolean;
}
// ---- Regex ------------------------------------------------------------------
// Timestamp VTT : HH:MM:SS.mmm ou MM:SS.mmm
const TIME_RE = /(\d{1,2}:\d{2}:\d{2}\.\d{3}|\d{2}:\d{2}\.\d{3})/;
const TIMESTAMP_LINE_RE = new RegExp(
`^${TIME_RE.source}\\s+-->\\s+${TIME_RE.source}\\s*(.*)$`
);
// Word timestamp Whisper : <HH:MM:SS.mmm>
const WORD_TIME_RE = /<(\d{1,2}:\d{2}:\d{2}\.\d{3}|\d{2}:\d{2}\.\d{3})>/g;
// Tag de classe Whisper : <c>...</c>
const CLASS_TAG_RE = /<\/?c>/g;
// Speaker tag : <v Nom> ou <v.classname Nom>
const SPEAKER_V_RE = /^<v(?:\.[^\s>]+)?\s+([^>]+)>/;
// Speaker bracket : [Nom]
const SPEAKER_BRACKET_RE = /^\[([^\]]+)\]/;
// Tout tag HTML/VTT générique
const ALL_TAGS_RE = /<[^>]+>/g;
// ---- Helpers ----------------------------------------------------------------
/** Normalise un timestamp "MM:SS.mmm" en "HH:MM:SS.mmm". */
function normalizeTime(t: string): string {
return t.includes(':') && t.split(':').length === 2 ? `00:${t}` : t;
}
/** Retire tous les tags VTT du texte pour obtenir le texte brut lisible. */
function stripTags(text: string): string {
return text.replace(ALL_TAGS_RE, '').trim();
}
/**
* Extrait les mots avec leurs timestamps Whisper depuis une ligne de texte cue.
* Si aucun timestamp n'est présent, retourne un tableau avec le texte complet.
*/
function extractWords(rawText: string): VttWord[] {
// Vérifier si des word timestamps sont présents
const hasWordTimes = WORD_TIME_RE.test(rawText);
WORD_TIME_RE.lastIndex = 0; // reset après test
if (!hasWordTimes) {
const clean = stripTags(rawText);
return clean ? [{ text: clean, time: '' }] : [];
}
// Découper en segments : [timestamp, texte, timestamp, texte, ...]
const words: VttWord[] = [];
const remaining = rawText;
let currentTime = '';
// Traiter segment par segment
const parts = remaining.split(WORD_TIME_RE);
// parts alterne : [texte_avant_premier_ts, ts1, texte1, ts2, texte2, ...]
for (let i = 0; i < parts.length; i++) {
const part = parts[i];
if (TIME_RE.test(part) && part.match(/^\d/)) {
// C'est un timestamp
currentTime = normalizeTime(part);
} else {
// C'est du texte — nettoyer les tags de classe
const clean = part.replace(CLASS_TAG_RE, '').replace(ALL_TAGS_RE, '');
if (clean.trim()) {
words.push({ text: clean, time: currentTime });
currentTime = '';
}
}
}
return words;
}
/**
* Extrait le locuteur et le texte brut (sans speaker tag) d'une ligne cue.
*/
function extractSpeakerAndText(line: string): { speaker: string | undefined; text: string } {
const vMatch = SPEAKER_V_RE.exec(line);
if (vMatch) {
return { speaker: vMatch[1].trim(), text: line.slice(vMatch[0].length) };
}
const bMatch = SPEAKER_BRACKET_RE.exec(line);
if (bMatch) {
return { speaker: bMatch[1].trim(), text: line.slice(bMatch[0].length).trim() };
}
return { speaker: undefined, text: line };
}
// ---- Parser -----------------------------------------------------------------
export class VttParser implements BaseTranscriptParser<VttDocument> {
parse(content: string): VttDocument {
const trailingNewline = content.endsWith('\n');
const normalized = content.replace(/\r\n/g, '\n').replace(/\r/g, '\n').trimEnd();
const lines = normalized.split('\n');
let i = 0;
// Sauter le header WEBVTT (et les métadonnées éventuelles)
while (i < lines.length && !lines[i].startsWith('WEBVTT')) i++;
i++; // sauter la ligne WEBVTT elle-même
const cues: VttCue[] = [];
while (i < lines.length) {
// Sauter les lignes vides
while (i < lines.length && lines[i].trim() === '') i++;
if (i >= lines.length) break;
// ID optionnel (ligne non-timestamp, non-vide avant le timestamp)
let cueId: string | undefined;
if (i < lines.length && !TIMESTAMP_LINE_RE.test(lines[i])) {
cueId = lines[i].trim();
i++;
}
// Ligne timestamp
if (i >= lines.length) break;
const tsMatch = TIMESTAMP_LINE_RE.exec(lines[i]);
if (!tsMatch) { i++; continue; } // ligne inattendue
const startTime = normalizeTime(tsMatch[1]);
const endTime = normalizeTime(tsMatch[2]);
i++;
// Lignes de texte de la cue
const rawLines: string[] = [];
while (i < lines.length && lines[i].trim() !== '') {
rawLines.push(lines[i]);
i++;
}
if (rawLines.length === 0) continue;
// Extraire locuteur depuis la première ligne
const firstLineResult = extractSpeakerAndText(rawLines[0]);
const speaker = firstLineResult.speaker;
// Reconstruire le texte complet de la cue (sans speaker tag sur la 1ère ligne)
const textLines = [firstLineResult.text, ...rawLines.slice(1)];
const fullRaw = textLines.join('\n');
// Extraire les word timestamps
const words = extractWords(fullRaw);
// Texte nettoyé pour pseudonymisation
const text = words.map((w) => w.text).join('');
cues.push({ id: cueId, startTime, endTime, speaker, text, words, rawLines });
}
return { cues, trailingNewline };
}
reconstruct(doc: VttDocument): string {
const parts: string[] = ['WEBVTT', ''];
for (const cue of doc.cues) {
if (cue.id !== undefined) parts.push(cue.id);
parts.push(`${cue.startTime} --> ${cue.endTime}`);
if (cue.words.length > 0 && cue.words.some((w) => w.time !== '')) {
// Reconstruire avec word timestamps depuis le tableau words
const line = this.reconstructWithWordTimestamps(cue);
parts.push(line);
} else {
// Pas de word timestamps — reconstruire depuis le texte pseudonymisé
// en préservant le speaker tag si présent
const speaker = cue.speaker;
const textLine = speaker ? `<v ${speaker}>${cue.text}` : cue.text;
parts.push(textLine);
}
parts.push('');
}
const body = parts.join('\n');
return doc.trailingNewline ? body : body.trimEnd();
}
private reconstructWithWordTimestamps(cue: VttCue): string {
let line = cue.speaker ? `<v ${cue.speaker}>` : '';
for (const word of cue.words) {
if (word.time) line += `<${word.time}>`;
line += `<c>${word.text}</c>`;
}
return line;
}
/**
* Met à jour le texte d'une cue après pseudonymisation.
* Propage le remplacement dans le tableau words pour maintenir le round-trip.
*/
static applyTextToWords(cue: VttCue, newText: string): void {
cue.text = newText;
// Si pas de word timestamps, on met tout dans le premier mot
if (cue.words.length === 0 || cue.words.every((w) => w.time === '')) {
cue.words = [{ text: newText, time: '' }];
return;
}
// Avec word timestamps : le remplacement affecte potentiellement plusieurs mots.
// Stratégie simple : on remplace le texte du premier mot et on vide les suivants.
// Une future version pourrait aligner mot à mot via diff.
cue.words[0].text = newText;
for (let i = 1; i < cue.words.length; i++) {
cue.words[i].text = '';
}
}
}

View file

@ -0,0 +1,31 @@
export const REDACTION_CHAR = '🀫';
/**
* Compte les syllabes d'un texte par heuristique de groupes de voyelles.
* Fonctionne pour le français et les principales langues romanes/germaniques.
* Minimum 1 syllabe retourné.
*/
export function countSyllables(text: string): number {
const groups = text.match(/[aeiouyàâäéèêëîïôùûüœæAEIOUYÀÂÄÉÈÊËÎÏÔÙÛÜŒÆ]+/g);
return Math.max(1, groups?.length ?? 1);
}
/**
* Génère un remplacement caviardé : 1 🀫 par syllabe de chaque mot.
* Les espaces entre mots sont préservés.
* Ex : "Marie Dupont" "🀫🀫 🀫🀫🀫" · "Saint-Jean" "🀫🀫🀫"
*/
export function generateRedaction(text: string): string {
return text
.split(/( +)/) // séparer sur les espaces en les conservant
.map((part) => {
if (/^ +$/.test(part)) return part; // préserver les espaces tels quels
return REDACTION_CHAR.repeat(countSyllables(part));
})
.join('');
}
/** Retourne true si un remplacement est un caviardage (commence par 🀫). */
export function isRedaction(replacement: string): boolean {
return replacement.startsWith(REDACTION_CHAR);
}

View file

@ -18,6 +18,39 @@ const TAG_TO_CATEGORY: Record<string, EntityCategory> = {
const MIN_ENTITY_LENGTH = 2;
// Préambules Markdown à stripper avant envoi au modèle NER.
// Chaque regex matche le début d'une ligne (format produit par les parsers du plugin).
// Le résultat : { cleanText, preambleLength } pour recaler les positions.
const PREAMBLE_PATTERNS: RegExp[] = [
// VTT noScribe : *HH:MM:SS.mmm → HH:MM:SS.mmm* **SPEAKER**
/^\*[\d:.]+\s*→\s*[\d:.]+\*(?:\s*\*\*[^*]+\*\*)?\s*/,
// SRT bloc de texte : **[N]** *HH:MM:SS,mmm → HH:MM:SS,mmm* (ligne à ignorer entièrement)
/^\*\*\[\d+\]\*\*\s*\*[^*]+\*/,
// CHAT tour de parole : **SPEAKER** :
/^\*\*[A-Z0-9_]+\*\*\s*:\s*/,
];
// Lignes à ignorer entièrement (pas de texte à analyser)
const SKIP_PATTERNS: RegExp[] = [
/^---/, // frontmatter YAML
/^pseudobs-/, // clés frontmatter du plugin
/^> /, // métadonnées CHAT (@, %)
/^\*\*\[\d+\]\*\*/, // en-tête de bloc SRT (index + timestamp sur la même ligne)
];
function stripMarkdownPreamble(line: string): { cleanText: string; preambleLength: number } {
// Lignes à ignorer complètement
for (const skip of SKIP_PATTERNS) {
if (skip.test(line)) return { cleanText: '', preambleLength: line.length };
}
// Stripper le préambule
for (const pat of PREAMBLE_PATTERNS) {
const m = pat.exec(line);
if (m) return { cleanText: line.slice(m[0].length), preambleLength: m[0].length };
}
return { cleanText: line, preambleLength: 0 };
}
type NerResult = {
entity_group: string;
score: number;
@ -103,6 +136,9 @@ export class OnnxNerScanner {
// numThreads=1 : SharedArrayBuffer non disponible dans Electron renderer
// sans COOP/COEP → pas de WASM threadé.
env.backends.onnx.wasm.numThreads = 1;
// proxy=true : délègue la compilation WASM à un worker thread Node.js
// → libère le thread renderer pendant l'initialisation du modèle.
env.backends.onnx.wasm.proxy = true;
}
// Autoriser le téléchargement depuis HuggingFace Hub
@ -147,18 +183,20 @@ export class OnnxNerScanner {
let offset = 0;
for (const line of lines) {
if (line.trim().length > 2) {
const { cleanText, preambleLength } = stripMarkdownPreamble(line);
if (cleanText.trim().length > 2) {
try {
const entities: NerResult[] = await _pipeline(line, { aggregation_strategy: 'simple' });
const entities: NerResult[] = await _pipeline(cleanText, { aggregation_strategy: 'simple' });
for (const ent of entities) {
if (ent.score < minScore) continue;
const word = ent.word.trim().replace(/^#+/, ''); // supprimer les ## de sous-mots
// Filtrer les artefacts de tokenisation BERT (mots fonctionnels, tokens trop courts)
const word = ent.word.trim().replace(/^#+/, '');
if (word.length < MIN_ENTITY_LENGTH) continue;
if (functionWords.has(word.toLowerCase())) continue;
const category = TAG_TO_CATEGORY[ent.entity_group] ?? 'custom';
const start = offset + ent.start;
const end = offset + ent.end;
// Positions recalées : offset de ligne + préambule strippé + position dans cleanText
const start = offset + preambleLength + ent.start;
const end = offset + preambleLength + ent.end;
const ctxLen = 45;
results.push({
id: `ner_${Date.now()}_${++_counter}`,

View file

@ -35,6 +35,13 @@ export interface OccurrenceRef {
status: MappingStatus;
}
/** Occurrence d'une règle explicitement ignorée par l'utilisateur. */
export interface IgnoredOccurrence {
text: string; // texte exact trouvé dans le document (sensible à la casse)
contextBefore: string;
contextAfter: string;
}
export interface MappingRule {
id: string;
source: string;
@ -50,6 +57,7 @@ export interface MappingRule {
updatedAt?: string;
notes?: string;
occurrences?: OccurrenceRef[];
ignoredOccurrences?: IgnoredOccurrence[];
}
export interface Occurrence {

View file

@ -248,7 +248,7 @@ export class DictScanReviewModal extends Modal {
const n = toCreate.length;
new Notice(t('notice.rulesCreated', String(n), n > 1 ? t('notice.rulesCreated.rules') : t('notice.rulesCreated.rule')));
void this.plugin.refreshHighlightData();
void this.plugin.refresh();
this.close();
}

View file

@ -128,7 +128,7 @@ export class EditRuleModal extends Modal {
await this.plugin.scopeResolver.saveStore(store, filePath);
new Notice(t('notice.ruleCreated', rule.source, this.replacement.trim()));
void this.plugin.refreshHighlightData();
void this.plugin.refresh();
this.close();
}
@ -136,8 +136,9 @@ export class EditRuleModal extends Modal {
const { store, filePath, rule } = this.location;
store.remove(rule.id);
await this.plugin.scopeResolver.saveStore(store, filePath);
new Notice(`✓ "${rule.source}"`);
void this.plugin.refreshHighlightData();
// Rétablir le texte original dans le fichier actif
await this.plugin.revertRuleInFile(rule.source, rule.replacement);
new Notice(t('notice.ruleDeleted', rule.source));
this.close();
}

View file

@ -1,13 +1,16 @@
import { App, Modal, Notice, TFile } from 'obsidian';
import { t } from '../i18n';
import type PseudObsPlugin from '../main';
import type { MappingRule } from '../types';
import { findSpansForRule } from '../pseudonymizer/ReplacementPlanner';
import type { MappingRule, Occurrence } from '../types';
import { resolveSpans, applySpans } from '../pseudonymizer/SpanProtector';
import type { ReplacementSpan } from '../types';
import { OccurrencesContextModal, type OccurrenceDecision } from './OccurrencesContextModal';
import type { IgnoredOccurrence } from '../types';
export interface MappingRuleResult {
rule: MappingRule;
matchCount: number;
occurrences: Occurrence[];
}
export class MappingScanReviewModal extends Modal {
@ -16,7 +19,11 @@ export class MappingScanReviewModal extends Modal {
private content: string;
private ruleResults: MappingRuleResult[];
private checked: boolean[];
// Décisions par règle : ruleId → occId → decision
private decisionsMap = new Map<string, Map<string, OccurrenceDecision>>();
private applyBtn!: HTMLButtonElement;
// Cellules de comptage — pour mise à jour live
private countCells: HTMLElement[] = [];
constructor(
app: App,
@ -31,6 +38,13 @@ export class MappingScanReviewModal extends Modal {
this.content = content;
this.ruleResults = ruleResults;
this.checked = ruleResults.map(() => true);
// Décisions initiales : toutes les occurrences validées
for (const { rule, occurrences } of ruleResults) {
const map = new Map<string, OccurrenceDecision>();
for (const occ of occurrences) map.set(occ.id, 'validated');
this.decisionsMap.set(rule.id, map);
}
}
onOpen(): void {
@ -41,7 +55,9 @@ export class MappingScanReviewModal extends Modal {
contentEl.createEl('h2', { text: t('mappingScanModal.title') });
const nr = this.ruleResults.length;
contentEl.createEl('p', {
text: t('mappingScanModal.summary', String(nr), nr > 1 ? t('mappingScanModal.summary.rules') : t('mappingScanModal.summary.rule'), this.file.name),
text: t('mappingScanModal.summary', String(nr),
nr > 1 ? t('mappingScanModal.summary.rules') : t('mappingScanModal.summary.rule'),
this.file.name),
cls: 'pseudobs-scan-summary',
});
contentEl.createEl('p', { text: t('mappingScanModal.hint'), cls: 'pseudobs-view-hint' });
@ -56,7 +72,7 @@ export class MappingScanReviewModal extends Modal {
);
const tbody = table.createEl('tbody');
this.ruleResults.forEach(({ rule, matchCount }, i) => {
this.ruleResults.forEach(({ rule, occurrences }, i) => {
const tr = tbody.createEl('tr', { cls: 'pseudobs-dict-review-row' });
// Checkbox
@ -72,11 +88,9 @@ export class MappingScanReviewModal extends Modal {
// Source
tr.createEl('td', { text: rule.source, cls: 'pseudobs-dict-review-term' });
// Flèche
tr.createEl('td', { text: '→', cls: 'pseudobs-dict-review-arrow' });
// Remplacement (lecture seule — déjà défini dans la règle)
// Remplacement
const repCell = tr.createEl('td');
const s = this.plugin.settings;
const displayRep = s.useMarkerInExport
@ -84,68 +98,156 @@ export class MappingScanReviewModal extends Modal {
: rule.replacement;
repCell.createEl('span', { text: displayRep, cls: 'pseudobs-dict-review-rep-static' });
// Occurrences
tr.createEl('td', { text: String(matchCount), cls: 'pseudobs-dict-review-count' });
// Occurrences — bouton cliquable qui ouvre la modale de contexte
const countCell = tr.createEl('td', { cls: 'pseudobs-dict-review-count' });
this.countCells.push(countCell);
this.renderCountCell(countCell, rule, occurrences, i);
});
const footer = contentEl.createDiv('pseudobs-dict-review-footer');
footer.createEl('button', { text: t('mappingScanModal.cancel') })
.addEventListener('click', () => this.close());
footer.createEl('button', { text: t('mappingScanModal.saveExceptions'), cls: 'pseudobs-save-exceptions-btn' })
.addEventListener('click', () => void this.saveExceptions());
this.applyBtn = footer.createEl('button', { cls: 'mod-cta' });
this.applyBtn.addEventListener('click', () => void this.apply());
this.updateApplyLabel();
}
private renderCountCell(
cell: HTMLElement,
rule: MappingRule,
occurrences: Occurrence[],
ruleIndex: number
): void {
cell.empty();
const decisions = this.decisionsMap.get(rule.id)!;
const validated = occurrences.filter((o) => decisions.get(o.id) === 'validated').length;
const total = occurrences.length;
if (total === 0) {
cell.createSpan({ text: '0' });
return;
}
const btn = cell.createEl('button', { cls: 'pseudobs-count-btn' });
// Afficher "N / total" si des occurrences ont été ignorées, "N" sinon
btn.setText(validated < total ? `${validated} / ${total}` : String(total));
btn.title = 'Voir et sélectionner les candidats';
btn.addEventListener('click', () => {
new OccurrencesContextModal(
this.app,
rule,
occurrences,
decisions,
(newDecisions) => {
this.decisionsMap.set(rule.id, newDecisions);
this.renderCountCell(cell, rule, occurrences, ruleIndex);
this.updateApplyLabel();
}
).open();
});
}
private countValidated(): number {
return this.ruleResults.reduce((sum, { rule, occurrences }, i) => {
if (!this.checked[i]) return sum;
const decisions = this.decisionsMap.get(rule.id)!;
return sum + occurrences.filter((o) => decisions.get(o.id) === 'validated').length;
}, 0);
}
private updateApplyLabel(): void {
const n = this.checked.filter(Boolean).length;
const total = this.ruleResults
.filter((_, i) => this.checked[i])
.reduce((sum, r) => sum + r.matchCount, 0);
this.applyBtn.textContent = n === 0
const rules = this.checked.filter(Boolean).length;
const total = this.countValidated();
this.applyBtn.textContent = rules === 0
? t('mappingScanModal.noRules')
: t('mappingScanModal.apply',
String(n), n > 1 ? t('mappingScanModal.apply.rules') : t('mappingScanModal.apply.rule'),
String(rules), rules > 1 ? t('mappingScanModal.apply.rules') : t('mappingScanModal.apply.rule'),
String(total), total > 1 ? t('mappingScanModal.apply.occurrences') : t('mappingScanModal.apply.occurrence'));
this.applyBtn.toggleClass('pseudobs-dict-review-btn-empty', n === 0);
this.applyBtn.toggleClass('pseudobs-dict-review-btn-empty', rules === 0);
}
/** Enregistre les exceptions dans le mapping sans appliquer de remplacements. */
private async saveExceptions(): Promise<void> {
await this.persistIgnoredOccurrences();
new Notice(t('mappingScanModal.exceptionsSaved'));
this.close();
}
/**
* Persiste les occurrences ignorées ( et ) dans le mapping.json de chaque règle.
* S'appuie sur findRuleByTerm pour localiser le bon mapping file.
* Les nouvelles exceptions sont fusionnées avec les existantes (déduplication par texte).
*/
private async persistIgnoredOccurrences(): Promise<void> {
for (let i = 0; i < this.ruleResults.length; i++) {
const { rule, occurrences } = this.ruleResults[i];
const decisions = this.decisionsMap.get(rule.id)!;
const newIgnored: IgnoredOccurrence[] = occurrences
.filter((occ) => {
const d = decisions.get(occ.id) ?? 'validated';
return d === 'ignored' || d === 'false_positive';
})
.map((occ) => ({ text: occ.text, contextBefore: occ.contextBefore, contextAfter: occ.contextAfter }));
if (newIgnored.length === 0) continue;
const location = await this.plugin.scopeResolver.findRuleByTerm(rule.source);
if (!location) continue;
const existing = location.rule.ignoredOccurrences ?? [];
const existingTexts = new Set(existing.map((o) => o.text));
const merged = [...existing, ...newIgnored.filter((o) => !existingTexts.has(o.text))];
location.store.update(rule.id, { ignoredOccurrences: merged });
await this.plugin.scopeResolver.saveStore(location.store, location.filePath);
}
void this.plugin.refresh();
}
private async apply(): Promise<void> {
const checkedRules = this.ruleResults
.filter((_, i) => this.checked[i])
.map((r) => r.rule);
if (checkedRules.length === 0) { this.close(); return; }
this.applyBtn.setAttr('disabled', 'true');
const s = this.plugin.settings;
const marker = s.useMarkerInExport
? { open: s.markerOpen, close: s.markerClose }
: undefined;
const wrap = (r: string) => s.useMarkerInExport
? `${s.markerOpen}${r}${s.markerClose}` : r;
// Collecter tous les spans (toutes règles cochées), résoudre les chevauchements,
// appliquer de droite à gauche comme le moteur principal
const allSpans = checkedRules.flatMap((rule) =>
findSpansForRule(this.content, rule, {
caseSensitive: s.caseSensitive,
wholeWordOnly: s.wholeWordOnly,
}).map((span) =>
marker
? { ...span, replacement: `${marker.open}${span.replacement}${marker.close}` }
: span
)
);
const spans: ReplacementSpan[] = [];
for (let i = 0; i < this.ruleResults.length; i++) {
if (!this.checked[i]) continue;
const { rule, occurrences } = this.ruleResults[i];
const decisions = this.decisionsMap.get(rule.id)!;
const resolved = resolveSpans(allSpans);
if (resolved.length === 0) {
for (const occ of occurrences) {
if ((decisions.get(occ.id) ?? 'validated') !== 'validated') continue;
spans.push({
start: occ.start,
end: occ.end,
source: occ.text,
replacement: wrap(rule.replacement),
mappingId: rule.id,
priority: rule.priority,
});
}
}
if (spans.length === 0) {
new Notice(t('notice.noOccurrences'));
this.close();
return;
}
this.applyBtn.setAttr('disabled', 'true');
const resolved = resolveSpans(spans);
const modified = applySpans(this.content, resolved);
await this.app.vault.modify(this.file, modified);
void this.plugin.refreshHighlightData();
// Persister les exceptions ignorées dans le mapping
await this.persistIgnoredOccurrences();
const total = resolved.length;
new Notice(t('notice.occurrencesPseudonymized',

View file

@ -0,0 +1,163 @@
/**
* Modale de contexte légère affiche les candidats (occurrences) d'une règle
* et retourne les décisions sans appliquer immédiatement.
*
* Utilisée depuis MappingScanReviewModal pour permettre une sélection fine
* occurrence par occurrence (ex : valider "Juste" nom propre, ignorer "juste" adjectif).
*/
import { App, Modal, Setting } from 'obsidian';
import type { MappingRule, Occurrence } from '../types';
export type OccurrenceDecision = 'validated' | 'ignored' | 'false_positive';
interface CardRef {
card: HTMLElement;
buttons: Map<OccurrenceDecision, HTMLElement>;
arrow: HTMLElement;
resLine: HTMLElement;
statusLabel: HTMLElement;
}
export class OccurrencesContextModal extends Modal {
private rule: MappingRule;
private occurrences: Occurrence[];
private decisions: Map<string, OccurrenceDecision>;
private onConfirm: (decisions: Map<string, OccurrenceDecision>) => void;
private cardRefs = new Map<string, CardRef>();
constructor(
app: App,
rule: MappingRule,
occurrences: Occurrence[],
existingDecisions: Map<string, OccurrenceDecision>,
onConfirm: (decisions: Map<string, OccurrenceDecision>) => void
) {
super(app);
this.rule = rule;
this.occurrences = occurrences;
this.decisions = new Map(existingDecisions);
this.onConfirm = onConfirm;
}
onOpen(): void {
const { contentEl } = this;
contentEl.addClass('pseudobs-ctx-modal');
contentEl.createEl('h3', {
text: `${this.rule.source}${this.rule.replacement}`,
cls: 'pseudobs-ctx-modal-title',
});
contentEl.createEl('p', {
text: `${this.occurrences.length} occurrence${this.occurrences.length > 1 ? 's' : ''} — sélectionnez celles à remplacer.`,
cls: 'pseudobs-view-hint',
});
// Boutons globaux
new Setting(contentEl)
.addButton((b) =>
b.setButtonText('Tout valider').onClick(() => {
for (const occ of this.occurrences) this.decisions.set(occ.id, 'validated');
this.updateAllCards();
})
)
.addButton((b) =>
b.setButtonText('Tout ignorer').onClick(() => {
for (const occ of this.occurrences) this.decisions.set(occ.id, 'ignored');
this.updateAllCards();
})
);
// Cartes
const scroll = contentEl.createDiv('pseudobs-ctx-modal-scroll');
for (const occ of this.occurrences) {
this.buildCard(scroll, occ);
}
contentEl.createEl('hr');
new Setting(contentEl)
.addButton((b) =>
b.setButtonText('Annuler').onClick(() => this.close())
)
.addButton((b) =>
b.setButtonText('Confirmer la sélection').setCta().onClick(() => {
this.onConfirm(new Map(this.decisions));
this.close();
})
);
}
private buildCard(container: HTMLElement, occ: Occurrence): void {
const card = container.createDiv('pseudobs-occ-card');
// Ligne source
const srcLine = card.createDiv('pseudobs-occ-line');
srcLine.createSpan({ text: occ.contextBefore, cls: 'pseudobs-ctx-side' });
srcLine.createSpan({ text: occ.text, cls: 'pseudobs-occ-term' });
srcLine.createSpan({ text: occ.contextAfter, cls: 'pseudobs-ctx-side' });
// Flèche + ligne résultat
const arrow = card.createDiv('pseudobs-occ-arrow');
arrow.setText('↓');
const resLine = card.createDiv('pseudobs-occ-line pseudobs-occ-result-line');
resLine.createSpan({ text: occ.contextBefore, cls: 'pseudobs-ctx-side' });
resLine.createSpan({ text: this.rule.replacement, cls: 'pseudobs-occ-replacement' });
resLine.createSpan({ text: occ.contextAfter, cls: 'pseudobs-ctx-side' });
const statusLabel = card.createDiv('pseudobs-occ-status-label');
card.createEl('small', { text: `ligne ${occ.line}`, cls: 'pseudobs-occ-meta' });
// Boutons décision
const actions = card.createDiv('pseudobs-occ-actions');
const btnRefs = new Map<OccurrenceDecision, HTMLElement>();
for (const [label, value, title] of [
['✓', 'validated', 'Valider'],
['✗', 'ignored', 'Ignorer'],
['⚠', 'false_positive', 'Faux positif'],
] as [string, OccurrenceDecision, string][]) {
const btn = actions.createEl('button', { text: label });
btn.title = title;
btn.addClass('pseudobs-occ-btn');
btn.addEventListener('click', () => {
this.decisions.set(occ.id, value);
this.updateCard(occ.id);
});
btnRefs.set(value, btn);
}
this.cardRefs.set(occ.id, { card, buttons: btnRefs, arrow, resLine, statusLabel });
this.updateCard(occ.id);
}
private updateCard(occId: string): void {
const ref = this.cardRefs.get(occId);
if (!ref) return;
const decision = this.decisions.get(occId) ?? 'validated';
ref.card.removeClass('pseudobs-occ-validated', 'pseudobs-occ-ignored', 'pseudobs-occ-false_positive');
ref.card.addClass(`pseudobs-occ-${decision}`);
for (const [value, btn] of ref.buttons) {
btn.toggleClass('pseudobs-occ-btn-active', value === decision);
}
const show = decision === 'validated';
ref.arrow.toggle(show);
ref.resLine.toggle(show);
ref.statusLabel.toggle(!show);
ref.statusLabel.setText(
decision === 'ignored' ? 'Conservé tel quel' : decision === 'false_positive' ? 'Faux positif — exclu' : ''
);
}
private updateAllCards(): void {
for (const occId of this.cardRefs.keys()) this.updateCard(occId);
}
onClose(): void {
this.contentEl.empty();
}
}

View file

@ -238,7 +238,7 @@ export class OccurrencesModal extends Modal {
const nv = validated.length, ni = ignored.length;
new Notice(`${nv} remplacement${nv > 1 ? 's' : ''} appliqué${nv > 1 ? 's' : ''}` + (ni > 0 ? `, ${ni} ignoré${ni > 1 ? 's' : ''}` : ''));
void this.plugin.refreshHighlightData();
void this.plugin.refresh();
this.close();
}

View file

@ -6,9 +6,10 @@ import { Decoration, DecorationSet, EditorView, ViewPlugin, ViewUpdate } from '@
export const highlightDataChanged = StateEffect.define<void>();
export interface HighlightData {
sources: string[]; // termes originaux encore présents → orange (à pseudonymiser)
replacements: string[]; // pseudonymes déjà appliqués → vert + souligné
nerCandidates: string[]; // entités détectées par NER → bleu (candidats à pseudonymiser)
sources: string[]; // termes originaux encore présents → orange
replacements: string[]; // pseudonymes déjà appliqués → vert + souligné
nerCandidates: string[]; // entités NER → bleu
ignoredTerms: string[]; // textes exacts des occurrences ignorées → rouge (sensible à la casse)
}
// Extension CodeMirror 6 qui surligne dans l'éditeur :
@ -34,17 +35,20 @@ export function createPseudonymHighlighter(getData: () => HighlightData): Extens
}
private build(view: EditorView): DecorationSet {
const { sources, replacements, nerCandidates } = getData();
if (sources.length === 0 && replacements.length === 0 && nerCandidates.length === 0)
const { sources, replacements, nerCandidates, ignoredTerms } = getData();
if (sources.length === 0 && replacements.length === 0
&& nerCandidates.length === 0 && ignoredTerms.length === 0)
return Decoration.none;
const text = view.state.doc.toString();
const lower = text.toLowerCase();
type Span = { from: number; to: number; cls: string };
// prio : 0 = plus haute priorité (gagne en cas de chevauchement)
type Span = { from: number; to: number; cls: string; prio: number };
const spans: Span[] = [];
const collect = (terms: string[], cls: string) => {
// Matching insensible à la casse (sources, remplacements, NER)
const collect = (terms: string[], cls: string, prio: number) => {
for (const term of terms) {
if (!term) continue;
const needle = term.toLowerCase();
@ -52,7 +56,21 @@ export function createPseudonymHighlighter(getData: () => HighlightData): Extens
while (pos < lower.length) {
const idx = lower.indexOf(needle, pos);
if (idx === -1) break;
spans.push({ from: idx, to: idx + term.length, cls });
spans.push({ from: idx, to: idx + term.length, cls, prio });
pos = idx + term.length;
}
}
};
// Matching SENSIBLE à la casse (exceptions — "juste" ≠ "Juste")
const collectExact = (terms: string[], cls: string, prio: number) => {
for (const term of terms) {
if (!term) continue;
let pos = 0;
while (pos < text.length) {
const idx = text.indexOf(term, pos);
if (idx === -1) break;
spans.push({ from: idx, to: idx + term.length, cls, prio });
pos = idx + term.length;
}
}
@ -73,12 +91,14 @@ export function createPseudonymHighlighter(getData: () => HighlightData): Extens
return !sourcesLower.some((src) => src !== cl && src.includes(cl));
});
collect(freshCandidates, 'pseudobs-ner-candidate');
collect(sources, 'pseudobs-source');
collect(replacements, 'pseudobs-replaced');
// Priorités explicites : 0 = gagne en cas de chevauchement
collectExact(ignoredTerms, 'pseudobs-exception', 0);
collect(replacements, 'pseudobs-replaced', 1);
collect(sources, 'pseudobs-source', 2);
collect(freshCandidates, 'pseudobs-ner-candidate', 3);
// Trier par position (RangeSetBuilder l'exige) et éliminer les chevauchements
spans.sort((a, b) => a.from - b.from || a.to - b.to);
// Trier par position, puis par priorité (0 = gagne) en cas d'égalité de position
spans.sort((a, b) => a.from - b.from || a.prio - b.prio || a.to - b.to);
const builder = new RangeSetBuilder<Decoration>();
let lastTo = -1;

View file

@ -94,6 +94,11 @@ export class PseudonymizationView extends ItemView {
await this.renderTab(tab);
}
/** Appelé par le plugin pour forcer un re-rendu de l'onglet actif. */
async refreshActiveTab(): Promise<void> {
if (!this._renderingTab) await this.renderTab(this.activeTab);
}
private async renderTab(tab: Tab): Promise<void> {
const pane = this.panes[tab];
pane.empty();
@ -153,14 +158,19 @@ export class PseudonymizationView extends ItemView {
caseSensitive: this.plugin.settings.caseSensitive,
wholeWordOnly: this.plugin.settings.wholeWordOnly,
});
const countByRule = new Map<string, number>();
const occsByRule = new Map<string, typeof occs>();
for (const occ of occs) {
const id = occ.mappingId ?? '';
countByRule.set(id, (countByRule.get(id) ?? 0) + 1);
if (!occsByRule.has(id)) occsByRule.set(id, []);
occsByRule.get(id)!.push(occ);
}
const ruleResults: MappingRuleResult[] = rules
.filter((r) => countByRule.has(r.id))
.map((r) => ({ rule: r, matchCount: countByRule.get(r.id)! }));
.filter((r) => occsByRule.has(r.id))
.map((r) => ({
rule: r,
matchCount: occsByRule.get(r.id)!.length,
occurrences: occsByRule.get(r.id)!,
}));
if (ruleResults.length === 0) { new Notice(t('notice.noOccurrences')); return; }
new MappingScanReviewModal(this.app, this.plugin, file, content, ruleResults).open();
} finally {
@ -240,6 +250,46 @@ export class PseudonymizationView extends ItemView {
editBtn.addEventListener('click', () => new EditRuleModal(this.app, this.plugin, loc).open());
}
}
// ---- Section Exceptions ----
const allIgnored = locations.flatMap(({ rule, store, filePath }) =>
(rule.ignoredOccurrences ?? []).map((occ) => ({ occ, rule, store, filePath }))
);
if (allIgnored.length > 0) {
el.createEl('h3', { text: t('panel.mappings.exceptions'), cls: 'pseudobs-mappings-scope-heading' });
el.createEl('p', { text: t('panel.mappings.exceptions.hint'), cls: 'pseudobs-view-hint' });
const exceptionsGrid = el.createDiv('pseudobs-exceptions-grid');
for (const { occ, rule, store, filePath } of allIgnored) {
const card = exceptionsGrid.createDiv('pseudobs-exception-card');
// En-tête : règle concernée
card.createEl('div', {
text: `${rule.source}${rule.replacement}`,
cls: 'pseudobs-exception-card-rule',
});
// Contexte
const ctx = card.createDiv('pseudobs-exception-card-ctx');
ctx.createSpan({ text: occ.contextBefore, cls: 'pseudobs-ctx-side' });
ctx.createSpan({ text: occ.text, cls: 'pseudobs-exception-card-term' });
ctx.createSpan({ text: occ.contextAfter, cls: 'pseudobs-ctx-side' });
// Bouton supprimer
const delBtn = card.createEl('button', { cls: 'pseudobs-exception-card-del' });
setIcon(delBtn, 'x');
delBtn.title = 'Supprimer cette exception';
delBtn.addEventListener('click', async () => {
const updated = (rule.ignoredOccurrences ?? []).filter((o) => o.text !== occ.text);
store.update(rule.id, { ignoredOccurrences: updated });
await this.plugin.scopeResolver.saveStore(store, filePath);
void this.plugin.refresh();
void this.renderTab('mappings');
});
}
}
}
// ---- Onglet Dictionnaires --------------------------------------
@ -386,10 +436,22 @@ export class PseudonymizationView extends ItemView {
const s = this.plugin.settings;
const nerScanBtn = el.createEl('button', { cls: 'pseudobs-view-action-btn mod-cta' });
setIcon(nerScanBtn, 'scan-search');
nerScanBtn.createSpan({ text: t('panel.ner.scanBtn') });
const nerScanIcon = nerScanBtn.createSpan();
setIcon(nerScanIcon, 'scan-search');
nerScanBtn.createSpan({ text: ` ${t('panel.ner.scanBtn')}` });
nerScanBtn.title = t('panel.ner.scanBtn');
nerScanBtn.addEventListener('click', () => void this.plugin.scanCurrentFileNer());
nerScanBtn.addEventListener('click', () => { void (async () => {
nerScanBtn.setAttr('disabled', 'true');
setIcon(nerScanIcon, 'loader-circle');
nerScanIcon.addClass('pseudobs-spin');
try {
await this.plugin.scanCurrentFileNer();
} finally {
nerScanBtn.removeAttribute('disabled');
setIcon(nerScanIcon, 'scan-search');
nerScanIcon.removeClass('pseudobs-spin');
}
})(); });
el.createEl('hr');

View file

@ -3,6 +3,7 @@ import type PseudObsPlugin from '../main';
import { MappingStore } from '../mappings/MappingStore';
import type { EntityCategory, MappingFile } from '../types';
import { t } from '../i18n';
import { generateRedaction } from '../pseudonymizer/Redaction';
type ApplyScope = 'occurrence' | 'file';
@ -18,19 +19,23 @@ export class QuickPseudonymizeModal extends Modal {
private applyScope: ApplyScope = 'file';
private suggestions: string[];
private isRedactionMode: boolean;
constructor(
app: App,
plugin: PseudObsPlugin,
editor: Editor,
prefillReplacement = '',
suggestions: string[] = []
suggestions: string[] = [],
isRedactionMode = false,
) {
super(app);
this.plugin = plugin;
this.editor = editor;
this.source = editor.getSelection();
this.replacement = prefillReplacement;
this.replacement = prefillReplacement || (isRedactionMode ? generateRedaction(this.source) : '');
this.suggestions = suggestions;
this.isRedactionMode = isRedactionMode;
if (suggestions.length > 0) this.category = 'first_name';
this.from = editor.getCursor('from');
this.to = editor.getCursor('to');
@ -48,7 +53,7 @@ export class QuickPseudonymizeModal extends Modal {
tx.inputEl.addClass('pseudobs-disabled-input');
});
let replacementInput: HTMLInputElement;
let replacementInput: HTMLInputElement | undefined;
if (this.suggestions.length > 0) {
const suggBox = contentEl.createDiv('pseudobs-suggestions-box');
@ -98,6 +103,30 @@ export class QuickPseudonymizeModal extends Modal {
d.onChange((v) => (this.applyScope = v as ApplyScope));
});
// Checkbox caviardage
const redactRow = contentEl.createDiv('pseudobs-redact-row');
const redactCb = redactRow.createEl('input');
redactCb.type = 'checkbox';
redactCb.checked = this.isRedactionMode;
redactCb.addClass('pseudobs-dict-review-cb');
const redactLabel = redactRow.createSpan({ text: ` ${t('redaction.checkbox')}` });
redactLabel.title = t('redaction.checkboxDesc');
redactCb.addEventListener('change', () => {
this.isRedactionMode = redactCb.checked;
if (this.isRedactionMode) {
this.replacement = generateRedaction(this.source);
if (replacementInput) replacementInput.value = this.replacement;
replacementInput?.setAttr('disabled', 'true');
} else {
this.replacement = '';
if (replacementInput) replacementInput.value = '';
replacementInput?.removeAttribute('disabled');
}
});
if (this.isRedactionMode && replacementInput) {
replacementInput.setAttr('disabled', 'true');
}
new Setting(contentEl).addButton((btn) =>
btn.setButtonText(t('quickModal.submit')).setCta().onClick(() => void this.apply())
);
@ -133,7 +162,7 @@ export class QuickPseudonymizeModal extends Modal {
new Notice(t('notice.appliedFile', this.source, marked, String(count), count > 1 ? 's' : ''));
}
void this.plugin.refreshHighlightData();
void this.plugin.refresh();
this.close();
}

View file

@ -1,5 +1,6 @@
import { App, Modal, Setting, TFile, Notice } from 'obsidian';
import { t } from '../i18n';
import { generateRedaction, REDACTION_CHAR } from '../pseudonymizer/Redaction';
import type PseudObsPlugin from '../main';
import { MappingStore } from '../mappings/MappingStore';
import type { EntityCategory, MappingFile, ScopeType } from '../types';
@ -211,6 +212,23 @@ export class RuleModal extends Modal {
tx.setValue('0').onChange((v) => { this.priority = parseInt(v, 10) || 0; })
);
// Checkbox caviardage
const redactRow = contentEl.createDiv('pseudobs-redact-row');
const redactCb = redactRow.createEl('input');
redactCb.type = 'checkbox';
redactCb.addClass('pseudobs-dict-review-cb');
redactRow.createSpan({ text: ` ${t('redaction.checkbox')}` }).title = t('redaction.checkboxDesc');
redactCb.addEventListener('change', () => {
if (redactCb.checked) {
this.replacement = generateRedaction(this.source || REDACTION_CHAR);
this.useClass = false;
if (replacementInput) { replacementInput.value = this.replacement; replacementInput.setAttr('disabled', 'true'); }
} else {
this.replacement = '';
if (replacementInput) { replacementInput.value = ''; replacementInput.removeAttribute('disabled'); }
}
});
new Setting(contentEl).addButton((btn) =>
btn.setButtonText(t('ruleModal.submit')).setCta().onClick(() => void this.createRule())
);
@ -283,7 +301,7 @@ export class RuleModal extends Modal {
}
new Notice(t('notice.ruleCreated', this.source.trim(), this.replacement.trim()));
void this.plugin.refreshHighlightData();
void this.plugin.refresh();
this.close();
}

View file

@ -7,6 +7,14 @@
outline: 1px solid rgba(80, 160, 255, 0.45);
}
/* Occurrences explicitement ignorées — exceptions (sensibles à la casse) */
.pseudobs-exception {
background-color: rgba(220, 50, 50, 0.15);
border-radius: 2px;
outline: 1px solid rgba(220, 50, 50, 0.4);
border-bottom: 2px solid rgba(220, 50, 50, 0.7);
}
/* Termes sources encore présents dans le texte (à pseudonymiser) */
.pseudobs-source {
background-color: rgba(255, 160, 0, 0.22);
@ -955,6 +963,7 @@
}
.pseudobs-onboarding-icon-btn-loading { cursor: wait; }
.pseudobs-onboarding-icon-btn-loading svg { animation: pseudobs-spin 0.9s linear infinite; }
.pseudobs-spin svg { animation: pseudobs-spin 0.9s linear infinite; }
/* ---- CorpusModal ---- */
@ -1048,3 +1057,71 @@
font-size: 0.95em;
line-height: 1.55;
}
/* ---- Section Exceptions (onglet Mappings) ---- */
.pseudobs-exceptions-grid {
display: flex;
flex-direction: column;
gap: 6px;
margin-bottom: 12px;
}
.pseudobs-exception-card {
display: flex;
flex-direction: column;
gap: 4px;
padding: 8px 10px;
border-radius: 6px;
border: 1px solid rgba(220, 50, 50, 0.35);
background: rgba(220, 50, 50, 0.06);
position: relative;
}
.pseudobs-exception-card-rule {
font-size: 0.8em;
color: var(--text-muted);
font-style: italic;
}
.pseudobs-exception-card-ctx {
font-size: 0.9em;
}
.pseudobs-exception-card-term {
background: rgba(220, 50, 50, 0.2);
border-radius: 2px;
outline: 1px solid rgba(220, 50, 50, 0.45);
padding: 0 2px;
font-weight: 500;
}
.pseudobs-exception-card-del {
position: absolute;
top: 6px;
right: 6px;
background: none;
border: none;
cursor: pointer;
color: var(--text-muted);
padding: 2px;
border-radius: 3px;
line-height: 1;
}
.pseudobs-exception-card-del:hover {
color: var(--text-error);
background: var(--background-modifier-error-hover);
}
.pseudobs-save-exceptions-btn {
margin-right: auto;
}
/* Ligne checkbox caviardage (QuickPseudonymizeModal, RuleModal) */
.pseudobs-redact-row {
display: flex;
align-items: center;
gap: 6px;
margin: 6px 0 10px;
font-size: 0.88em;
color: var(--text-muted);
}

298
tests/fixtures/fight_club.vtt vendored Normal file
View file

@ -0,0 +1,298 @@
WEBVTT fight_club
NOTE
Transcrit avec noScribe vers. 0.7
Fichier audio : /Users/axelleabbadie/SynologyDrive/Scientifique/Général/Tools/Pseudobsidianization/tests/fixtures/fight_club.mp3
(Langue : French (fr) | Détection des locuteurs : 3 | Chevauchements de parole 1 | Horodatage : 1 | Mots de remplissage : 1 | Marquer les pauses : 1)
NOTE media: /Users/axelleabbadie/SynologyDrive/Scientifique/Général/Tools/Pseudobsidianization/tests/fixtures/fight_club.mp3
1
00:00:00.530 --> 00:00:04.160
<v S00>S00:
2
00:00:00.530 --> 00:00:04.160
<v S00>[00:00:00]
3
00:00:00.530 --> 00:00:04.160
<v S00>Après un combat, tout le reste de la vie prenait beaucoup moins d&#x27;importance.
4
00:00:04.160 --> 00:00:07.584
<v S00>
5
00:00:00.530 --> 00:00:04.160
<v S00>S01:
6
00:00:00.530 --> 00:00:04.160
<v S00>[00:00:04]
7
00:00:00.530 --> 00:00:04.160
<v S00>
8
00:00:04.160 --> 00:00:07.584
<v S00>Qu&#x27;est-ce vous avez encore fichu hier soir ?
9
00:00:07.584 --> 00:00:07.990
<v S00>
10
00:00:07.584 --> 00:00:07.990
<v S00>S00:
11
00:00:00.530 --> 00:00:04.160
<v S00>[00:00:07]
12
00:00:00.530 --> 00:00:04.160
<v S00>
13
00:00:07.584 --> 00:00:07.990
<v S00>Quoi ?
14
00:00:07.584 --> 00:00:07.990
<v S00>S02:
15
00:00:00.530 --> 00:00:04.160
<v S00>[00:00:04]
16
00:00:00.530 --> 00:00:04.160
<v S00>
17
00:00:07.990 --> 00:00:09.312
<v S00>On pouvait endurer n&#x27;importe quoi.
18
00:00:09.312 --> 00:00:16.416
<v S00>
19
00:00:09.312 --> 00:00:16.416
<v S00>S01: [00:00:09] Vous avez fini ces rapports ?
20
00:00:16.416 --> 00:00:18.080
<v S01>S02:
21
00:00:16.416 --> 00:00:18.080
<v S01>[00:00:16]
22
00:00:16.416 --> 00:00:18.080
<v S01>Tu pouvais choisir ton adversaire, qui tu prendrais ?
23
00:00:18.080 --> 00:00:19.808
<v S01>(..)
24
00:00:19.808 --> 00:00:20.960
<v S01>
25
00:00:19.808 --> 00:00:20.960
<v S01>S00: Je prendrais sans doute mon patron.
26
00:00:21.824 --> 00:00:22.208
<v S01>
27
00:00:21.824 --> 00:00:22.208
<v S01>S02: Sérieux ?
28
00:00:22.208 --> 00:00:23.936
<v S01>(..)
29
00:00:23.936 --> 00:00:24.992
<v S00>S00:
30
00:00:23.936 --> 00:00:24.992
<v S00>[00:00:23]
31
00:00:23.936 --> 00:00:24.992
<v S00>Bah oui, pourquoi tu prendrais qui toi ?
32
00:00:25.696 --> 00:00:26.336
<v S00>
33
00:00:25.696 --> 00:00:26.336
<v S00>S02: Je prendrais mon père.
34
00:00:26.336 --> 00:00:27.616
<v S00>(.)
35
00:00:27.616 --> 00:00:28.224
<v S00>
36
00:00:27.616 --> 00:00:28.224
<v S00>S00: Je connais pas le mien.
37
00:00:28.224 --> 00:00:29.408
<v S00>(.)
38
00:00:29.408 --> 00:00:32.832
<v S00>Si, je le connais, mais il est parti quand je devais avoir 6 ans.
39
00:00:33.408 --> 00:00:34.816
<v S00>Il s&#x27;est remarié, il a eu d&#x27;autres gamins.
40
00:00:34.816 --> 00:00:36.160
<v S00>(.)
41
00:00:36.160 --> 00:00:36.960
<v S00>Il fait ça tous les 6 ans.
42
00:00:36.960 --> 00:00:38.016
<v S00>(.)
43
00:00:38.016 --> 00:00:39.740
<v S00>Il va dans une autre ville et il fonde une nouvelle famille.
44
00:00:39.960 --> 00:00:41.340
<v S01>S01:
45
00:00:39.960 --> 00:00:41.340
<v S01>[00:00:39]
46
00:00:39.960 --> 00:00:41.340
<v S01>Ce salaud, il monte des franchises.
47
00:00:41.340 --> 00:00:44.288
<v S01>(...)
48
00:00:44.288 --> 00:00:45.520
<v S01>Mon père, il est jamais allé en fac.
49
00:00:45.900 --> 00:00:47.552
<v S01>Alors pour lui, c&#x27;était important que j&#x27;y aille.
50
00:00:48.320 --> 00:00:48.990
<v S01>
51
00:00:48.320 --> 00:00:48.990
<v S01>S00: J&#x27;connais ce genre d&#x27;histoire.
52
00:00:48.990 --> 00:00:51.880
<v S01>
53
00:00:48.990 --> 00:00:51.880
<v S01>S01: Alors je passe ma licence, je l&#x27;appelle à l&#x27;autre bout du pays.
54
00:00:52.000 --> 00:00:52.992
<v S01>Je lui dis, papa, qu&#x27;est-ce que je fais ?
55
00:00:53.824 --> 00:00:54.650
<v S01>Il me dit, trouve-toi un boulot.
56
00:00:54.950 --> 00:00:55.750
<v S01>
57
00:00:54.950 --> 00:00:55.750
<v S01>S00: J&#x27;ai connu ça aussi.
58
00:00:55.990 --> 00:00:58.130
<v S01>
59
00:00:55.990 --> 00:00:58.130
<v S01>S01: Ensuite, j&#x27;ai 25 ans, je lui passe mon coup de fil annuel.
60
00:00:58.310 --> 00:01:00.670
<v S01>Je lui dis, papa, qu&#x27;est-ce que je fais ?
61
00:01:00.670 --> 00:01:02.770
<v S01>Il dit, je sais pas, tiens, marie-toi.
62
00:01:03.290 --> 00:01:04.192
<v S00>S00:
63
00:01:03.290 --> 00:01:04.192
<v S00>[00:01:03]
64
00:01:03.290 --> 00:01:04.192
<v S00>Ouais, pareil pour moi.
65
00:01:04.192 --> 00:01:05.440
<v S00>(.)
66
00:01:05.440 --> 00:01:08.928
<v S00>On peut pas se marier, je suis un gamin de 30 ans.
67
00:01:08.928 --> 00:01:10.176
<v S00>(.)
68
00:01:10.176 --> 00:01:12.032
<v S01>S01:
69
00:01:10.176 --> 00:01:12.032
<v S01>[00:01:10]
70
00:01:10.176 --> 00:01:12.032
<v S01>On a une génération d&#x27;hommes élevés par des femmes.
71
00:01:12.032 --> 00:01:13.376
<v S01>(.)
72
00:01:13.376 --> 00:01:15.840
<v S01>J&#x27;suis pas sûr qu&#x27;une autre femme soit la solution à nos problèmes.

13
tests/fixtures/juste-leblanc.html vendored Normal file

File diff suppressed because one or more lines are too long

BIN
tests/fixtures/juste-leblanc.mp3 vendored Normal file

Binary file not shown.

View file

@ -0,0 +1,104 @@
import { NoScribeHtmlParser } from '../../src/parsers/NoScribeHtmlParser';
const parser = new NoScribeHtmlParser();
// ---- Fixture ----------------------------------------------------------------
// Structure calquée sur une vraie sortie noScribe (Qt Rich Text HTML).
// Les timestamps sont en millisecondes depuis le début de l'audio.
const NOSCRIBE_HTML = `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<html><head><meta name="qrichtext" content="1" />
<meta charset="UTF-8" />
<meta name="audio_source" content="/Corpus/entretien_01.m4a" />
</head><body>
<div class="WordSection1">
<p><span style="font-weight: 600">entretien_01</span></p>
<p><span style="font-size: 0.8em; color: #909090">Transcrit avec noScribe vers. 0.5<br />
(Détection des locuteurs : 2)</span></p>
<p><a name="ts_0_2130_" ><span style="color: #000000">(..) </span></a></p>
<p><a name="ts_2130_7730_S00" ><span style="color: #000000">Expérimentateur </span></a><a name="ts_2130_7730_S00" ><span style="color: #78909c">[00:00:02]</span></a><a name="ts_2130_7730_S00" ><span style="color: #000000">: Ok. D'abord, merci d'avoir accepté.</span></a><a name="ts_8330_9730_S00" ><span style="color: #000000"> On s'était déjà parlé.</span></a><a name="ts_9730_12230_S00" ><span style="color: #000000"> (..)</span></a></p>
<p><a name="ts_17900_18660_S01" ><span style="color: #000000">SC01 : Si je me présente ?</span></a></p>
<p><a name="ts_1668980_1673180_S00" ><span style="color: #000000">Expérimentateur : si jamais demain j'arrive à être en contact avec quelqu'un.</span></a></p>
</div>
</body></html>`;
// ---- Tests ------------------------------------------------------------------
describe('NoScribeHtmlParser — isNoScribeHtml', () => {
test('reconnaît un HTML noScribe', () => {
expect(NoScribeHtmlParser.isNoScribeHtml(NOSCRIBE_HTML)).toBe(true);
});
test('rejette un HTML ordinaire', () => {
expect(NoScribeHtmlParser.isNoScribeHtml('<html><body><p>Bonjour</p></body></html>')).toBe(false);
});
});
describe('NoScribeHtmlParser — parse', () => {
test('ignore les paragraphes sans ancre ts_', () => {
const doc = parser.parse(NOSCRIBE_HTML);
// Les 2 paragraphes d'en-tête (titre + métadonnées) ne doivent pas produire de cue
// Les cues sont : pause, tour S00, tour S01, tour S00 long = 4 cues
expect(doc.cues.length).toBe(4);
});
test('extrait la pause sans locuteur', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const pause = doc.cues[0];
expect(pause.speaker).toBeUndefined();
expect(pause.text).toContain('..');
expect(pause.startTime).toBe('00:00:00.000');
expect(pause.endTime).toBe('00:00:02.130');
});
test('extrait le locuteur depuis "Nom : texte" multi-ancres', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const turn = doc.cues[1];
expect(turn.speaker).toBe('Expérimentateur');
expect(turn.text).not.toContain('Expérimentateur');
expect(turn.text).toContain('Ok. D\'abord');
});
test('extrait le locuteur depuis "Nom : texte" mono-ancre', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const turn = doc.cues[2];
expect(turn.speaker).toBe('SC01');
expect(turn.text).toContain('Si je me présente');
expect(turn.text).not.toContain('SC01');
});
test('convertit les timestamps en HH:MM:SS.mmm', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const turn = doc.cues[1];
expect(turn.startTime).toBe('00:00:02.130');
expect(turn.endTime).toBe('00:00:12.230');
});
test('timestamp long (> 27 min) correctement converti', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const turn = doc.cues[3];
// 1668980ms = 27min 48.980s
expect(turn.startTime).toBe('00:27:48.980');
expect(turn.endTime).toBe('00:27:53.180');
});
test('word timestamps produits pour chaque ancre distincte', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const turn = doc.cues[1]; // Expérimentateur avec 3 ancres ts_ distinctes
expect(turn.words.length).toBeGreaterThanOrEqual(2);
expect(turn.words[0].time).toBe('00:00:02.130');
});
test('ignore les timestamps d\'affichage [HH:MM:SS]', () => {
const doc = parser.parse(NOSCRIBE_HTML);
const turn = doc.cues[1];
expect(turn.text).not.toMatch(/\[\d{2}:\d{2}:\d{2}\]/);
});
});

View file

@ -0,0 +1,138 @@
import * as fs from 'fs';
import * as path from 'path';
import { NoScribeVttParser } from '../../src/parsers/NoScribeVttParser';
import { NoScribeHtmlParser } from '../../src/parsers/NoScribeHtmlParser';
import { noScribeHtmlToMarkdown } from '../../src/parsers/TranscriptConverter';
const VTT_FIXTURE = path.join(__dirname, '../fixtures/fight_club.vtt');
const HTML_FIXTURE = path.join(__dirname, '../fixtures/juste-leblanc.html');
// ---- Détection --------------------------------------------------------------
describe('NoScribeVttParser — détection', () => {
test('reconnaît un VTT noScribe', () => {
const content = fs.readFileSync(VTT_FIXTURE, 'utf-8');
expect(NoScribeVttParser.isNoScribeVtt(content)).toBe(true);
});
test('rejette un VTT standard', () => {
expect(NoScribeVttParser.isNoScribeVtt('WEBVTT\n\n1\n00:00:01.000 --> 00:00:02.000\nBonjour\n')).toBe(false);
});
test('extrait le chemin audio depuis NOTE media', () => {
const content = fs.readFileSync(VTT_FIXTURE, 'utf-8');
const src = NoScribeVttParser.extractAudioSource(content);
expect(src).toMatch(/fight_club\.mp3$/);
});
});
// ---- Parsing VTT fight_club -------------------------------------------------
describe('NoScribeVttParser — fight_club.vtt', () => {
let doc: ReturnType<NoScribeVttParser['parse']>;
beforeAll(() => {
const content = fs.readFileSync(VTT_FIXTURE, 'utf-8');
doc = new NoScribeVttParser().parse(content);
});
test('produit au moins 5 cues', () => {
expect(doc.cues.length).toBeGreaterThanOrEqual(5);
});
test('détecte les locuteurs S00, S01, S02', () => {
const speakers = new Set(doc.cues.map((c) => c.speaker).filter(Boolean));
expect(speakers.has('S00')).toBe(true);
expect(speakers.has('S01')).toBe(true);
expect(speakers.has('S02')).toBe(true);
});
test('premier cue : S00, timestamp correct, texte non vide', () => {
const first = doc.cues[0];
expect(first.speaker).toBe('S00');
expect(first.startTime).toBe('00:00:00.530');
expect(first.text).toContain('combat');
});
test('les timestamps d\'affichage [HH:MM:SS] sont absents du texte', () => {
for (const cue of doc.cues) {
expect(cue.text).not.toMatch(/\[\d{2}:\d{2}:\d{2}\]/);
}
});
test('les entités HTML sont décodées', () => {
const allText = doc.cues.map((c) => c.text).join(' ');
expect(allText).not.toContain('&#x27;');
expect(allText).toContain("'"); // apostrophes décodées
});
test('les labels SXX: sont absents du texte', () => {
for (const cue of doc.cues) {
expect(cue.text).not.toMatch(/^S\d+\s*:/);
}
});
test('chaque cue a un mot pour le timestamps word-level', () => {
for (const cue of doc.cues) {
expect(cue.words.length).toBeGreaterThan(0);
expect(cue.words[0].time).toBe(cue.startTime);
}
});
});
// ---- Parsing HTML juste-leblanc --------------------------------------------
describe('NoScribeHtmlParser — juste-leblanc.html', () => {
let doc: ReturnType<NoScribeHtmlParser['parse']>;
beforeAll(() => {
const content = fs.readFileSync(HTML_FIXTURE, 'utf-8');
doc = new NoScribeHtmlParser().parse(content);
});
test('produit au moins 8 cues', () => {
expect(doc.cues.length).toBeGreaterThanOrEqual(8);
});
test('détecte S00 et S01', () => {
const speakers = new Set(doc.cues.map((c) => c.speaker).filter(Boolean));
expect(speakers.has('S00')).toBe(true);
expect(speakers.has('S01')).toBe(true);
});
test('premier cue : S00, contient "Juste Leblanc"', () => {
const first = doc.cues.find((c) => c.speaker === 'S00');
expect(first?.text).toContain('Juste Leblanc');
});
test('les labels SXX: sont absents du texte', () => {
for (const cue of doc.cues) {
expect(cue.text).not.toMatch(/^S\d+\s*:/);
}
});
test('les timestamps [HH:MM:SS] sont absents du texte', () => {
for (const cue of doc.cues) {
expect(cue.text).not.toMatch(/\[\d{2}:\d{2}:\d{2}\]/);
}
});
test('audio source extrait correctement', () => {
const content = fs.readFileSync(HTML_FIXTURE, 'utf-8');
expect(NoScribeHtmlParser.extractAudioSource(content)).toMatch(/juste-leblanc\.mp3$/);
});
});
// ---- Markdown produit -------------------------------------------------------
describe('noScribeHtmlToMarkdown — juste-leblanc', () => {
test('format clean : pas de commentaires HTML', () => {
const content = fs.readFileSync(HTML_FIXTURE, 'utf-8');
const doc = new NoScribeHtmlParser().parse(content);
const md = noScribeHtmlToMarkdown(doc, 'juste-leblanc.html');
expect(md).toContain('pseudobs-format: html');
expect(md).not.toContain('<!--');
expect(md).toContain('**S00**');
expect(md).toContain('**S01**');
});
});

View file

@ -2,7 +2,9 @@ import * as fs from 'fs';
import * as path from 'path';
import { SrtParser } from '../../src/parsers/SrtParser';
import { ChatParser } from '../../src/parsers/ChatParser';
import { srtToMarkdown, chatToMarkdown } from '../../src/parsers/TranscriptConverter';
import { VttParser } from '../../src/parsers/VttParser';
import { NoScribeHtmlParser } from '../../src/parsers/NoScribeHtmlParser';
import { srtToMarkdown, chatToMarkdown, vttToMarkdown, noScribeHtmlToMarkdown, extractWordData, markdownToVtt } from '../../src/parsers/TranscriptConverter';
const SRT_FIXTURE = path.join(__dirname, '../fixtures/entretien_01.srt');
const CHA_FIXTURE = path.join(__dirname, '../fixtures/entretien_02.cha');
@ -91,3 +93,185 @@ describe('chatToMarkdown', () => {
expect(md.endsWith('\n')).toBe(true);
});
});
// ---- vttToMarkdown / noScribeHtmlToMarkdown --------------------------------
const VTT_FIXTURE = `WEBVTT
1
00:00:01.240 --> 00:00:05.800
<v Expérimentateur>Bonjour, je m'appelle Marie.
2
00:00:06.100 --> 00:00:09.300
<v SC01>Si je me présente ?
3
00:00:09.800 --> 00:00:11.200
(..)
`;
const NOSCRIBE_HTML_FIXTURE = `<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<html><head><meta name="qrichtext" content="1" /><meta charset="UTF-8" /></head><body>
<div class="WordSection1">
<p><a name="ts_0_2130_" ><span>(..) </span></a></p>
<p><a name="ts_2130_5800_S00" ><span>Expérimentateur </span></a><a name="ts_2130_5800_S00" ><span style="color: #78909c">[00:00:02]</span></a><a name="ts_2130_5800_S00" ><span>: Bonjour, je m'appelle Marie.</span></a></p>
<p><a name="ts_6100_9300_S01" ><span>SC01 : Si je me présente ?</span></a></p>
</div></body></html>`;
describe('vttToMarkdown — format noScribe', () => {
let md: string;
beforeAll(() => { md = vttToMarkdown(new VttParser().parse(VTT_FIXTURE), 'entretien.vtt'); });
it('frontmatter pseudobs-format: vtt', () => {
expect(md).toContain('pseudobs-format: vtt');
});
it('locuteur en gras suivi du timestamp [HH:MM:SS]', () => {
expect(md).toContain('**Expérimentateur** [00:00:01]');
expect(md).toContain('**SC01** [00:00:06]');
});
it('texte précédé de " : " quand il y a un locuteur', () => {
expect(md).toContain('**Expérimentateur** [00:00:01] : Bonjour');
});
it('cue sans locuteur : timestamp seul', () => {
expect(md).toContain('[00:00:09] (..)');
expect(md).not.toMatch(/\*\*\*\* \[/);
});
it('aucun commentaire HTML dans le corps', () => {
const body = md.split('---').slice(2).join('---');
expect(body).not.toContain('<!--');
});
it('termine par un saut de ligne', () => {
expect(md.endsWith('\n')).toBe(true);
});
});
describe('noScribeHtmlToMarkdown — format html', () => {
let md: string;
beforeAll(() => {
const doc = new NoScribeHtmlParser().parse(NOSCRIBE_HTML_FIXTURE);
md = noScribeHtmlToMarkdown(doc, 'entretien.html');
});
it('frontmatter pseudobs-format: html', () => {
expect(md).toContain('pseudobs-format: html');
expect(md).not.toContain('pseudobs-format: vtt');
});
it('locuteur en gras avec timestamp [HH:MM:SS]', () => {
expect(md).toContain('**Expérimentateur** [00:00:02]');
expect(md).toContain('**SC01** [00:00:06]');
});
it('texte présent après " : "', () => {
expect(md).toContain(': Bonjour, je m\'appelle Marie.');
expect(md).toContain(': Si je me présente ?');
});
it('aucun commentaire HTML dans le corps', () => {
const body = md.split('---').slice(2).join('---');
expect(body).not.toContain('<!--');
});
});
describe('extractWordData', () => {
const VTT_WITH_WORDS = `WEBVTT
1
00:00:01.240 --> 00:00:05.800
<v Expérimentateur><00:00:01.240><c> Bonjour,</c><00:00:02.100><c> Marie.</c>
2
00:00:06.100 --> 00:00:09.300
<v SC01>Si je me présente ?
`;
it('extrait les cues avec word timestamps', () => {
const doc = new VttParser().parse(VTT_WITH_WORDS);
const data = extractWordData(doc);
expect(data).toHaveLength(1); // cue 2 sans word timestamps exclue
expect(data[0].index).toBe(0);
expect(data[0].speaker).toBe('Expérimentateur');
expect(data[0].startTime).toBe('00:00:01.240');
expect(data[0].endTime).toBe('00:00:05.800');
expect(data[0].words.some((w) => w.time !== '')).toBe(true);
});
it('exclut les cues sans word timestamps', () => {
const doc = new VttParser().parse(VTT_WITH_WORDS);
const data = extractWordData(doc);
expect(data.every((c) => c.words.some((w) => w.time !== ''))).toBe(true);
});
it('produit un JSON sérialisable', () => {
const doc = new VttParser().parse(VTT_WITH_WORDS);
const data = extractWordData(doc);
expect(() => JSON.stringify(data)).not.toThrow();
});
});
// ---- markdownToVtt ----------------------------------------------------------
describe('markdownToVtt — re-export VTT', () => {
const WORDS_DATA = [
{ index: 0, startTime: '00:01:28.080', endTime: '00:01:33.536', speaker: 'S00',
words: [{ text: 'original', time: '00:01:28.080' }] },
{ index: 1, startTime: '00:01:32.672', endTime: '00:01:34.816', speaker: 'S01',
words: [{ text: 'original', time: '00:01:32.672' }] },
{ index: 2, startTime: '00:01:34.816', endTime: '00:01:45.920', speaker: 'S00',
words: [{ text: 'original', time: '00:01:34.816' }] },
];
const MD_PSEUDONYMIZED = `---
pseudobs-format: html
pseudobs-source: "juste-leblanc.html"
---
**S00** [00:01:28] : Je vais le faire moi-même. Il s'appelle Pierre Martin.
**S01** [00:01:32] : Ah bon, il n'a pas de prénom ?
**S00** [00:01:34] : Je viens de vous le dire, Pierre Martin.
`;
it('produit un WebVTT valide', () => {
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
expect(vtt).toMatch(/^WEBVTT/);
expect(vtt).toContain('00:01:28.080 --> 00:01:33.536');
expect(vtt).toContain('00:01:32.672 --> 00:01:34.816');
});
it('intègre le texte pseudonymisé', () => {
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
expect(vtt).toContain('Pierre Martin');
expect(vtt).not.toContain('Juste Leblanc');
});
it('préserve les locuteurs', () => {
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
expect(vtt).toContain('<v S00>');
expect(vtt).toContain('<v S01>');
});
it('utilise les timestamps du words.json, pas du Markdown', () => {
const { vtt } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
// Les ms précises viennent du words.json
expect(vtt).toContain('00:01:28.080');
expect(vtt).toContain('00:01:33.536');
});
it('signale une incohérence si le nombre de cues diffère', () => {
const { mismatch } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA.slice(0, 2));
expect(mismatch).toBe(true);
});
it('pas de mismatch si les comptes sont égaux', () => {
const { mismatch } = markdownToVtt(MD_PSEUDONYMIZED, WORDS_DATA);
expect(mismatch).toBe(false);
});
});

View file

@ -0,0 +1,124 @@
import { VttParser } from '../../src/parsers/VttParser';
const parser = new VttParser();
// ---- Helpers ----------------------------------------------------------------
function roundTrip(input: string): string {
return parser.reconstruct(parser.parse(input));
}
// ---- Fixtures ---------------------------------------------------------------
const VTT_SIMPLE = `WEBVTT
1
00:00:01.240 --> 00:00:05.800
Bonjour, je m'appelle Marie Dupont.
2
00:00:06.100 --> 00:00:09.300
Et tu habites à Lyon ?
`;
const VTT_SPEAKER = `WEBVTT
1
00:00:01.240 --> 00:00:05.800
<v SPEAKER_1>Bonjour, je m'appelle Marie Dupont.
2
00:00:06.100 --> 00:00:09.300
<v SPEAKER_2>Et tu habites à Lyon ?
`;
const VTT_WORD_TIMESTAMPS = `WEBVTT
1
00:00:01.240 --> 00:00:05.800
<v SPEAKER_1><00:00:01.240><c> Bonjour,</c><00:00:02.100><c> je</c><00:00:02.300><c> m'appelle</c><00:00:03.400><c> Marie</c><00:00:04.200><c> Dupont.</c>
2
00:00:06.100 --> 00:00:09.300
<v SPEAKER_2><00:00:06.100><c> Et</c><00:00:06.500><c> tu</c><00:00:06.800><c> habites</c><00:00:07.200><c> à</c><00:00:07.800><c> Lyon</c><00:00:08.200><c> ?</c>
`;
// ---- Tests ------------------------------------------------------------------
describe('VttParser — parse', () => {
test('VTT simple : 2 cues sans speaker', () => {
const doc = parser.parse(VTT_SIMPLE);
expect(doc.cues).toHaveLength(2);
expect(doc.cues[0].startTime).toBe('00:00:01.240');
expect(doc.cues[0].endTime).toBe('00:00:05.800');
expect(doc.cues[0].text).toContain('Marie Dupont');
expect(doc.cues[0].speaker).toBeUndefined();
});
test('VTT avec speakers : extraction correcte', () => {
const doc = parser.parse(VTT_SPEAKER);
expect(doc.cues[0].speaker).toBe('SPEAKER_1');
expect(doc.cues[1].speaker).toBe('SPEAKER_2');
expect(doc.cues[0].text).toContain('Marie Dupont');
expect(doc.cues[0].text).not.toContain('<v');
});
test('VTT avec word timestamps : extraction des mots', () => {
const doc = parser.parse(VTT_WORD_TIMESTAMPS);
const cue = doc.cues[0];
expect(cue.speaker).toBe('SPEAKER_1');
expect(cue.words.length).toBeGreaterThan(1);
expect(cue.words.some(w => w.time === '00:00:03.400')).toBe(true);
// Le mot "Marie" doit être associé au timestamp 00:00:03.400
const marie = cue.words.find(w => w.time === '00:00:03.400');
expect(marie?.text).toContain('Marie');
});
test('Header WEBVTT reconnu', () => {
const doc = parser.parse(VTT_SIMPLE);
expect(doc.cues.length).toBeGreaterThan(0);
});
test('Timestamps normalisés en HH:MM:SS.mmm', () => {
const vtt = `WEBVTT\n\n01:23.456 --> 01:27.800\nTest\n`;
const doc = parser.parse(vtt);
expect(doc.cues[0].startTime).toBe('00:01:23.456');
expect(doc.cues[0].endTime).toBe('00:01:27.800');
});
});
describe('VttParser — round-trip', () => {
test('VTT simple : round-trip exact', () => {
const doc = parser.parse(VTT_SIMPLE);
const out = parser.reconstruct(doc);
expect(out).toContain('WEBVTT');
expect(out).toContain('00:00:01.240 --> 00:00:05.800');
expect(out).toContain('Marie Dupont');
});
test('VTT avec speakers : speaker préservé', () => {
const doc = parser.parse(VTT_SPEAKER);
const out = parser.reconstruct(doc);
expect(out).toContain('<v SPEAKER_1>');
expect(out).toContain('<v SPEAKER_2>');
});
test('VTT word timestamps : reconstruction avec timestamps', () => {
const doc = parser.parse(VTT_WORD_TIMESTAMPS);
const out = parser.reconstruct(doc);
expect(out).toContain('<v SPEAKER_1>');
expect(out).toContain('00:00:03.400');
expect(out).toContain('Marie');
});
});
describe('VttParser — pseudonymisation', () => {
test('Remplacement dans le texte reflété dans la reconstruction', () => {
const doc = parser.parse(VTT_SPEAKER);
doc.cues[0].text = doc.cues[0].text.replace('Marie Dupont', 'Sophie Martin');
VttParser.applyTextToWords(doc.cues[0], doc.cues[0].text);
const out = parser.reconstruct(doc);
expect(out).toContain('Sophie Martin');
expect(out).not.toContain('Marie Dupont');
});
});

View file

@ -9,5 +9,6 @@
"0.1.1": "1.7.2",
"0.1.2": "1.7.2",
"0.1.3": "1.7.2",
"0.1.4": "1.7.2"
"0.1.4": "1.7.2",
"0.1.5": "1.7.2"
}