obsidian-plugin/callumalpass_tasknotes
1
0
Fork 0
mirror of https://github.com/callumalpass/tasknotes.git synced 2026-07-22 12:50:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: implement comprehensive i18n CI/CD system

Browse source 
- Add i18n management script with sync, verify, and status commands
- Integrate GitHub Actions workflow to enforce translation consistency
- Generate initial manifest (1209 keys) and state files for English/French
- Add package.json scripts for easy i18n workflow management
- Create comprehensive I18N_GUIDE.md with usage examples and best practices

Features:
- Automatic detection of missing/stale translations
- CI/CD enforcement preventing deployment of untranslated strings
- Hash-based tracking system for source string changes
- Support for multiple locales with individual progress tracking
- TypeScript file parsing with automatic cleanup

This ensures translation quality and prevents regressions while providing
clear workflows for both developers and translators.
This commit is contained in:
callumalpass 2025-09-19 23:47:38 +10:00
parent 38cdaca7f0
commit eb7be73987
6 changed files with 3050 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

36
.github/workflows/test.yml vendored
View file

@ -26,7 +26,41 @@ jobs:
- name: Install dependencies
run: npm ci --prefer-offline --no-audit
timeout-minutes: 5
- name: Check i18n manifest is up-to-date
run: |
echo "🔍 Checking if i18n manifest and state files are up-to-date..."
npm run i18n:sync
if [[ -n "$(git status --porcelain)" ]]; then
echo "❌ Error: i18n manifest or state files are out of date."
echo "📝 Files that need to be committed:"
git status --porcelain
echo ""
echo "🔧 Please run 'npm run i18n:sync' locally and commit the changes:"
echo " npm run i18n:sync"
echo " git add i18n.manifest.json i18n.state.json"
echo " git commit -m 'chore: update i18n manifest and state files'"
echo ""
echo "📊 Current diff:"
git diff --name-only
if git diff --quiet i18n.manifest.json; then
echo "ℹ️ No manifest changes"
else
echo "📄 Manifest changes:"
git diff i18n.manifest.json | head -20
fi
if git diff --quiet i18n.state.json; then
echo "ℹ️ No state changes"
else
echo "📄 State changes:"
git diff i18n.state.json | head -20
fi
exit 1
else
echo "✅ i18n files are up-to-date."
fi
timeout-minutes: 3
- name: Debug environment
run: |
echo "Node version: $(node --version)"

243
I18N_GUIDE.md Normal file
View file

@ -0,0 +1,243 @@
# Internationalization (i18n) CI/CD System
This project uses an automated CI/CD system to ensure translation consistency and prevent missing translations from being deployed.
## 📋 Overview
The i18n system tracks:
- **Manifest**: Hashes of all English (source) strings
- **State**: Translation status for each locale
- **CI Enforcement**: Automatic verification in GitHub Actions
## 🚀 Quick Start
### For Developers
When you modify English strings in `src/i18n/resources/en.ts`:
```bash
# 1. Edit the English translation file
vim src/i18n/resources/en.ts
# 2. Update the manifest and state files
npm run i18n:sync
# 3. Commit all changes (including generated files)
git add src/i18n/resources/en.ts i18n.manifest.json i18n.state.json
git commit -m "feat: add new translation keys for feature X"
```
### For Translators
To translate strings to another language:
```bash
# 1. Check what needs translation
npm run i18n:verify
# 2. Edit the translation file (e.g., French)
vim src/i18n/resources/fr.ts
# 3. Update state to mark translations as current
npm run i18n:sync
# 4. Commit your changes
git add src/i18n/resources/fr.ts i18n.state.json
git commit -m "feat: add French translations for feature X"
```
## 📜 Available Commands
| Command | Purpose | When to Use |
|---------|---------|-------------|
| `npm run i18n:sync` | Update manifest and state files | After changing ANY translation files |
| `npm run i18n:verify` | Check for missing/stale translations | Before releasing (fails on issues) |
| `npm run i18n:status` | Show translation coverage summary | To check overall progress |
## 🔍 How It Works
### 1. Manifest File (`i18n.manifest.json`)
Contains SHA1 hashes of all English strings:
```json
{
"common.appName": "6458145fdd07ad08ff52a2e72d531588936bdca6",
"common.cancel": "77dfd2135f4db726c47299bb55be26f7f4525a46"
}
```
### 2. State File (`i18n.state.json`)
Tracks translation status for each locale:
```json
{
"fr": {
"common.appName": "6458145fdd07ad08ff52a2e72d531588936bdca6", // ✅ Up-to-date
"common.cancel": "" // ❌ Missing
}
}
```
### 3. CI/CD Enforcement
The GitHub Actions workflow automatically:
- Runs `npm run i18n:sync` on every PR/push
- Fails the build if manifest/state files are out of date
- Forces developers to commit synchronized files
## 🔄 Workflow Examples
### Adding a New English String
```bash
# 1. Add to en.ts
export const en = {
common: {
newFeature: 'My new feature' // ← Add this
}
}
# 2. Sync
npm run i18n:sync
# ✓ Generated manifest from "en.ts" with 1210 keys.
# ✓ Updated state for locales: fr.
# 3. Check status
npm run i18n:status
# fr: 99% translated, 0% stale (1 new missing key)
# 4. Commit everything
git add src/i18n/resources/en.ts i18n.manifest.json i18n.state.json
git commit -m "feat: add newFeature translation key"
```
### Translating to French
```bash
# 1. See what needs translation
npm run i18n:verify
# ❌ Missing translations:
# [fr] 1 missing keys:
# - common.newFeature
# 2. Add French translation
# Edit fr.ts and add: newFeature: 'Ma nouvelle fonctionnalité'
# 3. Mark as current
npm run i18n:sync
# ✓ Updated state for locales: fr.
# 4. Verify completion
npm run i18n:verify
# ✅ All translations are up-to-date.
# 5. Commit
git add src/i18n/resources/fr.ts i18n.state.json
git commit -m "feat: add French translation for newFeature"
```
### When English String Changes
```bash
# 1. Modify existing English string
# Change "My new feature" → "My awesome feature"
# 2. Sync (this marks French as stale)
npm run i18n:sync
# 3. Check status
npm run i18n:verify
# ⚠️ Stale translations (source text changed):
# [fr] 1 stale keys:
# - common.newFeature
# 4. Update French translation
# Edit fr.ts: "Ma nouvelle fonctionnalité" → "Ma fonctionnalité géniale"
# 5. Mark as current
npm run i18n:sync
# 6. Commit all changes
git add src/i18n/resources/en.ts src/i18n/resources/fr.ts i18n.manifest.json i18n.state.json
git commit -m "feat: improve newFeature translation"
```
## 🛠️ Technical Details
### File Structure
```
├── scripts/i18n-manager.mjs # Core management script
├── i18n.manifest.json # Source string hashes
├── i18n.state.json # Translation state tracking
├── src/i18n/resources/
│ ├── en.ts # English (source) translations
│ └── fr.ts # French translations
└── .github/workflows/test.yml # CI/CD enforcement
```
### Translation Detection Logic
The system considers a translation:
- **Missing**: Key doesn't exist in translation file
- **Untranslated**: Value is identical to English source
- **Stale**: Source hash doesn't match state hash
- **Current**: Source hash matches state hash
### CI/CD Integration
The workflow step in `.github/workflows/test.yml`:
```yaml
- name: Check i18n manifest is up-to-date
run: |
npm run i18n:sync
if [[ -n "$(git status --porcelain)" ]]; then
echo "❌ Error: i18n files are out of date"
exit 1
fi
```
This ensures:
- No untranslated strings slip into production
- Translation state is always tracked
- Developers must explicitly acknowledge translation needs
## 🎯 Best Practices
1. **Always run `npm run i18n:sync`** after modifying translation files
2. **Commit generated files** (`i18n.manifest.json`, `i18n.state.json`) with your changes
3. **Use descriptive keys** like `features.taskList.emptyState` instead of generic ones
4. **Group related translations** using nested objects for better organization
5. **Test locally** with `npm run i18n:verify` before pushing
## 🐛 Troubleshooting
**CI fails with "i18n files are out of date"**
```bash
npm run i18n:sync
git add i18n.manifest.json i18n.state.json
git commit -m "chore: update i18n manifest and state files"
```
**"Could not load source locale" error**
- Check that `src/i18n/resources/en.ts` exists and has valid syntax
- Ensure the export follows the pattern: `export const en = { ... }`
**Wrong translation statistics**
- Run `npm run i18n:sync` to recalculate
- Check that translated strings are actually different from English
## 📈 Translation Progress
Use these commands to track progress:
```bash
# Quick overview
npm run i18n:status
# Detailed missing/stale report
npm run i18n:verify
# After making changes
npm run i18n:sync
```
---
This system ensures translation quality and consistency while making the workflow as smooth as possible for both developers and translators.

1211
i18n.manifest.json Normal file
View file

File diff suppressed because it is too large Load diff

1213
i18n.state.json Normal file
View file

File diff suppressed because it is too large Load diff

5
package.json
View file

@ -19,7 +19,10 @@
"test:performance": "jest --testPathPatterns=performance --detectOpenHandles --passWithNoTests",
"test:build": "node -e \"console.log('Build test passed - plugin bundle created successfully')\"",
"lint": "eslint src/ --ext .ts",
"typecheck": "tsc --noEmit"
"typecheck": "tsc --noEmit",
"i18n:sync": "node scripts/i18n-manager.mjs sync",
"i18n:verify": "node scripts/i18n-manager.mjs verify",
"i18n:status": "node scripts/i18n-manager.mjs status"
},
"keywords": [],
"author": "",

344
scripts/i18n-manager.mjs Normal file
View file

@ -0,0 +1,344 @@
import fs from 'fs';
import path from 'path';
import crypto from 'crypto';
// --- Configuration ---
const SOURCE_LOCALE = 'en';
const RESOURCES_DIR = path.resolve('src/i18n/resources');
const MANIFEST_PATH = path.resolve('i18n.manifest.json');
const STATE_PATH = path.resolve('i18n.state.json');
// --- Helper Functions ---
/** Flattens a nested translation object into a single-level key-value map. */
function flatten(tree, prefix = '') {
const entries = {};
for (const [key, value] of Object.entries(tree)) {
const fullKey = prefix ? `${prefix}.${key}` : key;
if (typeof value === 'string') {
entries[fullKey] = value;
} else if (value && typeof value === 'object' && !Array.isArray(value)) {
Object.assign(entries, flatten(value, fullKey));
}
}
return entries;
}
/** Hashes a string using SHA1. */
function hash(str) {
return crypto.createHash('sha1').update(str, 'utf8').digest('hex');
}
/** Dynamically imports a .ts file by converting it to a temporary .mjs file. */
async function loadLocaleModule(locale) {
const filePath = path.join(RESOURCES_DIR, `${locale}.ts`);
if (!fs.existsSync(filePath)) {
throw new Error(`Locale file not found: ${filePath}`);
}
// Read the TypeScript file
let content = fs.readFileSync(filePath, 'utf8');
// Remove imports (they won't work in our simple conversion)
content = content.replace(/^import\s+.*?;$/gm, '');
// Remove type exports at the end
content = content.replace(/export\s+type\s+.*?;$/gm, '');
// Convert TypeScript export to ES module format that Node can import
// Replace "export const locale = {" with "export default {"
content = content.replace(
new RegExp(`export\\s+const\\s+${locale}\\s*:\\s*\\w+\\s*=\\s*`, 'g'),
'export default '
);
// Also handle the case without type annotation
content = content.replace(
new RegExp(`export\\s+const\\s+${locale}\\s*=\\s*`, 'g'),
'export default '
);
// Write to temporary .mjs file
const tempPath = path.join(RESOURCES_DIR, `.${locale}.temp.mjs`);
fs.writeFileSync(tempPath, content);
try {
// Import the temporary file with cache busting - need absolute path with file:// protocol
const absolutePath = path.resolve(tempPath);
const module = await import(`file://${absolutePath}?v=${Date.now()}`);
return module.default;
} finally {
// Clean up temporary file
if (fs.existsSync(tempPath)) {
fs.unlinkSync(tempPath);
}
}
}
/** Loads and flattens a locale file. */
async function getLocaleMap(locale) {
try {
const module = await loadLocaleModule(locale);
return flatten(module);
} catch (error) {
console.error(`Error loading locale "${locale}":`, error.message);
return {};
}
}
/** Safely reads a JSON file. */
function loadJson(filePath) {
if (!fs.existsSync(filePath)) return {};
try {
return JSON.parse(fs.readFileSync(filePath, 'utf8'));
} catch (error) {
console.error(`Error parsing JSON file ${filePath}:`, error.message);
return {};
}
}
/** Writes a JSON file with standardized formatting. */
function saveJson(filePath, data) {
fs.writeFileSync(filePath, JSON.stringify(data, null, 2) + '\n');
}
/** Gets all available locale files. */
function getAvailableLocales() {
if (!fs.existsSync(RESOURCES_DIR)) {
console.error(`Resources directory not found: ${RESOURCES_DIR}`);
return [];
}
return fs.readdirSync(RESOURCES_DIR)
.filter(file => file.endsWith('.ts') && !file.startsWith('.'))
.map(file => path.basename(file, '.ts'));
}
// --- Commands ---
/** `sync`: Updates manifest and state files. */
async function sync() {
console.log('Syncing i18n files...');
// 1. Update Manifest from source locale
const sourceMap = await getLocaleMap(SOURCE_LOCALE);
if (Object.keys(sourceMap).length === 0) {
console.error(`❌ Error: Could not load source locale "${SOURCE_LOCALE}". Check that the file exists and exports valid data.`);
process.exit(1);
}
const newManifest = {};
for (const key in sourceMap) {
newManifest[key] = hash(sourceMap[key]);
}
saveJson(MANIFEST_PATH, newManifest);
console.log(`✓ Generated manifest from "${SOURCE_LOCALE}.ts" with ${Object.keys(newManifest).length} keys.`);
// 2. Update State file for all other locales
const allLocales = getAvailableLocales();
const otherLocales = allLocales.filter(l => l !== SOURCE_LOCALE);
if (otherLocales.length === 0) {
console.log('ℹ️ No other locale files found. Only manifest updated.');
return;
}
const currentState = loadJson(STATE_PATH);
const newState = {};
for (const locale of otherLocales) {
console.log(`Processing locale: ${locale}`);
newState[locale] = {};
const translationMap = await getLocaleMap(locale);
for (const key in newManifest) {
const sourceHash = newManifest[key];
const translatedValue = translationMap[key];
if (translatedValue === undefined) {
// Key is missing from translation file
newState[locale][key] = ''; // Mark as untranslated
} else if (translatedValue === sourceMap[key]) {
// Value is identical to source, likely untranslated
newState[locale][key] = ''; // Mark as untranslated
} else {
// Check if the translation is stale
const previousStateHash = currentState[locale]?.[key];
if (previousStateHash === sourceHash) {
// Translation is up-to-date
newState[locale][key] = sourceHash;
} else {
// Translation exists but might be stale. We assume it's stale.
// A more advanced system might check if the translation itself has changed.
// For now, we preserve the old hash to signify it's stale.
newState[locale][key] = previousStateHash || '';
}
}
}
}
saveJson(STATE_PATH, newState);
console.log(`✓ Updated state for locales: ${otherLocales.join(', ')}.`);
console.log('\nSync complete. Review and commit the updated i18n files.');
}
/** `verify`: Checks for stale or missing translations. */
async function verify() {
console.log('Verifying i18n status...');
const manifest = loadJson(MANIFEST_PATH);
const state = loadJson(STATE_PATH);
if (Object.keys(manifest).length === 0) {
console.error('❌ Error: No manifest found. Run "npm run i18n:sync" first.');
process.exit(1);
}
const staleTranslations = [];
const missingTranslations = [];
const localeStats = {};
for (const locale in state) {
localeStats[locale] = { total: 0, translated: 0, stale: 0 };
for (const key in manifest) {
const sourceHash = manifest[key];
const stateHash = state[locale]?.[key];
localeStats[locale].total++;
if (!stateHash || stateHash === '') {
missingTranslations.push({ locale, key });
} else if (sourceHash !== stateHash) {
staleTranslations.push({ locale, key });
localeStats[locale].stale++;
} else {
localeStats[locale].translated++;
}
}
}
// Print statistics
console.log('\n📊 Translation Statistics:');
for (const [locale, stats] of Object.entries(localeStats)) {
const percentage = Math.round((stats.translated / stats.total) * 100);
console.log(` ${locale}: ${stats.translated}/${stats.total} (${percentage}%) up-to-date, ${stats.stale} stale`);
}
const hasIssues = staleTranslations.length > 0 || missingTranslations.length > 0;
if (missingTranslations.length > 0) {
console.error('\n❌ Missing translations:');
const byLocale = {};
missingTranslations.forEach(({ locale, key }) => {
if (!byLocale[locale]) byLocale[locale] = [];
byLocale[locale].push(key);
});
for (const [locale, keys] of Object.entries(byLocale)) {
console.error(` [${locale}] ${keys.length} missing keys:`);
keys.slice(0, 10).forEach(key => console.error(` - ${key}`));
if (keys.length > 10) {
console.error(` ... and ${keys.length - 10} more`);
}
}
}
if (staleTranslations.length > 0) {
console.error('\n⚠️ Stale translations (source text changed):');
const byLocale = {};
staleTranslations.forEach(({ locale, key }) => {
if (!byLocale[locale]) byLocale[locale] = [];
byLocale[locale].push(key);
});
for (const [locale, keys] of Object.entries(byLocale)) {
console.error(` [${locale}] ${keys.length} stale keys:`);
keys.slice(0, 10).forEach(key => console.error(` - ${key}`));
if (keys.length > 10) {
console.error(` ... and ${keys.length - 10} more`);
}
}
}
if (!hasIssues) {
console.log('\n✅ All translations are up-to-date.');
process.exit(0);
} else {
console.error(`\nPlease update translations and run 'npm run i18n:sync' to mark them as current.`);
process.exit(1);
}
}
/** `status`: Shows a summary of translation status without failing. */
async function status() {
console.log('Translation Status Report');
console.log('========================');
const manifest = loadJson(MANIFEST_PATH);
const state = loadJson(STATE_PATH);
if (Object.keys(manifest).length === 0) {
console.log('No manifest found. Run "npm run i18n:sync" to generate.');
return;
}
console.log(`Source locale: ${SOURCE_LOCALE}`);
console.log(`Total keys: ${Object.keys(manifest).length}`);
if (Object.keys(state).length === 0) {
console.log('No translation state found. Run "npm run i18n:sync" to generate.');
return;
}
console.log('\nTranslation Coverage:');
for (const locale in state) {
let translated = 0;
let stale = 0;
const total = Object.keys(manifest).length;
for (const key in manifest) {
const sourceHash = manifest[key];
const stateHash = state[locale]?.[key];
if (stateHash && stateHash !== '') {
if (sourceHash === stateHash) {
translated++;
} else {
stale++;
}
}
}
const percentage = Math.round((translated / total) * 100);
const stalePercentage = Math.round((stale / total) * 100);
console.log(` ${locale}: ${percentage}% translated, ${stalePercentage}% stale`);
}
}
// --- Main Execution ---
const command = process.argv[2];
(async () => {
try {
if (command === 'sync') {
await sync();
} else if (command === 'verify') {
await verify();
} else if (command === 'status') {
await status();
} else {
console.error(`Unknown command: ${command}`);
console.error('Available commands:');
console.error(' sync - Update manifest and state files');
console.error(' verify - Check for missing or stale translations (fails on issues)');
console.error(' status - Show translation status summary (non-failing)');
process.exit(1);
}
} catch (error) {
console.error('❌ Script failed:', error.message);
if (process.env.NODE_ENV === 'development') {
console.error(error.stack);
}
process.exit(1);
}
})();