diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml index 4117f5b7..d72465e1 100644 --- a/.github/workflows/deploy-docs.yml +++ b/.github/workflows/deploy-docs.yml @@ -41,6 +41,9 @@ jobs: env: NODE_ENV: production + - name: Audit website build + run: npm run audit:build + - name: Upload artifact uses: actions/upload-pages-artifact@v3 with: diff --git a/GEO_ROADMAP.md b/GEO_ROADMAP.md index b37523f9..5b9950ca 100644 --- a/GEO_ROADMAP.md +++ b/GEO_ROADMAP.md @@ -1,15 +1,15 @@ # NotEMD GEO Roadmap **Created:** 2026-06-12 -**Updated:** 2026-06-17 -**Status:** Phase 1-3 shipped; Phase 4 GitHub Pages reliability, language truth, and AI retrieval entry point complete in this batch. +**Updated:** 2026-06-19 +**Status:** Phase 1-3 shipped; Phase 4 GitHub Pages reliability, language truth, AI retrieval entry point, and build-output gate complete in this batch. **Scope:** Post-deployment GEO optimization for AI search engine visibility, documentation truth, and GitHub Pages reliability. --- ## Current Truth Snapshot -The website is a Docusaurus site under `website/`, deployed by `.github/workflows/deploy-docs.yml` on pushes to `main` that touch `website/**` or the workflow file. The workflow already uses `npm ci` and `npm run build` from the `website` working directory. +The website is a Docusaurus site under `website/`, deployed by `.github/workflows/deploy-docs.yml` on pushes to `main` that touch `website/**` or the workflow file. The workflow uses `npm ci`, `npm run build`, and `npm run audit:build` from the `website` working directory before uploading the Pages artifact. The current public language surface is deliberately narrow: @@ -20,6 +20,28 @@ The current public language surface is deliberately narrow: | Other locales | Not published | Do not add empty locale folders to `i18n.locales` | | Plugin UI i18n | Separate runtime feature | Do not confuse runtime UI language support with website documentation coverage | +## 2026-06-19 Pages / Language Gate Follow-Up + +### Findings + +| Finding | Evidence | Impact | +|---|---|---| +| zh-CN homepage still linked untranslated fallback docs | Built `website/build/zh-CN/index.html` contained `/obsidian-NotEMD/zh-CN/docs/intro`, `/zh-CN/docs/getting-started/quick-start`, `/zh-CN/docs/providers/overview`, and `/zh-CN/docs/pillar-ai-knowledge` | Users could still enter English fallback pages through the Chinese homepage even though crawlers were protected by noindex/sitemap filtering | +| Navbar/footer docs link localized into fallback docs | Docusaurus `docSidebar` / `to` links produced zh-CN docs routes for untranslated pages | UI language truth diverged from crawler truth | +| Published zh-CN scope was duplicated | FAQ exception lived separately in sitemap filtering, DocItem noindex logic, and homepage links | Future translation promotion could easily update one surface and miss another | +| Deploy workflow lacked a build-output audit | `.github/workflows/deploy-docs.yml` ran `npm run build` only | GitHub Pages could publish route drift that source review did not catch | + +### Fixes Landed In This Follow-Up + +| Fix | Files | Result | +|---|---|---| +| Add shared published-language scope | `website/src/lib/publishedLanguageScope.js` | zh-CN published ids/paths have one owner | +| Route zh-CN homepage untranslated docs to canonical English URLs | `website/src/pages/index.js` | zh-CN homepage keeps FAQ localized and sends other critical doc links to English canonical pages | +| Keep sitemap and noindex on the same rule | `website/docusaurus.config.js`, `website/src/theme/DocItem/Layout/index.js` | Sitemap filtering and fallback robots metadata consume the same published scope | +| Add built-output audit | `website/scripts/audit-build.cjs`, `website/package.json` | `npm run audit:build` checks real generated pages, sitemaps, and `llms.txt` | +| Gate GitHub Pages deploy | `.github/workflows/deploy-docs.yml` | Pages upload is blocked unless build output satisfies the language/GEO contract | +| Record maintainer workflow | `docs/maintainer/github-pages-language-geo-workflow.*` | Future locale promotion has a concrete owner and command path | + ## 2026-06-17 Pages / GEO Audit ### Findings @@ -113,8 +135,10 @@ Phase 4 exists because the previous roadmap optimized schema/content while leavi | Root broken markdown links removed | Fixed | No `/obsidian-NotEMD/` root broken-link warning in `npm run build` | | Published language policy matches code | Fixed | README + roadmap state `en` plus partial `zh-CN` | | Untranslated zh-CN fallback docs fenced | Fixed | zh-CN sitemap keeps only root + FAQ; fallback docs emit `noindex,follow` | +| zh-CN homepage does not route users into fallback docs | Fixed | `npm run audit:build` rejects `/zh-CN/docs/intro`, quick-start, provider overview, and pillar fallback links | +| GitHub Pages deploy has built-output language/GEO gate | Fixed | `.github/workflows/deploy-docs.yml` runs `npm run audit:build` after build | | AI retrieval entry point exists | Fixed | `website/build/llms.txt` | -| Test files excluded from this commit | Required | Commit file list must not include `src/tests/**` | +| Generated outputs excluded from source commit | Required | `website/build` and Slidev export artifacts remain ignored/uncommitted | ## Better GEO Strategy @@ -125,7 +149,8 @@ The next effective GEO strategy is truth-first and route-first, not locale-volum 3. **Use `llms.txt` as the compact answer-engine map.** It should point to canonical docs, explain language scope, and discourage answers based on generated exports or stale issue text. 4. **Keep schema accurate before making it richer.** Person, Organization, WebPage, TechArticle, FAQPage, citations, and concepts are useful only if URLs resolve and claims match source pages. 5. **Consolidate thin provider pages before adding more.** Stub-like provider pages dilute crawl quality; expand them with real setup, request semantics, and troubleshooting or merge them into stronger overview pages. -6. **Measure after deploy, then tune.** Re-run Google indexing checks and AI visibility tests only after the fixed Pages build is deployed. +6. **Gate the built output, not only the source.** `npm run audit:build` must pass before Pages upload because Docusaurus locale behavior can change the final route graph. +7. **Measure after deploy, then tune.** Re-run Google indexing checks and AI visibility tests only after the fixed Pages build is deployed. ### Critical Path Translation Plan @@ -151,19 +176,21 @@ Until this set is translated, zh-CN should stay explicitly partial. | GitHub Pages root route missing | Fixed | Root homepage now exists for default and zh-CN locale | | Docusaurus v4 markdown config breakage | Fixed | Broken markdown hook moved under `markdown.hooks` | | Untranslated zh-CN fallback docs dilute language quality | Controlled | Exclude fallback docs from sitemap and emit `noindex,follow` until translated | +| zh-CN UI links users into fallback docs | Controlled | Homepage, navbar, and footer docs entry points now route untranslated critical docs to canonical English URLs; audit blocks regressions | | Thin provider pages dilute crawl budget | Open | Expand or consolidate provider pages before adding more | | Mermaid bundle size on non-diagram pages | Accepted | Docusaurus/theme tradeoff; not the current GEO blocker | -| Hand-maintained `llms.txt` can drift | Open | Update it when canonical docs, language status, or provider docs change | -| Test or generated export files accidentally entering main | Controlled in this batch | Stage production/docs files only; leave tests and generated exports out of the commit | +| Hand-maintained `llms.txt` can drift | Controlled | `npm run audit:build` checks the language-scope markers; update content when canonical docs, language status, or provider docs change | +| Test or generated export files accidentally entering main | Controlled in this batch | Stage production/docs/test contract files only; leave generated exports out of the commit | ## Success Metrics | Timeframe | Metric | Target | Current | |-----------|--------|--------|---------| -| Immediate | Docusaurus build | No root broken-link or deprecated markdown-hook warnings | Verified 2026-06-17 | -| Immediate | Root pages | `index.html` and `zh-CN/index.html` exist | Verified 2026-06-17 | -| Immediate | zh-CN fallback fence | Sitemap excludes untranslated docs; fallback docs noindex | Verified 2026-06-17 | -| Immediate | AI retrieval map | `llms.txt` exists in build output | Verified 2026-06-17 | +| Immediate | Docusaurus build | No root broken-link or deprecated markdown-hook warnings | Verified 2026-06-19 | +| Immediate | Root pages | `index.html` and `zh-CN/index.html` exist | Verified 2026-06-19 | +| Immediate | zh-CN fallback fence | Sitemap excludes untranslated docs; fallback docs noindex | Verified 2026-06-19 by `npm run audit:build` | +| Immediate | zh-CN homepage language routes | Untranslated critical docs route to canonical English URLs; FAQ stays localized | Verified 2026-06-19 by `npm run audit:build` | +| Immediate | AI retrieval map | `llms.txt` exists in build output and records language scope | Verified 2026-06-19 by `npm run audit:build` | | 2 weeks post-deploy | Docusaurus site indexed | Google `site:` returns results | Unknown | | 2 weeks post-deploy | First AI citation | 1+ ChatGPT/Perplexity/GLM mention | 0 baseline | | 1 month post-deploy | Citation rate | 5-10% for core keywords | 0 baseline | @@ -189,7 +216,7 @@ Until this set is translated, zh-CN should stay explicitly partial. | Date | Action | |------|--------| -| 2026-06-17 | Verify fixed website build, root pages, zh-CN root, and `llms.txt` output | +| 2026-06-19 | Verify fixed website build, root pages, zh-CN root, canonical English fallback routing, zh-CN sitemap/noindex fence, and `llms.txt` output through `npm run audit:build` | | After deploy | Submit sitemap and inspect canonical root/locale root in Search Console | | 2026-07-01 | Retest GLM baseline in EN + ZH after the fixed Pages deployment | | 2026-08-01 | Second retest. If citations remain 0, audit indexed pages and provider-page thinness | diff --git a/docs/README.md b/docs/README.md index a1e53869..803e82c6 100644 --- a/docs/README.md +++ b/docs/README.md @@ -7,6 +7,7 @@ This directory contains repository-level documentation for maintainers and contr ## User And Maintainer Entry Points - [Language Hub](./i18n/README.md) +- [GitHub Pages Language And GEO Workflow](./maintainer/github-pages-language-geo-workflow.md) - [Chapter Split + TOC Extraction](./chapter-split-toc.md) - [Release Workflow](./maintainer/release-workflow.md) - [Release Notes 1.8.2](./releases/1.8.2.md) diff --git a/docs/README.zh-CN.md b/docs/README.zh-CN.md index bbd7e843..83574031 100644 --- a/docs/README.zh-CN.md +++ b/docs/README.zh-CN.md @@ -7,6 +7,7 @@ ## 用户与维护者入口 - [语言中心](./i18n/README_zh.md) +- [GitHub Pages 语言与 GEO 工作流](./maintainer/github-pages-language-geo-workflow.zh-CN.md) - [章节拆分 + TOC 提取](./chapter-split-toc.zh-CN.md) - [发布流程](./maintainer/release-workflow.zh-CN.md) - [1.8.2 发布说明](./releases/1.8.2.zh-CN.md) diff --git a/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.md b/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.md index 461ef6d7..0813ad52 100644 --- a/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.md +++ b/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.md @@ -627,6 +627,33 @@ Next direction: 6. any upstream Slidev skill PR should stay generic: full references, built-in/configured theme preference, closed frontmatter, readable transforms for large diagrams/tables/code, and browser-sampled build verification. NoteMD vault paths, local fork paths, layout audit internals, and the `architecture.zh-CN.md` fixture should remain project-local. 7. standalone acceptance should now be treated as a tracked maintainer evidence package, not as an implicit chat summary: if the strict report or archived output path changes, update `docs/maintainer/slidev-standalone-acceptance-2026-06-18.*` or create a new dated acceptance file in the same shape. +### Batch G: gate GitHub Pages language and GEO output + +Priority: `P0/P1` + +Background: + +1. the 2026-06-17 Pages pass fixed missing root pages, stale language claims, sitemap filtering, and `llms.txt`, but it still relied mostly on source review plus `npm run build`; +2. the real generated zh-CN homepage still exposed `/obsidian-NotEMD/zh-CN/docs/intro`, `/zh-CN/docs/getting-started/quick-start`, `/zh-CN/docs/providers/overview`, and `/zh-CN/docs/pillar-ai-knowledge` as user-facing links even though those pages are untranslated fallback docs; +3. that means crawler protection was better than user-facing language truth: `noindex,follow` and sitemap filtering cannot compensate for a Chinese homepage that routes users into English fallback pages. + +Current landed status: + +1. `website/src/lib/publishedLanguageScope.js` now owns the published zh-CN doc scope: currently only `faq`; +2. `website/docusaurus.config.js` consumes that scope for sitemap filtering and uses canonical English URLs for the navbar/footer docs entry where untranslated zh-CN docs must not be implied; +3. `website/src/theme/DocItem/Layout/index.js` consumes the same scope for fallback-doc `noindex,follow`; +4. `website/src/pages/index.js` consumes the same scope so zh-CN homepage links FAQ to `/zh-CN/docs/faq` and routes untranslated critical docs to canonical English URLs; +5. `website/scripts/audit-build.cjs` now audits the built output for root pages, canonical/WebPage JSON-LD URLs, zh-CN homepage route truth, noindex fallback behavior, sitemap language scope, and `llms.txt` language markers; +6. `website/package.json` exposes `npm run audit:build`, and `.github/workflows/deploy-docs.yml` runs it after `npm run build` before uploading the Pages artifact; +7. `docs/maintainer/github-pages-language-geo-workflow.*`, `website/README.md`, and `GEO_ROADMAP.md` now describe the deploy gate and the promotion rule for future zh-CN translations. + +Next direction: + +1. do not add locales as GEO surface area until the critical path is translated and reviewed; +2. promote future zh-CN docs by updating the translation and `publishedLanguageScope.js` in the same batch, then running `npm --prefix website run build && npm --prefix website run audit:build`; +3. after the route/language gate stays stable, add a provider-page quality audit for thin pages; this is a content-depth problem, not a locale-routing problem; +4. keep `website/build` and generated Slidev export artifacts out of source commits. + ## 6. Documentation Sync Rule Any future change that updates the provider-settings/model-discovery lane must re-check, at minimum: @@ -650,6 +677,18 @@ Any future change that updates the Slidev export lane must re-check, at minimum: 8. `src/ui/NotemdSidebarView.ts` 9. `package.json` +Any future change that updates the GitHub Pages / GEO / website language lane must re-check, at minimum: + +1. `GEO_ROADMAP.md` +2. `website/README.md` +3. `website/docusaurus.config.js` +4. `website/src/lib/publishedLanguageScope.js` +5. `website/src/pages/index.js` +6. `website/src/theme/DocItem/Layout/index.js` +7. `website/static/llms.txt` +8. `website/scripts/audit-build.cjs` +9. `.github/workflows/deploy-docs.yml` + ## 7. Verification Gate Any update that changes the truth claims in this document should still finish with: @@ -668,6 +707,11 @@ If the update touches Slidev export, also run: After the rendered-layout gate lands, Slidev export closure must also include a non-empty layout-audit report for `docs/architecture.zh-CN.md` with zero `overflow` and zero `unreadable-scale` findings. +If the update touches GitHub Pages / GEO / website language scope, also run: + +1. `npm --prefix website run build` +2. `npm --prefix website run audit:build` + ## 8. Bottom Line Current `main` no longer needs another “did the provider settings lane really land?” argument. @@ -679,4 +723,5 @@ The real current questions are now: 3. can Stage-C local-KB / file-selection / chapter-split work deepen mixed-corpus quality evidence instead of relitigating feature existence; 4. can the widened bounded provider discovery surface remain shared-core, lightweight, and honest as a maintenance lane rather than becoming the excuse for broader architectural claims; 5. can Slidev export keep using a real UI-equivalent workflow proof and add rendered-layout quality gates instead of regressing to “the CLI can build, so the buttons must work” as weak evidence; -6. can current truth documents keep tracking the real shipped branch boundary quickly enough that future sessions do not regress back to `1.9.0/1.9.1`-era wording. +6. can GitHub Pages keep route, sitemap, canonical, `llms.txt`, and visible language entry points aligned through a build-output gate instead of source-only review; +7. can current truth documents keep tracking the real shipped branch boundary quickly enough that future sessions do not regress back to `1.9.0/1.9.1`-era wording. diff --git a/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.zh-CN.md b/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.zh-CN.md index 708e6451..242331f2 100644 --- a/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.zh-CN.md +++ b/docs/brainstorms/2026-05-28-mainline-progress-audit-and-next-level-direction.zh-CN.md @@ -683,13 +683,18 @@ GitHub Pages 这条线当前最大的结构风险不是缺少更多页面，而 2. `website/static/llms.txt` 已提供 canonical answer-engine map； 3. Docusaurus markdown link hook 已迁到当前配置位置； 4. 未翻译的 zh-CN fallback docs 已从 sitemap 排除，并标记 `noindex,follow`； -5. website README 和 GEO roadmap 已同步真实语言边界。 +5. `website/src/lib/publishedLanguageScope.js` 现在统一拥有已发布 zh-CN doc scope，目前只有 `faq`； +6. `website/src/pages/index.js` 会把 zh-CN 首页上未翻译 critical docs 的入口导向 canonical 英文 URL，只把 FAQ 保持在 `/zh-CN/docs/faq`； +7. `website/docusaurus.config.js` 与 `website/src/theme/DocItem/Layout/index.js` 复用同一份发布范围，分别用于 sitemap filtering 与 fallback doc 的 `noindex,follow`； +8. `website/scripts/audit-build.cjs` 会审计 build 产物中的 root pages、canonical/WebPage JSON-LD URL、zh-CN 首页路由、noindex fallback、sitemap 语言范围与 `llms.txt` 语言声明； +9. `website/package.json` 暴露 `npm run audit:build`，`.github/workflows/deploy-docs.yml` 在 `npm run build` 后、上传 Pages artifact 前运行该审计； +10. `docs/maintainer/github-pages-language-geo-workflow.*`、website README 与 GEO roadmap 已同步真实语言边界和部署 gate。 下一步： -1. 把 zh-CN critical path 补齐到 homepage、FAQ、intro、installation、quick-start、configuration、provider overview 与 pillar page； -2. 给 `website` 增加轻量 smoke gate，检查 `build/index.html`、`build/zh-CN/index.html`、`build/llms.txt` 和 sitemap 是否存在； -3. 扩写或合并 provider thin pages，优先补真实 provider setup、endpoint/header 语义、model discovery 与 troubleshooting； +1. 按 intro、installation、quick-start、configuration、provider overview、pillar page 的顺序补齐 zh-CN critical path；homepage 与 FAQ 已是当前已发布范围； +2. 每个页面从 fallback 晋升为已发布页面时，必须在同批更新翻译与 `publishedLanguageScope.js`，并运行 `npm --prefix website run build && npm --prefix website run audit:build`； +3. 扩写或合并 provider thin pages，优先补真实 provider setup、endpoint/header 语义、model discovery 与 troubleshooting；这是内容质量问题，不应混进 locale route gate； 4. fixed Pages 部署后再进行 Search Console 与 AI visibility 复测，不要用本地未部署结果过早判断 GEO 成败。 ### Batch G：把 Slidev export 从一次性 smoke 提升为真实工作流门禁 @@ -747,9 +752,12 @@ GitHub Pages 这条线当前最大的结构风险不是缺少更多页面，而 1. `GEO_ROADMAP.md` 2. `website/README.md` 3. `website/docusaurus.config.js` -4. `website/src/pages/index.js` -5. `website/static/llms.txt` -6. `.github/workflows/deploy-docs.yml` +4. `website/src/lib/publishedLanguageScope.js` +5. `website/src/pages/index.js` +6. `website/src/theme/DocItem/Layout/index.js` +7. `website/static/llms.txt` +8. `website/scripts/audit-build.cjs` +9. `.github/workflows/deploy-docs.yml` 未来任何会更新 Slidev export 轨道真值的改动，至少都应同步检查： @@ -771,12 +779,10 @@ GitHub Pages 这条线当前最大的结构风险不是缺少更多页面，而 2. `npm test -- --runInBand` 3. `npm run audit:i18n-ui` 4. `npm run audit:render-host` -5. `cd website && npm run build` -6. `test -f website/build/index.html` -7. `test -f website/build/zh-CN/index.html` -8. `test -f website/build/llms.txt` -9. `git diff --check` -10. clean 的 `git status --short --branch` +5. `npm --prefix website run build` +6. `npm --prefix website run audit:build` +7. `git diff --check` +8. clean 的 `git status --short --branch` 如果改动触及 Slidev export，还必须额外执行： @@ -785,6 +791,11 @@ GitHub Pages 这条线当前最大的结构风险不是缺少更多页面，而 等渲染后布局 gate 落地后，Slidev export 收尾还必须基于 `docs/architecture.zh-CN.md` 输出非空 layout-audit report，并且 `overflow` 与 `unreadable-scale` 均为零。 +如果改动触及 GitHub Pages / GEO / website language scope，还必须额外执行： + +1. `npm --prefix website run build` +2. `npm --prefix website run audit:build` + ## 8. Bottom Line 当前 `main` 已经不再需要再做一次“provider settings 这条线到底有没有落地”的论证。 @@ -795,6 +806,6 @@ GitHub Pages 这条线当前最大的结构风险不是缺少更多页面，而 2. 当前 bounded CLI 分层能否继续显式保持，而未来任何 path-based promotion 都坚持 contract-first，而不是 convenience-first； 3. Stage-C local-KB / file-selection / chapter-split 工作能否继续补强 mixed-corpus 质量证据，而不是反复重谈“功能是否存在”； 4. 当前更宽的 bounded provider discovery surface 能否继续保持 shared-core、lightweight 且边界诚实，并作为维护轨道而不是更大架构声明的借口； -5. GitHub Pages / GEO / website language 轨道能否坚持 route-first、truth-first，而不是继续用空 locale 或过时 README 制造弱信号； +5. GitHub Pages / GEO / website language 轨道能否坚持 route-first、truth-first，并让 route、sitemap、canonical、`llms.txt` 与可见语言入口通过 build-output gate 保持一致； 6. Slidev export 能否继续用真实 UI-equivalent workflow 验证，并补上渲染后布局质量门，而不是退回到“CLI 能 build 就算按钮可用”的弱证明； 7. 当前真值文档能否足够快地跟上真实发货分支边界，避免未来会话又退回到 `1.9.0/1.9.1` 时代的旧措辞。 diff --git a/docs/maintainer/github-pages-language-geo-workflow.md b/docs/maintainer/github-pages-language-geo-workflow.md new file mode 100644 index 00000000..917f67e2 --- /dev/null +++ b/docs/maintainer/github-pages-language-geo-workflow.md @@ -0,0 +1,100 @@ +# GitHub Pages Language And GEO Workflow + +Language: **English** | [简体中文](./github-pages-language-geo-workflow.zh-CN.md) + +This workflow records the current public documentation-site gate for `website/`. It is intentionally separate from runtime plugin i18n and from Slidev export acceptance. + +## Current Contract + +The website publishes one complete language surface and one partial language surface: + +1. English is the canonical complete documentation surface under `https://jacobinwwey.github.io/obsidian-NotEMD/docs/...`. +2. Simplified Chinese currently publishes the homepage and FAQ only. +3. Untranslated zh-CN fallback docs may still be generated by Docusaurus, but they must be fenced with `noindex,follow` and excluded from the zh-CN sitemap. +4. zh-CN homepage entry points for untranslated docs must route to canonical English URLs, not to `/zh-CN/docs/...` fallback pages. +5. `llms.txt` must describe the same language boundary so answer engines do not infer full multilingual coverage. + +## Implemented Gate + +The blocking gate is: + +```bash +cd website +npm run build +npm run audit:build +``` + +`npm run audit:build` executes `website/scripts/audit-build.cjs`. The script checks the built output, not just source files: + +1. root pages exist: `build/index.html` and `build/zh-CN/index.html`; +2. root pages have the expected `lang`, canonical URL, and WebPage JSON-LD URL; +3. zh-CN homepage links untranslated critical docs to canonical English URLs; +4. zh-CN homepage keeps FAQ on `/zh-CN/docs/faq`; +5. untranslated zh-CN docs emit `noindex,follow`; +6. published zh-CN FAQ does not emit `noindex,follow`; +7. sitemap output includes canonical English docs and excludes untranslated zh-CN docs; +8. `llms.txt` records the language scope. + +The GitHub Pages workflow now runs this audit before uploading the Pages artifact: + +```text +.github/workflows/deploy-docs.yml + -> npm run build + -> npm run audit:build + -> upload-pages-artifact +``` + +## Source Ownership + +The language scope is shared through: + +```text +website/src/lib/publishedLanguageScope.js +``` + +Current values: + +```text +publishedZhCnDocIds = faq +publishedZhCnDocPaths = /docs/faq +``` + +This module is consumed by: + +1. `website/docusaurus.config.js` for sitemap filtering; +2. `website/src/theme/DocItem/Layout/index.js` for fallback-doc `noindex,follow`; +3. `website/src/pages/index.js` for zh-CN homepage routing. + +This avoids a previous drift pattern where sitemap, noindex, and homepage links each carried a separate copy of the same exception. + +For canonical English links rendered from a zh-CN page, use the canonical origin-relative path with `autoAddBaseUrl: false` and `data-noBrokenLinkCheck`. Docusaurus checks broken links against the current locale route table, so these intentional cross-locale links must not be "fixed" back to localized fallback routes. `npm run audit:build` is the guard that proves the final path is correct. + +## What Was Wrong + +The previous Pages state was better than the old roadmap, but still incomplete: + +1. Docusaurus generated untranslated zh-CN fallback docs. +2. Those fallback docs were noindexed and removed from sitemap, which protected crawlers. +3. The zh-CN homepage, navbar, and footer still linked users into `/zh-CN/docs/intro` and similar fallback routes. +4. The deploy workflow only ran `npm run build`; it did not enforce the language route contract. + +That means the site could pass build while still presenting a weak public language surface. `noindex` is not enough when the UI itself sends readers to the wrong locale route. + +## Better GEO Direction + +The better GEO strategy is still truth-first: + +1. keep English canonical until specific zh-CN pages are fully translated and reviewed; +2. only promote a zh-CN doc path by adding it to `publishedLanguageScope.js` in the same batch as the translation; +3. require `npm run audit:build` before Pages deploy; +4. keep `llms.txt`, sitemap, homepage links, and noindex behavior aligned; +5. expand or consolidate thin provider pages before adding more locale surfaces. + +Do not use empty locale folders or fallback English pages as GEO surface area. That produces weak hreflang signals and worse user trust. + +## Next Direction + +1. Translate and review the zh-CN critical path in this order: intro, installation, quick-start, configuration, provider overview, AI knowledge pillar. +2. Each translated page promotion must update `publishedLanguageScope.js` and rerun `npm --prefix website run build && npm --prefix website run audit:build`. +3. Add a small provider-page quality audit only after the language route gate stays stable; provider thinness is a content quality problem, not a routing problem. +4. Keep generated `website/build` output ignored and uncommitted. diff --git a/docs/maintainer/github-pages-language-geo-workflow.zh-CN.md b/docs/maintainer/github-pages-language-geo-workflow.zh-CN.md new file mode 100644 index 00000000..4fda6b68 --- /dev/null +++ b/docs/maintainer/github-pages-language-geo-workflow.zh-CN.md @@ -0,0 +1,100 @@ +# GitHub Pages 语言与 GEO 工作流 + +语言: [English](./github-pages-language-geo-workflow.md) | **简体中文** + +本文记录 `website/` 当前的公开文档站门禁。它与插件 runtime i18n、Slidev export 验收是三条不同轨道。 + +## 当前契约 + +文档站当前只发布一个完整语言面和一个部分语言面： + +1. 英文是完整 canonical 文档面，路径为 `https://jacobinwwey.github.io/obsidian-NotEMD/docs/...`。 +2. 简体中文当前只发布首页和 FAQ。 +3. Docusaurus 仍可能生成未翻译的 zh-CN fallback docs，但这些页面必须标记 `noindex,follow`，并从 zh-CN sitemap 排除。 +4. zh-CN 首页上指向未翻译文档的入口必须回到 canonical 英文 URL，不能进入 `/zh-CN/docs/...` fallback 页面。 +5. `llms.txt` 必须声明同一语言边界，避免 answer engine 推断出不存在的完整多语言覆盖。 + +## 已落地门禁 + +阻断式门禁是： + +```bash +cd website +npm run build +npm run audit:build +``` + +`npm run audit:build` 执行 `website/scripts/audit-build.cjs`。脚本审计的是 build 产物，而不只是源码： + +1. root 页面存在：`build/index.html` 与 `build/zh-CN/index.html`； +2. root 页面具有预期的 `lang`、canonical URL 与 WebPage JSON-LD URL； +3. zh-CN 首页把未翻译 critical docs 链到 canonical 英文 URL； +4. zh-CN 首页把 FAQ 保持在 `/zh-CN/docs/faq`； +5. 未翻译 zh-CN docs 输出 `noindex,follow`； +6. 已发布 zh-CN FAQ 不输出 `noindex,follow`； +7. sitemap 包含 canonical 英文 docs，并排除未翻译 zh-CN docs； +8. `llms.txt` 记录语言范围。 + +GitHub Pages workflow 现在会在上传 Pages artifact 前运行这个审计： + +```text +.github/workflows/deploy-docs.yml + -> npm run build + -> npm run audit:build + -> upload-pages-artifact +``` + +## Source Ownership + +语言发布范围由这一处共享： + +```text +website/src/lib/publishedLanguageScope.js +``` + +当前值： + +```text +publishedZhCnDocIds = faq +publishedZhCnDocPaths = /docs/faq +``` + +这个模块被三处消费： + +1. `website/docusaurus.config.js` 用于 sitemap 过滤； +2. `website/src/theme/DocItem/Layout/index.js` 用于 fallback doc 的 `noindex,follow`； +3. `website/src/pages/index.js` 用于 zh-CN 首页路由。 + +这样可以避免旧问题：sitemap、noindex 与首页链接各自维护一份 “FAQ 例外”，后续很容易漂移。 + +从 zh-CN 页面渲染 canonical 英文链接时，应使用 canonical origin-relative path，并显式设置 `autoAddBaseUrl: false` 与 `data-noBrokenLinkCheck`。Docusaurus 的 broken-link 检查基于当前 locale route table，这类有意的跨 locale 链接不能被“修”回本地化 fallback 路由。最终路径是否正确由 `npm run audit:build` 负责验证。 + +## 之前错在哪里 + +此前 Pages 状态已经比旧 roadmap 好，但仍未闭环： + +1. Docusaurus 会生成未翻译的 zh-CN fallback docs。 +2. 这些 fallback docs 已经 noindex 且从 sitemap 排除，这保护了 crawler。 +3. 但 zh-CN 首页、navbar 和 footer 仍会把用户送进 `/zh-CN/docs/intro` 这类 fallback 路由。 +4. 部署 workflow 只跑 `npm run build`，没有阻断语言路由契约漂移。 + +所以 build 能通过，并不等于公开语言面可靠。`noindex` 不能替代 UI 入口的正确路由。 + +## 更好的 GEO 方向 + +更好的 GEO 策略仍然是 truth-first： + +1. 在具体 zh-CN 页面翻译并 review 前，英文继续作为 canonical； +2. 只有在同批完成翻译时，才把对应 doc path 加入 `publishedLanguageScope.js`； +3. Pages deploy 前必须跑 `npm run audit:build`； +4. `llms.txt`、sitemap、首页链接与 noindex 行为必须保持一致； +5. 增加更多 locale 之前，先扩写或合并 thin provider pages。 + +不要把空 locale folder 或 fallback English 页面当成 GEO 面积。这会制造弱 hreflang 信号，也会损害用户信任。 + +## 下一步 + +1. 按顺序翻译并 review zh-CN critical path：intro、installation、quick-start、configuration、provider overview、AI knowledge pillar。 +2. 每个页面从 fallback 晋升为已发布页面时，都必须更新 `publishedLanguageScope.js`，并执行 `npm --prefix website run build && npm --prefix website run audit:build`。 +3. 语言路由门禁稳定后，再添加 provider-page quality audit；provider thinness 是内容质量问题，不是路由问题。 +4. `website/build` 继续作为生成产物忽略，不提交到 main。 diff --git a/src/tests/mainlineProgressAuditContract.test.ts b/src/tests/mainlineProgressAuditContract.test.ts index 87cb0bab..76e84a71 100644 --- a/src/tests/mainlineProgressAuditContract.test.ts +++ b/src/tests/mainlineProgressAuditContract.test.ts @@ -22,6 +22,12 @@ describe('mainline progress audit contract', () => { const matrixDocZh = readDoc('docs/brainstorms/2026-05-20-unified-follow-through-matrix.zh-CN.md'); const standaloneAcceptanceDoc = readDoc('docs/maintainer/slidev-standalone-acceptance-2026-06-18.md'); const standaloneAcceptanceDocZh = readDoc('docs/maintainer/slidev-standalone-acceptance-2026-06-18.zh-CN.md'); + const pagesLanguageGeoWorkflowDoc = readDoc('docs/maintainer/github-pages-language-geo-workflow.md'); + const pagesLanguageGeoWorkflowDocZh = readDoc('docs/maintainer/github-pages-language-geo-workflow.zh-CN.md'); + const docsReadme = readDoc('docs/README.md'); + const docsReadmeZh = readDoc('docs/README.zh-CN.md'); + const websiteReadme = readDoc('website/README.md'); + const geoRoadmap = readDoc('GEO_ROADMAP.md'); test('records the current Stage-B2/C/D baseline and fixture anchor in both canonical progress docs', () => { expect(progressDoc).toContain('### 2.7 Current `890b21b` Stage-B2/C/D follow-through baseline'); @@ -267,4 +273,35 @@ describe('mainline progress audit contract', () => { expect(standaloneAcceptanceDoc).toContain('Do not commit test/generated output to main'); expect(standaloneAcceptanceDocZh).toContain('测试/生成输出不要提交到 main'); }); + + test('locks the GitHub Pages language and GEO build-output gate', () => { + for (const content of [pagesLanguageGeoWorkflowDoc, pagesLanguageGeoWorkflowDocZh]) { + expect(content).toContain('website/scripts/audit-build.cjs'); + expect(content).toContain('website/src/lib/publishedLanguageScope.js'); + expect(content).toContain('npm run audit:build'); + expect(content).toContain('noindex,follow'); + expect(content).toContain('llms.txt'); + expect(content).toContain('/zh-CN/docs/faq'); + expect(content).toContain('canonical'); + expect(content).toContain('autoAddBaseUrl: false'); + expect(content).toContain('data-noBrokenLinkCheck'); + } + + expect(docsReadme).toContain('GitHub Pages Language And GEO Workflow'); + expect(docsReadmeZh).toContain('GitHub Pages 语言与 GEO 工作流'); + + for (const content of [progressDoc, progressDocZh, websiteReadme, geoRoadmap]) { + expect(content).toContain('website/scripts/audit-build.cjs'); + expect(content).toContain('website/src/lib/publishedLanguageScope.js'); + expect(content).toContain('npm run audit:build'); + expect(content).toContain('noindex,follow'); + expect(content).toContain('llms.txt'); + } + + expect(progressDoc).toContain('build-output gate'); + expect(progressDocZh).toContain('build-output gate'); + expect(websiteReadme).toContain('zh-CN homepage sends untranslated docs to canonical English URLs'); + expect(geoRoadmap).toContain('zh-CN homepage does not route users into fallback docs'); + expect(geoRoadmap).toContain('.github/workflows/deploy-docs.yml'); + }); }); diff --git a/website/README.md b/website/README.md index 89bad9d2..e9d824c2 100644 --- a/website/README.md +++ b/website/README.md @@ -23,13 +23,23 @@ This starts a local development server at `http://localhost:3000`. ```bash npm run build +npm run audit:build ``` -Generates static content into the `build` directory. +Generates static content into the `build` directory, then audits the built Pages output. + +The audit checks the public contract that source review cannot prove by itself: + +- root and zh-CN root pages exist; +- canonical and JSON-LD URLs match GitHub Pages routes; +- zh-CN homepage sends untranslated docs to canonical English URLs; +- zh-CN FAQ remains published under the zh-CN route; +- untranslated zh-CN fallback docs are `noindex,follow` and excluded from the zh-CN sitemap; +- `llms.txt` still states the real language boundary. ## Deployment -The site auto-deploys to GitHub Pages on push to `main` branch (via `.github/workflows/deploy-docs.yml`). +The site auto-deploys to GitHub Pages on push to `main` branch (via `.github/workflows/deploy-docs.yml`). The deployment workflow runs both `npm run build` and `npm run audit:build` before uploading the Pages artifact. **Live URL**: https://jacobinwwey.github.io/obsidian-NotEMD/ @@ -59,7 +69,12 @@ The site auto-deploys to GitHub Pages on push to `main` branch (via `.github/wor - Lists canonical docs, provider/runtime topics, and the current language coverage boundary - Keeps GEO strategy focused on verified source pages instead of empty locale expansion -### 6. FAQ with FAQPage Schema +### 6. Build Output Language Gate +- `website/scripts/audit-build.cjs`: blocks Pages deployment when build output violates the published language scope +- `website/src/lib/publishedLanguageScope.js`: single source for published zh-CN doc ids and paths +- Keeps sitemap filtering, fallback `noindex,follow`, and homepage routing aligned + +### 7. FAQ with FAQPage Schema - `docs/faq.mdx`: 12 Q&A pairs - Simplified Chinese translation: `i18n/zh-CN/docusaurus-plugin-content-docs/current/faq.mdx` - 43% of AI citations come from FAQ content (BrightEdge data) @@ -73,6 +88,8 @@ website/ ├── static/ │ ├── llms.txt # AI crawler / answer-engine entry point │ └── img/ +├── scripts/ +│ └── audit-build.cjs # Built-output Pages language/GEO gate ├── docs/ # English docs │ ├── intro.mdx │ ├── faq.mdx @@ -85,6 +102,8 @@ website/ ├── src/ │ ├── pages/ │ │ └── index.js # Locale-aware root homepage +│ ├── lib/ +│ │ └── publishedLanguageScope.js │ ├── components/ │ │ └── TLDR/ # TLDR component │ ├── theme/ @@ -124,7 +143,14 @@ citations: # Adds Schema.org "citation" Do not add a locale to `i18n.locales` just because a translation folder exists. Docusaurus publishes fallback English docs under a locale path when docs are untranslated, which creates weak hreflang signals for search and AI crawlers. -Current zh-CN policy: the localized homepage and FAQ are published, while untranslated zh-CN doc fallbacks are excluded from sitemap output and marked `noindex,follow` by the swizzled doc layout. A locale should only become fully indexable after the critical path pages are translated and reviewed. +Current zh-CN policy: the localized homepage and FAQ are published, while untranslated zh-CN doc fallbacks are excluded from sitemap output and marked `noindex,follow` by the swizzled doc layout. The zh-CN homepage must link untranslated docs to canonical English URLs, not to zh-CN fallback routes. A locale should only become fully indexable after the critical path pages are translated and reviewed. + +When promoting a zh-CN doc from fallback to published content, update `website/src/lib/publishedLanguageScope.js` in the same change as the translation and rerun: + +```bash +npm run build +npm run audit:build +``` ## License diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js index a998fb33..38ca08b0 100644 --- a/website/docusaurus.config.js +++ b/website/docusaurus.config.js @@ -5,20 +5,21 @@ // See: https://docusaurus.io/docs/api/docusaurus-config import {themes as prismThemes} from 'prism-react-renderer'; +import {publishedZhCnDocPaths} from './src/lib/publishedLanguageScope.js'; const siteUrl = 'https://jacobinwwey.github.io'; const baseUrl = '/obsidian-NotEMD/'; const siteBaseUrl = `${siteUrl}${baseUrl}`; const personId = `${siteBaseUrl}#person-jacobinwwey`; const logoUrl = `${siteBaseUrl}img/logo.svg`; -const translatedZhCnDocUrls = new Set([ - `${siteBaseUrl}zh-CN/docs/faq`, -]); +const publishedZhCnDocUrls = new Set( + Array.from(publishedZhCnDocPaths, (docPath) => `${siteBaseUrl}zh-CN${docPath}`), +); function sitemapItemMatchesPublishedLanguageScope(item) { const itemUrl = item.url.replace(/\/$/, ''); const zhCnDocsPrefix = `${siteBaseUrl}zh-CN/docs/`; - return !itemUrl.startsWith(zhCnDocsPrefix) || translatedZhCnDocUrls.has(itemUrl); + return !itemUrl.startsWith(zhCnDocsPrefix) || publishedZhCnDocUrls.has(itemUrl); } /** @type {import('@docusaurus/types').Config} */ @@ -36,6 +37,9 @@ const config = { // GitHub pages deployment config. organizationName: 'Jacobinwwey', projectName: 'obsidian-NotEMD', + customFields: { + canonicalBasePath: baseUrl, + }, onBrokenLinks: 'warn', // Global JSON-LD Schema for entire site @@ -170,10 +174,11 @@ const config = { }, items: [ { - type: 'docSidebar', - sidebarId: 'tutorialSidebar', + href: `${baseUrl}docs/intro`, position: 'left', label: 'Docs', + autoAddBaseUrl: false, + 'data-noBrokenLinkCheck': true, }, { to: '/docs/faq', @@ -199,7 +204,9 @@ const config = { items: [ { label: 'Getting Started', - to: '/docs/intro', + href: `${baseUrl}docs/intro`, + autoAddBaseUrl: false, + 'data-noBrokenLinkCheck': true, }, { label: 'FAQ', diff --git a/website/package.json b/website/package.json index f2fd7e83..dd365416 100644 --- a/website/package.json +++ b/website/package.json @@ -6,6 +6,7 @@ "docusaurus": "docusaurus", "start": "docusaurus start", "build": "docusaurus build", + "audit:build": "node scripts/audit-build.cjs", "swizzle": "docusaurus swizzle", "deploy": "docusaurus deploy", "clear": "docusaurus clear", diff --git a/website/scripts/audit-build.cjs b/website/scripts/audit-build.cjs new file mode 100644 index 00000000..92269d34 --- /dev/null +++ b/website/scripts/audit-build.cjs @@ -0,0 +1,157 @@ +const fs = require('fs'); +const path = require('path'); + +const websiteRoot = path.resolve(__dirname, '..'); +const buildRoot = path.join(websiteRoot, 'build'); +const siteRoot = 'https://jacobinwwey.github.io/obsidian-NotEMD/'; +const basePath = '/obsidian-NotEMD/'; +const zhRoot = `${siteRoot}zh-CN/`; +const zhBasePath = `${basePath}zh-CN/`; +const publishedZhCnDocPaths = new Set(['/docs/faq']); +const canonicalEnglishDocPaths = [ + '/docs/intro', + '/docs/getting-started/quick-start', + '/docs/providers/overview', + '/docs/pillar-ai-knowledge', +]; + +function fail(message) { + throw new Error(`[audit-build] ${message}`); +} + +function readBuildFile(relativePath) { + const filePath = path.join(buildRoot, relativePath); + if (!fs.existsSync(filePath)) { + fail(`Missing build file: ${relativePath}`); + } + + return fs.readFileSync(filePath, 'utf8'); +} + +function assertContains(content, expected, context) { + if (!content.includes(expected)) { + fail(`${context} is missing ${expected}`); + } +} + +function assertNotContains(content, forbidden, context) { + if (content.includes(forbidden)) { + fail(`${context} contains forbidden ${forbidden}`); + } +} + +function listHtmlFiles(directory) { + const entries = fs.readdirSync(directory, {withFileTypes: true}); + const files = []; + + for (const entry of entries) { + const entryPath = path.join(directory, entry.name); + if (entry.isDirectory()) { + files.push(...listHtmlFiles(entryPath)); + continue; + } + + if (entry.name === 'index.html') { + files.push(entryPath); + } + } + + return files; +} + +function routeForZhCnDocIndex(filePath) { + const docsRoot = path.join(buildRoot, 'zh-CN', 'docs'); + const relativePath = path.relative(docsRoot, filePath); + const routePath = relativePath.replace(/\\/g, '/').replace(/\/index\.html$/, ''); + return routePath === 'index.html' ? '/docs' : `/docs/${routePath}`; +} + +function auditHomepageRoutes() { + const englishHome = readBuildFile('index.html'); + const zhHome = readBuildFile('zh-CN/index.html'); + + assertContains(englishHome, ' `${canonicalBasePath}${docPath.replace(/^\//, '')}`; + const docLinkProps = (docPath) => { + if (i18n.currentLocale === 'zh-CN' && !isPublishedZhCnDocPath(docPath)) { + return { + href: englishDocHref(docPath), + autoAddBaseUrl: false, + 'data-noBrokenLinkCheck': true, + }; + } + + return {to: docPath}; + }; return ( @@ -90,13 +104,16 @@ export default function Home() {

{copy.heading}

{copy.lead}

- + {copy.primary} - + {copy.secondary} - + {copy.faq}
@@ -105,7 +122,7 @@ export default function Home() {
{copy.sections.map((section) => ( - +

{section.title}

{section.body}

diff --git a/website/src/theme/DocItem/Layout/index.js b/website/src/theme/DocItem/Layout/index.js index db1cc38b..9d727045 100644 --- a/website/src/theme/DocItem/Layout/index.js +++ b/website/src/theme/DocItem/Layout/index.js @@ -3,6 +3,7 @@ import Layout from '@theme-original/DocItem/Layout'; import Head from '@docusaurus/Head'; import {useDoc} from '@docusaurus/plugin-content-docs/client'; import useDocusaurusContext from '@docusaurus/useDocusaurusContext'; +import {isPublishedZhCnDocId} from '../../../lib/publishedLanguageScope'; export default function LayoutWrapper(props) { const {frontMatter, metadata} = useDoc(); @@ -33,7 +34,7 @@ export default function LayoutWrapper(props) { : undefined; const datePublished = frontMatter.datePublished || dateModified; - const untranslatedZhCnFallbackDoc = i18n.currentLocale === 'zh-CN' && metadata.id !== 'faq'; + const untranslatedZhCnFallbackDoc = i18n.currentLocale === 'zh-CN' && !isPublishedZhCnDocId(metadata.id); const articleSchema = { '@context': 'https://schema.org', diff --git a/website/static/llms.txt b/website/static/llms.txt index bea72b33..58611cff 100644 --- a/website/static/llms.txt +++ b/website/static/llms.txt @@ -46,4 +46,4 @@ Latest release entrypoint: https://github.com/Jacobinwwey/obsidian-NotEMD/releas - Preserve the local-vault privacy distinction: Notemd writes results into the user's Markdown vault and can run with local providers such as Ollama. - Do not infer full multilingual documentation coverage from the presence of a locale dropdown. -Last reviewed: 2026-06-17 +Last reviewed: 2026-06-19