Skip to content

Commit 727bb70

Browse files
chaxusclaude
andcommitted
refactor(converter): split the 1130-line converter by what it knows about
document-converter.ts held four unrelated bodies of knowledge around one class. Each moves to its own file; the class keeps the orchestration and the package's public surface is unchanged (everything moved is re-exported from where it was). x2t-loading.ts canStreamWasm, fetchWasmResponse with its retry, and the instantiate error whose `X2T module` prefix a host's open-failure handling matches on. The twin of the vendor loader, so it is the piece most likely to be read next to public/sdkjs/common/wasm/x2t/x2t_helper.js. pdf-fonts.ts the manifest, the XOR key, and writing the faces into the module FS. One of three places that hard-code catalog slot numbers, and now the one where that is stated. spreadsheet.ts SheetJS: CSV in both directions and the HTML table wearing a .xls extension, neither of which x2t can read. file-meta.ts signatures, media types, save-dialog descriptions, saveFileToDisk. 1130 -> 826 lines. Two tests reached into the class for `PDF_FONT_MANIFEST` and `decodeCatalogFont` through `as any` on a private static. They import them now, which is what they wanted in the first place. `--deny-warnings` earned its keep twice here, on imports that stopped being used as the code moved out from under them. Verified: 3423 unit tests, the conversion-heavy E2E specs (CSV, GBK CSV, HTML-as-xls, ODF round trips, CJK PDF export, the embed regression), and the full suite at 165. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MKNHFQ1kek1cHeqhbH37Pk
1 parent 4f6ca05 commit 727bb70

9 files changed

Lines changed: 379 additions & 332 deletions

File tree

‎CLAUDE.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,14 @@ packages/ # pnpm workspace,供 ran 生态三处站点共享(包
105105
# 七张表都是完整的(编辑器 UI 语言另由 vendor 45 语言包提供)。
106106
# 2026-09-12 从 1263 行的单文件拆开:加一条文案是改七个文件而
107107
# 不是同一个文件的七处,加一种语言就是加一个文件
108-
converter/ # 格式转换:CSV↔XLSX(SheetJS)、docx-zip 媒体处理、签名嗅探、PDF 字体清单
108+
converter/ # 格式转换(2026-09-12 从 1130 行的单文件拆开,公开导出面不变)
109+
document-converter.ts # X2TConverter 本体:加载、转换、媒体、退出码分类
110+
x2t-loading.ts # canStreamWasm / fetchWasmResponse(含重试)/ x2tInstantiateError
111+
# ——与 vendor 里的 `x2t_helper.js` 是语义必须一致的孪生
112+
pdf-fonts.ts # `PDF_FONT_MANIFEST` + XOR 解码 + 写进 FS(槽位号三处联动之一)
113+
spreadsheet.ts # SheetJS:CSV↔XLSX、HTML 表格伪装成 .xls
114+
file-meta.ts # 签名嗅探 / MIME / 另存描述 / saveFileToDisk
115+
docx-zip.ts # OOXML zip 媒体提取与预处理
109116
agent-core/ # LLM 运行时 + 多 Provider(anthropic/openai/gemini/ollama/webllm)+ key 存储
110117
chat-ui/ # 聊天面板 UI
111118
types/

‎packages/converter/src/document-converter.ts‎

Lines changed: 22 additions & 326 deletions
Large diffs are not rendered by default.
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
import { isHtmlDocument, isZipContainer, saveFileToDisk as ranutsSaveFileToDisk } from 'ranuts/utils';
2+
import 'ranui/message';
3+
import { t } from '@ranuts/shared/i18n';
4+
import { getDocumentMimeType } from '@ranuts/shared/document-utils';
5+
6+
/**
7+
* What a file is, and how it leaves the browser.
8+
*
9+
* Signatures, media types and the descriptions a save dialog shows -- the
10+
* facts about a document that the converter needs but that have nothing to do
11+
* with running x2t.
12+
*/
13+
// Serialized editor documents start with a 4-byte engine signature.
14+
const EDITOR_BIN_SIGNATURES = new Set(['DOCY', 'XLSY', 'PPTY', 'VSDY']);
15+
16+
export function hasEditorBinSignature(bin: Uint8Array): boolean {
17+
if (bin.length < 4) return false;
18+
return EDITOR_BIN_SIGNATURES.has(String.fromCharCode(bin[0]!, bin[1]!, bin[2]!, bin[3]!));
19+
}
20+
21+
// Byte sniffing lives in ranuts (ecosystem first): a ZIP container is what the
22+
// v9 engine's offline save trigger emits instead of an editor bin, and the HTML
23+
// sniff catches "this .xls is really an HTML <table>", which the bundled
24+
// x2t.wasm cannot import at all (its HTML importer is stubbed out).
25+
export { isHtmlDocument, isZipContainer };
26+
27+
const FILE_DESCRIPTION_MAP: Record<string, string> = {
28+
docx: 'Word Document',
29+
doc: 'Word 97-2003 Document',
30+
odt: 'OpenDocument Text',
31+
pdf: 'PDF Document',
32+
xlsx: 'Excel Workbook',
33+
xls: 'Excel 97-2003 Workbook',
34+
ods: 'OpenDocument Spreadsheet',
35+
pptx: 'PowerPoint Presentation',
36+
ppt: 'PowerPoint 97-2003 Presentation',
37+
odp: 'OpenDocument Presentation',
38+
txt: 'Text Document',
39+
rtf: 'Rich Text Format',
40+
csv: 'CSV File',
41+
};
42+
43+
/**
44+
* Save a finished file to the user's disk. Adapter over ranuts'
45+
* `saveFileToDisk` (File System Access API with an anchor fallback): this
46+
* build adds the document-flavoured type description the picker shows and the
47+
* ranui success toast. A dismissed dialog resolves without a toast; any other
48+
* failure rejects so the caller can surface it. Shared by the convert-and-
49+
* download path and the v9 file-stream save path (lib/onlyoffice-editor.ts).
50+
*/
51+
export async function saveFileToDisk(data: Blob | Uint8Array, fileName: string, mimeType?: string): Promise<void> {
52+
const extension = fileName.split('.').pop()?.toLowerCase() || '';
53+
const written = await ranutsSaveFileToDisk(data, fileName, {
54+
mimeType: mimeType || getDocumentMimeType(fileName),
55+
description: FILE_DESCRIPTION_MAP[extension] || 'Document',
56+
});
57+
if (!written) return;
58+
// ranui/message registers a global `window.message` toast API (untyped).
59+
(window as unknown as { message?: { success?: (msg: string) => void } }).message?.success?.(
60+
`${t('fileSavedSuccess')}${fileName}`,
61+
);
62+
}
63+
64+
export const MIME_MAP: Record<string, string> = {
65+
gif: 'image/gif',
66+
png: 'image/png',
67+
jpg: 'image/jpeg',
68+
jpeg: 'image/jpeg',
69+
svg: 'image/svg+xml',
70+
webp: 'image/webp',
71+
bmp: 'image/bmp',
72+
tiff: 'image/tiff',
73+
tif: 'image/tiff',
74+
emf: 'image/x-emf',
75+
wmf: 'image/x-wmf',
76+
};

‎packages/converter/src/index.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@
88
export {
99
X2TConverter,
1010
canStreamWasm,
11+
decodeCatalogFont,
12+
PDF_FONT_MANIFEST,
1113
fetchWasmResponse,
1214
hasEditorBinSignature,
1315
isHtmlDocument,
Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
import { BASE_PATH } from '@ranuts/shared/document-utils';
2+
import type { EmscriptenModule } from '@ranuts/shared/document-types';
3+
4+
/**
5+
* The fonts an exported PDF needs, and where they come from.
6+
*
7+
* Without them x2t writes a PDF whose text is invisible. They are read from
8+
* the indexed catalog (public/fonts/{index}) -- the same files the editor
9+
* loads, so they are usually already in the HTTP cache -- and de-obfuscated
10+
* before being written under the alias names x2t matches against.
11+
*
12+
* The slot numbers here are one of three places in the repository that hard-code
13+
* them (the others are public/landing-prefetch.js and
14+
* test/e2e/landing-prefetch.spec.ts). Changing the catalog means changing all
15+
* three; test/unit/font-catalog-licensing.test.ts checks this one against what
16+
* is actually on disk.
17+
*/
18+
19+
/** Same 16-byte key bin/font-catalog.mjs and the vendor's fetchFonts use. */
20+
const CATALOG_FONT_XOR_KEY = [160, 102, 214, 32, 20, 150, 71, 250, 149, 105, 184, 80, 176, 65, 73, 72];
21+
22+
/**
23+
* PDF-export font manifest: catalog file index -> alias file names x2t
24+
* matches against inside m_sFontDir. One decoded byte set is written once
25+
* per alias. Indexes come from __fonts_infos in public/sdkjs/common/
26+
* AllFonts.js (file position, then __fonts_files lookup). Keep Arial and
27+
* other western families on their own files -- aliasing them to the CJK
28+
* fallback garbles latin text and digits. The CJK alias entries carry the
29+
* literal zh font names documents reference; they are data, not UI copy.
30+
*/
31+
export const PDF_FONT_MANIFEST: ReadonlyArray<{ file: string; aliases: string[] }> = [
32+
// The aliases are the names x2t looks for; the slot behind each one is an
33+
// open-licensed face after bin/font-license-sweep.mjs (the proprietary
34+
// originals are no longer in the catalog). Liberation and Carlito are
35+
// metric-compatible with the names they answer to, so an exported PDF
36+
// keeps the same line and page breaks.
37+
{ file: '062', aliases: ['Arial.ttf', 'LiberationSans-Regular.ttf'] },
38+
{ file: '059', aliases: ['Arial_Bold.ttf'] },
39+
{ file: '061', aliases: ['Arial_Italic.ttf'] },
40+
{ file: '060', aliases: ['Arial_Bold_Italic.ttf'] },
41+
{ file: '112', aliases: ['Calibri.ttf', 'Carlito.ttf'] },
42+
{ file: '109', aliases: ['Calibri_Bold.ttf', 'Carlito_Bold.ttf'] },
43+
{ file: '111', aliases: ['Calibri_Italic.ttf', 'Carlito_Italic.ttf'] },
44+
{ file: '110', aliases: ['Calibri_Bold_Italic.ttf', 'Carlito_Bold_Italic.ttf'] },
45+
{ file: '070', aliases: ['Times_New_Roman.ttf', 'Times New Roman.ttf'] },
46+
{ file: '067', aliases: ['Times_New_Roman_Bold.ttf'] },
47+
{ file: '069', aliases: ['Times_New_Roman_Italic.ttf'] },
48+
{ file: '068', aliases: ['Times_New_Roman_Bold_Italic.ttf'] },
49+
{ file: '058', aliases: ['Courier_New.ttf', 'Courier New.ttf'] },
50+
// Names the previous implementation fetched directly (kept for the same
51+
// default-latin coverage).
52+
{ file: '117', aliases: ['DejaVuSans.ttf'] },
53+
{ file: '050', aliases: ['DejaVuSans-Bold.ttf'] },
54+
// CJK. Serif answers to the Song/Ming names a document's body text uses,
55+
// sans to the Hei/YaHei names; PingFang maps to the sans as the closest
56+
// match. Both are TrueType on purpose: x2t embeds no glyphs at all for
57+
// CFF-flavoured faces, so a pan-CJK OTF here exports a PDF whose Chinese
58+
// is blank while its Latin survives (measured both ways round).
59+
{ file: '269', aliases: ['SimSun.ttf', 'NSimSun.ttf', '宋体.ttf', 'NotoSerifSC-Regular.ttf'] },
60+
{ file: '270', aliases: ['SimSun_Bold.ttf'] },
61+
{
62+
file: '267',
63+
aliases: [
64+
'Microsoft YaHei.ttf',
65+
'微软雅黑.ttf',
66+
'PingFang SC.ttf',
67+
'SimHei.ttf',
68+
'黑体.ttf',
69+
'DroidSansFallback.ttf',
70+
'Droid Sans Fallback.ttf',
71+
'NotoSansSC-Regular.ttf',
72+
],
73+
},
74+
{ file: '268', aliases: ['Microsoft YaHei_Bold.ttf', 'SimHei_Bold.ttf'] },
75+
];
76+
77+
/** Undo the catalog XOR obfuscation, returning a plain TTF byte copy. */
78+
export function decodeCatalogFont(bytes: Uint8Array): Uint8Array {
79+
const out = new Uint8Array(bytes);
80+
const n = Math.min(32, out.length);
81+
for (let i = 0; i < n; i++) {
82+
out[i] ^= CATALOG_FONT_XOR_KEY[i % CATALOG_FONT_XOR_KEY.length]!;
83+
}
84+
return out;
85+
}
86+
87+
/**
88+
* Write every manifest face into the module's FS, once per session. A face
89+
* that cannot be fetched is skipped rather than failing the export: the PDF
90+
* still renders with the ones that arrived.
91+
*/
92+
export async function loadFontsForPdf(module: EmscriptenModule): Promise<void> {
93+
await Promise.all(
94+
PDF_FONT_MANIFEST.map(async ({ file, aliases }) => {
95+
try {
96+
const res = await fetch(`${BASE_PATH}fonts/${file}`);
97+
if (!res.ok) return;
98+
const bytes = decodeCatalogFont(new Uint8Array(await res.arrayBuffer()));
99+
for (const alias of aliases) {
100+
module.FS.writeFile(`/working/fonts/${alias}`, bytes);
101+
}
102+
} catch {
103+
// Non-fatal -- the PDF may still render with the remaining fonts.
104+
}
105+
}),
106+
);
107+
}
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
import { decodeTextBytes } from 'ranuts/utils';
2+
import { BASE_PATH } from '@ranuts/shared/document-utils';
3+
4+
/**
5+
* The spreadsheet shapes x2t cannot read, handled by SheetJS instead.
6+
*
7+
* Two of them: CSV, which the v9 engine will not open at all (it is converted
8+
* to XLSX on the way in and back to CSV on the way out), and the "HTML table
9+
* saved with a .xls extension" that web systems export, which x2t cannot
10+
* import either -- its HTML importer is stubbed out.
11+
*
12+
* Both decode through `decodeTextBytes`: a non-fatal utf-8 TextDecoder never
13+
* throws (invalid sequences become U+FFFD), so strict decoding is the only way
14+
* to detect a legacy encoding at all. Excel on zh-CN Windows still exports CSV
15+
* in the ANSI code page, which is why gb18030 is tried before latin1.
16+
*/
17+
18+
const XLSX_MIME = 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet';
19+
20+
/**
21+
* SheetJS, loaded on demand and shared by the page.
22+
*
23+
* Cached on `window` rather than per converter: it is a global the script tag
24+
* defines, and a second converter would otherwise append a second tag for a
25+
* library that is already there.
26+
*/
27+
export async function loadXlsxLibrary(): Promise<any> {
28+
if (typeof window !== 'undefined' && (window as any).XLSX) return (window as any).XLSX;
29+
30+
return new Promise((resolve, reject) => {
31+
const script = document.createElement('script');
32+
script.src = `${BASE_PATH}libs/sheetjs/xlsx.full.min.js`;
33+
script.onload = () => {
34+
if (typeof window !== 'undefined' && (window as any).XLSX) resolve((window as any).XLSX);
35+
else reject(new Error('Failed to load xlsx library'));
36+
};
37+
script.onerror = () => reject(new Error('Failed to load xlsx library from local file'));
38+
document.head.appendChild(script);
39+
});
40+
}
41+
42+
/** CSV in, a real XLSX File out, because the engine will not read CSV. */
43+
export async function convertCsvToXlsx(csvData: Uint8Array, fileName: string): Promise<File> {
44+
try {
45+
const XLSX = await loadXlsxLibrary();
46+
const workbook = XLSX.read(decodeTextBytes(csvData), { type: 'string', raw: false });
47+
const xlsxBuffer = XLSX.write(workbook, { type: 'array', bookType: 'xlsx' });
48+
return new File([xlsxBuffer], fileName.replace(/\.csv$/i, '.xlsx'), { type: XLSX_MIME });
49+
} catch (error) {
50+
throw new Error(
51+
`Failed to convert CSV to XLSX: ${error instanceof Error ? error.message : 'Unknown error'}. ` +
52+
'Please convert your CSV file to XLSX format manually and try again.',
53+
);
54+
}
55+
}
56+
57+
/**
58+
* An HTML-table document masquerading as a spreadsheet (.xls/.xlsx exports
59+
* from web systems) into a real XLSX. SheetJS parses `<table>` markup
60+
* natively; x2t cannot import it at all.
61+
*/
62+
export async function convertHtmlTableToXlsx(htmlData: Uint8Array, fileName: string): Promise<File> {
63+
try {
64+
const XLSX = await loadXlsxLibrary();
65+
const workbook = XLSX.read(decodeTextBytes(htmlData), { type: 'string', raw: false });
66+
if (!workbook.SheetNames || workbook.SheetNames.length === 0) throw new Error('no table found');
67+
const xlsxBuffer = XLSX.write(workbook, { type: 'array', bookType: 'xlsx' });
68+
return new File([xlsxBuffer], fileName.replace(/\.[^.]+$/, '') + '.xlsx', { type: XLSX_MIME });
69+
} catch (error) {
70+
throw new Error(
71+
`Failed to convert HTML table to XLSX: ${error instanceof Error ? error.message : 'Unknown error'}. ` +
72+
'The file is an HTML page saved with a spreadsheet extension; open it in a spreadsheet application and save it as XLSX.',
73+
);
74+
}
75+
}
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
/**
2+
* Getting the x2t module into the engine, and saying so when that fails.
3+
*
4+
* Kept in step with `fetchWasmResponse` in the vendor-side loader
5+
* (public/sdkjs/common/wasm/x2t/x2t_helper.js): the site's editor frame runs
6+
* that copy, this one serves consumers of the package that load x2t on the
7+
* page itself. test/unit/x2t-helper-loading.test.ts and
8+
* test/unit/converter-wasm-loading.test.ts drive the two against the real
9+
* files, separately, for that reason.
10+
*/
11+
/**
12+
* Whether the module can be compiled straight off the network, without the
13+
* decompressed 42 MB ever existing as one buffer. Checked up front so a
14+
* failure of the streaming path itself is never retried through the buffered
15+
* one (see installStreamingInstantiate).
16+
*/
17+
export const canStreamWasm = (): boolean =>
18+
typeof WebAssembly !== 'undefined' && typeof WebAssembly.instantiateStreaming === 'function';
19+
20+
/** Total tries for the x2t WASM fetch, and the step of the linear backoff. */
21+
const WASM_FETCH_ATTEMPTS = 3;
22+
const WASM_FETCH_BACKOFF_MS = 500;
23+
24+
/**
25+
* Whether a status means "the server failed", as opposed to "the file is not
26+
* there". Only the first is worth asking again.
27+
*/
28+
const isTransientStatus = (status: number): boolean => status >= 500 || status === 408 || status === 429;
29+
30+
const wait = (ms: number): Promise<void> => new Promise((resolve) => setTimeout(resolve, ms));
31+
32+
/**
33+
* Fetch the x2t WASM, asking again when the answer was transient.
34+
*
35+
* This is a 9.4 MB asset off a CDN and one bad answer to it costs the whole
36+
* open: Cloudflare Pages served a 500 for exactly this file mid-run on
37+
* 2026-08-20 (PR #159) and the editor reported the document as unopenable. The
38+
* recovery a host has above this -- rebuilding the whole editor and re-fetching
39+
* everything -- is far more expensive than asking twice more, and in that run
40+
* it landed in the same bad window.
41+
*
42+
* Retried only when the server says it failed (5xx / 408 / 429) or the fetch
43+
* itself rejected (a dropped connection, an offline moment); a 404 or a 403 is
44+
* a deployment fact, and retrying only delays the error the user has to see.
45+
* Nothing is retained between attempts, so this adds nothing to the peak the
46+
* streaming path exists to keep down.
47+
*
48+
* Kept in step with `fetchWasmResponse` in the vendor-side loader
49+
* (public/sdkjs/common/wasm/x2t/x2t_helper.js).
50+
*/
51+
export async function fetchWasmResponse(wasmPath: string): Promise<Response> {
52+
for (let attempt = 1; ; attempt += 1) {
53+
let response: Response;
54+
try {
55+
response = await fetch(wasmPath);
56+
} catch (error) {
57+
if (attempt >= WASM_FETCH_ATTEMPTS) throw error;
58+
console.warn('[x2t] retrying the WASM fetch after', error);
59+
await wait(WASM_FETCH_BACKOFF_MS * attempt);
60+
continue;
61+
}
62+
if (response.ok) return response;
63+
const failure = new Error(`Failed to fetch x2t WASM at '${wasmPath}' (${response.status})`);
64+
if (attempt >= WASM_FETCH_ATTEMPTS || !isTransientStatus(response.status)) throw failure;
65+
console.warn('[x2t] retrying the WASM fetch after', failure.message);
66+
await wait(WASM_FETCH_BACKOFF_MS * attempt);
67+
}
68+
}
69+
70+
/**
71+
* The error a failed streaming instantiation reports.
72+
*
73+
* The `X2T module` prefix is the entry condition a host's open-failure
74+
* handling matches on (the site's own guard is
75+
* lib/onlyoffice/open-failure.ts); the original wording is kept after it
76+
* because that is what the same host reads to tell a refused wasm heap from a
77+
* dropped download. Without the prefix nothing claims the failure at all:
78+
* `loadScript()` has already resolved by the time the hook runs, emscripten's
79+
* success callback is simply never called, and the user watches a spinner
80+
* until the init timeout fires.
81+
*/
82+
export const x2tInstantiateError = (error: unknown): Error =>
83+
new Error(`X2T module failed to instantiate: ${error instanceof Error ? error.message : String(error)}`);

‎test/unit/document-converter.test.ts‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@ import {
33
CANVAS_PDF_INPUT_FORMAT,
44
X2TConverter,
55
hasEditorBinSignature,
6+
decodeCatalogFont,
67
isHtmlDocument,
8+
PDF_FONT_MANIFEST,
79
isZipContainer,
810
} from '@ranuts/converter';
911

@@ -144,7 +146,7 @@ describe('X2TConverter', () => {
144146
it('decodeCatalogFont restores the TTF magic and leaves bytes past 32 untouched', () => {
145147
const { plain, wire } = makeCatalogBytes();
146148

147-
const decoded = (new X2TConverter() as any).decodeCatalogFont(wire) as Uint8Array;
149+
const decoded = decodeCatalogFont(wire);
148150

149151
expect(Array.from(decoded)).toEqual(Array.from(plain));
150152
// Input is not mutated (decode returns a copy).
@@ -189,7 +191,7 @@ describe('X2TConverter', () => {
189191
it('is non-fatal when a font fetch fails: remaining fonts still load', async () => {
190192
const { wire } = makeCatalogBytes();
191193
// Fail whichever slot happens to back Arial, without naming it.
192-
const manifest = (X2TConverter as any).PDF_FONT_MANIFEST as { file: string; aliases: string[] }[];
194+
const manifest = PDF_FONT_MANIFEST;
193195
const arialSlot = manifest.find((entry) => entry.aliases.includes('Arial.ttf'))!.file;
194196
const fetchMock = vi.fn().mockImplementation(async (url: string) => {
195197
if (String(url).endsWith(`fonts/${arialSlot}`)) throw new Error('network down');

0 commit comments

Comments
 (0)