refactor(tools): fetch 的 HTML 转换换 turndown DOM 解析

- 截断/畸形 HTML 不再漏残标签与半截 script 正文(正则 [\s\S]*? 遇缺闭标签整体失配的必然缺陷)
- class/id 页壳启发式限定块级容器,不再连坐删正文
- 相对 href/src 按重定向终址绝对化,抓回的链接可直接作为下一次 fetch 参数
- 标题走 DOM 规则取(修复先删 head 后取 title 会取空的问题)
- 新增解析硬帽 10MB:超限与解析抛错均降级轻量剥标签,不空手而归
- 截断提示文案区分「下载少了」与「解析坏了」
This commit is contained in:
2026-10-01 10:06:02 +08:00
parent c3331f1e23
commit 2cab7747c0
3 changed files with 112 additions and 57 deletions
+1
View File
@@ -13,6 +13,7 @@
"express": "4.21.2",
"multer": "1.4.5-lts.1",
"sharp": "0.35.4",
"turndown": "7.2.0",
"yaml": "2.6.1"
}
}