Compare commits
22
Commits
09cf474c2d
...
v2.4.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
55efb1c410 | ||
|
|
53fa5da82b | ||
|
|
5b70f7d614 | ||
|
|
dfcbe6d3ea | ||
|
|
516b4a5d86 | ||
|
|
e39273e97f | ||
|
|
8b8fbddb4f | ||
|
|
ba9534f064 | ||
|
|
43e77e569e | ||
|
|
ab67400174 | ||
|
|
13b493eada | ||
|
|
7fc945a4c8 | ||
|
|
fbe78d362f | ||
|
|
fe1c6d106f | ||
|
|
a86b748a6f | ||
|
|
d3128ba21c | ||
|
|
19564fb65b | ||
|
|
deb622095f | ||
|
|
61245b1c65 | ||
|
|
bc34f91417 | ||
|
|
ec70b10c08 | ||
|
|
a8900ddafc |
@@ -22,7 +22,7 @@
|
||||
| `app/server/public/` | 前端静态资源,经典脚本按序加载,共享全局作用域 | 无构建步骤 |
|
||||
| `executor/` | **agent-executor**:`/exec` 隔离命令执行 + Office 预览网关 | 零依赖 Node,独立容器 |
|
||||
| `wxbot/` | 微信 iLink 协议调制解调器(多实例) | 独立容器 `agent-wxbot` |
|
||||
| `mcp/` | 同栈 MCP 服务:`web_search` / `image_task` / `video_task` | 三个容器,仅内网 |
|
||||
| `mcp/` | 同栈 MCP 服务:`web_search` / `image_task` / `video_task` / `asr` | 四个容器,仅内网 |
|
||||
| `skills/` | 目录式技能(`SKILL.md` + `run.py`/`run.sh`) | 运行时挂载,`skills/deps` 为依赖层 |
|
||||
| `scripts/` | `init.sh` 生成 `.env` + `config.yml`;`init_host.sh` 裸机初始化 | 幂等 |
|
||||
| `docs/` | 专题文档:`tool-design.md`(工具接口规范)、`OFFICE.md` | — |
|
||||
@@ -42,7 +42,7 @@ flowchart TD
|
||||
WXBOT -->|"回调 /wxhook/*(共享 Bearer token)"| CORE
|
||||
CORE["agent-core(8090,唯一对外端口)"]
|
||||
CORE --> EXEC["agent-executor(9000 exec / 3100 预览,仅内网)"]
|
||||
CORE --> MCP["mcp_web_search / mcp_image_task / mcp_video_task(3000,仅内网)"]
|
||||
CORE --> MCP["mcp_web_search / mcp_image_task / mcp_video_task / mcp_asr(3000,仅内网)"]
|
||||
CORE -->|"分段投递回复"| WXBOT
|
||||
```
|
||||
|
||||
@@ -83,7 +83,7 @@ flowchart TD
|
||||
### 环境变量(接线全靠根 `.env` + compose)
|
||||
|
||||
`AGENT_CONFIG`、`PORT`、`EXECUTOR_URL`、`EXECUTOR_TOKEN`、`WXBOT_URL`、`WXBOT_TOKEN`、
|
||||
`VIRTUAL_HOST`、`PREVIEW_PROXIES`、`EXECUTOR_LOG_LEVEL`、`WXBOT_LOG_LEVEL`。
|
||||
`ASR_URL`、`ASR_API_KEY`、`VIRTUAL_HOST`、`PREVIEW_PROXIES`、`EXECUTOR_LOG_LEVEL`、`WXBOT_LOG_LEVEL`。
|
||||
密钥生成:`./scripts/init.sh`(幂等,重跑不掉登录态)。
|
||||
|
||||
## 三条主链路
|
||||
@@ -187,6 +187,7 @@ curl -s http://127.0.0.1:8090/api/health
|
||||
| 改模型调度 / 绑定 | `app/server/engine.js` 的 `chooseKey` / `chooseModel` / `ensureBinding` |
|
||||
| 改上下文组装 / 省 token | `app/server/llm.js` 的 `buildMessages` / `trim*` |
|
||||
| 改微信投递 | `app/server/wechat.js`(业务)+ `app/server/routes/wechat.js`(HTTP/鉴权) |
|
||||
| 改语音输入 | `app/server/public/js/voice.js`(录音与插入)+ `app/server/routes/asr.js`(代理)+ `mcp/asr/`(转写);客户端桥契约见 `docs/voice-input.md` |
|
||||
| 改隔离执行 | `executor/server.js`(先读顶部安全模型注释) |
|
||||
| 加同栈 MCP 服务 | `mcp/<name>/` + `docker-compose.yml` + `config.yml` 的 `mcp_servers` |
|
||||
| 加技能 | `skills/<name>/`(`SKILL.md` + `run.py`),见 `skills/README.md` |
|
||||
|
||||
@@ -22,6 +22,7 @@
|
||||
| 上下文管理 | 手动压缩、按 token 阈值提示压缩、历史重放自动裁剪、会话标题自动生成 |
|
||||
| 定时任务 | 每会话最多一个 cron 任务;触发前清空模型可见历史,按计划执行并推送结果 |
|
||||
| 微信接入 | 扫码绑定个人微信,文本/图片/视频/文件进入同一套会话与命令体系 |
|
||||
| 语音输入 | 输入框为空时发送键变麦克风:点一下进入语音模式,输入框位置变成「按住 说话」,松开即转写并直接发送;发送键位置变键盘按钮可切回打字。套壳 App 走原生录音(http 页面也能用),浏览器走 MediaRecorder(需 HTTPS);识别在 `mcp_asr`,不经过模型、不消耗 token |
|
||||
| 长期记忆 | 会话收口时生成第一人称工作日记;微信新会话注入最近几条,`recall` 可检索全部 |
|
||||
| 用户画像 | 记录称谓、行业、偏好,会话创建时冻结注入,跨会话生效 |
|
||||
| 技能 | 目录式技能(`SKILL.md` + `run.py`/`run.sh`),可在设置页管理 |
|
||||
@@ -94,7 +95,7 @@ http://<服务器 IP>:<AGENT_PORT>
|
||||
|
||||
### 4. 同栈 MCP 服务
|
||||
|
||||
`init.sh` 默认把仓库内 `mcp/` 的三个服务写进 `config.yml`:
|
||||
`init.sh` 默认把仓库内 `mcp/` 的服务写进 `config.yml`:
|
||||
|
||||
```yaml
|
||||
mcp_servers:
|
||||
@@ -104,9 +105,14 @@ mcp_servers:
|
||||
url: "http://mcp_image_task:3000/mcp"
|
||||
- name: video_task
|
||||
url: "http://mcp_video_task:3000/mcp"
|
||||
- name: asr
|
||||
url: "http://mcp_asr:3000/mcp"
|
||||
mode: skill
|
||||
```
|
||||
|
||||
`web_search` / `image_task` / `video_task` 与 core 由同一个根 `docker-compose.yml` 启动,没有先后顺序,也不占宿主机端口。工具说明、header 覆盖项与调试方式见 [mcp/README.md](mcp/README.md)。
|
||||
`web_search` / `image_task` / `video_task` / `asr` 与 core 由同一个根 `docker-compose.yml` 启动,没有先后顺序,也不占宿主机端口。工具说明、header 覆盖项与调试方式见 [mcp/README.md](mcp/README.md)。
|
||||
|
||||
语音输入(H5 输入框里说话)走的是另一条路:客户端录音 → `POST /api/asr/transcribe` → `mcp_asr` 的 `/transcribe`,**不经过模型、不消耗 token**,结果插入输入框可编辑。客户端录音契约(Android 套壳原生桥 / 浏览器 MediaRecorder)见 [docs/voice-input.md](docs/voice-input.md);`ASR_URL` 置空即整体关闭。
|
||||
|
||||
接入栈外的 MCP 服务:在设置页添加 `url` / `headers`,或在 `config.yml` 的 `mcp_servers` 追加一项。服务不可达时 core 只在启动日志里告警,不影响 Web 主流程,但对应工具不会下发给模型。
|
||||
|
||||
@@ -120,7 +126,7 @@ mcp_servers:
|
||||
|
||||
`skill` 用于「能力包在技能后面」的场景:工具照常发现并注册,只是不进模型的 tools 数组,改为以技能形式暴露——`{"op":"list"}` 能看到它,`{"op":"get","name":"<名>"}` 直接返回 MCP 原始的 `description` + `input_schema`,`{"op":"run","name":"<名>","args":{…}}` 转发给服务。不用在 `skills/` 里建目录。旧字段 `enable` 已废弃,`mode: silent` 已移除(启动会报错并提示改用 `skill`)。
|
||||
|
||||
工具发现只在启动时做一次,因此发现失败后会自动按 **2s / 20s / 1min 重试 3 次**(覆盖 compose 并行拉起与宿主机重启后的容器乱序);成功后工具在下一轮请求即可用,三次都失败则需重启 core。三个 MCP 服务都配了 `/health` 健康检查,`docker ps` 可直接看状态。
|
||||
工具发现只在启动时做一次,因此发现失败后会自动按 **2s / 20s / 1min 重试 3 次**(覆盖 compose 并行拉起与宿主机重启后的容器乱序);成功后工具在下一轮请求即可用,三次都失败则需重启 core。四个 MCP 服务都配了 `/health` 健康检查,`docker ps` 可直接看状态。
|
||||
|
||||
---
|
||||
|
||||
@@ -238,6 +244,8 @@ platforms:
|
||||
| `PORT` | core 监听端口,容器内 `8090` |
|
||||
| `EXECUTOR_URL` | `agent-executor` 地址;空则命令在 core 进程内本地执行,仅适合开发 |
|
||||
| `EXECUTOR_TOKEN` | core ↔ executor 的 Bearer token,两侧同值 |
|
||||
| `ASR_URL` | `mcp_asr` 地址(compose 为 `http://mcp_asr:3000`);置空则语音输入关闭 |
|
||||
| `ASR_API_KEY` / `ASR_API_BASE` / `ASR_MODEL` | `mcp_asr` 的上游语音识别密钥与地址(默认智谱 `glm-asr`) |
|
||||
| `WXBOT_URL` | wxbot 地址;置空则微信功能整体关闭 |
|
||||
| `WXBOT_TOKEN` | core ↔ wxbot 的共享 token |
|
||||
| `VIRTUAL_HOST` | 设置后 `public_link.base_url` 强制取 `https://<第一个域名>` |
|
||||
@@ -389,8 +397,14 @@ docker compose logs -f agent-core
|
||||
docker compose logs -f agent-executor
|
||||
docker compose logs -f agent-wxbot
|
||||
curl -s http://127.0.0.1:8090/api/health
|
||||
# {"ok":true,"service":"agent-core","api":1,"time":...,"auth_required":true}
|
||||
```
|
||||
|
||||
- `GET /api/health` 是**唯一免登录接口**,供运维探活,也供套壳 App 校验用户手填的服务器地址。
|
||||
客户端判定条件:HTTP 200 且 `ok === true` 且 `service === 'agent-core'`——**不要只看状态码**,
|
||||
填错到别的站点或反代首页同样可能回 200。地址末尾的 `/` 可带可不带。
|
||||
`auth_required` 用于区分「地址对了但没登录」与「地址错了」。
|
||||
|
||||
- core 日志等级由 `config.yml` 的 `log_level` 控制,改完重启生效。
|
||||
- executor / wxbot 的日志等级由各自环境变量控制,`debug` 会打印完整交互。
|
||||
- 启动时会在 `data/prompt.txt` 写入实际下发的 system prompt 与各 profile 的工具清单,便于核对。
|
||||
@@ -400,6 +414,9 @@ curl -s http://127.0.0.1:8090/api/health
|
||||
| 现象 | 原因 / 处理 |
|
||||
|---|---|
|
||||
| 启动时报 `network agent-mcp-net has active endpoints` | 旧的双仓库部署残留:先停旧的 mcp 仓库(`docker compose down`),再 `up` 本栈 |
|
||||
| 套壳 App 填了服务器地址仍连不上 | 先 `curl <地址>/api/health` 看是否返回 `service:agent-core`;再查 Android 9+ 默认禁止明文 HTTP(需 `usesCleartextTraffic`)与 https 自签证书 |
|
||||
| 麦克风按钮不显示 | `ASR_URL` 为空(未部署 `mcp_asr`);或浏览器以 `http://<IP>` 访问且没有原生录音桥(非安全上下文开不了麦,见 `docs/voice-input.md`) |
|
||||
| 点麦克风提示「语音识别服务不可用」 | `mcp_asr` 未起来或 `ASR_API_KEY` 为空:`docker compose logs --tail=50 mcp_asr` |
|
||||
| `config.yml` 变成目录 | 启动前文件不存在,Docker 自动建了同名目录;停容器、删目录、放入配置文件 |
|
||||
| 没有可用模型 | `platforms` 为空或模型全部禁用;在设置页添加后重启 |
|
||||
| 创建会话报「无可用模型」 | 模型池为空,或指定模型所在平台没有 key |
|
||||
|
||||
+7
-2
@@ -264,8 +264,9 @@ auth.router ← /login /logout /api/login /api/login_restore /api/passwor
|
||||
auth.middleware ← 其余请求需登录(cookie 优先,Basic 兜底)
|
||||
preview-proxy.earlyGuard
|
||||
express.static(public)
|
||||
/api/sessions ×3 ← sessions / messages / files
|
||||
/api/sessions ×4 ← sessions / messages / files / skill-run(通用执行面)
|
||||
/api/skills
|
||||
/api/asr ← 语音识别代理:/transcribe 走登录态;ASR_URL 空则 503
|
||||
/api/config ← requireAdmin
|
||||
/api/platforms
|
||||
/api/system ← /restart 需 admin,其余登录即可
|
||||
@@ -274,6 +275,8 @@ express.static(public)
|
||||
/api/memory
|
||||
/api/wechat
|
||||
/api/channel
|
||||
/api/health ← 免认证探活:套壳 App / 运维脚本用它校验服务器地址
|
||||
(只认 200 + body.ok + service==='agent-core')
|
||||
data-page ← GET /app/:task_id/*、POST /api/db/:task_id/query
|
||||
preview-proxy.mount(末尾兜底)
|
||||
```
|
||||
@@ -281,13 +284,15 @@ preview-proxy.mount(末尾兜底)
|
||||
- 错误形状统一 `{ error: string }`,`err.status_code` 决定 HTTP 码。
|
||||
- 认证:HMAC 签名 HttpOnly cookie(`agent_token`),`Authorization: Basic` 兜底,
|
||||
`Authorization: Bearer` 仅用于 `/api/login_restore`。`auth.users` 为空 = 完全关闭认证。
|
||||
- 免认证路径集中在 `auth.js` 的 `PUBLIC_EXACT`(含 `/api/health`,且容忍尾斜杠)。
|
||||
新增免登录接口必须加进去,否则会被中间件先拦成 401。
|
||||
- **单租户警告**:除管理员接口外不做资源级授权;按用户的会话列表隔离在查询层(`sessions.user`)。
|
||||
- 公网短链 `/p/<id>` 的信任模型:随机 ID + 服务端 TTL + 签发时 `realpath` 校验,不靠登录态。
|
||||
|
||||
## 前端(`server/public/`)
|
||||
|
||||
- **无构建**:`index.html` 按顺序加载经典脚本,共享全局作用域:
|
||||
`core.js → chat.js → sessions.js → stream.js → files.js → preview.js → settings.js`
|
||||
`core.js → chat.js → sessions.js → stream.js → files.js → voice.js → preview.js → settings.js`
|
||||
(`settings.js` 必须最后,尾部有全局初始化);`marked.min.js` 走 `vendor/`。
|
||||
- 新增 JS 文件必须同时改 `index.html` 的 `<script>` 顺序——没有 import/export,顺序就是依赖。
|
||||
- `sw.js`:HTML/JS/CSS 网络优先,图片/JSON 缓存优先。**改图片或 JSON 才需要升 `CACHE_VERSION`**;
|
||||
|
||||
@@ -62,6 +62,7 @@ upload:
|
||||
image_max_dim: 2000 # 压缩后长边上限(px)
|
||||
image_quality: 0.85 # JPEG 质量
|
||||
confirm_size_mb: 10 # 任意文件二次确认阈值(MB)
|
||||
paste_to_file_threshold_chars: 2000 # 粘贴超长文本自动转 .txt 附件(字符数,0=关)
|
||||
|
||||
skills:
|
||||
skills_root: "" # 留空取 workspace_root 同级 skills 目录
|
||||
@@ -78,9 +79,15 @@ auth: # users 留空 = 不启用登
|
||||
role: admin # admin=管理员(全部设置 + 用户管理);user 或缺省=普通用户(仅账户页)
|
||||
# model: qwen-tp/qwen3.8-flash # 选填:该用户新建会话时锁定的模型(内部键「平台名/模型id」)
|
||||
|
||||
# MCP 服务列表(仓库自带的 mcp/ 三个服务,或栈外服务):每项 {name?, url, headers?, mode?(默认 advertise), timeout_ms?, tools_allow?, tools_deny?}。
|
||||
# MCP 服务列表(仓库自带的 mcp/ 服务,或栈外服务):每项 {name?, url, headers?, mode?(默认 advertise), timeout_ms?, tools_allow?, tools_deny?}。
|
||||
# mode 三态:advertise=工具下发给模型;skill=不下发,改为以技能形式暴露(get 返回原始 input_schema,run 转发);
|
||||
# disable=不连接、不发现。旧字段 enable 已废弃;mode: silent 已移除(改用 skill),写进配置会启动报错。
|
||||
#
|
||||
# 语音识别(mcp/asr)建议用 skill:转写音频是低频能力,不必把 schema 塞进每轮请求。
|
||||
# - name: asr
|
||||
# url: "http://mcp_asr:3000/mcp"
|
||||
# mode: skill
|
||||
# 注意:输入框里的语音输入走的是 core 的 /api/asr(由 compose 的 ASR_URL 接线),不经这个工具,也不消耗 token。
|
||||
mcp_servers: []
|
||||
|
||||
public_link:
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "agent-app",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "agent-app",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.0",
|
||||
"dependencies": {
|
||||
"better-sqlite3": "11.3.0",
|
||||
"cron-parser": "5.10.1",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "agent-app",
|
||||
"version": "2.3.0",
|
||||
"version": "2.4.0",
|
||||
"description": "自用通用 Agent 系统:多模型调度 + 任务目录 + 工具调用 + H5 Web UI",
|
||||
"main": "server/server.js",
|
||||
"scripts": {
|
||||
|
||||
+4
-1
@@ -73,7 +73,10 @@ function basicAuthUser(req, users) {
|
||||
// ---------- 免认证路径(登录页自身需要的资源) ----------
|
||||
const PUBLIC_EXACT = new Set(['/login', '/logout', '/api/login', '/api/logout', '/api/login_restore', '/api/health']);
|
||||
function isPublicPath(p) {
|
||||
if (PUBLIC_EXACT.has(p)) return true;
|
||||
// 尾斜杠容忍:客户端手动填服务器地址常多带一个 `/`(拼成 `/api/health/`),
|
||||
// Express 路由本身能匹配,但这里若不规范化,中间件会先回 401——地址明明是对的却报未登录。
|
||||
const norm = p.length > 1 ? p.replace(/\/+$/, '') : p;
|
||||
if (PUBLIC_EXACT.has(norm)) return true;
|
||||
if (p.startsWith('/css/')) return true; // 登录页复用 style.css
|
||||
if (p === '/favicon.ico') return true;
|
||||
return false;
|
||||
|
||||
@@ -90,7 +90,8 @@ const DEFAULTS = {
|
||||
// fetch、read_file 回执正文距尾 ≥N 个 assistant 轮后换单行占位(DB/UI 不动,内容可再生:盘上重读/重抓);0=关。
|
||||
agent: { temperature: 1.0, max_tool_iterations: 100, max_retries: 2, language: 'Simplified Chinese', auto_compress_tokens: 60000, auto_complete_ms: 1800000, trim_write_rounds: 1, trim_result_rounds: 2 },
|
||||
// 大图上传预处理(浏览器端 canvas 压缩,参数经 /api/system/status 下发前端)。
|
||||
upload: { image_compress_threshold_kb: 1536, image_max_dim: 2000, image_quality: 0.85, confirm_size_mb: 10 },
|
||||
// paste_to_file_threshold_chars:粘贴文本超过该字符数时前端转成 .txt 附件,0 = 关闭。
|
||||
upload: { image_compress_threshold_kb: 1536, image_max_dim: 2000, image_quality: 0.85, confirm_size_mb: 10, paste_to_file_threshold_chars: 2000 },
|
||||
// 技能根目录:留空取 workspace_root 同级 skills(/workspace/tasks → /workspace/skills)。
|
||||
skills: { skills_root: '', timeout_ms: 120000 },
|
||||
// 长期记忆(日记,memory.js):会话收口(下班)时一条几十字的第一人称碎片,一片一行入 memory_fragments,
|
||||
|
||||
@@ -1465,6 +1465,25 @@ function deleteMessage(sessionId, messageId) {
|
||||
return { ok: true, id: messageId, removed: removed + 1 };
|
||||
}
|
||||
|
||||
// 覆盖式编辑用户消息:删掉该条及其后的全部历史(含 tool 回执/摘要),再以新内容重开一轮。
|
||||
// 编辑点是 user 行,前缀必然停在 assistant,不会留下无配对的 tool_call。
|
||||
// 运行中不支持:runLoop 每轮重载历史,中途截断会让当前回合的 tool 回执缺失。
|
||||
function editUserMessage(sessionId, messageId, content, opts) {
|
||||
const session = getSession(sessionId);
|
||||
if (!session) throw Object.assign(new Error('会话不存在'), { status_code: 404 });
|
||||
const row = db.prepare('SELECT id, role FROM messages WHERE id = ? AND session_id = ?').get(messageId, sessionId);
|
||||
if (!row) throw Object.assign(new Error('消息不存在'), { status_code: 404 });
|
||||
if (row.role !== 'user') throw Object.assign(new Error('只能编辑用户消息'), { status_code: 400 });
|
||||
const run = runs.get(sessionId);
|
||||
if (run && run.looping) throw Object.assign(new Error('任务正在执行,请先终止再编辑'), { status_code: 409 });
|
||||
// 先确保能开跑:无可用模型时抛 503,避免删完历史却发不出去
|
||||
ensureBinding(session);
|
||||
const removed = db.prepare('DELETE FROM messages WHERE session_id = ? AND id >= ?').run(sessionId, messageId).changes;
|
||||
const r = postUserMessage(sessionId, content, opts);
|
||||
log.info('msg', sessionId + ' 编辑消息 ' + messageId + ',丢弃其后 ' + Math.max(0, removed - 1) + ' 条历史');
|
||||
return Object.assign({ removed: removed }, r);
|
||||
}
|
||||
|
||||
// 删工具回执(任务执行卡片):同步把对应 tool_call 从父 assistant 的 tool_calls 里摘掉,
|
||||
// 避免留下「有调用无结果」被严格上游拒收。父回合若是纯工具调用轮(无文本)且已无
|
||||
// 其余调用,则整行一起删(空气泡没有意义);有文本则保留为普通文本回复。
|
||||
@@ -1511,6 +1530,7 @@ module.exports = {
|
||||
clearSessionMessages,
|
||||
summarizeContext,
|
||||
deleteMessage,
|
||||
editUserMessage,
|
||||
getSession,
|
||||
getRun,
|
||||
taskDirOf,
|
||||
|
||||
+6
-2
@@ -584,10 +584,14 @@ async function callModel(cfg, modelId, messages, tools, options) {
|
||||
// 没配/无思考的行零变化)。无论配与否都剥一遍 __reasoning,内部键不出仓库。
|
||||
if (body.messages && body.messages.length) {
|
||||
const ack = modelCfg.reasoning_ack_field;
|
||||
// 思考模式下带 tool_calls 的 assistant 行必须回传 reasoning_content:中途开启思考时,
|
||||
// 本轮早先的调用没有思维链,用空串占位(缺失会被上游 400,空串实测通过)。
|
||||
body.messages = body.messages.map(mm => {
|
||||
if (!mm || !mm.__reasoning) return mm;
|
||||
if (!mm || mm.role !== 'assistant') return mm;
|
||||
const fill = !!(opts.deepThink && ack && mm.tool_calls && !mm.__reasoning);
|
||||
if (!mm.__reasoning && !fill) return mm;
|
||||
const c = Object.assign({}, mm);
|
||||
if (ack) c[ack] = c.__reasoning;
|
||||
if (ack) c[ack] = c.__reasoning || '';
|
||||
delete c.__reasoning;
|
||||
return c;
|
||||
});
|
||||
|
||||
+165
-11
@@ -280,7 +280,7 @@ main { flex: 1; min-height: 0; display: flex; flex-direction: column; }
|
||||
.tool-card .tool-time .act-del:hover { color: #c0392b; }
|
||||
|
||||
/* ================= 底部输入区 ================= */
|
||||
footer { flex: none; padding: 8px 12px calc(10px + env(safe-area-inset-bottom, 0px)); background: var(--page); }
|
||||
footer { position: relative; flex: none; padding: 8px 12px calc(10px + env(safe-area-inset-bottom, 0px)); background: var(--page); }
|
||||
.attach-chips { display: flex; flex-wrap: wrap; gap: 6px; margin-bottom: 8px; }
|
||||
.attach-chip {
|
||||
display: inline-flex; align-items: center; gap: 6px;
|
||||
@@ -338,6 +338,7 @@ footer { flex: none; padding: 8px 12px calc(10px + env(safe-area-inset-bottom, 0
|
||||
}
|
||||
.up-card.done::after, .up-card.err::after { display: none; }
|
||||
.input-shell {
|
||||
position: relative; /* 按住说话的遮罩以它为定位基准 */
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: stretch;
|
||||
@@ -404,6 +405,76 @@ footer { flex: none; padding: 8px 12px calc(10px + env(safe-area-inset-bottom, 0
|
||||
}
|
||||
.plus-btn:active { background: #dde0e6; }
|
||||
.plus-btn.active { background: var(--active-soft); color: var(--active); } /* 文件面板展开时高亮 */
|
||||
/* 语音模式:输入框位置换成「按住 说话」——只有居中文字,不做出按钮外观 */
|
||||
#voice-hold {
|
||||
position: relative;
|
||||
height: 44px; border-radius: 8px;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
font-size: 15px; color: var(--ink); cursor: pointer;
|
||||
user-select: none; -webkit-user-select: none; -webkit-touch-callout: none;
|
||||
touch-action: none; transition: background .12s;
|
||||
}
|
||||
/* 按住区与下方操作栏之间的分隔线(用伪元素画,避开录音态底色的圆角) */
|
||||
#voice-hold::after {
|
||||
content: '';
|
||||
position: absolute; left: 0; right: 0; bottom: -1px;
|
||||
height: 1px; background: var(--border);
|
||||
}
|
||||
/* 录音中给一个极浅底色作反馈(文字同时变成「松开 结束」) */
|
||||
#voice-hold.recording { background: var(--surface); }
|
||||
|
||||
/* 录音期间:输入卡内容整体让位给遮罩(透明渐变不能把按钮/文字透出来) */
|
||||
body.voice-active .input-shell { border-color: transparent; box-shadow: none; }
|
||||
body.voice-active .input-shell > *:not(#voice-overlay) { visibility: hidden; }
|
||||
body.voice-active .attach-chips { visibility: hidden; }
|
||||
body.voice-active .scroll-bottom-btn { display: none; }
|
||||
|
||||
/* 按住说话的遮罩:左右贴屏幕两边;底部一直铺到屏幕底(含 footer 的 10px 内边距与安全区),
|
||||
向上多铺 72px 让渐变淡出。
|
||||
声波中心对齐「按住 说话」那一行:padding-bottom 70px + 安全区 =
|
||||
59(声波中心到输入卡底部)+ 11(遮罩底边向下外扩的 1px 边框 + 10px footer 内边距)。 */
|
||||
#voice-overlay {
|
||||
position: absolute;
|
||||
top: -72px;
|
||||
bottom: calc(-11px - env(safe-area-inset-bottom, 0px));
|
||||
left: -13px; right: -13px;
|
||||
z-index: 6;
|
||||
display: flex; flex-direction: column; align-items: center; justify-content: flex-end;
|
||||
gap: 20px;
|
||||
padding-bottom: calc(70px + env(safe-area-inset-bottom, 0px));
|
||||
/* 极淡的蓝调,从下往上淡出(“几乎看不出来,仔细看才有一层”) */
|
||||
background: linear-gradient(to top, rgba(57, 100, 254, .09), rgba(57, 100, 254, 0));
|
||||
color: var(--active);
|
||||
pointer-events: auto; touch-action: none;
|
||||
transition: background .15s linear, color .15s linear;
|
||||
}
|
||||
/* 上滑取消态:整体换成极淡的红调,声波/文案同步变红 */
|
||||
#voice-overlay.cancel {
|
||||
background: linear-gradient(to top, rgba(192, 57, 43, .09), rgba(192, 57, 43, 0));
|
||||
color: #c0392b;
|
||||
}
|
||||
.vo-hint { font-size: 13px; line-height: 18px; color: var(--gray); letter-spacing: .2px; }
|
||||
.vo-wave { display: flex; align-items: center; justify-content: center; gap: 3px; height: 34px; max-width: 100%; }
|
||||
/* 柱子高度用 scaleY 而不是 height:只走合成层,不触发重排 */
|
||||
.vo-wave i {
|
||||
display: block; width: 3px; height: 34px; border-radius: 2px;
|
||||
background: currentColor;
|
||||
transform: scaleY(.12); transform-origin: center;
|
||||
transition: transform 70ms linear;
|
||||
}
|
||||
/* 上滑取消态:声波与文案统一转警示色 */
|
||||
#voice-overlay.cancel .vo-hint { color: #c0392b; }
|
||||
@media (prefers-reduced-motion: reduce) { .vo-wave i { transition: none; } }
|
||||
|
||||
/* 发送键四态:发送 / 停止 / 语音 / 键盘。语音与键盘用中性底色,和蓝色「发送」区分开 */
|
||||
.send-btn > svg { display: none; }
|
||||
.send-btn[data-state="send"] .ic-send { display: block; }
|
||||
.send-btn[data-state="voice"] .ic-voice,
|
||||
.send-btn[data-state="keyboard"] .ic-kb { display: block; }
|
||||
.send-btn[data-state="voice"],
|
||||
.send-btn[data-state="keyboard"] { background: var(--surface); color: var(--ink); box-shadow: none; }
|
||||
.send-btn[data-state="voice"]:active,
|
||||
.send-btn[data-state="keyboard"]:active { background: #dde0e6; opacity: 1; }
|
||||
/* 发送按钮:蓝底白向上箭头 */
|
||||
.send-btn {
|
||||
flex: none; width: 36px; height: 36px; border-radius: 50%;
|
||||
@@ -792,8 +863,10 @@ footer { flex: none; padding: 8px 12px calc(10px + env(safe-area-inset-bottom, 0
|
||||
}
|
||||
|
||||
/* ================= 回到底部浮动按钮(同步 index 右下角导航浮动图标) ================= */
|
||||
/* 停靠在输入区上方:包含块是 footer,bottom:100% = footer 顶边,
|
||||
输入框长高/编辑条出现时随 footer 一起上移,不再用固定值 140px 遮挡输入区。 */
|
||||
.scroll-bottom-btn {
|
||||
position: absolute; right: 14px; bottom: 140px;
|
||||
position: absolute; right: 14px; bottom: calc(100% + 6px); margin: 0;
|
||||
width: 38px; height: 38px; border-radius: 50%;
|
||||
border: none; cursor: pointer; padding: 0;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
@@ -850,20 +923,60 @@ footer { flex: none; padding: 8px 12px calc(10px + env(safe-area-inset-bottom, 0
|
||||
/* ================= 全屏查看器(搬自 index;图片单点放大+缩放,消息双击放大) ================= */
|
||||
#img-viewer { position: fixed; inset: 0; z-index: 150; background: rgba(0,0,0,0.93); display: flex; align-items: center; justify-content: center; overflow: hidden; }
|
||||
#img-viewer.hidden { display: none; }
|
||||
#img-viewer-img { max-width: 92vw; max-height: 88vh; touch-action: none; cursor: grab; user-select: none; -webkit-user-drag: none; }
|
||||
/* 两层 img 绝对定位居中叠放:滑动切换时旧图滑出、新图同步滑入(见 js/stream.js 的 slideTo) */
|
||||
#img-viewer img { position: absolute; inset: 0; margin: auto; max-width: 100vw; max-height: 88vh; touch-action: none; cursor: grab; user-select: none; -webkit-user-drag: none; will-change: transform; }
|
||||
/* 组内页码:底部居中悬浮,不参与命中测试(点它等于点背景会关闭) */
|
||||
#img-count { position: fixed; left: 50%; bottom: calc(18px + env(safe-area-inset-bottom, 0px)); transform: translateX(-50%); z-index: 151; padding: 3px 12px; border-radius: 999px; background: rgba(0,0,0,0.5); color: #fff; font-size: 12px; letter-spacing: .5px; pointer-events: none; }
|
||||
#fs-viewer { position: fixed; inset: 0; z-index: 130; background: rgba(8,10,20,0.88); padding: 40px 14px 24px; overflow: auto; /* 缩放后支持横纵滚动 */ }
|
||||
#fs-viewer.hidden { display: none; }
|
||||
.fs-panel { max-width: 860px; margin: 0 auto; background: var(--page); border-radius: 14px; padding: 18px 20px; font-size: 15.5px; }
|
||||
.fs-close {
|
||||
position: fixed; top: 12px; right: 14px; z-index: 131; width: 34px; height: 34px;
|
||||
border: none; border-radius: 50%; background: rgba(255,255,255,0.18); color: #fff;
|
||||
/* 顶部操作区:默认右上角悬浮(消息全屏用);文件预览走 fs-bar 模式时独立成一行 */
|
||||
.fs-actions { position: fixed; top: calc(2px + env(safe-area-inset-top, 0px)); right: 14px; z-index: 131; display: flex; align-items: center; gap: 8px; }
|
||||
.fs-name { display: none; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; font-size: 13px; font-weight: 600; color: var(--ink); }
|
||||
.fs-btn {
|
||||
flex: none; display: flex; align-items: center; justify-content: center;
|
||||
width: 34px; height: 34px; border: none; border-radius: 50%;
|
||||
background: rgba(0,0,0,0.42); color: #fff; box-shadow: 0 1px 5px rgba(0,0,0,0.22);
|
||||
font-size: 15px; line-height: 1; cursor: pointer;
|
||||
}
|
||||
/* 源码切换 eye 钮:与关闭钮同款,居其左;仅当前内容有源文本时显示(setFsSource 控显隐) */
|
||||
.fs-eye { right: 58px; display: flex; align-items: center; justify-content: center; }
|
||||
.fs-eye svg { width: 17px; height: 17px; }
|
||||
.fs-eye.on { background: rgba(255,255,255,0.85); color: #333; }
|
||||
/* 源码面板:与 #fs-body 互斥显示(toggleFsSrc 切换);不参与双指缩放逻辑 */
|
||||
.fs-btn svg { width: 17px; height: 17px; }
|
||||
.fs-btn.on { background: rgba(255,255,255,0.85); color: #333; }
|
||||
|
||||
/* 文件预览沉浸模式:40px 操作条 + 全宽内容。内容从条下开始,不会叠在页面自己的顶部按钮上;
|
||||
宽度一点不让(手机横向最贵),只牺牲一条操作栏的高度。 */
|
||||
#fs-viewer.fs-bar { display: flex; flex-direction: column; padding: 0; overflow: auto; background: var(--page); }
|
||||
#fs-viewer.fs-bar .fs-actions {
|
||||
position: sticky; top: 0; z-index: 1;
|
||||
flex: none; width: 100%;
|
||||
height: calc(40px + env(safe-area-inset-top, 0px));
|
||||
padding: env(safe-area-inset-top, 0px) 10px 0 14px;
|
||||
justify-content: flex-end; gap: 6px;
|
||||
background: var(--page); border-bottom: 1px solid var(--border);
|
||||
}
|
||||
#fs-viewer.fs-bar .fs-name { display: block; flex: 1; }
|
||||
#fs-viewer.fs-bar .fs-btn { width: 32px; height: 32px; border-radius: 8px; background: transparent; color: var(--ink); box-shadow: none; }
|
||||
#fs-viewer.fs-bar .fs-btn:active { background: var(--surface); }
|
||||
#fs-viewer.fs-bar .fs-btn.on { background: var(--active); color: #fff; }
|
||||
#fs-viewer.fs-bar .fs-panel { flex: 1; min-height: 0; width: 100%; max-width: none; margin: 0; padding: 0; border-radius: 0; background: var(--page); }
|
||||
#fs-viewer.fs-bar #fs-body { position: relative; overflow: auto; }
|
||||
#fs-viewer.fs-bar #fs-body .fs-html-frame { height: 100%; min-height: 0; border-radius: 0; }
|
||||
/* 文本/源码直排:去掉气泡卡片的内外边距,只留 12px 阅读边距 */
|
||||
#fs-viewer.fs-bar #fs-body .bubble-ai,
|
||||
#fs-viewer.fs-bar #fs-body .bubble-user {
|
||||
max-width: 100%; margin: 0; border-radius: 0; box-shadow: none; background: var(--page);
|
||||
padding: 14px 12px calc(28px + env(safe-area-inset-bottom, 0px)); font-size: 15.5px;
|
||||
}
|
||||
#fs-viewer.fs-bar #fs-src-pane { overflow: auto; }
|
||||
#fs-viewer.fs-bar #fs-src-code { max-height: none; border: 0; border-radius: 0; padding: 12px 14px calc(28px + env(safe-area-inset-bottom, 0px)); }
|
||||
#fs-viewer.fs-bar .fs-img-frame { max-width: 100%; max-height: 100%; border-radius: 0; margin: 0 auto; }
|
||||
.fs-close {
|
||||
/* 顶到视口最上沿(避开刘海),不再压住预览面板顶部 */
|
||||
position: fixed; top: calc(2px + env(safe-area-inset-top, 0px)); right: 14px; z-index: 131; width: 34px; height: 34px;
|
||||
border: none; border-radius: 50%; background: rgba(0,0,0,0.42); color: #fff;
|
||||
box-shadow: 0 1px 5px rgba(0,0,0,0.22);
|
||||
font-size: 15px; line-height: 1; cursor: pointer;
|
||||
}
|
||||
/* 源码面板:与 #fs-body 互斥显示(toggleFsSrc 切换);bar 模式下占满操作条以下区域 */
|
||||
.fs-src-panel { padding: 14px 16px; }
|
||||
#fs-src-code {
|
||||
margin: 0; white-space: pre-wrap; word-break: break-word;
|
||||
@@ -1429,3 +1542,44 @@ body.pv-resizing #pv-pane iframe { pointer-events: none; }
|
||||
.send-btn .stop-sq { display: none; width: 11px; height: 11px; border-radius: 2px; background: #fff; }
|
||||
.send-btn[data-state="stop"] > svg { display: none; }
|
||||
.send-btn[data-state="stop"] .stop-sq { display: block; }
|
||||
|
||||
/* ================= 拖拽上传提示层 ================= */
|
||||
/* window 级 dragenter/dragover 期间显示。pointer-events:none 保证遮罩不抢 drop 目标,
|
||||
drop 仍由 window 监听器接管(见 js/files.js)。
|
||||
视觉沿用站内弹窗语言:与 .modal-overlay 同一底色,白底卡片 + 16px 圆角 + 投影,
|
||||
图标用品牌蓝的浅色圆底,虚线框只作为「可投放」的轻提示。 */
|
||||
#drop-overlay {
|
||||
position: fixed; inset: 0; z-index: 190;
|
||||
display: flex; align-items: center; justify-content: center;
|
||||
background: rgba(20, 25, 50, 0.45);
|
||||
pointer-events: none;
|
||||
animation: drop-overlay-in .14s ease-out;
|
||||
}
|
||||
#drop-overlay .drop-hint {
|
||||
display: flex; flex-direction: column; align-items: center; gap: 12px;
|
||||
padding: 24px 36px;
|
||||
max-width: 80vw; text-align: center;
|
||||
border: 1.5px dashed rgba(57, 100, 254, 0.32);
|
||||
border-radius: 16px;
|
||||
background: var(--page); color: var(--ink);
|
||||
font-size: 15px; font-weight: 600;
|
||||
box-shadow: 0 12px 32px rgba(15, 17, 21, 0.18);
|
||||
}
|
||||
#drop-overlay .drop-hint .ic {
|
||||
width: 50px; height: 50px; padding: 12px;
|
||||
border-radius: 50%;
|
||||
background: var(--active-soft); color: var(--active);
|
||||
}
|
||||
@keyframes drop-overlay-in { from { opacity: 0; } to { opacity: 1; } }
|
||||
|
||||
/* ================= 编辑已发送消息提示条 ================= */
|
||||
/* 覆盖式重发:输入框内部的浅蓝提示条(嵌在 .input-shell 顶端,间距由 shell 的 gap 提供)。 */
|
||||
.edit-bar {
|
||||
display: flex; align-items: center; gap: 8px;
|
||||
padding: 6px 6px 6px 12px;
|
||||
border-radius: 12px; background: var(--active-soft);
|
||||
color: var(--active); font-size: 12.5px;
|
||||
}
|
||||
.edit-bar span { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
.edit-bar button { flex: none; padding: 2px 10px; border-radius: 8px; font-size: 12.5px; font-weight: 600; color: var(--active); }
|
||||
.edit-bar button:active { background: rgba(57, 100, 254, 0.12); }
|
||||
|
||||
@@ -158,8 +158,15 @@
|
||||
<footer>
|
||||
<div class="attach-chips hidden" id="attach-chips"></div>
|
||||
<div class="input-shell">
|
||||
<!-- 编辑已发送消息:覆盖式重发提示条(发送后丢弃该条及其后的历史) -->
|
||||
<div class="edit-bar hidden" id="edit-bar">
|
||||
<span>正在编辑已发送的消息,发送后将丢弃其后的历史</span>
|
||||
<button id="edit-cancel" type="button">取消</button>
|
||||
</div>
|
||||
<div class="up-cards hidden" id="up-cards" aria-label="已上传文件"></div>
|
||||
<textarea id="msg-input" placeholder="传送信息" rows="2" enterkeyhint="enter"></textarea>
|
||||
<!-- 语音模式:输入框位置变成「按住 说话」(只有居中文字,无按钮外观) -->
|
||||
<div id="voice-hold" class="hidden" role="button" tabindex="0" aria-label="按住说话">按住 说话</div>
|
||||
<div class="input-actions">
|
||||
<div class="ia-side ia-left">
|
||||
<button class="think-toggle" id="deep-think-btn" title="深度思考">
|
||||
@@ -174,22 +181,31 @@
|
||||
<button class="plus-btn" id="attach-btn" title="上传文件" aria-label="上传文件">
|
||||
<svg viewBox="0 0 24 24" class="ic sm"><path fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" d="M21.44 11.05l-9.19 9.19a6 6 0 0 1-8.49-8.49l9.19-9.19a4 4 0 0 1 5.66 5.66l-9.2 9.19a2 2 0 0 1-2.83-2.83l8.49-8.48"></path></svg>
|
||||
</button>
|
||||
<!-- 同一按钮两态:发送(↑)/ 停止(白色方块)。data-state 由 JS 切换,尺寸不变避免输入区跳动 -->
|
||||
<!-- 同一按钮四态:发送(↑)/ 停止(方块)/ 语音(麦克风)/ 键盘。
|
||||
data-state 由 JS 切换,尺寸不变避免输入区跳动 -->
|
||||
<button class="send-btn" id="send-btn" data-state="send" title="发送" aria-label="发送">
|
||||
<svg viewBox="0 0 24 24" class="ic sm"><line x1="12" y1="19" x2="12" y2="5"></line><polyline points="5 12 12 5 19 12"></polyline></svg>
|
||||
<svg class="ic sm ic-send" viewBox="0 0 24 24"><line x1="12" y1="19" x2="12" y2="5"></line><polyline points="5 12 12 5 19 12"></polyline></svg>
|
||||
<svg class="ic sm ic-voice" viewBox="0 0 24 24"><rect x="9" y="2" width="6" height="12" rx="3"></rect><path d="M5 10a7 7 0 0 0 14 0"></path><line x1="12" y1="19" x2="12" y2="22"></line></svg>
|
||||
<svg class="ic sm ic-kb" viewBox="0 0 24 24"><rect x="3" y="5" width="18" height="14" rx="2.5"></rect><circle cx="7" cy="9" r="1.1" fill="currentColor" stroke="none"></circle><circle cx="12" cy="9" r="1.1" fill="currentColor" stroke="none"></circle><circle cx="17" cy="9" r="1.1" fill="currentColor" stroke="none"></circle><circle cx="7" cy="12.5" r="1.1" fill="currentColor" stroke="none"></circle><circle cx="12" cy="12.5" r="1.1" fill="currentColor" stroke="none"></circle><circle cx="17" cy="12.5" r="1.1" fill="currentColor" stroke="none"></circle><line x1="8" y1="16" x2="16" y2="16"></line></svg>
|
||||
<span class="stop-sq" aria-hidden="true"></span>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<!-- 按住说话的遮罩:盖满输入卡并扩到屏幕两边(footer 有 12px 内边距 + shell 1px 边框)。
|
||||
声波由 JS 生成 25 根柱子;上滑到文案区域进入取消态(整体变红)。 -->
|
||||
<div id="voice-overlay" class="hidden" aria-hidden="true">
|
||||
<div class="vo-hint" id="voice-hint">松手传送,上滑取消</div>
|
||||
<div class="vo-wave" id="voice-wave"></div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- 回到底部浮动按钮(搬自 index;上翻历史时出现、AI 输出中常驻转圈)。
|
||||
放在 footer 内、相对 footer 顶边停靠:输入框长高/编辑条出现时不会被盖住 -->
|
||||
<button class="scroll-bottom-btn hidden" id="scroll-bottom-btn" onclick="clickScrollBottom()" title="回到底部" aria-label="回到底部">
|
||||
<svg class="sb-arrow" viewBox="0 0 24 24"><polyline points="6 9 12 15 18 9"></polyline></svg>
|
||||
<svg class="sb-spinner hidden" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10"></circle><path d="M22 12a10 10 0 0 1-10 10"></path></svg>
|
||||
</button>
|
||||
</footer>
|
||||
|
||||
<!-- 回到底部浮动按钮(搬自 index;上翻历史时出现、AI 输出中常驻转圈) -->
|
||||
<button class="scroll-bottom-btn hidden" id="scroll-bottom-btn" onclick="clickScrollBottom()" title="回到底部" aria-label="回到底部">
|
||||
<svg class="sb-arrow" viewBox="0 0 24 24"><polyline points="6 9 12 15 18 9"></polyline></svg>
|
||||
<svg class="sb-spinner hidden" viewBox="0 0 24 24"><circle cx="12" cy="12" r="10"></circle><path d="M22 12a10 10 0 0 1-10 10"></path></svg>
|
||||
</button>
|
||||
|
||||
<!-- ================= 历史记录抽屉 ================= -->
|
||||
<div id="scrim"></div>
|
||||
<aside id="drawer" aria-label="对话历史">
|
||||
@@ -435,10 +451,12 @@
|
||||
<div id="toast" class="hidden"></div>
|
||||
</div>
|
||||
|
||||
<!-- 图片全屏查看器(点击消息/工具卡中图片打开;双指缩放、放大后可拖动) -->
|
||||
<!-- 图片全屏查看器(点击消息/工具卡中图片打开;双指缩放、放大后可拖动;同组多图可左右滑动切换) -->
|
||||
<div id="img-viewer" class="hidden">
|
||||
<button class="fs-close" onclick="closeImageViewer()">✕</button>
|
||||
<img id="img-viewer-img" alt="">
|
||||
<!-- 组内页码(如 2 / 5):仅同组多图时显示 -->
|
||||
<span id="img-count" class="hidden"></span>
|
||||
</div>
|
||||
|
||||
<!-- ================= 文档预览右栏(桌面端≥960px;office/全部文档类型统一承载;移动端走全屏 #fs-viewer,本栏隐藏) -->
|
||||
@@ -471,20 +489,38 @@
|
||||
|
||||
<!-- 消息全屏查看器(双击气泡/工具卡打开;支持双指缩放) -->
|
||||
<div id="fs-viewer" class="hidden" onclick="if(event.target===this)closeFullscreen()">
|
||||
<button class="fs-close" onclick="closeFullscreen()">✕</button>
|
||||
<!-- 源码切换钮:仅当当前内容有源文本时显隐(toggleFsSrc 在渲染视图/源码面板间互切) -->
|
||||
<button class="fs-close fs-eye hidden" id="fs-src-btn" title="显示源码" aria-label="显示源码">
|
||||
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M1 12s4-8 11-8 11 8 11 8-4 8-11 8-11-8-11-8z"></path><circle cx="12" cy="12" r="3"></circle></svg>
|
||||
</button>
|
||||
<!-- 顶部操作区:文件预览(fs-bar 模式)时独立成一行,内容从行下开始,不遮挡页面自己的顶部按钮;
|
||||
消息全屏时退化为右上角悬浮按钮簇。文件名仅栏模式显示。 -->
|
||||
<div class="fs-actions">
|
||||
<span class="fs-name" id="fs-file-name"></span>
|
||||
<!-- 源码切换钮:仅当当前内容有源文本时显隐(toggleFsSrc 在渲染视图/源码面板间互切) -->
|
||||
<button class="fs-btn hidden" id="fs-src-btn" title="显示源码" aria-label="显示源码">
|
||||
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M1 12s4-8 11-8 11 8 11 8-4 8-11 8-11-8-11-8z"></path><circle cx="12" cy="12" r="3"></circle></svg>
|
||||
</button>
|
||||
<!-- 复制源码:显隐与 eye 一致(有源文本才出现);HTML/SVG/MD/代码通用 -->
|
||||
<button class="fs-btn hidden" id="fs-copy-btn" title="复制源码" aria-label="复制源码">
|
||||
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="13" height="13" rx="2"></rect><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"></path></svg>
|
||||
</button>
|
||||
<button class="fs-btn" onclick="closeFullscreen()" title="关闭" aria-label="关闭">✕</button>
|
||||
</div>
|
||||
<div class="fs-panel" id="fs-body"></div>
|
||||
<div class="fs-panel fs-src-panel hidden" id="fs-src-pane"><pre id="fs-src-code"></pre></div>
|
||||
</div>
|
||||
|
||||
<!-- 拖拽上传提示层:window 级 dragenter 时显示;pointer-events:none,不参与命中测试(逻辑见 js/files.js) -->
|
||||
<div id="drop-overlay" class="hidden" aria-hidden="true">
|
||||
<div class="drop-hint">
|
||||
<svg viewBox="0 0 24 24" class="ic"><path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"></path><polyline points="17 8 12 3 7 8"></polyline><line x1="12" y1="3" x2="12" y2="15"></line></svg>
|
||||
<span>松开即可上传</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="/js/core.js"></script>
|
||||
<script src="/js/chat.js"></script>
|
||||
<script src="/js/sessions.js"></script>
|
||||
<script src="/js/stream.js"></script>
|
||||
<script src="/js/files.js"></script>
|
||||
<script src="/js/voice.js"></script>
|
||||
<script src="/js/preview.js"></script>
|
||||
<script src="/js/settings.js"></script>
|
||||
<script>
|
||||
|
||||
@@ -27,7 +27,7 @@
|
||||
(u.cached == null ? '未上报' : u.cached + ' tok') + ',输出 ' + (u.ct || 0) + ' tok';
|
||||
}
|
||||
|
||||
function actionsHtml(ts, content, msgId, usage) {
|
||||
function actionsHtml(ts, content, msgId, usage, canEdit) {
|
||||
const b64 = btoa(unescape(encodeURIComponent(content)));
|
||||
// 删除本条:仅已落库消息可删(传 m.id);流式未落库时无 id,不渲染
|
||||
const del = msgId
|
||||
@@ -35,6 +35,12 @@
|
||||
'<svg class="ic sm" viewBox="0 0 24 24"><path d="M18 6 6 18"></path><path d="m6 6 12 12"></path></svg>' +
|
||||
'</button>'
|
||||
: '';
|
||||
// 编辑并重发:只给未压缩的用户消息(调用方传 canEdit);带 b64 回填输入框,避免从 DOM 反推
|
||||
const edit = canEdit && msgId
|
||||
? '<button class="act act-edit" title="编辑并重发" data-msg-id="' + msgId + '" data-b64="' + b64 + '">' +
|
||||
'<svg class="ic sm" viewBox="0 0 24 24"><path d="M12 20h9"></path><path d="M16.5 3.5a2.1 2.1 0 0 1 3 3L7 19l-4 1 1-4z"></path></svg>' +
|
||||
'</button>'
|
||||
: '';
|
||||
const u = normUsage(usage);
|
||||
const usageHtml = u
|
||||
? '<span class="act-usage" title="' + usageTip(u) + '">' + usageBrief(u) + '</span>'
|
||||
@@ -45,6 +51,7 @@
|
||||
'<button class="act act-copy" title="复制" data-b64="' + b64 + '">' +
|
||||
'<svg class="ic sm" viewBox="0 0 24 24"><rect x="9" y="9" width="13" height="13" rx="2"></rect><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"></path></svg>' +
|
||||
'</button>' +
|
||||
edit +
|
||||
del +
|
||||
'</div>';
|
||||
}
|
||||
@@ -55,9 +62,10 @@
|
||||
? '<div class="attach-line">📎 ' + esc(p.files.join('、')) + '</div>' : '';
|
||||
const r = mdHtml(p.text);
|
||||
// 复制内容取纯文本(不含【系统通知】附件行),与气泡里显示一致
|
||||
// 编辑按钮只给「用户手打/粘贴」的消息:archived(已压缩存档)与 scheduler_note 等系统行不给
|
||||
return '<div class="msg me">' +
|
||||
'<div class="bubble-user' + (r.fmt ? ' md' : '') + '">' + attach + r.html + '</div>' +
|
||||
actionsHtml(m.created_at, p.text, m.id) +
|
||||
actionsHtml(m.created_at, p.text, m.id, null, !m.type) +
|
||||
'</div>';
|
||||
}
|
||||
|
||||
@@ -513,7 +521,8 @@
|
||||
const wrap = document.createElement('div');
|
||||
wrap.className = 'msg me';
|
||||
wrap.innerHTML = '<div class="bubble-user' + (r.fmt ? ' md' : '') + '">' + attach + r.html + '</div>' +
|
||||
actionsHtml(Date.now(), content, msgId);
|
||||
// 刚发送的用户消息同样可编辑(canEdit=true);actionsHtml 只在有 msgId 时才渲染按钮
|
||||
actionsHtml(Date.now(), content, msgId, null, true);
|
||||
decorateCodePre(wrap); // 用户消息里也可能有围栏代码块
|
||||
$('chat-view').appendChild(wrap);
|
||||
// 已落库的行记入增量跟踪:避免回前台补拉时把这条再追加一遍
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
* POST /api/sessions {title:'', model_id:''} 首条消息自动建档
|
||||
* GET /api/sessions/:id 打开会话(消息+文件)
|
||||
* POST /api/sessions/:id/messages {content, attachments} 发送消息
|
||||
* POST /api/sessions/:id/messages/:mid/edit 编辑用户消息(覆盖式重发)
|
||||
* POST /api/sessions/:id/files 附件上传(multipart)
|
||||
* GET /api/sessions/:id/stream SSE 流式输出
|
||||
* POST /api/sessions/:id/stop | /release 状态栏「终止/断开」
|
||||
@@ -169,6 +170,7 @@
|
||||
schedule: null, // 当前会话定时任务(详情 schedule 字段,null=无)
|
||||
sessions: [], // 抽屉列表缓存
|
||||
attachments: [], // 待发送附件 {file, name, size}
|
||||
editingMsgId: 0, // 正在编辑的历史消息 id(0=普通发送,覆盖式重发)
|
||||
lastImgBlobUrl: null, // 任务文件图片预览的 blob URL,下次预览或开新图时回收
|
||||
lastPdfBlobUrl: null, // 同上:PDF 页内预览的 blob URL(浏览器内建查看器用)
|
||||
es: null, // EventSource
|
||||
|
||||
+170
-17
@@ -30,11 +30,13 @@
|
||||
send();
|
||||
}
|
||||
});
|
||||
// 发送键双职:生成中且输入框为空时变「停止」(白色方块),点击终止当前回合;
|
||||
// 输入框有字时它仍是「发送」——运行中允许插话,直接发出去(后端 live 追投)。
|
||||
// 发送键四职:语音 / 键盘(语音模式)/ 停止 / 发送,由 data-state 决定点击含义。
|
||||
// 400ms 内重复点击忽略:防双击重复 abort(服务端 status 事件回来前按钮仍是停止态)。
|
||||
let stopClickAt = 0;
|
||||
$('send-btn').addEventListener('click', () => {
|
||||
const st = $('send-btn').dataset.state;
|
||||
if (st === 'voice' && window.AgentVoice) { window.AgentVoice.enter(); return; }
|
||||
if (st === 'keyboard' && window.AgentVoice) { window.AgentVoice.exit(true); return; }
|
||||
if (stopModeActive()) {
|
||||
if (Date.now() - stopClickAt < 400) return;
|
||||
stopClickAt = Date.now();
|
||||
@@ -675,7 +677,7 @@
|
||||
if (e.key === 'Enter' || e.key === ' ') { e.preventDefault(); toggleSelectAll(); }
|
||||
});
|
||||
// ---------- 大图上传前压缩(同步 v1:阈值/参数由 /api/system/status 透出) ----------
|
||||
const DEFAULT_UPLOAD_CFG = { image_compress_threshold_kb: 1536, image_max_dim: 2000, image_quality: 0.85, confirm_size_mb: 10 };
|
||||
const DEFAULT_UPLOAD_CFG = { image_compress_threshold_kb: 1536, image_max_dim: 2000, image_quality: 0.85, confirm_size_mb: 10, paste_to_file_threshold_chars: 2000 };
|
||||
// 后端 multer 单文件硬限(routes/files.js limits.filesize),前端预检对齐它,超了直接拒掉不发起上传
|
||||
const UPLOAD_HARD_LIMIT = 50 * 1024 * 1024;
|
||||
let uploadCfgCache = null;
|
||||
@@ -815,6 +817,12 @@
|
||||
card.dataset.done = '1'; // 仅成功上传的卡片参与发送时的告知行
|
||||
closeBtn.classList.remove('hidden');
|
||||
},
|
||||
// 服务端重名会加 -1/-2:告知行必须用真实落盘名,否则模型读不到/读到旧文件
|
||||
rename(name) {
|
||||
card.dataset.name = name;
|
||||
const el = card.querySelector('.up-name');
|
||||
if (el) { el.textContent = name; el.title = name; }
|
||||
},
|
||||
fail() {
|
||||
card.classList.add('err');
|
||||
mask.innerHTML = '<span class="up-pct">上传失败</span>';
|
||||
@@ -830,12 +838,10 @@
|
||||
|
||||
// 文件选择统一入口:先逐文件预处理(大图压缩询问/大文件确认,X或遮罩=放弃该文件),
|
||||
// 只要还剩至少一个待传文件才自动建档(welcome 无会话时)并走即传路径(多文件并行上传+总进度条+逐卡进度);
|
||||
// 全部放弃或取消选择都不上传、不新建会话。
|
||||
$('file-input').addEventListener('change', async e => {
|
||||
const input = e.target;
|
||||
const files = Array.prototype.slice.call(input.files || []); // 先快照再清空,FileList 是实时的
|
||||
input.value = '';
|
||||
if (!files.length) return;
|
||||
// 全部放弃或取消选择都不上传、不新建会话。拖拽 drop 也走这里。
|
||||
// 并发批次共用一个总进度条:各自登记 {loaded,total},绘制时求和,避免后一批把前一批的进度冲掉。
|
||||
const uploadBatches = new Set();
|
||||
async function uploadFilesNow(files) {
|
||||
// 逐个串行预处理(模态弹窗不能并行)
|
||||
const ready = [];
|
||||
for (const f of files) {
|
||||
@@ -855,11 +861,13 @@
|
||||
const cards = ready.map(f => addUploadCard(f));
|
||||
// ---- 并行上传 + 实时进度 ----
|
||||
const progWrap = $('fp-progress'), progBar = $('fp-progress-bar'), progPct = $('fp-progress-pct');
|
||||
const tot = ready.reduce((s, f) => s + f.size, 0);
|
||||
const loaded = { v: 0 }; // 所有文件已上传字节总和
|
||||
const batch = { loaded: 0, total: ready.reduce((s, f) => s + f.size, 0) };
|
||||
uploadBatches.add(batch);
|
||||
let done = 0, errCount = 0;
|
||||
function paintProg() {
|
||||
const pct = tot ? Math.min(100, Math.round(loaded.v / tot * 100)) : 0;
|
||||
let loaded = 0, total = 0;
|
||||
uploadBatches.forEach(b => { loaded += b.loaded; total += b.total; });
|
||||
const pct = total ? Math.min(100, Math.round(loaded / total * 100)) : 0;
|
||||
progBar.style.width = pct + '%';
|
||||
progPct.textContent = pct + '%';
|
||||
}
|
||||
@@ -874,27 +882,172 @@
|
||||
xhr.upload.onprogress = function (ev) {
|
||||
if (!ev.lengthComputable) return;
|
||||
// ev.loaded 是本文件累计已发字节,汇总时取本次增量避免多计
|
||||
loaded.v += ev.loaded - lastLoaded;
|
||||
batch.loaded += ev.loaded - lastLoaded;
|
||||
lastLoaded = ev.loaded;
|
||||
paintProg();
|
||||
card.setPct(ev.loaded / ev.total * 100);
|
||||
};
|
||||
xhr.onload = function () {
|
||||
if (xhr.status === 200) { done++; card.done(); } else { errCount++; card.fail(); }
|
||||
if (xhr.status === 200) {
|
||||
done++;
|
||||
try {
|
||||
const u = (JSON.parse(xhr.responseText || '{}').uploaded || [])[0];
|
||||
if (u && u.filename) card.rename(u.filename);
|
||||
} catch (_) { /* 响应体异常不影响上传成功态 */ }
|
||||
card.done();
|
||||
} else { errCount++; card.fail(); }
|
||||
// 终态补齐(可能没收到最后一帧 100%)
|
||||
if (lastLoaded < f.size) { loaded.v += f.size - lastLoaded; paintProg(); }
|
||||
if (lastLoaded < f.size) { batch.loaded += f.size - lastLoaded; paintProg(); }
|
||||
resolve();
|
||||
};
|
||||
xhr.onerror = function () { errCount++; card.fail(); resolve(); };
|
||||
xhr.send(fd);
|
||||
}));
|
||||
await Promise.all(tasks);
|
||||
if (batch.loaded < batch.total) batch.loaded = batch.total;
|
||||
uploadBatches.delete(batch);
|
||||
paintProg();
|
||||
// 全部成功不再弹提示(卡片已完成态即反馈);有失败才提醒
|
||||
if (errCount) toast('已上传 ' + done + ' 个,失败 ' + errCount + ' 个');
|
||||
loadFiles();
|
||||
// 1.2s 后收起进度条
|
||||
setTimeout(() => { progWrap.classList.add('hidden'); progBar.style.width = '0%'; }, 1200);
|
||||
// 1.2s 后收起进度条(还有别的批次在传就不收)
|
||||
setTimeout(() => {
|
||||
if (uploadBatches.size) return;
|
||||
progWrap.classList.add('hidden'); progBar.style.width = '0%';
|
||||
}, 1200);
|
||||
}
|
||||
|
||||
$('file-input').addEventListener('change', e => {
|
||||
const input = e.target;
|
||||
const files = Array.prototype.slice.call(input.files || []); // 先快照再清空,FileList 是实时的
|
||||
input.value = '';
|
||||
if (files.length) uploadFilesNow(files).catch(er => toast('上传失败:' + (er && er.message || er)));
|
||||
});
|
||||
|
||||
// ---------- 拖拽文件到页面即上传(仅桌面浏览器,触屏没有 drag 事件) ----------
|
||||
// 必须挂在 window 上:文件落到输入框/面板/抽屉等任何位置,若没有 drop 处理,
|
||||
// 浏览器默认行为是导航去打开它,当前草稿和正在流式的回合会一起丢失。
|
||||
function pickDroppedFiles(dt) {
|
||||
// dataTransfer 只在事件回调内有效,这里同步取完;目录在 files 里是 0 字节条目,必须跳过
|
||||
const items = dt.items ? Array.prototype.slice.call(dt.items) : [];
|
||||
if (!items.length) return { files: Array.prototype.slice.call(dt.files || []), dirs: 0 };
|
||||
const files = [];
|
||||
let dirs = 0;
|
||||
for (const it of items) {
|
||||
if (it.kind !== 'file') continue;
|
||||
let entry = null;
|
||||
try { entry = it.webkitGetAsEntry && it.webkitGetAsEntry(); } catch (_) {}
|
||||
if (entry && entry.isDirectory) { dirs++; continue; }
|
||||
const f = it.getAsFile();
|
||||
if (f) files.push(f);
|
||||
}
|
||||
return { files, dirs };
|
||||
}
|
||||
(function bindDropUpload() {
|
||||
const overlay = $('drop-overlay');
|
||||
if (!overlay) return;
|
||||
let depth = 0, resetTimer = 0;
|
||||
// 拖动页面内选中的文字也会触发 dragenter,只认带 Files 的拖拽
|
||||
function hasFiles(e) {
|
||||
const types = e.dataTransfer && e.dataTransfer.types;
|
||||
return !!types && Array.prototype.indexOf.call(types, 'Files') !== -1;
|
||||
}
|
||||
function show() {
|
||||
clearTimeout(resetTimer);
|
||||
depth = 1;
|
||||
overlay.classList.remove('hidden');
|
||||
}
|
||||
function reset() {
|
||||
depth = 0;
|
||||
clearTimeout(resetTimer);
|
||||
overlay.classList.add('hidden');
|
||||
}
|
||||
window.addEventListener('dragenter', e => {
|
||||
if (!hasFiles(e)) return;
|
||||
e.preventDefault(); // 旧版 Chrome/Safari 需要它才让区域接受拖放
|
||||
if (!depth) depth = 1;
|
||||
clearTimeout(resetTimer);
|
||||
overlay.classList.remove('hidden');
|
||||
});
|
||||
window.addEventListener('dragover', e => {
|
||||
if (!hasFiles(e)) return;
|
||||
e.preventDefault(); // 缺这一步 drop 永远不会触发
|
||||
e.dataTransfer.dropEffect = 'copy';
|
||||
if (overlay.classList.contains('hidden')) show();
|
||||
});
|
||||
window.addEventListener('dragleave', e => {
|
||||
if (!hasFiles(e)) return;
|
||||
// 拖出窗口时浏览器不保证再配对一次 dragenter,relatedTarget 为 null 直接收干净
|
||||
if (!e.relatedTarget) { reset(); return; }
|
||||
depth = Math.max(0, depth - 1);
|
||||
if (depth) return;
|
||||
resetTimer = setTimeout(reset, 150);
|
||||
});
|
||||
window.addEventListener('dragend', reset);
|
||||
window.addEventListener('drop', e => {
|
||||
if (!hasFiles(e)) return;
|
||||
e.preventDefault(); // 阻止浏览器默认的“打开文件”
|
||||
reset();
|
||||
const picked = pickDroppedFiles(e.dataTransfer);
|
||||
if (picked.files.length) {
|
||||
uploadFilesNow(picked.files).catch(er => toast('上传失败:' + (er && er.message || er)));
|
||||
}
|
||||
if (picked.dirs) toast('暂不支持拖入文件夹' + (picked.files.length ? ',已忽略' : ''));
|
||||
});
|
||||
})();
|
||||
|
||||
// ---------- 粘贴超长文本自动转 .txt 附件 ----------
|
||||
// 延迟到发送时上传:粘贴不建档、不落盘,chip 可随时撤掉;阈值由 upload 配置下发(0=关闭)。
|
||||
let pasteThreshold = Number(DEFAULT_UPLOAD_CFG.paste_to_file_threshold_chars) || 2000;
|
||||
getUploadCfg().then(function (cfg) {
|
||||
const n = Number(cfg.paste_to_file_threshold_chars);
|
||||
if (Number.isFinite(n) && n >= 0) pasteThreshold = Math.floor(n);
|
||||
}).catch(function () { /* 取不到配置就用默认阈值 */ });
|
||||
function pastedTextName() {
|
||||
const d = new Date(), p = function (n) { return String(n).padStart(2, '0'); };
|
||||
return 'pasted-text-' + d.getFullYear() + p(d.getMonth() + 1) + p(d.getDate()) + '-' +
|
||||
p(d.getHours()) + p(d.getMinutes()) + p(d.getSeconds()) + '.txt';
|
||||
}
|
||||
function addAttachmentFiles(files, titleSeed) {
|
||||
for (const f of files) {
|
||||
// 同名同大小视为同一次粘贴的重复表示(剪贴板可能给多份)
|
||||
if (state.attachments.some(a => a.name === f.name && a.file && a.file.size === f.size)) continue;
|
||||
state.attachments.push({ file: f, name: f.name, size: f.size, titleSeed: titleSeed || '' });
|
||||
}
|
||||
renderAttachChips();
|
||||
updateSendBtn();
|
||||
}
|
||||
ta.addEventListener('paste', e => {
|
||||
if (!pasteThreshold || !e.clipboardData) return; // 0 = 关闭捕获,走浏览器默认粘贴
|
||||
// clipboardData 只在事件回调内有效:文本与文件都必须同步取完
|
||||
const text = e.clipboardData.getData('text/plain') || '';
|
||||
const files = [];
|
||||
const items = e.clipboardData.items ? Array.prototype.slice.call(e.clipboardData.items) : [];
|
||||
for (const it of items) {
|
||||
if (it.kind !== 'file') continue;
|
||||
const f = it.getAsFile();
|
||||
if (f) files.push(f);
|
||||
}
|
||||
// 部分浏览器(iOS)只给 files 不给 items
|
||||
if (!items.length && e.clipboardData.files) {
|
||||
Array.prototype.push.apply(files, Array.prototype.slice.call(e.clipboardData.files));
|
||||
}
|
||||
const trimmed = text.trim();
|
||||
if (trimmed.length <= pasteThreshold) {
|
||||
// 文本不长:保持默认可编辑粘贴;纯文件剪贴板则拦掉浏览器插入文件名的行为
|
||||
if (files.length) {
|
||||
if (!trimmed.length) e.preventDefault();
|
||||
addAttachmentFiles(files);
|
||||
}
|
||||
return;
|
||||
}
|
||||
// 超阈值:输入框不再灌大段文本;原文(不 trim,保留首尾空白与行尾)包成 .txt
|
||||
e.preventDefault();
|
||||
addAttachmentFiles([new File([text], pastedTextName(), { type: 'text/plain' })], trimmed.slice(0, 24));
|
||||
// 同时带文件时文本已保住内容,只补非图片文件(Word/PPT 里的附件);图片多为排版噪音,略过
|
||||
const extra = files.filter(f => !/^image\//.test(f.type || ''));
|
||||
if (extra.length) addAttachmentFiles(extra);
|
||||
toast('长文本已转为附件,发送时一并上传');
|
||||
});
|
||||
|
||||
|
||||
|
||||
@@ -35,6 +35,23 @@ function pvKind(name) {
|
||||
return 'none';
|
||||
}
|
||||
|
||||
// 文件面板「当前目录」里的图片作为一组,供查看器左右滑动切换。
|
||||
// 同组直接用下载直链(<img> 不受 Content-Disposition: attachment 影响),不必逐张取 blob。
|
||||
// 依赖 files.js 的 filesNodeAt / filesCurDir / assetDownloadUrl(同为全局,加载顺序在前)。
|
||||
function pvImgGroup(name) {
|
||||
try {
|
||||
if (typeof filesNodeAt !== 'function') return null;
|
||||
const node = filesNodeAt(typeof filesCurDir === 'string' ? filesCurDir : '');
|
||||
if (!node) return null;
|
||||
const paths = node.files.map(f => f.filename).filter(p => PV_IMG.test(p));
|
||||
if (paths.length < 2) return null;
|
||||
return {
|
||||
list: paths.map(p => ({ src: assetDownloadUrl(p), alt: p.split('/').pop() })),
|
||||
at: paths.indexOf(name),
|
||||
};
|
||||
} catch (_) { return null; }
|
||||
}
|
||||
|
||||
// 刷新/新窗口两个钮对**所有**预览类型统一开放:
|
||||
// refresh → pvState.reload(各类型都注入「按同一路径重开」,与地址条回车同一条路)
|
||||
// popup → pvState.url(office=预览网关;文本/HTML/图片/PDF=blob URL;不支持的类型=下载地址)
|
||||
@@ -107,12 +124,21 @@ function pvCloseDoc() {
|
||||
}
|
||||
|
||||
// 移动端全屏承载:把节点塞进 #fs-viewer 并展开
|
||||
function pvOverlayShow(node) {
|
||||
// opts.bar → 文件预览沉浸模式:顶部独立操作条,内容全宽占满剩余空间
|
||||
// opts.name → 操作条左侧的文件名(仅 bar 模式可见)
|
||||
// opts.source → 源文本,交给 eye / 复制钮(未传则两者隐藏)
|
||||
function pvOverlayShow(node, opts) {
|
||||
opts = opts || {};
|
||||
const viewer = document.getElementById('fs-viewer');
|
||||
const body = document.getElementById('fs-body');
|
||||
viewer.classList.toggle('fs-bar', !!opts.bar);
|
||||
const nm = document.getElementById('fs-file-name');
|
||||
nm.textContent = opts.name ? String(opts.name) : '';
|
||||
nm.title = nm.textContent;
|
||||
body.innerHTML = ''; body.style.zoom = '';
|
||||
if (typeof setFsSource === 'function') setFsSource(null);
|
||||
if (typeof setFsSource === 'function') setFsSource(opts.source == null ? null : opts.source);
|
||||
body.appendChild(node);
|
||||
document.getElementById('fs-viewer').classList.remove('hidden');
|
||||
viewer.classList.remove('hidden');
|
||||
}
|
||||
|
||||
// ---------- office:iframe 直挂预览网关(HEAD 预探顺便触发冷槽复活) ----------
|
||||
@@ -128,15 +154,23 @@ async function pvOpenOffice(name, sizeText) {
|
||||
document.getElementById('pv-body').appendChild(load); // 开窗即刻可见,冷启动转圈等首帧
|
||||
} else {
|
||||
load = pvLoadingBox(); load.classList.add('inline');
|
||||
pvOverlayShow(load);
|
||||
pvOverlayShow(load, { bar: true, name: name });
|
||||
if (!pvState.mobilePush) { try { history.pushState({ pvOffice: 1 }, ''); pvState.mobilePush = true; } catch (_) {} }
|
||||
}
|
||||
let head = null;
|
||||
try { head = await fetch(url, { method: 'HEAD' }); } catch (_) {}
|
||||
if (my !== pvSeq) return;
|
||||
// 404 = 预览服务里没有这份文档。文件可能正被 AI 的写盘链路整体替换(rename 瞬间找不到),
|
||||
// 探针给一次短重试再落占位卡——比让用户自己反复点刷新更接近「多刷几次就出来了」的实情。
|
||||
if (head && head.status === 404) {
|
||||
await new Promise((r) => setTimeout(r, 600));
|
||||
if (my !== pvSeq) return;
|
||||
try { head = await fetch(url, { method: 'HEAD' }); } catch (_) {}
|
||||
if (my !== pvSeq) return;
|
||||
}
|
||||
const missing = head && head.status === 404;
|
||||
if (missing) {
|
||||
const node = pvInfoNode(name, sizeText, '预览服务中暂无这份文档 —— 只有 AI 在本任务中生成/编辑过的文档才能在线预览(较旧的文档可能已被自动清理)。');
|
||||
const node = pvInfoNode(name, sizeText, '预览服务中暂无这份文档 —— 文件可能已被删除、改名,或不在本任务目录下。');
|
||||
if (desk) {
|
||||
// pvClearBody 会清掉 reload/url:占位卡仍要保留刷新/新窗口,否则按钮可见但点了没反应
|
||||
pvClearBody(); document.getElementById('pv-body').appendChild(node);
|
||||
@@ -179,7 +213,7 @@ async function openFilePreview(name, sizeText) {
|
||||
pvClearBody(); pvHeader(name, sizeText);
|
||||
document.getElementById('pv-body').appendChild(node);
|
||||
pvToolbar(tb || tbAll);
|
||||
} else pvOverlayShow(node);
|
||||
} else pvOverlayShow(node, { bar: true, name: name });
|
||||
};
|
||||
if (kind === 'none') {
|
||||
const low = String(name).toLowerCase();
|
||||
@@ -225,7 +259,8 @@ async function openFilePreview(name, sizeText) {
|
||||
newBlobUrl = u;
|
||||
const im = document.createElement('img');
|
||||
im.className = 'pv-img'; im.src = u; im.alt = String(name);
|
||||
im.addEventListener('click', () => openImageViewer(u, String(name).split('/').pop()));
|
||||
const g = pvImgGroup(name);
|
||||
im.addEventListener('click', () => openImageViewer(u, String(name).split('/').pop(), g && g.list, g && g.at));
|
||||
node = im; tb = Object.assign({}, tbAll, { url: u });
|
||||
} else { // text/md/代码/json/csv
|
||||
node = await pvTextNode(name, sizeText, r);
|
||||
@@ -312,7 +347,7 @@ async function pvTextNode(name, sizeText, r) {
|
||||
return holder;
|
||||
}
|
||||
|
||||
// 移动端非 office:保持原全屏行为(#fs-viewer)
|
||||
// 移动端非 office:统一走 #fs-viewer 沉浸式(顶部操作条 + 全宽内容)
|
||||
async function pvRenderLegacy(name, r, my) {
|
||||
if (/\.html?$/i.test(name)) {
|
||||
const html = await r.text();
|
||||
@@ -320,8 +355,7 @@ async function pvRenderLegacy(name, r, my) {
|
||||
iframe.className = 'fs-html-frame';
|
||||
iframe.setAttribute('sandbox', 'allow-same-origin allow-scripts');
|
||||
iframe.srcdoc = rewriteHtmlDocSrcs(html, name);
|
||||
pvOverlayShow(iframe);
|
||||
setFsSource(html); // 源码钮可看页面原码
|
||||
pvOverlayShow(iframe, { bar: true, name: name, source: html }); // 全宽沉浸,源码可看/可复制
|
||||
if (my !== pvSeq) return;
|
||||
} else if (/\.pdf$/i.test(name)) {
|
||||
const u = URL.createObjectURL(await r.blob());
|
||||
@@ -331,12 +365,23 @@ async function pvRenderLegacy(name, r, my) {
|
||||
frame.className = 'fs-html-frame';
|
||||
frame.title = name;
|
||||
frame.src = u;
|
||||
pvOverlayShow(frame);
|
||||
pvOverlayShow(frame, { bar: true, name: name });
|
||||
} else if (/\.svg$/i.test(name)) {
|
||||
// SVG 当图片显示(<img> 不执行内嵌脚本),同时保留源码给 eye/复制钮
|
||||
const text = await r.text();
|
||||
const u = URL.createObjectURL(new Blob([text], { type: 'image/svg+xml;charset=utf-8' }));
|
||||
if (state.lastImgBlobUrl) { try { URL.revokeObjectURL(state.lastImgBlobUrl); } catch (_) {} }
|
||||
state.lastImgBlobUrl = u;
|
||||
const im = document.createElement('img');
|
||||
im.className = 'fs-img-frame'; im.src = u; im.alt = String(name);
|
||||
im.addEventListener('click', () => openImageViewer(u, String(name).split('/').pop()));
|
||||
pvOverlayShow(im, { bar: true, name: name, source: text });
|
||||
} else if (PV_IMG.test(name)) {
|
||||
const u = URL.createObjectURL(await r.blob());
|
||||
if (state.lastImgBlobUrl) { try { URL.revokeObjectURL(state.lastImgBlobUrl); } catch (_) {} }
|
||||
state.lastImgBlobUrl = u;
|
||||
openImageViewer(u, name);
|
||||
const g = pvImgGroup(name);
|
||||
openImageViewer(u, name, g && g.list, g && g.at);
|
||||
} else {
|
||||
const text = await r.text();
|
||||
let cls = 'bubble-ai', html;
|
||||
@@ -352,7 +397,7 @@ async function pvRenderLegacy(name, r, my) {
|
||||
wrap.className = 'msg other';
|
||||
wrap.innerHTML = '<div class="' + cls + '">' + html + '</div>';
|
||||
decorateCodePre(wrap);
|
||||
openFullscreen(wrap.firstChild, text); // 文件原文供 eye 钮看源码
|
||||
pvOverlayShow(wrap.firstChild, { bar: true, name: name, source: text }); // 全宽沉浸,eye/复制看原文
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -86,6 +86,7 @@
|
||||
state.lastRenderedMsgId = 0;
|
||||
state.renderedSigs = {};
|
||||
state.streaming = false;
|
||||
if (typeof clearEditState === 'function') clearEditState();
|
||||
setView('welcome'); // 复用 newChat 同样的视图切换路径
|
||||
}
|
||||
toast('已删除会话');
|
||||
@@ -200,6 +201,7 @@
|
||||
state.toolCallStreaming = {}; // 流式参数早卡映射随会话/回合重置
|
||||
state.streaming = false;
|
||||
state.attachments = [];
|
||||
if (typeof clearEditState === 'function') clearEditState();
|
||||
userPaused = false;
|
||||
state.chatHasMore = false; // 分页状态先复位:防详情返回前用旧会话的游标/标志拉到错会话的消息
|
||||
state.chatOldestId = null;
|
||||
@@ -249,6 +251,7 @@
|
||||
state.schedule = null;
|
||||
state.pendingModel = '';
|
||||
state.attachments = [];
|
||||
if (typeof clearEditState === 'function') clearEditState();
|
||||
state.streamEl = null;
|
||||
state.streamText = '';
|
||||
state.toolCards = {};
|
||||
@@ -294,15 +297,29 @@
|
||||
// 运行中允许插话(后端 postUserMessage 返回 delivered:'live',下一轮迭代会带上这条消息)。
|
||||
// 输入框按 trim 判空:纯空白等同于空(与 send() 的裁剪一致,不能因为一个空格就发不了消息)。
|
||||
function stopModeActive() {
|
||||
return state.status === 'active' && !(($('msg-input') || {}).value || '').trim();
|
||||
// 有附件时按钮保持「发送」:粘贴长文本转成的附件不应被当成停止操作
|
||||
return state.status === 'active' && !(($('msg-input') || {}).value || '').trim() && !state.attachments.length;
|
||||
}
|
||||
// 发送键四态:停止 / 键盘(语音模式)/ 语音(空输入且支持录音)/ 发送。
|
||||
// 停止优先:生成中且输入框为空时,用户最可能想停止,不能被语音入口顶掉。
|
||||
// 语音与键盘两态由 voice.js 暴露的 window.AgentVoice 提供(脚本加载顺序在 sessions 之后,
|
||||
// 初始化阶段它还不存在,此处必须容错——那时按普通「发送」渲染即可)。
|
||||
function updateSendBtn() {
|
||||
const sb = $('send-btn');
|
||||
if (!sb) return;
|
||||
const stop = stopModeActive();
|
||||
sb.dataset.state = stop ? 'stop' : 'send';
|
||||
sb.title = stop ? '停止生成' : '发送';
|
||||
sb.setAttribute('aria-label', stop ? '停止生成' : '发送');
|
||||
const v = window.AgentVoice;
|
||||
const empty = !(($('msg-input') || {}).value || '').trim() && !state.attachments.length;
|
||||
let mode;
|
||||
if (stopModeActive()) mode = 'stop';
|
||||
else if (v && v.isVoiceMode()) mode = 'keyboard';
|
||||
else if (empty && v && v.isAvailable()) mode = 'voice';
|
||||
else mode = 'send';
|
||||
sb.dataset.state = mode;
|
||||
const label = mode === 'stop' ? '停止生成'
|
||||
: mode === 'keyboard' ? '切换到键盘输入'
|
||||
: mode === 'voice' ? '语音输入' : '发送';
|
||||
if (sb.title !== label) sb.title = label;
|
||||
if (sb.getAttribute('aria-label') !== label) sb.setAttribute('aria-label', label);
|
||||
}
|
||||
|
||||
function updateHeaderActions() {
|
||||
|
||||
@@ -615,6 +615,7 @@
|
||||
{ path: 'upload.image_max_dim', label: '压缩后长边上限(px)', type: 'int' },
|
||||
{ path: 'upload.image_quality', label: 'JPEG 质量(0~1)', type: 'float' },
|
||||
{ path: 'upload.confirm_size_mb', label: '大文件确认阈值(MB)', type: 'int' },
|
||||
{ path: 'upload.paste_to_file_threshold_chars', label: '粘贴转文件阈值(字符,0=关)', type: 'int' },
|
||||
] },
|
||||
];
|
||||
function kvPlain(label, v) {
|
||||
|
||||
+291
-57
@@ -2,22 +2,25 @@
|
||||
'use strict';
|
||||
// ---------- 发送 ----------
|
||||
async function uploadFiles(sessionId, files) {
|
||||
const fd = new FormData();
|
||||
for (const f of files) fd.append('files', f.file, f.name);
|
||||
// 大文件上传在慢速网络上可以远超普通请求超时:放宽到 120s
|
||||
const r = await req('POST', '/api/sessions/' + sessionId + '/files', fd, true, 120000);
|
||||
return (r.uploaded || []).map(u => ({ name: u.filename, size: u.size }));
|
||||
const out = [];
|
||||
// 后端 multer 单请求最多 10 个文件(limits 见 routes/files.js),超过就分批
|
||||
for (let i = 0; i < files.length; i += 10) {
|
||||
const fd = new FormData();
|
||||
for (const f of files.slice(i, i + 10)) fd.append('files', f.file, f.name);
|
||||
// 大文件上传在慢速网络上可以远超普通请求超时:放宽到 120s
|
||||
const r = await req('POST', '/api/sessions/' + sessionId + '/files', fd, true, 120000);
|
||||
out.push(...(r.uploaded || []));
|
||||
}
|
||||
return out.map(u => ({ name: u.filename, size: u.size }));
|
||||
}
|
||||
|
||||
async function send() {
|
||||
const ta = $('msg-input');
|
||||
const content = ta.value.trim();
|
||||
// 没有文字就不发(避免产生空消息);上传文件告知行在发送时拼在正文前(见下)。
|
||||
// 运行中也是这条规则:有字就发(后端 live 追投),没字时按钮本身是「停止」。
|
||||
if (!content) {
|
||||
if (state.attachments.length) toast('附件需随一条文字消息发送');
|
||||
return;
|
||||
}
|
||||
const editingId = state.editingMsgId || 0;
|
||||
// 没有文字就不发(避免产生空消息);但只有附件时允许发送——粘贴长文本转成的 .txt 需要能单独发出。
|
||||
// 运行中也是这条规则:有字就发(后端 live 追投),没字又没附件时按钮本身是「停止」。
|
||||
if (!content && !state.attachments.length) return;
|
||||
|
||||
const pending = state.attachments.slice();
|
||||
state.attachments = [];
|
||||
@@ -26,6 +29,7 @@
|
||||
autosize();
|
||||
updateSendBtn(); // 清空后若仍在生成中,按钮应回到「停止」
|
||||
|
||||
let domDropped = false;
|
||||
try {
|
||||
const id = await ensureSession();
|
||||
let atts = [];
|
||||
@@ -33,17 +37,30 @@
|
||||
// 输入区上传卡片里成功入库的文件:正文前拼一行告知,让 AI 知道任务目录多了哪些文件
|
||||
const upNames = Array.from($('up-cards').querySelectorAll('.up-card[data-done="1"]'))
|
||||
.map(c => c.dataset.name).filter(Boolean);
|
||||
const text = upNames.length
|
||||
? '用户上传了文件:' + upNames.join('、') + '\n\n' + content
|
||||
// 输入区上传卡片 + 本条消息随附的附件(粘贴转文件等)都要告知;
|
||||
// atts 用的是服务端返回的真实文件名(重名会自动加序号)。
|
||||
const names = upNames.concat(atts.map(a => a.name));
|
||||
const text = names.length
|
||||
? '用户上传了文件:' + names.join('、') + (content ? '\n\n' + content : '')
|
||||
: content;
|
||||
const body = { content: text, deepThink: !!state.deepThink };
|
||||
const sent = await req('POST', '/api/sessions/' + id + '/messages', body);
|
||||
let sent;
|
||||
if (editingId) {
|
||||
// 覆盖式编辑:先撤掉本地该条及其后的渲染,再让服务端删除同范围历史并以新内容重开一轮
|
||||
dropDomFromMessage(editingId);
|
||||
domDropped = true;
|
||||
sent = await req('POST', '/api/sessions/' + id + '/messages/' + editingId + '/edit', body);
|
||||
clearEditState();
|
||||
} else {
|
||||
sent = await req('POST', '/api/sessions/' + id + '/messages', body);
|
||||
}
|
||||
clearUploadCards(); // 消息已发出(告知行已拼入正文),卡片使命完成清掉
|
||||
|
||||
appendUserMsg(text, atts.map(a => a.name), sent && sent.message_id);
|
||||
if (!state.sessionTitle && content) {
|
||||
// 标题取用户原文,不带文件告知行
|
||||
state.sessionTitle = content.length > 24 ? content.slice(0, 24) : content;
|
||||
// 标题取用户原文,不带文件告知行;纯附件发送时退回粘贴文本的前缀
|
||||
const seed = content || (pending[0] && pending[0].titleSeed) || '';
|
||||
if (!state.sessionTitle && seed) {
|
||||
state.sessionTitle = seed.length > 24 ? seed.slice(0, 24) : seed;
|
||||
}
|
||||
renderHeader();
|
||||
// 运行中插话:正文字流进行中已有一个气泡作「活着」的信号,不再挂打字占位
|
||||
@@ -58,8 +75,10 @@
|
||||
if (content) {
|
||||
ta.value = content + (ta.value ? '\n' + ta.value : '');
|
||||
autosize();
|
||||
updateSendBtn(); // 文字回填了:按钮回到「发送」(即使仍在生成中)
|
||||
}
|
||||
updateSendBtn(); // 文字/附件都可能被还原:按当前状态刷新发送键(附件在时不能停在停止态)
|
||||
// 编辑请求失败(如运行中 409):服务端历史未动,把被乐观撤掉的节点重拉回来
|
||||
if (domDropped) restoreChatView();
|
||||
toast('发送失败:' + e.message);
|
||||
}
|
||||
}
|
||||
@@ -429,6 +448,69 @@
|
||||
}
|
||||
});
|
||||
|
||||
// ---------- 编辑已发送的用户消息(覆盖式) ----------
|
||||
// 发送时服务端会删除该条及其后的全部历史(含工具回执/摘要),前端同步丢弃同范围的 DOM。
|
||||
// 已压缩存档的消息(type='archived')不给编辑入口:删掉摘要后它前面的行仍不发模型,上下文会断。
|
||||
function clearEditState() {
|
||||
state.editingMsgId = 0;
|
||||
const bar = $('edit-bar');
|
||||
if (bar) bar.classList.add('hidden');
|
||||
}
|
||||
function setEditState(msgId) {
|
||||
state.editingMsgId = msgId;
|
||||
const bar = $('edit-bar');
|
||||
if (bar) bar.classList.remove('hidden');
|
||||
}
|
||||
// 乐观撤场:删掉该条及之后的所有节点(旧工具卡/打字点/流式气泡都在后面),
|
||||
// 并把增量渲染游标回退到编辑点之前,免得后续软补拉把已丢的行当成还在。
|
||||
function dropDomFromMessage(msgId) {
|
||||
const box = $('chat-view');
|
||||
const btn = box && box.querySelector('[data-msg-id="' + msgId + '"]');
|
||||
const node = btn && btn.closest('.msg');
|
||||
if (!node) return false;
|
||||
let n = node;
|
||||
while (n) { const next = n.nextElementSibling; n.remove(); n = next; }
|
||||
let max = 0;
|
||||
for (const k of Object.keys(state.renderedSigs)) {
|
||||
const id = Number(k);
|
||||
if (id >= msgId) delete state.renderedSigs[k];
|
||||
else if (id > max) max = id;
|
||||
}
|
||||
state.lastRenderedMsgId = max;
|
||||
return true;
|
||||
}
|
||||
// 编辑失败(如运行中 409):服务端没动历史,重拉已加载量恢复被乐观撤掉的节点。
|
||||
async function restoreChatView() {
|
||||
try {
|
||||
const detail = await req('GET', '/api/sessions/' + state.sessionId + '?limit=' + Math.max(CHAT_PAGE_SIZE, state.chatLoadedCount));
|
||||
if (state.view === 'chat') renderDetail(detail);
|
||||
} catch (_) { /* 恢复失败不阻断主流程 */ }
|
||||
}
|
||||
$('chat-view').addEventListener('click', e => {
|
||||
const btn = e.target.closest('.act-edit');
|
||||
if (!btn) return;
|
||||
const msgId = Number(btn.dataset.msgId);
|
||||
if (!msgId || !state.sessionId) return;
|
||||
if (state.status === 'active') { toast('任务执行中,暂不支持编辑'); return; }
|
||||
let text = '';
|
||||
try { text = decodeURIComponent(escape(atob(btn.dataset.b64 || ''))); } catch (_) { return; }
|
||||
const ta = $('msg-input');
|
||||
ta.value = text;
|
||||
autosize();
|
||||
updateSendBtn();
|
||||
setEditState(msgId);
|
||||
ta.focus();
|
||||
try { ta.setSelectionRange(text.length, text.length); } catch (_) {}
|
||||
});
|
||||
$('edit-cancel').addEventListener('click', () => {
|
||||
clearEditState();
|
||||
const ta = $('msg-input');
|
||||
ta.value = '';
|
||||
autosize();
|
||||
updateSendBtn();
|
||||
ta.focus();
|
||||
});
|
||||
|
||||
// ---------- 全屏查看器(双击气泡/工具卡放大;图片单击放大;均同步 index) ----------
|
||||
// 源码查看:当前内容有源文本时(消息/文件预览)右上角 eye 钮可点,源码面板与渲染视图互切。
|
||||
// 源文本两条来路:预览调用方直接传(文件原文);双击消息时从尾部操作行复制钮取
|
||||
@@ -445,6 +527,8 @@
|
||||
btn.classList.remove('on');
|
||||
btn.title = '显示源码';
|
||||
btn.setAttribute('aria-label', '显示源码');
|
||||
// 复制钮与 eye 同显隐:有源码才可复制
|
||||
$('fs-copy-btn').classList.toggle('hidden', fsSrcText == null);
|
||||
}
|
||||
function toggleFsSrc() {
|
||||
if (fsSrcText == null) return;
|
||||
@@ -459,8 +543,15 @@
|
||||
btn.setAttribute('aria-label', lbl);
|
||||
}
|
||||
$('fs-src-btn').addEventListener('click', toggleFsSrc);
|
||||
// 复制源码:HTML/SVG/MD/代码通用,拿到的就是 eye 面板里那份文本
|
||||
$('fs-copy-btn').addEventListener('click', async () => {
|
||||
if (fsSrcText == null) return;
|
||||
const ok = await copyText(fsSrcText);
|
||||
toast(ok ? '已复制' : '复制失败');
|
||||
});
|
||||
|
||||
function openFullscreen(el, src) {
|
||||
$('fs-viewer').classList.remove('fs-bar'); // 消息全屏沿用悬浮按钮,清掉文件预览的栏模式
|
||||
const body = $('fs-body');
|
||||
body.innerHTML = '';
|
||||
body.style.zoom = '';
|
||||
@@ -470,8 +561,8 @@
|
||||
clone.querySelectorAll('[id]').forEach(function (n) { n.removeAttribute('id'); });
|
||||
// 全屏查看直接展开参数/结果,方便观看
|
||||
clone.querySelectorAll('details').forEach(function (d) { d.open = true; });
|
||||
// 克隆里摘掉删除钮:删除的委托绑在 chat-view,查看器里点了无响应,不留死按钮
|
||||
clone.querySelectorAll('.act-del').forEach(function (n) { n.remove(); });
|
||||
// 克隆里摘掉删除/编辑钮:二者的委托绑在 chat-view,查看器里点了无响应,不留死按钮
|
||||
clone.querySelectorAll('.act-del, .act-edit').forEach(function (n) { n.remove(); });
|
||||
// 源文本:调用方传入优先(文件预览),否则从消息自带 b64 现取(双击场景)
|
||||
let s = src;
|
||||
if (s == null) {
|
||||
@@ -484,6 +575,7 @@
|
||||
}
|
||||
function closeFullscreen() {
|
||||
setFsSource(null); // 顺手收起源码面板/隐藏 eye 钮,下轮打开不残留
|
||||
$('fs-viewer').classList.remove('fs-bar');
|
||||
$('fs-viewer').classList.add('hidden');
|
||||
$('fs-viewer').dispatchEvent(new Event('fs:close'));
|
||||
}
|
||||
@@ -498,67 +590,209 @@
|
||||
});
|
||||
|
||||
// 点击消息/工具卡里的图片 → 图片全屏查看器(#fs-viewer 里的克隆图也生效)
|
||||
// 同一气泡/工具卡内的图作为一组,查看器里可左右滑动切换
|
||||
function imgViewerDelegate(e) {
|
||||
const im = e.target.closest('.bubble-ai img, .bubble-user img, .tool-card img');
|
||||
if (im && im.src) openImageViewer(im.src, im.alt || '');
|
||||
if (!im || !im.src) return;
|
||||
const box = im.closest('.msg') || im.closest('.tool-card') || $('fs-body');
|
||||
const list = Array.prototype.map.call(box.querySelectorAll('img'), x => ({ src: x.src, alt: x.alt || '' }))
|
||||
.filter(x => x.src);
|
||||
openImageViewer(im.src, im.alt || '', list);
|
||||
}
|
||||
$('chat-view').addEventListener('click', imgViewerDelegate);
|
||||
$('fs-viewer').addEventListener('click', imgViewerDelegate);
|
||||
|
||||
// 图片全屏查看器:transform 缩放+平移。双指捻合缩放(1~6x)、放大后单指/鼠标拖动、双击切 2.5x。
|
||||
// 第 3 参 list([{src,alt}] 或 [src]) 传入「同组图片」时开启左右滑动切换;第 4 参 at 指定起始下标。
|
||||
// 切图用双层 img 做真正的分页位移(旧图跟手滑出、新图同步滑入),并在开图时预取相邻图,
|
||||
// 避免「松手→瞬移复位→闪白→图才下载出来」这三连卡顿。
|
||||
let openImageViewer, closeImageViewer;
|
||||
(function () {
|
||||
const viewer = document.getElementById('img-viewer');
|
||||
const img = document.getElementById('img-viewer-img');
|
||||
// 第二层由脚本建(不写进 HTML):旧版缓存的 HTML 里没有它也不会把查看器搞挂
|
||||
const img2 = document.createElement('img');
|
||||
img2.className = 'hidden'; img2.alt = '';
|
||||
viewer.appendChild(img2);
|
||||
const count = document.getElementById('img-count');
|
||||
let cur = img, other = img2; // cur=当前层;other=滑动时接替的层
|
||||
let scale = 1, tx = 0, ty = 0, mode = null;
|
||||
let sx = 0, sy = 0, stx = 0, sty = 0, pDist = 0, pScale = 1;
|
||||
// gallery=当前组;gi=组内下标;wx/wy/wdx=横滑起点与实时位移;dragDir/pending=本次拖动方向与待接替下标
|
||||
let gallery = [], gi = 0, wx = 0, wy = 0, wdx = 0, dragDir = 0, pending = -1;
|
||||
const preloads = new Map(); // src → Image:提前下载并解码,切图时不再等网络
|
||||
let settleTimer = 0;
|
||||
const tdist = t => Math.hypot(t[0].clientX - t[1].clientX, t[0].clientY - t[1].clientY);
|
||||
const W = () => window.innerWidth;
|
||||
function apply() {
|
||||
img.style.transform = 'translate(' + tx + 'px,' + ty + 'px) scale(' + scale + ')';
|
||||
img.style.cursor = scale > 1 ? 'grab' : 'zoom-in';
|
||||
cur.style.transition = 'none'; // 缩放/拖动要跟手,不能被上一段滑动动画的过渡拖着走
|
||||
cur.style.transform = 'translate(' + tx + 'px,' + ty + 'px) scale(' + scale + ')';
|
||||
cur.style.cursor = scale > 1 ? 'grab' : 'zoom-in';
|
||||
}
|
||||
openImageViewer = function (src, alt) {
|
||||
scale = 1; tx = ty = 0; mode = null;
|
||||
function drawCount() {
|
||||
if (!count) return; // 旧版 HTML 缓存里没有计数节点时不报错
|
||||
const on = gallery.length > 1;
|
||||
count.textContent = on ? (gi + 1) + ' / ' + gallery.length : '';
|
||||
count.classList.toggle('hidden', !on);
|
||||
}
|
||||
// 预取相邻图:Image 对象常驻,解码结果留在内存里,滑过去就是现成的
|
||||
function preload(src) {
|
||||
if (!src || preloads.has(src) || /^(data|blob):/i.test(src)) return;
|
||||
const im = new Image();
|
||||
im.decoding = 'async';
|
||||
im.src = src;
|
||||
preloads.set(src, im);
|
||||
}
|
||||
function prefetchAround() {
|
||||
[gi + 1, gi + 2, gi - 1].forEach(i => { if (gallery[i]) preload(gallery[i].src); });
|
||||
}
|
||||
// 把某层摆到「相对屏幕中心偏移 offset 像素」处(1x、无过渡)
|
||||
function place(el, offset) {
|
||||
el.style.transition = 'none';
|
||||
el.style.transform = 'translate(' + offset + 'px,0) scale(1)';
|
||||
}
|
||||
function showLayer(el, item) {
|
||||
el.src = item.src; el.alt = item.alt || '';
|
||||
el.classList.remove('hidden');
|
||||
}
|
||||
// 动画收尾:旧层收起、两层归位(滑到一半又开始新手势时也靠它清场)
|
||||
function finishSettle() {
|
||||
clearTimeout(settleTimer); settleTimer = 0;
|
||||
other.classList.add('hidden');
|
||||
place(other, 0); place(cur, 0);
|
||||
apply();
|
||||
img.src = src; img.alt = alt || '';
|
||||
prefetchAround();
|
||||
}
|
||||
// 滑到第 i 张(dir=+1 从右侧进,-1 从左侧进);已到边界就弹回
|
||||
function slideTo(i, dir) {
|
||||
if (settleTimer) finishSettle(); // 上一张还在动画里(连点方向键):先收尾,避免接替层被瞬移
|
||||
const t = Math.max(0, Math.min(gallery.length - 1, i));
|
||||
if (t === gi) return settleBack(dir);
|
||||
const from = cur, to = other;
|
||||
if (pending !== t) { place(to, dir * W()); showLayer(to, gallery[t]); }
|
||||
void to.offsetWidth; // 先把起点落位再加过渡,否则首帧会被合并成瞬移
|
||||
const ease = 'transform .2s cubic-bezier(.22,.61,.36,1)';
|
||||
from.style.transition = ease; to.style.transition = ease;
|
||||
from.style.transform = 'translate(' + (-dir * W()) + 'px,0) scale(1)';
|
||||
to.style.transform = 'translate(0,0) scale(1)';
|
||||
cur = to; other = from;
|
||||
gi = t; scale = 1; tx = ty = 0; dragDir = 0; pending = -1;
|
||||
drawCount();
|
||||
settleTimer = setTimeout(finishSettle, 220);
|
||||
}
|
||||
// 没过阈值/到边界:当前层弹回原位,接替层退回屏幕外
|
||||
function settleBack(dir) {
|
||||
const ease = 'transform .2s ease-out';
|
||||
cur.style.transition = ease;
|
||||
cur.style.transform = 'translate(0,0) scale(1)';
|
||||
if (dir && !other.classList.contains('hidden')) {
|
||||
other.style.transition = ease;
|
||||
other.style.transform = 'translate(' + dir * W() + 'px,0) scale(1)';
|
||||
}
|
||||
scale = 1; tx = ty = 0; dragDir = 0; pending = -1;
|
||||
settleTimer = setTimeout(finishSettle, 220);
|
||||
}
|
||||
openImageViewer = function (src, alt, list, at) {
|
||||
const raw = Array.isArray(list) && list.length ? list : [{ src: src, alt: alt }];
|
||||
gallery = raw.map(x => (typeof x === 'string' ? { src: x, alt: '' } : x)).filter(x => x && x.src);
|
||||
// 起始下标优先用调用方给的(文件面板那张是 blob URL,与组里的直链对不上),否则按 src 匹配
|
||||
let i = typeof at === 'number' && at >= 0 ? at : gallery.findIndex(x => x.src === src);
|
||||
if (i < 0) i = 0;
|
||||
clearTimeout(settleTimer); settleTimer = 0;
|
||||
cur = img; other = img2;
|
||||
gi = i; scale = 1; tx = ty = 0; mode = null; dragDir = 0; pending = -1;
|
||||
other.classList.add('hidden'); place(other, 0);
|
||||
showLayer(cur, gallery[gi] || { src: src, alt: alt });
|
||||
place(cur, 0); apply();
|
||||
drawCount();
|
||||
viewer.classList.remove('hidden');
|
||||
prefetchAround(); // 开图即把前后两张拉下来,滑的时候是现成的
|
||||
};
|
||||
closeImageViewer = function () {
|
||||
viewer.classList.add('hidden');
|
||||
gallery = []; gi = 0; dragDir = 0; pending = -1;
|
||||
preloads.clear(); // 断开引用,解码位图随之释放
|
||||
drawCount();
|
||||
};
|
||||
closeImageViewer = function () { viewer.classList.add('hidden'); };
|
||||
// 点背景(非图片)关闭
|
||||
viewer.addEventListener('click', function (e) { if (e.target === viewer) closeImageViewer(); });
|
||||
img.addEventListener('dblclick', function () {
|
||||
scale = scale > 1 ? 1 : 2.5;
|
||||
if (scale === 1) { tx = ty = 0; }
|
||||
apply();
|
||||
});
|
||||
img.addEventListener('touchstart', function (e) {
|
||||
if (e.touches.length === 2) { mode = 'pinch'; pDist = tdist(e.touches) || 1; pScale = scale; }
|
||||
else if (e.touches.length === 1 && scale > 1) { mode = 'pan'; sx = e.touches[0].clientX; sy = e.touches[0].clientY; stx = tx; sty = ty; }
|
||||
else mode = null;
|
||||
}, { passive: true });
|
||||
img.addEventListener('touchmove', function (e) {
|
||||
if (mode === 'pinch' && e.touches.length >= 2) {
|
||||
e.preventDefault();
|
||||
scale = Math.min(6, Math.max(1, pScale * tdist(e.touches) / pDist));
|
||||
// 两层都要绑:切图后当前层会换成另一张 img,事件不能只挂在第一层上
|
||||
function bindLayer(el) {
|
||||
el.addEventListener('dblclick', function () {
|
||||
scale = scale > 1 ? 1 : 2.5;
|
||||
if (scale === 1) { tx = ty = 0; }
|
||||
apply();
|
||||
} else if (mode === 'pan' && e.touches.length === 1) {
|
||||
});
|
||||
el.addEventListener('touchstart', function (e) {
|
||||
if (settleTimer) finishSettle(); // 上一次滑动还没收尾:先清场再新手势
|
||||
if (e.touches.length === 2) { mode = 'pinch'; pDist = tdist(e.touches) || 1; pScale = scale; }
|
||||
else if (e.touches.length === 1 && scale > 1) { mode = 'pan'; sx = e.touches[0].clientX; sy = e.touches[0].clientY; stx = tx; sty = ty; }
|
||||
else if (e.touches.length === 1 && gallery.length > 1) { mode = 'swipe'; wx = e.touches[0].clientX; wy = e.touches[0].clientY; wdx = 0; dragDir = 0; pending = -1; }
|
||||
else mode = null;
|
||||
}, { passive: true });
|
||||
el.addEventListener('touchmove', function (e) {
|
||||
if (mode === 'pinch' && e.touches.length >= 2) {
|
||||
e.preventDefault();
|
||||
scale = Math.min(6, Math.max(1, pScale * tdist(e.touches) / pDist));
|
||||
if (scale === 1) { tx = ty = 0; }
|
||||
apply();
|
||||
} else if (mode === 'pan' && e.touches.length === 1) {
|
||||
e.preventDefault();
|
||||
tx = stx + e.touches[0].clientX - sx;
|
||||
ty = sty + e.touches[0].clientY - sy;
|
||||
apply();
|
||||
} else if (mode === 'swipe' && e.touches.length === 1) {
|
||||
const dx = e.touches[0].clientX - wx, dy = e.touches[0].clientY - wy;
|
||||
if (Math.abs(dx) <= Math.abs(dy)) return; // 竖直为主:不接管
|
||||
wdx = dx;
|
||||
const dir = dx < 0 ? 1 : -1; // 左滑看下一张
|
||||
if (dir !== dragDir) {
|
||||
dragDir = dir;
|
||||
const next = gallery[gi + dir];
|
||||
// 方向一确定就把相邻图摆到另一侧待命(已预取,几乎是现成的),跟手滑进来
|
||||
if (next) { pending = gi + dir; place(other, dir * W()); showLayer(other, next); }
|
||||
else { pending = -1; other.classList.add('hidden'); } // 到头了:没有下一张,只做橡皮筋
|
||||
}
|
||||
const off = pending >= 0 ? dx : dx * 0.35;
|
||||
cur.style.transition = 'none';
|
||||
cur.style.transform = 'translate(' + off + 'px,0) scale(1)';
|
||||
if (pending >= 0) {
|
||||
other.style.transition = 'none';
|
||||
other.style.transform = 'translate(' + (off + dragDir * W()) + 'px,0) scale(1)';
|
||||
}
|
||||
}
|
||||
}, { passive: false });
|
||||
el.addEventListener('touchend', function (e) {
|
||||
if (mode === 'swipe') {
|
||||
const dir = dragDir;
|
||||
if (pending >= 0 && Math.abs(wdx) > 45) slideTo(pending, dir);
|
||||
else settleBack(dir);
|
||||
mode = null;
|
||||
return;
|
||||
}
|
||||
if (e.touches.length < 2) { if (mode === 'pinch') mode = null; }
|
||||
if (!e.touches.length) mode = null;
|
||||
}, { passive: true });
|
||||
el.addEventListener('touchcancel', function () {
|
||||
if (mode === 'swipe') settleBack(dragDir);
|
||||
mode = null;
|
||||
}, { passive: true });
|
||||
// 桌面:放大后按住拖动平移
|
||||
el.addEventListener('mousedown', function (e) {
|
||||
if (scale <= 1) return;
|
||||
e.preventDefault();
|
||||
tx = stx + e.touches[0].clientX - sx;
|
||||
ty = sty + e.touches[0].clientY - sy;
|
||||
apply();
|
||||
}
|
||||
}, { passive: false });
|
||||
img.addEventListener('touchend', function (e) { if (e.touches.length < 2) { if (mode === 'pinch') mode = null; } if (!e.touches.length) mode = null; }, { passive: true });
|
||||
img.addEventListener('touchcancel', function () { mode = null; }, { passive: true });
|
||||
// 桌面:放大后按住拖动平移
|
||||
img.addEventListener('mousedown', function (e) {
|
||||
if (scale <= 1) return;
|
||||
e.preventDefault();
|
||||
const x0 = e.clientX, y0 = e.clientY, tx0 = tx, ty0 = ty;
|
||||
const mv = function (ev) { tx = tx0 + ev.clientX - x0; ty = ty0 + ev.clientY - y0; apply(); };
|
||||
const up = function () { window.removeEventListener('mousemove', mv); window.removeEventListener('mouseup', up); };
|
||||
window.addEventListener('mousemove', mv); window.addEventListener('mouseup', up);
|
||||
const x0 = e.clientX, y0 = e.clientY, tx0 = tx, ty0 = ty;
|
||||
const mv = function (ev) { tx = tx0 + ev.clientX - x0; ty = ty0 + ev.clientY - y0; apply(); };
|
||||
const up = function () { window.removeEventListener('mousemove', mv); window.removeEventListener('mouseup', up); };
|
||||
window.addEventListener('mousemove', mv); window.addEventListener('mouseup', up);
|
||||
});
|
||||
}
|
||||
bindLayer(img); bindLayer(img2);
|
||||
// 桌面没有滑动:组内多图时用 ←/→ 切换(单图不响应)
|
||||
document.addEventListener('keydown', function (e) {
|
||||
if (viewer.classList.contains('hidden') || gallery.length < 2) return;
|
||||
if (e.key === 'ArrowLeft') slideTo(gi - 1, -1);
|
||||
else if (e.key === 'ArrowRight') slideTo(gi + 1, 1);
|
||||
});
|
||||
})();
|
||||
|
||||
|
||||
@@ -0,0 +1,559 @@
|
||||
// 语音输入(按住说话)
|
||||
//
|
||||
// 交互:
|
||||
// · 输入框为空时,发送键变成麦克风;点一下进入语音模式
|
||||
// · 语音模式下输入框位置变成「按住 说话」:按住后整块输入区被全宽遮罩盖住,
|
||||
// 显示实时电平声波 + 「松手传送,上滑取消」
|
||||
// · 手指上滑进入文案区域 → 取消态(声波与文案变红,文案改「松开取消」),滑回下方恢复
|
||||
// · 松手:取消态 → 丢弃 + 轻提示;否则转写后**直接发送**
|
||||
// · 发送键位置变成键盘按钮,点一下回文本模式并聚焦输入框(移动端弹键盘)
|
||||
//
|
||||
// 录音在客户端:套壳原生桥(http 页面也能用)或 HTTPS 下的 MediaRecorder;
|
||||
// 转写在服务端 mcp_asr(经 core 的 /api/asr 中转)。
|
||||
// 电平:浏览器用 AnalyserNode 算 RMS;套壳上报 onNativeVoiceAmp(原始 getMaxAmplitude)。
|
||||
// 归一化/门限/平滑全部在页面侧做,壳不做任何换算。
|
||||
//
|
||||
// 包在 IIFE 里:前端是共享全局作用域的经典脚本,顶层声明会和其他文件撞名。
|
||||
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
const hold = $('voice-hold');
|
||||
const ta = $('msg-input');
|
||||
const overlay = $('voice-overlay');
|
||||
const hintEl = $('voice-hint');
|
||||
const waveEl = $('voice-wave');
|
||||
if (!hold || !ta) return;
|
||||
|
||||
const native = (window.AgentNative && typeof window.AgentNative.startVoice === 'function')
|
||||
? window.AgentNative : null;
|
||||
const webOk = !!(window.isSecureContext && navigator.mediaDevices
|
||||
&& typeof navigator.mediaDevices.getUserMedia === 'function' && window.MediaRecorder);
|
||||
|
||||
const MAX_MS = 60000; // 单次录音上限
|
||||
const MIN_MS = 500; // 短于这个时长按误触处理,不上传
|
||||
const NO_RESULT_MS = 3000; // 松开后壳迟迟不回音频(比如还在等权限)→ 复位
|
||||
const BAR_COUNT = 45; // 声波柱数(要足够宽,手指按住时两侧仍能看到)
|
||||
|
||||
// 两端矮、中间高的包络:看起来像一段声波而不是一排等高柱子
|
||||
const WEIGHTS = [];
|
||||
for (let i = 0; i < BAR_COUNT; i++) {
|
||||
WEIGHTS.push(0.45 + 0.55 * Math.sin(Math.PI * (i + 0.5) / BAR_COUNT));
|
||||
}
|
||||
const bars = [];
|
||||
if (waveEl) {
|
||||
for (let i = 0; i < BAR_COUNT; i++) {
|
||||
const b = document.createElement('i');
|
||||
waveEl.appendChild(b);
|
||||
bars.push(b);
|
||||
}
|
||||
}
|
||||
|
||||
let available = false; // 有能力 + 服务端接线:发送键才显示麦克风
|
||||
let mode = false; // 是否处于语音模式
|
||||
let phase = 'idle'; // idle | arming(等 getMedia)| recording | busy
|
||||
let pressing = false;
|
||||
let pressY = 0; // 本次按压的起始 Y(判断“确实上滑了”)
|
||||
let cancelArmed = false; // 已上滑到取消区
|
||||
let discard = false; // 本次录音太短/已取消:结果到了也丢掉
|
||||
let recordStartAt = 0;
|
||||
let stopTimer = null;
|
||||
let hardTimer = null;
|
||||
let watchdog = null;
|
||||
let rec = null;
|
||||
let stream = null;
|
||||
let chunks = [];
|
||||
let mime = '';
|
||||
|
||||
// 电平与渲染
|
||||
let rafId = 0;
|
||||
let levelTarget = 0;
|
||||
let levelCur = 0;
|
||||
let lastRealLevelAt = 0;
|
||||
let audioCtx = null;
|
||||
let analyser = null;
|
||||
let audioBuf = null;
|
||||
let srcNode = null;
|
||||
|
||||
function clamp01(v) { return v < 0 ? 0 : v > 1 ? 1 : v; }
|
||||
|
||||
// 电平归一化的唯一实现:浏览器(RMS)和壳(峰值幅度)都在这里落到 0~1。
|
||||
// 低于 -55dB 视为静音,-5dB 及以上视为满幅;要调灵敏度只改这两个常量。
|
||||
const LEVEL_FLOOR_DB = -55;
|
||||
const LEVEL_RANGE_DB = 50;
|
||||
function dbToLevel(db) {
|
||||
return clamp01((db - LEVEL_FLOOR_DB) / LEVEL_RANGE_DB);
|
||||
}
|
||||
|
||||
function setHoldText(s) {
|
||||
if (hold.textContent !== s) hold.textContent = s;
|
||||
}
|
||||
|
||||
// ---------- 遮罩 / 声波 ----------
|
||||
function setOverlayMode(m) { // 'send' | 'cancel' | 'busy'
|
||||
cancelArmed = m === 'cancel';
|
||||
if (overlay) overlay.classList.toggle('cancel', m === 'cancel');
|
||||
if (hintEl) {
|
||||
hintEl.textContent = m === 'cancel' ? '松开取消'
|
||||
: m === 'busy' ? '正在识别…' : '松手传送,上滑取消';
|
||||
}
|
||||
}
|
||||
|
||||
function renderBars(level, animate) {
|
||||
if (!bars.length) return;
|
||||
const t = Date.now() / 1000;
|
||||
// 只有真的收到可听电平(大于门限)才加流动感;安静时保持完全静止,不靠抖动“装活”
|
||||
const flow = animate && level > 0.02;
|
||||
for (let i = 0; i < bars.length; i++) {
|
||||
const ripple = flow ? (0.88 + 0.12 * Math.sin(t * 6 + i * 0.55)) : 1;
|
||||
const h = Math.max(0.08, Math.min(1, (0.15 + 0.85 * level) * WEIGHTS[i] * ripple));
|
||||
bars[i].style.transform = 'scaleY(' + h.toFixed(3) + ')';
|
||||
}
|
||||
}
|
||||
|
||||
function startLoop() {
|
||||
if (rafId || !overlay || overlay.classList.contains('hidden')) return;
|
||||
if (typeof requestAnimationFrame !== 'function') return;
|
||||
rafId = requestAnimationFrame(tickMeter);
|
||||
}
|
||||
|
||||
function tickMeter() {
|
||||
rafId = 0;
|
||||
if (!overlay || overlay.classList.contains('hidden')) return;
|
||||
|
||||
let animate = false;
|
||||
if (analyser && audioBuf) {
|
||||
analyser.getFloatTimeDomainData(audioBuf);
|
||||
let sum = 0;
|
||||
for (let i = 0; i < audioBuf.length; i++) sum += audioBuf[i] * audioBuf[i];
|
||||
const rms = Math.sqrt(sum / audioBuf.length);
|
||||
const db = 20 * Math.log10(rms + 1e-7); // 静音约 -140,正常说话 -35 ~ -12
|
||||
levelTarget = dbToLevel(db);
|
||||
animate = true;
|
||||
} else if (Date.now() - lastRealLevelAt <= 300) {
|
||||
animate = true; // 壳刚上报过电平
|
||||
}
|
||||
|
||||
if (!animate) {
|
||||
// 没有电平来源(壳没实现 onNativeVoiceAmp / AnalyserNode 不可用):
|
||||
// 静默停在基线,不再空转(也绝不能假动)
|
||||
levelCur = 0;
|
||||
renderBars(0, false);
|
||||
return;
|
||||
}
|
||||
|
||||
// 快攻慢放:起音跟得紧,收音回落慢,柱体不抽筋
|
||||
levelCur += (levelTarget - levelCur) * (levelTarget > levelCur ? 0.55 : 0.18);
|
||||
renderBars(levelCur, true);
|
||||
rafId = requestAnimationFrame(tickMeter);
|
||||
}
|
||||
|
||||
function attachAnalyser() {
|
||||
if (!stream) return;
|
||||
try {
|
||||
const AC = window.AudioContext || window.webkitAudioContext;
|
||||
if (!AC) return;
|
||||
audioCtx = new AC();
|
||||
analyser = audioCtx.createAnalyser();
|
||||
analyser.fftSize = 512;
|
||||
audioBuf = new Float32Array(analyser.fftSize);
|
||||
srcNode = audioCtx.createMediaStreamSource(stream);
|
||||
srcNode.connect(analyser); // 不接 destination,不会回放
|
||||
if (audioCtx.state === 'suspended') audioCtx.resume();
|
||||
startLoop(); // 之前可能在静默态停掉了循环
|
||||
} catch (_) {
|
||||
analyser = null;
|
||||
audioBuf = null;
|
||||
}
|
||||
}
|
||||
|
||||
function startMeter() {
|
||||
stopMeter(false);
|
||||
levelTarget = 0;
|
||||
levelCur = 0;
|
||||
lastRealLevelAt = 0;
|
||||
startLoop();
|
||||
}
|
||||
|
||||
function stopMeter(flatten) {
|
||||
if (rafId) { cancelAnimationFrame(rafId); rafId = 0; }
|
||||
if (srcNode) { try { srcNode.disconnect(); } catch (_) { /* 已断 */ } srcNode = null; }
|
||||
analyser = null;
|
||||
audioBuf = null;
|
||||
if (audioCtx) { try { audioCtx.close(); } catch (_) { /* 已关 */ } audioCtx = null; }
|
||||
if (flatten !== false) {
|
||||
for (let i = 0; i < bars.length; i++) bars[i].style.transform = 'scaleY(.12)';
|
||||
}
|
||||
}
|
||||
|
||||
function showOverlay() {
|
||||
if (overlay) overlay.classList.remove('hidden');
|
||||
// 录音期间隐藏输入卡内容(否则透明渐变会把按钮/文字透出来)
|
||||
if (document.body) document.body.classList.add('voice-active');
|
||||
setOverlayMode('send');
|
||||
startMeter();
|
||||
}
|
||||
|
||||
function hideOverlay() {
|
||||
if (overlay) overlay.classList.add('hidden');
|
||||
if (document.body) document.body.classList.remove('voice-active');
|
||||
stopMeter();
|
||||
cancelArmed = false;
|
||||
}
|
||||
|
||||
function resetUI() {
|
||||
phase = 'idle';
|
||||
pressing = false;
|
||||
pressY = 0;
|
||||
discard = false;
|
||||
if (stopTimer) { clearTimeout(stopTimer); stopTimer = null; }
|
||||
if (hardTimer) { clearTimeout(hardTimer); hardTimer = null; }
|
||||
if (watchdog) { clearTimeout(watchdog); watchdog = null; }
|
||||
hold.classList.remove('recording');
|
||||
setHoldText('按住 说话');
|
||||
hideOverlay();
|
||||
updateSendBtn();
|
||||
}
|
||||
|
||||
function releaseStream() {
|
||||
if (!stream) return;
|
||||
try { stream.getTracks().forEach(function (t) { t.stop(); }); } catch (_) { /* 已关 */ }
|
||||
stream = null;
|
||||
}
|
||||
|
||||
function pickMime() {
|
||||
const list = ['audio/webm;codecs=opus', 'audio/webm', 'audio/mp4', 'audio/ogg;codecs=opus'];
|
||||
for (let i = 0; i < list.length; i++) {
|
||||
try { if (MediaRecorder.isTypeSupported(list[i])) return list[i]; } catch (_) { /* 忽略 */ }
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
// ---------- 模式切换 ----------
|
||||
function enter() {
|
||||
if (!available || mode) return;
|
||||
mode = true;
|
||||
ta.classList.add('hidden');
|
||||
hold.classList.remove('hidden');
|
||||
try { ta.blur(); } catch (_) { /* 忽略 */ }
|
||||
resetUI();
|
||||
updateSendBtn();
|
||||
}
|
||||
|
||||
function exit(focus) {
|
||||
if (!mode) return;
|
||||
cancelHold();
|
||||
mode = false;
|
||||
hold.classList.add('hidden');
|
||||
ta.classList.remove('hidden');
|
||||
autosize();
|
||||
updateSendBtn();
|
||||
// 必须在点击手势里同步 focus,移动端才会弹键盘
|
||||
if (focus) { try { ta.focus(); } catch (_) { /* 忽略 */ } }
|
||||
}
|
||||
|
||||
// ---------- 录音 ----------
|
||||
function startHold() {
|
||||
if (phase !== 'idle') return;
|
||||
pressing = true;
|
||||
discard = false;
|
||||
hold.classList.add('recording');
|
||||
setHoldText('松开 结束');
|
||||
showOverlay();
|
||||
|
||||
if (native) {
|
||||
// 原生侧即使要弹权限也先当「已开始」:松开时 stopVoice(),壳若还在等权限会忽略,
|
||||
// 由 watchdog 兜底复位(只有首次授权那一次会遇到)。
|
||||
phase = 'recording';
|
||||
recordStartAt = Date.now();
|
||||
stopTimer = setTimeout(finishHold, MAX_MS);
|
||||
hardTimer = setTimeout(function () {
|
||||
try { if (native.cancelVoice) native.cancelVoice(); } catch (_) { /* 忽略 */ }
|
||||
resetUI();
|
||||
toast('录音超时');
|
||||
}, MAX_MS + NO_RESULT_MS);
|
||||
try {
|
||||
native.startVoice();
|
||||
} catch (e) {
|
||||
resetUI();
|
||||
toast('无法开始录音:' + String((e && e.message) || e));
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
phase = 'arming'; // 等 getUserMedia/MediaRecorder 起来
|
||||
startWeb();
|
||||
}
|
||||
|
||||
async function startWeb() {
|
||||
try {
|
||||
stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
||||
} catch (e) {
|
||||
resetUI();
|
||||
const denied = e && (e.name === 'NotAllowedError' || e.name === 'SecurityError');
|
||||
toast(denied ? '麦克风权限被拒绝' : '无法打开麦克风:' + String((e && e.message) || e));
|
||||
return;
|
||||
}
|
||||
if (!pressing) { releaseStream(); resetUI(); return; } // 松开得比授权快
|
||||
attachAnalyser(); // 声波接同一路 MediaStream
|
||||
|
||||
mime = pickMime();
|
||||
try {
|
||||
rec = mime ? new MediaRecorder(stream, { mimeType: mime }) : new MediaRecorder(stream);
|
||||
} catch (_) {
|
||||
rec = new MediaRecorder(stream);
|
||||
mime = rec.mimeType || '';
|
||||
}
|
||||
chunks = [];
|
||||
rec.ondataavailable = function (ev) {
|
||||
if (ev.data && ev.data.size) chunks.push(ev.data);
|
||||
};
|
||||
rec.onstop = function () {
|
||||
const type = mime || 'audio/webm';
|
||||
const blob = new Blob(chunks, { type: type });
|
||||
chunks = [];
|
||||
rec = null;
|
||||
releaseStream();
|
||||
if (discard) { discard = false; resetUI(); toast('说话时间太短'); return; }
|
||||
upload(blob);
|
||||
};
|
||||
rec.onerror = function () {
|
||||
releaseStream();
|
||||
resetUI();
|
||||
toast('录音出错');
|
||||
};
|
||||
try {
|
||||
rec.start();
|
||||
} catch (e) {
|
||||
releaseStream();
|
||||
resetUI();
|
||||
toast('无法开始录音:' + String((e && e.message) || e));
|
||||
return;
|
||||
}
|
||||
|
||||
phase = 'recording';
|
||||
recordStartAt = Date.now();
|
||||
stopTimer = setTimeout(finishHold, MAX_MS);
|
||||
hardTimer = setTimeout(resetUI, MAX_MS + NO_RESULT_MS);
|
||||
}
|
||||
|
||||
// 松手:结束录音(太短则丢弃)
|
||||
function finishHold() {
|
||||
if (phase !== 'recording') return;
|
||||
if (stopTimer) { clearTimeout(stopTimer); stopTimer = null; }
|
||||
|
||||
const held = Date.now() - recordStartAt;
|
||||
if (held < MIN_MS) discard = true;
|
||||
phase = 'busy';
|
||||
hold.classList.remove('recording');
|
||||
setHoldText(discard ? '按住 说话' : '正在识别…');
|
||||
// 遮罩留着,文案换成「正在识别…」;柱子停在低位,等结果回来再收
|
||||
setOverlayMode(discard ? 'send' : 'busy');
|
||||
stopMeter();
|
||||
|
||||
if (native) {
|
||||
try {
|
||||
native.stopVoice();
|
||||
} catch (e) {
|
||||
resetUI();
|
||||
toast('结束录音失败:' + String((e && e.message) || e));
|
||||
return;
|
||||
}
|
||||
// 壳若因为权限弹窗没有真正开录,stopVoice() 会被忽略、结果不会回来
|
||||
watchdog = setTimeout(function () {
|
||||
if (phase !== 'busy') return;
|
||||
try { if (native.cancelVoice) native.cancelVoice(); } catch (_) { /* 忽略 */ }
|
||||
resetUI();
|
||||
}, NO_RESULT_MS);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
if (rec && rec.state !== 'inactive') rec.stop();
|
||||
else { releaseStream(); resetUI(); }
|
||||
} catch (_) {
|
||||
releaseStream();
|
||||
resetUI();
|
||||
}
|
||||
}
|
||||
|
||||
// 放弃本次录音(切模式 / 离开页面 / 上滑取消 / 壳回调 cancelled)
|
||||
function cancelHold() {
|
||||
if (phase === 'idle') return;
|
||||
const r = rec;
|
||||
rec = null;
|
||||
chunks = [];
|
||||
if (r) { r.onstop = null; r.onerror = null; }
|
||||
releaseStream();
|
||||
resetUI();
|
||||
try {
|
||||
if (r && r.state !== 'inactive') r.stop();
|
||||
else if (native && native.cancelVoice) native.cancelVoice();
|
||||
} catch (_) { /* 已停 */ }
|
||||
}
|
||||
|
||||
// ---------- 上传 + 自动发送 ----------
|
||||
function upload(blob) {
|
||||
if (!blob || !blob.size) { resetUI(); toast('没有录到声音'); return; }
|
||||
phase = 'busy';
|
||||
setOverlayMode('busy');
|
||||
req('POST', '/api/asr/transcribe', blob, true, 120000)
|
||||
.then(function (r) {
|
||||
const text = String((r && r.text) || '').replace(/\s+/g, ' ').trim();
|
||||
// 先收遮罩再做后续:toast 固定在 bottom:96px,遮罩还在时弹出来会压住声波
|
||||
resetUI();
|
||||
if (!text) { toast('没有识别到内容'); return; }
|
||||
ta.value = text;
|
||||
send();
|
||||
})
|
||||
.catch(function (e) {
|
||||
resetUI();
|
||||
toast('语音识别失败:' + String((e && e.message) || e));
|
||||
});
|
||||
}
|
||||
|
||||
// ---------- 按住 / 滑动 / 松开 ----------
|
||||
hold.addEventListener('pointerdown', function (e) {
|
||||
if (e && e.button != null && e.button !== 0) return; // 只响应左键/触摸
|
||||
e.preventDefault();
|
||||
if (phase !== 'idle') return;
|
||||
pressY = e.clientY || 0;
|
||||
try { hold.setPointerCapture(e.pointerId); } catch (_) { /* 忽略 */ }
|
||||
startHold();
|
||||
});
|
||||
|
||||
function onMove(e) {
|
||||
if (!pressing) return;
|
||||
if (phase !== 'recording' && phase !== 'arming') return;
|
||||
if (!overlay || overlay.classList.contains('hidden') || !hintEl) return;
|
||||
// 阈值取文案元素的实际下沿;同时要求相对按下点确实上滑了一段,
|
||||
// 否则首次 pointermove 的坐标噪声或极端布局会把状态直接判成取消(一进来就红)。
|
||||
const line = hintEl.getBoundingClientRect().bottom;
|
||||
const limit = pressY ? Math.min(line - 4, pressY - 24) : (line - 4);
|
||||
const y = e.clientY;
|
||||
if (!cancelArmed && y < limit) setOverlayMode('cancel');
|
||||
else if (cancelArmed && y > line + 4) setOverlayMode('send');
|
||||
}
|
||||
|
||||
function onRelease(e) {
|
||||
// 必须先进 pressing 判断:这个监听器也挂在 window 上,
|
||||
// 对无关的 pointerup 调 preventDefault 会吞掉全局 click 事件。
|
||||
if (!pressing) return;
|
||||
if (e) e.preventDefault();
|
||||
pressing = false;
|
||||
|
||||
if (cancelArmed) {
|
||||
// 不弹提示:toast 会盖住声波(取消本身已经有颜色/文案反馈)
|
||||
cancelHold();
|
||||
return;
|
||||
}
|
||||
if (phase === 'recording') finishHold();
|
||||
else if (phase === 'arming') cancelHold(); // 还没真正开录:当误触
|
||||
}
|
||||
|
||||
hold.addEventListener('pointerup', onRelease);
|
||||
hold.addEventListener('pointercancel', onRelease);
|
||||
hold.addEventListener('pointermove', onMove);
|
||||
// 兜底:个别环境下 setPointerCapture 不生效,手指移出按钮后也要跟手
|
||||
window.addEventListener('pointerup', onRelease);
|
||||
window.addEventListener('pointercancel', onRelease);
|
||||
window.addEventListener('pointermove', onMove);
|
||||
|
||||
// 长按不弹系统菜单(移动端)
|
||||
hold.addEventListener('contextmenu', function (e) { e.preventDefault(); });
|
||||
|
||||
// 键盘可达:空格/回车按下开始,松开结束
|
||||
hold.addEventListener('keydown', function (e) {
|
||||
if (e.repeat) return;
|
||||
if (e.key !== ' ' && e.key !== 'Enter') return;
|
||||
e.preventDefault();
|
||||
if (!pressing) startHold();
|
||||
});
|
||||
hold.addEventListener('keyup', function (e) {
|
||||
if (e.key !== ' ' && e.key !== 'Enter') return;
|
||||
e.preventDefault();
|
||||
onRelease(null);
|
||||
});
|
||||
|
||||
// 录音中离开页面:释放麦克风
|
||||
window.addEventListener('pagehide', cancelHold);
|
||||
|
||||
// ---------- 原生桥回调(契约见 docs/voice-input.md) ----------
|
||||
// 实时幅度:套壳每 ~100ms 上报一次 MediaRecorder.getMaxAmplitude() 的**原始值**(0~32767)。
|
||||
// 换算、门限、平滑全部在页面侧做——规则只存在这一份,壳不要做任何数学。
|
||||
function onNativeAmp(raw) {
|
||||
if (!native) return;
|
||||
const amp = Number(raw);
|
||||
if (!isFinite(amp)) return;
|
||||
lastRealLevelAt = Date.now();
|
||||
if (amp <= 0) {
|
||||
levelTarget = 0;
|
||||
} else {
|
||||
const db = 20 * Math.log10(Math.min(amp, 32767) / 32767);
|
||||
levelTarget = dbToLevel(db);
|
||||
}
|
||||
startLoop(); // 静默态下循环已停,下一帧数据到了再启
|
||||
}
|
||||
window.onNativeVoiceAmp = onNativeAmp;
|
||||
window.onNativeVoiceLevel = onNativeAmp; // 旧名兼容:同样按原始幅度解释
|
||||
|
||||
window.onNativeVoiceResult = function (b64, mimeType) {
|
||||
if (phase !== 'busy' && phase !== 'recording') return;
|
||||
if (watchdog) { clearTimeout(watchdog); watchdog = null; }
|
||||
if (discard) { discard = false; resetUI(); toast('说话时间太短'); return; }
|
||||
try {
|
||||
const bin = atob(String(b64 || ''));
|
||||
const arr = new Uint8Array(bin.length);
|
||||
for (let i = 0; i < bin.length; i++) arr[i] = bin.charCodeAt(i);
|
||||
upload(new Blob([arr], { type: mimeType || 'audio/mp4' }));
|
||||
} catch (_) {
|
||||
resetUI();
|
||||
toast('语音数据解析失败');
|
||||
}
|
||||
};
|
||||
|
||||
window.onNativeVoiceState = function (s) {
|
||||
if (!native || s !== 'recording' || phase !== 'recording') return;
|
||||
// 真正开录:60s 与「太短」判定从这一刻算(首次授权弹窗的时间不算)
|
||||
recordStartAt = Date.now();
|
||||
if (stopTimer) clearTimeout(stopTimer);
|
||||
stopTimer = setTimeout(finishHold, MAX_MS);
|
||||
if (!pressing) finishHold(); // 弹窗期间已经松手:立刻收口(held≈0 → 丢弃)
|
||||
};
|
||||
|
||||
window.onNativeVoiceError = function (code, message) {
|
||||
if (phase === 'idle') return;
|
||||
resetUI();
|
||||
// 生命周期打断(切后台/换页)属预期,不弹提示
|
||||
if (code === 'cancelled' || code === 'canceled') return;
|
||||
toast(code === 'too_short'
|
||||
? '说话时间太短,请按住再说'
|
||||
: '录音失败:' + String(message || code || '未知错误'));
|
||||
};
|
||||
|
||||
// ---------- 对外暴露给发送键 / sessions.js 的状态查询 ----------
|
||||
window.AgentVoice = {
|
||||
isAvailable: function () { return available; },
|
||||
isVoiceMode: function () { return mode; },
|
||||
enter: enter,
|
||||
exit: exit
|
||||
};
|
||||
|
||||
// 能力判定:桥优先;没有桥就得是安全上下文(浏览器不允许 http 页面开麦)
|
||||
if (!native && !webOk) {
|
||||
console.info('[voice] 语音入口未启用:没有 window.AgentNative,且当前不是安全上下文(需 HTTPS,或由套壳 App 注入原生桥)');
|
||||
return;
|
||||
}
|
||||
// 服务端没接线(core 的 ASR_URL 为空)就不给入口;探测失败则照常启用
|
||||
req('GET', '/api/asr/status').then(function (s) {
|
||||
if (s && s.enabled === false) {
|
||||
console.info('[voice] 语音入口未启用:服务端未配置语音识别(agent-core 的 ASR_URL 为空)');
|
||||
return;
|
||||
}
|
||||
available = true;
|
||||
updateSendBtn();
|
||||
}).catch(function () {
|
||||
available = true;
|
||||
updateSendBtn();
|
||||
});
|
||||
})();
|
||||
@@ -0,0 +1,71 @@
|
||||
// 语音识别代理:浏览器 / 套壳 App 录好的音频 → agent-core → mcp_asr → 文字。
|
||||
//
|
||||
// 为什么要 core 中转、而不是前端直连 ASR 容器:mcp_asr 只挂在 agent-mcp-net 内网、
|
||||
// 一个宿主端口都不占(它自身没有鉴权,暴露即被白嫖)。对外唯一的入口仍是 core,
|
||||
// 鉴权沿用全局登录态(auth.middleware 已挂在 /api 之前)。
|
||||
//
|
||||
// 开关:core 的 ASR_URL 为空 = 未部署语音识别,本路由一律 503,前端据此不显示麦克风。
|
||||
const express = require('express');
|
||||
const log = require('../log');
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
const ASR_URL = String(process.env.ASR_URL || '').trim().replace(/\/+$/, '');
|
||||
// 与 mcp_asr 的上限保持一致(60s 的 AAC 约 240KB,这里给长音频留余量)
|
||||
const MAX_BYTES = 20 * 1024 * 1024;
|
||||
const TIMEOUT_MS = 120000;
|
||||
|
||||
// 只接音频二进制。全局 express.json(2mb) 只解析 application/json,两者不冲突。
|
||||
// 注意 WebView 可能发 audio/mp4、audio/webm;codecs=opus,统一按 audio/* 接。
|
||||
const rawAudio = express.raw({
|
||||
type: ['audio/*', 'application/octet-stream', 'application/mp4'],
|
||||
limit: '20mb',
|
||||
});
|
||||
|
||||
// 前端可用性探测(按钮是否显示)。不碰网络,纯看接线。
|
||||
router.get('/status', function (req, res) {
|
||||
res.json({ enabled: !!ASR_URL, max_bytes: MAX_BYTES });
|
||||
});
|
||||
|
||||
router.post('/transcribe', rawAudio, async function (req, res) {
|
||||
if (!ASR_URL) {
|
||||
return res.status(503).json({ error: '语音识别未配置(core 未设置 ASR_URL)' });
|
||||
}
|
||||
const buf = req.body;
|
||||
if (!Buffer.isBuffer(buf) || !buf.length) {
|
||||
return res.status(400).json({ error: '未收到音频数据(Content-Type 需为 audio/*)' });
|
||||
}
|
||||
try {
|
||||
const r = await fetch(ASR_URL + '/transcribe', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': req.headers['content-type'] || 'application/octet-stream' },
|
||||
body: buf,
|
||||
signal: AbortSignal.timeout(TIMEOUT_MS),
|
||||
});
|
||||
const text = await r.text();
|
||||
if (!r.ok) {
|
||||
// ASR 服务回的是 {error:string}:解出来再转,不然前端会看到两层套嵌的 JSON
|
||||
let msg = text.slice(0, 300);
|
||||
try { msg = JSON.parse(text).error || msg; } catch (_) { /* 非 JSON 保留原文 */ }
|
||||
log.warn('asr', '识别失败 HTTP ' + r.status + ':' + msg);
|
||||
// 4xx(静音 / 坏音频 / 过载)原样透传;5xx 统一当上游故障
|
||||
const code = r.status >= 400 && r.status < 500 ? r.status : 502;
|
||||
return res.status(code).json({ error: msg });
|
||||
}
|
||||
// 原样透传 ASR 的 {text, ms, backend}
|
||||
res.type('application/json').send(text);
|
||||
} catch (e) {
|
||||
log.warn('asr', '调用 ASR 服务失败:' + String(e && e.message || e));
|
||||
res.status(502).json({ error: '语音识别服务不可用:' + String(e && e.message || e) });
|
||||
}
|
||||
});
|
||||
|
||||
// express.raw 超限时抛 entity.too.large,默认会回 HTML;这里翻成前端能读的 JSON。
|
||||
router.use(function (err, req, res, next) {
|
||||
if (err && (err.type === 'entity.too.large' || err.status === 413)) {
|
||||
return res.status(413).json({ error: '音频过大(上限 ' + Math.round(MAX_BYTES / 1048576) + 'MB)' });
|
||||
}
|
||||
next(err);
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
@@ -1,7 +1,9 @@
|
||||
// data-page 技能的服务端承载,两条通道(配套 skills/data-page/SKILL.md 的契约):
|
||||
// 1. GET /app/{task_id}/... —— 静态托管任务目录的 .app/ 子树(index.html 兑底,不给列目录);
|
||||
// 目录名刻意带点:任务文件面板默认隐藏(?hidden=1 才列);
|
||||
// 只暴露 .app/ 而非整个任务目录,其余任务文件不从 URL 泄漏。
|
||||
// 只暴露 .app/ 而非整个任务目录,其余任务文件不从 URL 泄漏;
|
||||
// 允许符号链接(技能页面链回技能目录,不必每会话复制),
|
||||
// 但 realpath 必须落在会话目录或技能根目录内。
|
||||
// 2. POST /api/db/{task_id}/query —— 任务 data.db 的 SQL API(页面前端 fetch 调):任意单语句,
|
||||
// 执行核心与 db 工具共用 runTaskSql(executor.js),语义单一事实源。
|
||||
// 认证:挂在 server.js 的 auth.middleware 之后——信任模型与 files 路由一致(登录 ≈ 运维,可跨会话)。
|
||||
@@ -13,6 +15,7 @@ const fs = require('fs');
|
||||
const path = require('path');
|
||||
const engine = require('../engine');
|
||||
const { resolveInTask, runTaskSql } = require('../tools/executor');
|
||||
const { skillsRoot } = require('../tools/skills');
|
||||
|
||||
// 页面忘写 LIMIT 的服务端兜底:iterate 到量即停(不全量物化),超限报错引导分页。
|
||||
const API_MAX_ROWS = 2000;
|
||||
@@ -23,6 +26,19 @@ function notFound(res, msg) {
|
||||
res.status(404).json({ error: msg });
|
||||
}
|
||||
|
||||
// 符号链接白名单:技能页面不必每个会话复制一份,发布时链回技能目录(见各技能的 run.py)。
|
||||
// 链接只是字符串:executor 里看不到的目标也能被链上,而解析发生在 core 的命名空间
|
||||
// (例如 core 把 ./config.yml 挂在 /app/config.yml),所以 realpath 后必须落在
|
||||
// 「本会话目录」或「技能根目录」内,其余一律 404。
|
||||
function realInside(abs, roots) {
|
||||
let real;
|
||||
try { real = fs.realpathSync(abs); } catch (_) { return null; }
|
||||
for (const root of roots) {
|
||||
if (real === root || real.startsWith(root + path.sep)) return real;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------- /app/{task_id}/...:任务 .app/ 子树静态托管(URL 路径与目录名解耦)----------
|
||||
router.use('/app', (req, res) => {
|
||||
try {
|
||||
@@ -44,7 +60,15 @@ router.use('/app', (req, res) => {
|
||||
try { st = fs.statSync(abs); } catch (_) { return notFound(res, '页面不存在'); }
|
||||
}
|
||||
if (!st.isFile()) return notFound(res, '页面不存在');
|
||||
res.sendFile(abs, err => {
|
||||
const candidates = [engine.taskDirOf(sessionId)];
|
||||
try { candidates.push(skillsRoot({ cfg: engine.getConfig() })); } catch (_) {}
|
||||
const roots = [];
|
||||
for (const dir of candidates) {
|
||||
try { roots.push(fs.realpathSync(dir)); } catch (_) {}
|
||||
}
|
||||
const real = realInside(abs, roots);
|
||||
if (!real) return notFound(res, '页面不存在');
|
||||
res.sendFile(real, err => {
|
||||
if (err && !res.headersSent) notFound(res, '页面不存在');
|
||||
});
|
||||
} catch (err) {
|
||||
|
||||
@@ -28,6 +28,17 @@ router.post('/:id/messages', wrap((req, res) => {
|
||||
}));
|
||||
|
||||
|
||||
// 覆盖式编辑:删除该条及其后的历史,再以新内容重开一轮(运行中 409)。
|
||||
router.post('/:id/messages/:mid/edit', wrap((req, res) => {
|
||||
const content = String((req.body && req.body.content) || '').trim();
|
||||
if (!content) return res.status(400).json({ error: 'content 不能为空' });
|
||||
const deepThink = !!(req.body && req.body.deepThink);
|
||||
const r = engine.editUserMessage(req.params.id, Number(req.params.mid), content, { deepThink });
|
||||
log.info('msg', (req.user || '-') + ' ' + req.params.id + ' 编辑消息 ' + req.params.mid +
|
||||
'(丢弃其后 ' + Math.max(0, (r.removed || 1) - 1) + ' 条)');
|
||||
res.json(r);
|
||||
}));
|
||||
|
||||
router.post('/:id/stop', wrap((req, res) => {
|
||||
const r = engine.stop(req.params.id);
|
||||
log.info('msg', (req.user || '-') + ' ' + req.params.id + ' 用户终止');
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
// 通用技能执行面:POST /api/sessions/:id/skills/:name/run
|
||||
//
|
||||
// 与 /api/db 的定位对称:data-page 给了页面「通用数据面」,这里补上「通用执行面」。
|
||||
// 页面类技能(如文档审核页)需要用户点一下就在服务端跑一次技能,而不经过模型:
|
||||
// - 不新开技能专属路由,core 不感知任何技能名、参数语义或产物路径;
|
||||
// - 参数校验、相对路径规整、远端 executor、超时/取消全部复用 tools/skills.runSkill,
|
||||
// 与模型侧 skill(op=run) 是同一事实源,不存在第二套执行逻辑;
|
||||
// - 本文件只做会话存在性、并发闸门、HTTP 错误映射。
|
||||
//
|
||||
// 认证:挂在 server.js 的 auth.middleware 之后,信任模型与 data-page/files 一致
|
||||
// (登录 ≈ 运维;能登录的人本来也能通过对话让模型跑任意技能,这里不新增权限面)。
|
||||
// 刻意不加 CORS:跨站 application/json POST 会被预检挡下。
|
||||
//
|
||||
// 并发:同一 (session, skill) 同时只允许一个请求;页面重复点按钮得到 409 而不是双跑。
|
||||
// 单实例内存 Set 即可(仓库整体就是单进程内存模型,重启即清空)。
|
||||
//
|
||||
// 响应契约:
|
||||
// - 进程成功退出 → HTTP 200 {ok:true, result}
|
||||
// stdout 能解析成 JSON 就返回对象,否则原样返回字符串;
|
||||
// - 技能自身业务失败(脚本以 exit 0 + {"ok":false} 表达)仍算进程成功,
|
||||
// 调用方查 result.ok(与 skill 工具的约定一致);
|
||||
// - 进程失败/超时 → HTTP 200 {ok:false, error}(与 data-page 的「HTTP 200 + error」一致);
|
||||
// - 会话不存在 / 参数形状错误 → 4xx {ok:false, error}。
|
||||
'use strict';
|
||||
const express = require('express');
|
||||
const engine = require('../engine');
|
||||
const skills = require('../tools/skills');
|
||||
const log = require('../log');
|
||||
|
||||
const router = express.Router();
|
||||
const inflight = new Set();
|
||||
|
||||
function sendErr(res, code, msg) {
|
||||
res.status(code).json({ ok: false, error: msg });
|
||||
}
|
||||
|
||||
router.post('/:id/skills/:name/run', async (req, res) => {
|
||||
const sessionId = req.params.id;
|
||||
const name = String(req.params.name || '').trim();
|
||||
if (!engine.getSession(sessionId)) return sendErr(res, 404, '会话不存在');
|
||||
if (!name) return sendErr(res, 400, '技能名不能为空');
|
||||
|
||||
const body = req.body || {};
|
||||
const args = body.args === undefined ? {} : body.args;
|
||||
if (args === null || typeof args !== 'object' || Array.isArray(args)) {
|
||||
return sendErr(res, 400, 'args 必须是 JSON 对象');
|
||||
}
|
||||
|
||||
const key = sessionId + '\n' + name;
|
||||
if (inflight.has(key)) return sendErr(res, 409, '该技能正在执行,请稍候');
|
||||
inflight.add(key);
|
||||
const t0 = Date.now();
|
||||
try {
|
||||
const ctx = {
|
||||
sessionId,
|
||||
taskDir: engine.taskDirOf(sessionId),
|
||||
cfg: engine.getConfig(),
|
||||
};
|
||||
const out = await skills.runSkill(ctx, name, args);
|
||||
let result = out;
|
||||
try { result = JSON.parse(out); } catch (_) { /* 非 JSON 技能:原样返回字符串 */ }
|
||||
log.info('skills', sessionId + ' HTTP 运行 ' + name + ' 完成(' + (Date.now() - t0) + 'ms)');
|
||||
res.json({ ok: true, result });
|
||||
} catch (err) {
|
||||
const msg = String((err && err.message) || err || '技能执行失败');
|
||||
if (err && err.name === 'SkillNotFoundError') return sendErr(res, 404, msg);
|
||||
// 脚本失败/超时不冒充 5xx:调用方统一查 ok/error(与 data-page 契约一致)
|
||||
log.warn('skills', sessionId + ' HTTP 运行 ' + name + ' 失败(' + (Date.now() - t0) + 'ms):' + msg.split('\n')[0]);
|
||||
res.json({ ok: false, error: msg });
|
||||
} finally {
|
||||
inflight.delete(key);
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
@@ -15,7 +15,7 @@ const router = express.Router();
|
||||
const STARTED_AT = Date.now();
|
||||
// 版本以本文件字面量为准:容器部署只挂载 app/server,镜像里的 package.json 可能是旧的,
|
||||
// 读它会一直显示发版时的版本号(之前 2.1.0 改完仍显 2.0.0 即此因)。发版时记得同步这里。
|
||||
const VERSION = '2.3.0';
|
||||
const VERSION = '2.4.0';
|
||||
// git 提交号单独返回(提交时 post-commit 钩子自动生成),前端只显示简洁版本号
|
||||
const VERSION_COMMIT = versionInfo.commit || '';
|
||||
|
||||
|
||||
+22
-1
@@ -67,7 +67,13 @@ app.use(express.static(path.join(__dirname, 'public')));
|
||||
app.use('/api/sessions', require('./routes/sessions'));
|
||||
app.use('/api/sessions', require('./routes/messages'));
|
||||
app.use('/api/sessions', require('./routes/files'));
|
||||
// 通用技能执行面(页面类技能用,不经模型):POST /api/sessions/:id/skills/:name/run。
|
||||
// 与 /api/db 对称;执行核心仍复用 tools/skills.runSkill,core 不感知具体技能。
|
||||
app.use('/api/sessions', require('./routes/skill-run'));
|
||||
app.use('/api/skills', require('./routes/skills'));
|
||||
// 语音识别(可选):录音在客户端(套壳 App 原生桥或浏览器 MediaRecorder),转写在 mcp_asr。
|
||||
// ASR_URL 为空时接口回 503,前端据此不显示麦克风按钮(见 routes/asr.js)。
|
||||
app.use('/api/asr', require('./routes/asr'));
|
||||
// 系统配置含全部用户(脱敏密码)与平台密钥,且可整份改写 YAML:仅管理员可访问。
|
||||
app.use('/api/config', auth.requireAdmin, require('./routes/config'));
|
||||
app.use('/api/platforms', require('./routes/platforms'));
|
||||
@@ -142,7 +148,22 @@ app.get('/api/models', (req, res) => {
|
||||
res.json(rows);
|
||||
});
|
||||
|
||||
app.get('/api/health', (req, res) => res.json({ ok: true, time: Date.now() }));
|
||||
// 探活 + 服务器地址校验(免认证,见 auth.js 的 PUBLIC_EXACT)。
|
||||
// 套壳 App / 运维脚本会手动填服务器地址,必须能判断“这个地址到底是不是本服务”:
|
||||
// 只认 HTTP 200 + body.ok===true + service==='agent-core',不能只看状态码——
|
||||
// 填到别的站点、反代首页或劫持页同样可能回 200。
|
||||
// no-store:避免 WebView / 反向代理 / Service Worker 把旧响应当成“服务在线”。
|
||||
// auth_required 让客户端区分“连上了但没登录”和“地址填错了”。
|
||||
app.get('/api/health', (req, res) => {
|
||||
res.setHeader('Cache-Control', 'no-store');
|
||||
res.json({
|
||||
ok: true,
|
||||
service: 'agent-core',
|
||||
api: 1, // 接口代际:未来出现不兼容改动时升,客户端据此提示升级
|
||||
time: Date.now(),
|
||||
auth_required: !!(cfg.auth && cfg.auth.users && cfg.auth.users.length),
|
||||
});
|
||||
});
|
||||
|
||||
const PORT = process.env.PORT || cfg.port;
|
||||
|
||||
|
||||
@@ -1,109 +1,74 @@
|
||||
You are a general-purpose task assistant running on a private server via a web
|
||||
interface. You help with file generation, data processing, web fetching,
|
||||
You are a general-purpose task assistant on a private server, reached through a
|
||||
web interface. You help with file generation, data processing, web fetching,
|
||||
running commands, writing scripts, and similar tasks.
|
||||
|
||||
You have a dedicated working directory for each conversation. All file paths in
|
||||
tool calls are relative to that directory unless otherwise specified.
|
||||
Each conversation has its own working directory; tool paths are relative to it
|
||||
unless stated otherwise. Uploaded files appear there.
|
||||
|
||||
## Scope & Capability Boundaries
|
||||
## Capabilities
|
||||
|
||||
You are a file / data / information processing assistant. The tools available
|
||||
to you vary by session — always check the live tool list before answering.
|
||||
Your capabilities are exactly what the live tool list and the skills list below
|
||||
expose — they vary by session, so check them before answering. Do not assume a
|
||||
capability that isn't there.
|
||||
|
||||
In scope: understanding images (describe, OCR, read charts); deterministic image
|
||||
operations (format conversion, resize, crop, watermark); generative image
|
||||
creation and editing when an image skill/tool is available; charts and diagrams
|
||||
from data; document processing and conversion (PDF / Markdown / Word / Excel /
|
||||
PowerPoint); data processing; running scripts and commands; fetching and
|
||||
summarizing web information; audio/video transcoding and basic editing;
|
||||
generative media when a tool or skill for it is present in the current session
|
||||
(read that tool's or skill's own description for its capabilities and limits);
|
||||
SQLite operations.
|
||||
Never fake results: no placeholder files, no claiming a file was created, no
|
||||
crude substitutes (ASCII art, hand-drawn SVG) for what the user actually wants.
|
||||
|
||||
Out of scope: capabilities no available tool exposes in this session. Do not
|
||||
fake results — no placeholder files, no claiming a file was created, no crude
|
||||
substitutes (ASCII art, hand-drawn SVG) for what the user actually wants.
|
||||
|
||||
When the user asks for something that seems out of scope:
|
||||
|
||||
- Before concluding you can't do it, check for combinations: chaining tools, or
|
||||
combining tools with the user's own environment (their browser or device).
|
||||
Some requests are impossible from the server alone but become possible when
|
||||
the work is split between the server and the user's side.
|
||||
- If an available skill might handle it, check the skill before declaring it out
|
||||
of scope.
|
||||
- If it truly can't be done, say so in one or two sentences and stop — do not
|
||||
call tools just to reject a request, do not over-apologize. If a third-party
|
||||
tool covers it, mention one briefly. If part of the request is doable, offer
|
||||
that part.
|
||||
|
||||
## Communication
|
||||
|
||||
- Be concise and direct. Use plain language; format with Markdown when useful.
|
||||
- Be truthful. If unsure, say so.
|
||||
- Do not over-apologize.
|
||||
If something seems impossible, check relevant skills first, then whether tools
|
||||
can be combined, or whether the work can be split with the user's own
|
||||
environment (their browser or device). If it truly can't be done, say so in one
|
||||
or two sentences and offer whatever part is doable; do not call tools just to
|
||||
reject a request.
|
||||
|
||||
## Work Principles
|
||||
|
||||
- **Choose the simplest reliable path.** Prefer standard library or built-in
|
||||
tools. If a required package is missing, install it into a session-local venv
|
||||
when possible; otherwise use a fallback or report the limitation.
|
||||
- **Stop after repeated failures.** If the same approach fails more than once or
|
||||
twice, switch to a different approach or report failure. Do not keep retrying
|
||||
the same method.
|
||||
- **Report progress.** If a task will take many steps, state your plan briefly
|
||||
first and give periodic updates. Do not run silently for a long stretch.
|
||||
- **Match effort to the task.** Do not write complex scripts where a few
|
||||
commands suffice, and do not verify more deeply than the task requires. One
|
||||
basic check that the output is usable is enough.
|
||||
- Choose the simplest reliable path; do not write a script where a few commands
|
||||
suffice.
|
||||
- If the same approach fails twice, switch approaches or report the failure —
|
||||
do not keep retrying.
|
||||
- For multi-step work, state the plan briefly and give periodic updates.
|
||||
- Match effort to the task: one basic check that the output is usable is enough.
|
||||
|
||||
## Tool Use
|
||||
## Tools
|
||||
|
||||
- Use the most appropriate tool for each step.
|
||||
- Independent tool calls can be made in parallel.
|
||||
- Follow the parameter schemas exactly; parameters are passed as top-level
|
||||
fields of the tool call arguments.
|
||||
- Skills marked **required** in the skill list below must be used for their
|
||||
scope — do not hand-write scripts to accomplish what a required skill already
|
||||
does. For other skills, prefer them when they match the task.
|
||||
- `mint_public_link` creates a public, unauthenticated URL — never use it for
|
||||
files containing secrets. For image editing, minting the reference image is
|
||||
the intended path; the link is temporary but publicly reachable.
|
||||
- Independent calls can run in parallel. Parameters are top-level fields of the
|
||||
tool call arguments — follow the schema exactly.
|
||||
- `mint_public_link` makes a file publicly reachable without authentication —
|
||||
never use it for files containing secrets.
|
||||
|
||||
## Skill Use
|
||||
|
||||
Before writing a script or installing a package, check the skill list below.
|
||||
If a skill covers the task, use it instead of reimplementing the same job in
|
||||
ad-hoc code — skills already handle the server-specific pitfalls. Read its
|
||||
interface with the `skill` tool (`{"op":"get","name":"<skill>"}`) before the
|
||||
first run, once per skill per conversation. `required` marks the skills whose
|
||||
domain must never be hand-rolled.
|
||||
|
||||
{APP_SKILLS_SECTION}
|
||||
|
||||
## File Management
|
||||
## Communication
|
||||
|
||||
- Work inside the working directory. Use clear file names.
|
||||
- Uploaded files appear in the working directory; check with `terminal` (e.g.
|
||||
`ls`) if needed.
|
||||
- Be concise and direct; Markdown when useful. If unsure, say so. Don't
|
||||
over-apologize.
|
||||
- When done, briefly state what was done and list created/modified files. Offer
|
||||
follow-ups only when useful.
|
||||
|
||||
## Completion
|
||||
## Environment
|
||||
|
||||
- After finishing, briefly state what was done and list created/modified files.
|
||||
- Offer follow-up options only if useful; do not do extra work automatically.
|
||||
|
||||
## System Environment
|
||||
|
||||
- OS: linux. Shell commands run under POSIX sh (`/bin/sh` is dash); use `bash`
|
||||
for scripts (`run.sh`). Keep `terminal` one-liners POSIX-compatible.
|
||||
- Isolated container: a command can access the session working directory, a
|
||||
session-scoped `/tmp`, read-only `/workspace/skills`, and read-only system
|
||||
paths. It cannot see other sessions.
|
||||
- `/tmp` is usable for scratch across `terminal` calls in the same conversation.
|
||||
`/dev/shm` and `/var/tmp` reset on every `terminal` call — do not use them to
|
||||
carry state across calls.
|
||||
- Python 3.11 is available; the system Python and its packages are read-only.
|
||||
To add packages, create a virtualenv inside the working directory
|
||||
(`python3 -m venv --system-site-packages .venv`, then `.venv/bin/pip` and `.venv/bin/python`).
|
||||
Keep it in the working directory, not `/tmp`.
|
||||
The venv inherits system packages; do not reinstall packages that are
|
||||
already importable. Only install what is missing.
|
||||
- System packages cannot be installed at runtime (`apt-get install` fails on the
|
||||
read-only root filesystem). If a required CLI tool is missing, tell the user
|
||||
it must be added on the server.
|
||||
- No C/C++ compiler or `make` — `pip install` works only for packages that ship
|
||||
a prebuilt wheel or are pure Python.
|
||||
- For Chinese text in matplotlib: set `font.sans-serif` to `Noto Sans CJK SC`.
|
||||
- For Chinese ZIP files with GBK filenames, prefer `unar` or `7z x` to avoid
|
||||
garbled names; in Python, use automatic encoding detection.
|
||||
- Linux container, isolated per conversation: you see the working directory, a
|
||||
session-scoped `/tmp`, read-only `/workspace/skills`, and system paths — never
|
||||
other sessions.
|
||||
- `terminal` runs POSIX sh (`/bin/sh` is dash); use `bash` for scripts.
|
||||
- `/tmp` persists across calls in a conversation; `/dev/shm` and `/var/tmp` are
|
||||
wiped on every call.
|
||||
- Python 3.11; system packages are read-only. Add packages in a venv in the
|
||||
working directory: `python3 -m venv --system-site-packages .venv`, then
|
||||
`.venv/bin/pip` / `.venv/bin/python`. Don't reinstall packages that are
|
||||
already importable.
|
||||
- System packages can't be installed at runtime (`apt-get` fails). A missing CLI
|
||||
tool must be added on the server — tell the user.
|
||||
- No C/C++ compiler or `make`: `pip install` only works for pure-Python or
|
||||
prebuilt-wheel packages.
|
||||
- Chinese text in matplotlib: set `font.sans-serif` to `Noto Sans CJK SC`. For
|
||||
ZIPs with GBK filenames, use `unar` or `7z x` to avoid garbled names.
|
||||
|
||||
@@ -24,6 +24,8 @@ services:
|
||||
# 微信接入
|
||||
- WXBOT_URL=http://agent-wxbot:8093
|
||||
- WXBOT_TOKEN=${WXBOT_TOKEN:?WXBOT_TOKEN not set - run scripts/init.sh}
|
||||
# 语音识别(可选):mcp_asr 的内网地址。置空 = 不启用,前端不显示麦克风按钮。
|
||||
- ASR_URL=http://mcp_asr:3000
|
||||
networks:
|
||||
- agent-mcp-net
|
||||
|
||||
@@ -206,6 +208,30 @@ services:
|
||||
networks:
|
||||
- agent-mcp-net
|
||||
|
||||
# 语音识别:录音在客户端(套壳 App 原生桥 / 浏览器 MediaRecorder),转写在这里。
|
||||
# 同进程两个入口(见 mcp/asr/src/server.js 头注释):
|
||||
# POST /transcribe ← agent-core 的 /api/asr 转发(输入框,低延迟、不经过模型)
|
||||
# POST /mcp ← 模型的 MCP 工具 transcribe_audio({url})(转写任务目录里的音频)
|
||||
# 转写走上游 OpenAI 兼容的 /audio/transcriptions(默认智谱 glm-asr,换平台改 ASR_API_* 即可)。
|
||||
mcp_asr:
|
||||
build: ./mcp/asr
|
||||
container_name: mcp_asr
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:3000/health').then(r => process.exit(r.ok ? 0 : 1)).catch(() => process.exit(1))"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 5s
|
||||
environment:
|
||||
- ASR_API_KEY=${ASR_API_KEY:-}
|
||||
- ASR_API_BASE=${ASR_API_BASE:-https://open.bigmodel.cn/api/paas/v4}
|
||||
- ASR_MODEL=${ASR_MODEL:-glm-asr}
|
||||
- ASR_LOG_LEVEL=${ASR_LOG_LEVEL:-info}
|
||||
- TZ=Asia/Shanghai
|
||||
networks:
|
||||
- agent-mcp-net
|
||||
|
||||
networks:
|
||||
# 全部服务共用一个网络;由本仓库创建(固定名字,避免被加上 compose 项目名前缀)。
|
||||
# - core 与 executor 靠它按容器名访问 MCP 服务(如 http://mcp_web_search:3000/)
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# 文件上传(选择器 / 拖拽 / 粘贴)
|
||||
|
||||
前端上传只有一个落点:`app/server/public/js/files.js` 的 `uploadFilesNow(files)`。
|
||||
文件选择器和拖拽 drop 都只负责凑出 `File[]`,之后的大图压缩询问、大文件确认、
|
||||
自动建档、逐卡进度、上传与刷新文件面板全部共用同一条链路。
|
||||
|
||||
## 入口
|
||||
|
||||
| 入口 | 位置 | 说明 |
|
||||
|---|---|---|
|
||||
| 回形针按钮 / 面板上传图标 | `#file-input` 的 `change` | 选中后快照 `input.files` → `uploadFilesNow` |
|
||||
| 拖拽文件到页面任意位置 | `window` 的 `drop` | 从 `dataTransfer` 同步取 `File[]` → `uploadFilesNow` |
|
||||
| 粘贴超长文本 | `#msg-input` 的 `paste` | 包成 `File` 放进 `state.attachments`,发送时随消息上传 |
|
||||
|
||||
发布时 `send()` 会把已成功卡片的 `dataset.name` 拼成「用户上传了文件:…」告知行;
|
||||
卡片名以服务端返回的 `uploaded[].filename` 为准(重名时后端会加 `-1`/`-2`)。
|
||||
|
||||
## 拖拽实现要点
|
||||
|
||||
- **监听在 `window` 上**,不是某个 dropZone。文件落到输入框、文件面板、抽屉上时,
|
||||
没有处理的 `drop` 会让浏览器导航去打开该文件,当前草稿和正在流式的回合都会丢。
|
||||
- **只认带 `Files` 的拖拽**:拖动页面内选中的文字同样触发 `dragenter`,不判断会误弹遮罩。
|
||||
- `dragover` 必须 `preventDefault()`,否则 `drop` 不触发;同时设 `dropEffect = 'copy'`。
|
||||
- 遮罩 `#drop-overlay` 是 `pointer-events: none` 的固定层,不参与命中测试,drop 仍由 window 接管。
|
||||
- `dragenter` / `dragleave` 在子元素间移动会成对冒泡到 window,用 150ms 兜底定时器收尾;
|
||||
`relatedTarget === null`(拖出窗口)直接收干净。
|
||||
- **不支持拖入目录**:目录在 `dataTransfer.files` 里是 0 字节条目,直接传会落一个空文件。
|
||||
实现用 `webkitGetAsEntry().isDirectory` 识别并跳过,有文件时提示「暂不支持拖入文件夹,已忽略」。
|
||||
- 触屏设备没有 drag 事件,遮罩只在真实拖拽时出现,不影响移动端布局。
|
||||
- 并发批次(上一次还在传又拖入一批)共用一个总进度条:各批登记 `{loaded,total}`,
|
||||
绘制时求和,最后一批结束后才收起。
|
||||
|
||||
## 后端约束(前端已对齐)
|
||||
|
||||
- 单文件硬限 50MB(`routes/files.js` 的 `multer.limits.filesize`),超限在预处理阶段直接拒掉。
|
||||
- 大于 `upload.confirm_size_mb`(默认 10MB)先二次确认;大图大于
|
||||
`upload.image_compress_threshold_kb`(默认 1536KB)走本地 canvas 压缩询问。
|
||||
- 每个文件一个 XHR 请求,所以即时上传路径不会撞上 `upload.array('files', 10)` 的上限;
|
||||
发送时随消息上传的附件是「一个请求带多个文件」,`uploadFiles()` 会按 10 个一批切分。
|
||||
- 落盘名经 `sanitizeName` 清洗,重名自动追加序号;前端把响应里的真实文件名回写到卡片,
|
||||
避免告知行指向同名的旧文件。
|
||||
|
||||
## 已知限制
|
||||
|
||||
- 一次拖入多张大图时,`prepareUploadFile` 会逐张弹压缩确认(串行,不做批量合并)。
|
||||
- 拖拽期间会立刻 `ensureSession()` 建档(与回形针一致),即使用户随后放弃上传;
|
||||
全部文件被放弃时不会建档。
|
||||
|
||||
## 长文本粘贴转文件
|
||||
|
||||
在 `#msg-input` 上监听 `paste`(不挂 `document`,避免劫持搜索框等输入):
|
||||
|
||||
- 剪贴板文本 trim 后超过 `upload.paste_to_file_threshold_chars`(默认 2000 字符,`0` 关闭)时
|
||||
`preventDefault()`,用 `new File([原始文本], 'pasted-text-<时间戳>.txt')` 构造附件。
|
||||
阈值判断用 trim 后的长度,写入必须用原始文本(保留首尾空白与行尾)。
|
||||
- 附件进入 `state.attachments`,**不提前建档、不落盘**,chip 可随时撤掉;发送时才随消息上传。
|
||||
- 剪贴板同时有文件时:文本超阈值则文本转 txt(保住内容),另外只补非图片文件;
|
||||
文本很短时保持默认粘贴,剪贴板里的截图/文件另行挂成附件;纯文件则拦掉文件名插入。
|
||||
- 发送约束:`send()` 允许「只有附件」的消息,告知行合并已上传卡片与本条附件(用服务端返回的
|
||||
真实文件名);纯附件发送时标题退回粘贴文本的前 24 字。
|
||||
- 上传/发送失败时附件会被还原回 chip(`send()` 的失败分支),内容不丢。
|
||||
@@ -0,0 +1,393 @@
|
||||
# 语音输入:客户端录音契约
|
||||
|
||||
链路是「按住说话 → 客户端录音 → `POST /api/asr/transcribe` → 转写结果**自动发送**」。
|
||||
服务端只收音频字节,不关心是谁录的。`public/js/voice.js` 按能力自己选录音方式:
|
||||
|
||||
| 页面环境 | 录音方式 | 说明 |
|
||||
|---|---|---|
|
||||
| Android 套壳 App(WebView) | **原生录音 + JS 桥** | WebView 没有 Web Speech API;http 页面也允许 `getUserMedia` 的替代路径 |
|
||||
| 浏览器 / PWA(HTTPS) | `MediaRecorder` | 标准 `getUserMedia`,不用改壳 |
|
||||
| 纯浏览器(`http://<IP>`) | 不可用 | 非安全上下文,浏览器拒绝开麦 |
|
||||
|
||||
**发送键的四种状态**(`sessions.js` 的 `updateSendBtn` 决定):
|
||||
|
||||
| `data-state` | 什么时候 | 点击行为 |
|
||||
|---|---|---|
|
||||
| `stop` | 生成中且输入框为空 | 停止当前回合(优先级最高,不被语音入口顶掉) |
|
||||
| `voice` | 空闲、无文字无附件、且语音可用 | 进入语音模式 |
|
||||
| `keyboard` | 已在语音模式 | 回键盘模式,聚焦输入框(移动端弹键盘) |
|
||||
| `send` | 其余情况 | 发送 |
|
||||
|
||||
「语音可用」= ①有录音能力(原生桥,或安全上下文下的 `MediaRecorder`)②`GET /api/asr/status`
|
||||
返回 `enabled !== false`。所以**套壳 App 里即使页面是 `http://`,只要桥注入成功,麦克风入口照样出现**;
|
||||
纯浏览器 `http://<IP>` 不会出现(非安全上下文,浏览器禁止开麦)。
|
||||
|
||||
语音模式下输入框被 `#voice-hold` 取代(只有居中文字,无按钮外观):按住后整块输入区被
|
||||
**全宽遮罩**盖住(`#voice-overlay`,向左右各扩到屏幕边缘),遮罩里是「实时电平声波 + 提示文案」。
|
||||
手指上滑进入文案区域 → 取消态(声波与文案变红,文案改为「松开取消」),滑回下方恢复;
|
||||
松手时取消态则**直接丢弃**(不弹 toast——它会盖住声波,取消本身已经有变色/文案反馈),否则转写并直接发送。
|
||||
松手后遮罩保持到识别结束,文案换成「正在识别…」。发送键变成键盘按钮。
|
||||
|
||||
---
|
||||
|
||||
## 一、桥的契约
|
||||
|
||||
### 1. 注入时机
|
||||
|
||||
必须在 **`loadUrl()` 之前**注册,不要在 `onPageFinished` 里用 `evaluateJavascript` 注入:
|
||||
|
||||
```java
|
||||
webView.setWebViewClient(/* ... */);
|
||||
webView.addJavascriptInterface(new VoiceBridge(activity, webView), "AgentNative");
|
||||
webView.loadUrl(startUrl);
|
||||
```
|
||||
|
||||
`addJavascriptInterface` 的绑定对所有后续导航生效;在 `onPageFinished` 里再注入,首屏那次
|
||||
加载会拿不到 `window.AgentNative`(页面脚本已经跑完了)。
|
||||
|
||||
### 2. 方法(壳实现)
|
||||
|
||||
```js
|
||||
window.AgentNative = {
|
||||
startVoice(), // 开始录音;失败/拒绝通过 onNativeVoiceError 回报
|
||||
stopVoice(), // 结束录音;结果通过 onNativeVoiceResult 回传
|
||||
cancelVoice() // 放弃本次录音(无回调;页面自己已经把状态复位了)
|
||||
};
|
||||
```
|
||||
|
||||
**线程模型**:这三个方法都跑在 WebView 的 JavaBridge 子线程上,**不是主线程**。
|
||||
`MediaRecorder` 的所有调用必须收敛到主线程:方法入口先 `mainHandler.post(...)`,
|
||||
读临时文件 + Base64 才放回子线程。
|
||||
|
||||
### 3. 回调(页面定义,壳调用)
|
||||
|
||||
```js
|
||||
window.onNativeVoiceResult(base64, mime) // mime 例:'audio/mp4'
|
||||
window.onNativeVoiceError(code, message) // 见下方错误码表
|
||||
window.onNativeVoiceState('recording') // 建议实现:真正开始录时调一次
|
||||
window.onNativeVoiceAmp(amp) // 建议实现:原始幅度 0~32767,驱动遮罩里的声波
|
||||
```
|
||||
|
||||
- `onNativeVoiceAmp(amp)`:壳只上报 `MediaRecorder.getMaxAmplitude()` 的**原始值**
|
||||
(0~32767,主线程每 100ms 一次),**不要做任何换算**。dB 归一化、静音门限、快攻慢放平滑
|
||||
全部在页面侧——浏览器(RMS)和壳(峰值)两条路的规则只存在一份。
|
||||
旧名 `onNativeVoiceLevel` 仍作为别名接受(同样按原始幅度解释)。
|
||||
不实现也能用:声波静默停在基线上(不空转、不假动)。
|
||||
JS 侧还要自己做快攻慢放平滑,壳不需要平滑。
|
||||
- `onNativeVoiceState('recording')`:**按住说话的时长判定依赖它**。页面用这个回调重置
|
||||
「60 秒上限」和「太短丢弃」的起点,避免首次授权弹窗的时间被算进录音。不实现也不会崩:
|
||||
页面按点击时刻计时,代价是首次授权那一次会被当成「太短」丢弃(用户再按一次即可)。
|
||||
- 页面收到 `onNativeVoiceResult` 后会直接发送消息(不经过输入框、不可编辑)。
|
||||
- 所有回调都必须在**主线程**调 `webView.evaluateJavascript`。
|
||||
- 字符串一律用 `JSONObject.quote(...)` 拼,不要手写 `'` 转义——异常 message 里可能有
|
||||
`\`、换行、U+2028,会把整段脚本打断。
|
||||
|
||||
### 4. 状态机与错误码
|
||||
|
||||
壳侧状态:`IDLE → (WAITING_PERMISSION) → RECORDING → ENCODING → IDLE`。
|
||||
`startVoice` 只在 `IDLE` 生效,`stopVoice` 只在 `RECORDING` 生效(其余静默忽略)。
|
||||
|
||||
| code | 触发时机 | 页面表现 |
|
||||
|---|---|---|
|
||||
| `no_permission` | 用户拒绝 / 取消授权弹窗 | toast「需要麦克风权限」 |
|
||||
| `start_failed` | `prepare()` / `start()` 抛异常(含回退 MIC 后仍失败) | toast |
|
||||
| `too_short` | `stop()` 抛异常(录音 < ~1s) | toast「说话时间太短,请按住再说」,**不上传** |
|
||||
| `read_failed` | 读临时文件失败 | toast |
|
||||
| `cancelled` | 生命周期打断(切后台 / 换页) | **静默复位,不弹提示** |
|
||||
| 其他未知 code | — | toast 里显示 `message`,没有就显示 code |
|
||||
|
||||
**取消语义**:页面在 `stopVoice()` 之后会一直等结果(直到点击后 90 秒的硬超时),
|
||||
所以壳在生命周期里取消录音时**必须**回调 `onNativeVoiceError('cancelled','')`,
|
||||
否则按钮会卡在「录音中」。`cancelVoice()` 是页面主动发起的,壳不需要回报。
|
||||
|
||||
### 5. 体积与时序
|
||||
|
||||
- 16kHz / 32kbps / 单声道 AAC:60 秒 ≈ **240KB**,Base64 后 ≈ **320KB**。
|
||||
- `evaluateJavascript` 走 Binder,单次约 1MB 上限;320KB 安全,但**不要再延长单次时长**。
|
||||
真要 2 分钟以上,改成按段回传或本地 HTTP 回传。
|
||||
- 松开后页面最多等 3 秒拿音频(等不到就复位,覆盖「壳还在等权限」的情况);
|
||||
拿到后上传+识别,正常 1~2 秒。
|
||||
- 按住时间 < 500ms 视为误触:页面直接丢弃,不上传;壳自己的 `too_short` 是第二道保险。
|
||||
|
||||
---
|
||||
|
||||
## 二、Java 参考骨架(纯 Java / minSdk 21)
|
||||
|
||||
只演示契约要求的部分;`UrlPolicy`、埋点、通知等按各自工程接。
|
||||
|
||||
```java
|
||||
// AndroidManifest.xml
|
||||
// <uses-permission android:name="android.permission.RECORD_AUDIO" />
|
||||
|
||||
public final class VoiceBridge {
|
||||
private static final int REQ_MIC = 1001;
|
||||
private static final long MAX_MS = 60_000;
|
||||
|
||||
private enum State { IDLE, WAITING_PERMISSION, RECORDING, ENCODING }
|
||||
|
||||
private final Activity activity;
|
||||
private final WebView webView;
|
||||
private final Handler main = new Handler(Looper.getMainLooper());
|
||||
private final Runnable autoStop = this::stopOnMain;
|
||||
|
||||
// 实时电平:每 100ms 读一次 getMaxAmplitude() 交给页面画声波
|
||||
private static final long LEVEL_INTERVAL_MS = 100;
|
||||
private final Runnable levelTick = this::tickLevel;
|
||||
|
||||
private void tickLevel() {
|
||||
MediaRecorder r = recorder;
|
||||
if (state != State.RECORDING || r == null) return;
|
||||
int amp = 0;
|
||||
try { amp = r.getMaxAmplitude(); } catch (Exception ignored) { }
|
||||
// 原始幅度直接给页面:dB 归一化/门限/平滑全在 JS,壳不做任何数学
|
||||
js("window.onNativeVoiceAmp && window.onNativeVoiceAmp(" + amp + ")");
|
||||
main.postDelayed(levelTick, LEVEL_INTERVAL_MS);
|
||||
}
|
||||
|
||||
private State state = State.IDLE;
|
||||
private MediaRecorder recorder;
|
||||
private File file;
|
||||
|
||||
public VoiceBridge(Activity activity, WebView webView) {
|
||||
this.activity = activity;
|
||||
this.webView = webView;
|
||||
}
|
||||
|
||||
// ---- JS 入口:JavaBridge 线程 → 主线程 ----
|
||||
@JavascriptInterface public void startVoice() { main.post(this::startOnMain); }
|
||||
@JavascriptInterface public void stopVoice() { main.post(this::stopOnMain); }
|
||||
@JavascriptInterface public void cancelVoice() { main.post(() -> cancelOnMain(true)); }
|
||||
|
||||
// ---- 生命周期:由 Activity / WebViewClient 调用 ----
|
||||
// 注意:只取消真正在录/在编码的状态。授权弹窗期间某些 ROM 会走 onPause,
|
||||
// 那里如果一并取消,首次授权的两段式流程就断了。
|
||||
public void onPause() {
|
||||
main.post(() -> {
|
||||
if (state == State.RECORDING || state == State.ENCODING) cancelOnMain(true);
|
||||
});
|
||||
}
|
||||
public void onDestroy() { main.post(() -> cancelOnMain(false)); }
|
||||
public void onPageChanged() { main.post(() -> cancelOnMain(true)); }
|
||||
|
||||
public void onRequestPermissionsResult(int requestCode, int[] grantResults) {
|
||||
if (requestCode != REQ_MIC) return;
|
||||
main.post(() -> {
|
||||
if (state != State.WAITING_PERMISSION) return;
|
||||
state = State.IDLE;
|
||||
boolean ok = grantResults != null && grantResults.length > 0
|
||||
&& grantResults[0] == PackageManager.PERMISSION_GRANTED;
|
||||
if (!ok) { fail("no_permission", "需要麦克风权限"); return; }
|
||||
beginRecording(); // 授权后直接开录(体验优先,不要求用户再点一次)
|
||||
});
|
||||
}
|
||||
|
||||
private void startOnMain() {
|
||||
if (state != State.IDLE) return;
|
||||
// 第三方 CDN 脚本同样能调这个桥:只允许自家页面 + 前台时录音
|
||||
if (!UrlPolicy.isAllowed(webView.getUrl()) || !activity.hasWindowFocus()) {
|
||||
fail("forbidden", "当前页面不允许录音");
|
||||
return;
|
||||
}
|
||||
if (ContextCompat.checkSelfPermission(activity, Manifest.permission.RECORD_AUDIO)
|
||||
!= PackageManager.PERMISSION_GRANTED) {
|
||||
state = State.WAITING_PERMISSION;
|
||||
ActivityCompat.requestPermissions(
|
||||
activity, new String[]{Manifest.permission.RECORD_AUDIO}, REQ_MIC);
|
||||
return;
|
||||
}
|
||||
beginRecording();
|
||||
}
|
||||
|
||||
@SuppressLint("NewApi")
|
||||
private void beginRecording() {
|
||||
// VOICE_RECOGNITION 关掉 AGC/降噪,对 ASR 更友好;个别机型不支持则回退 MIC
|
||||
int[] sources = {MediaRecorder.AudioSource.VOICE_RECOGNITION,
|
||||
MediaRecorder.AudioSource.MIC};
|
||||
for (int source : sources) {
|
||||
File f = null;
|
||||
MediaRecorder r = null;
|
||||
try {
|
||||
f = File.createTempFile("voice", ".m4a", activity.getCacheDir());
|
||||
r = Build.VERSION.SDK_INT >= 31
|
||||
? new MediaRecorder(activity) // 无参构造在 API 31 起废弃
|
||||
: new MediaRecorder();
|
||||
r.setAudioSource(source);
|
||||
r.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
|
||||
r.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
|
||||
r.setAudioSamplingRate(16000);
|
||||
r.setAudioEncodingBitRate(32000);
|
||||
r.setAudioChannels(1);
|
||||
r.setOutputFile(f.getAbsolutePath());
|
||||
r.prepare();
|
||||
r.start();
|
||||
|
||||
recorder = r;
|
||||
file = f;
|
||||
state = State.RECORDING;
|
||||
main.postDelayed(autoStop, MAX_MS);
|
||||
main.postDelayed(levelTick, LEVEL_INTERVAL_MS);
|
||||
// 让页面的 60s 从“真的开始录”算(授权弹窗的时间不算)
|
||||
js("window.onNativeVoiceState && window.onNativeVoiceState('recording')");
|
||||
// 不要再额外弹原生 Toast:页面遮罩已经给了全部反馈,弹窗会压住声波
|
||||
return;
|
||||
} catch (Exception e) {
|
||||
// 局部变量释放:失败时 recorder 字段还没赋值,releaseQuietly() 管不到它
|
||||
if (r != null) {
|
||||
try { r.stop(); } catch (Exception ignored) { }
|
||||
try { r.release(); } catch (Exception ignored) { }
|
||||
}
|
||||
delete(f);
|
||||
}
|
||||
}
|
||||
state = State.IDLE;
|
||||
fail("start_failed", "无法开始录音");
|
||||
}
|
||||
|
||||
private void stopOnMain() {
|
||||
if (state != State.RECORDING) return;
|
||||
main.removeCallbacks(autoStop);
|
||||
main.removeCallbacks(levelTick); // 停止上报电平
|
||||
state = State.ENCODING;
|
||||
|
||||
MediaRecorder r = recorder;
|
||||
File f = file;
|
||||
recorder = null;
|
||||
file = null;
|
||||
|
||||
boolean stopped = true;
|
||||
try { r.stop(); } catch (RuntimeException e) { stopped = false; } // 太短会抛
|
||||
try { r.release(); } catch (Exception ignored) { }
|
||||
|
||||
if (!stopped) {
|
||||
delete(f);
|
||||
state = State.IDLE;
|
||||
fail("too_short", "没有录到声音");
|
||||
return;
|
||||
}
|
||||
new Thread(() -> {
|
||||
try {
|
||||
byte[] bytes = readAll(f);
|
||||
String b64 = Base64.encodeToString(bytes, Base64.NO_WRAP);
|
||||
delete(f);
|
||||
main.post(() -> {
|
||||
state = State.IDLE;
|
||||
js("window.onNativeVoiceResult(" + JSONObject.quote(b64) + ",'audio/mp4')");
|
||||
});
|
||||
} catch (Exception e) {
|
||||
delete(f);
|
||||
main.post(() -> { state = State.IDLE; fail("read_failed", "读取录音失败"); });
|
||||
}
|
||||
}, "voice-encode").start();
|
||||
}
|
||||
|
||||
/** notify=true:页面在等结果,必须回 cancelled 让它复位;onDestroy 时页面已走,不必回。 */
|
||||
private void cancelOnMain(boolean notify) {
|
||||
main.removeCallbacks(autoStop);
|
||||
main.removeCallbacks(levelTick);
|
||||
boolean wasActive = state != State.IDLE;
|
||||
releaseQuietly();
|
||||
delete(file);
|
||||
file = null;
|
||||
state = State.IDLE;
|
||||
if (notify && wasActive) js("window.onNativeVoiceError('cancelled','')");
|
||||
}
|
||||
|
||||
private void releaseQuietly() {
|
||||
if (recorder == null) return;
|
||||
try { recorder.stop(); } catch (Exception ignored) { }
|
||||
try { recorder.release(); } catch (Exception ignored) { }
|
||||
recorder = null;
|
||||
}
|
||||
|
||||
private void fail(String code, String message) {
|
||||
js("window.onNativeVoiceError(" + JSONObject.quote(code) + ","
|
||||
+ JSONObject.quote(message) + ")");
|
||||
}
|
||||
|
||||
private void js(String script) {
|
||||
webView.post(() -> webView.evaluateJavascript(script, null));
|
||||
}
|
||||
|
||||
private static byte[] readAll(File f) throws Exception {
|
||||
try (FileInputStream in = new FileInputStream(f);
|
||||
ByteArrayOutputStream out = new ByteArrayOutputStream()) {
|
||||
byte[] buf = new byte[8192];
|
||||
int n;
|
||||
while ((n = in.read(buf)) > 0) out.write(buf, 0, n);
|
||||
return out.toByteArray();
|
||||
}
|
||||
}
|
||||
|
||||
private static void delete(File f) {
|
||||
if (f != null) f.delete();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
接线:
|
||||
|
||||
```java
|
||||
// setupWebView() 里,loadUrl 之前
|
||||
webView.addJavascriptInterface(new VoiceBridge(this, webView), "AgentNative");
|
||||
|
||||
@Override public void onRequestPermissionsResult(int code, String[] p, int[] r) {
|
||||
super.onRequestPermissionsResult(code, p, r);
|
||||
bridge.onRequestPermissionsResult(code, r);
|
||||
}
|
||||
@Override protected void onPause() { bridge.onPause(); super.onPause(); }
|
||||
@Override protected void onDestroy() { bridge.onDestroy(); super.onDestroy(); }
|
||||
// WebViewClient.onPageStarted:URL 变了就 bridge.onPageChanged()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 三、壳侧必须同步的清单
|
||||
|
||||
| 项 | 内容 |
|
||||
|---|---|
|
||||
| `AndroidManifest.xml` | `<uses-permission android:name="android.permission.RECORD_AUDIO" />` |
|
||||
| `proguard-rules.pro` | 保留桥:`-keepclassmembers class * { @android.webkit.JavascriptInterface <methods>; }`,并删掉「本项目未使用 addJavascriptInterface」的注释 |
|
||||
| README 安全表 | 从「不使用 JS 桥」改为「仅暴露语音桥:三个无参方法,无文件/存储读取能力」 |
|
||||
| README 权限行 | 「只有 INTERNET」→「INTERNET + RECORD_AUDIO」 |
|
||||
| 桥的防护 | `UrlPolicy.isAllowed(webView.getUrl())` + `activity.hasWindowFocus()` + 单实例状态机;建议再加一个原生 Toast 提示录音已开始(外部 CDN 脚本与页面同源,能调桥,但用户至少看得见) |
|
||||
| 页面导航 | `onPageStarted` 且 URL 变化时取消,避免 base64 打到新页面的回调上 |
|
||||
|
||||
> 触摸时间戳门槛(最近 N 秒内有触摸才允许录音)能进一步防静默录音,但会让无障碍操作不可用,
|
||||
> 不建议默认开。
|
||||
|
||||
---
|
||||
|
||||
## 四、iOS(以后若有 WKWebView 壳)
|
||||
|
||||
契约相同,换注入方式:
|
||||
|
||||
```swift
|
||||
// 注入:WKUserScript(source: "window.AgentNative = {...}", injectionTime: .atDocumentStart, ...)
|
||||
// 录音:AVAudioRecorder(16kHz / 单声道 / m4a)
|
||||
// 回传:webView.evaluateJavaScript("window.onNativeVoiceResult('\(b64)','audio/mp4')")
|
||||
```
|
||||
|
||||
`Info.plist` 需要 `NSMicrophoneUsageDescription`。
|
||||
|
||||
## 五、服务端接口
|
||||
|
||||
```http
|
||||
POST /api/asr/transcribe
|
||||
Content-Type: audio/mp4 # 或 audio/webm / audio/wav / application/octet-stream
|
||||
Cookie: <登录态> # 与页面同一份 HttpOnly cookie
|
||||
|
||||
<音频二进制>
|
||||
```
|
||||
|
||||
```json
|
||||
{ "text": "打开今天的日报", "ms": 842, "model": "glm-asr" }
|
||||
```
|
||||
|
||||
- 上限 20MB,超时 120s;识别失败返回 `{"error": "..."}`(前端直接 toast 出来)。
|
||||
- `GET /api/asr/status` → `{"enabled": true}`:`ASR_URL` 没接线时为 `false`。
|
||||
|
||||
## 六、为什么不用浏览器的 Web Speech API
|
||||
|
||||
- Android WebView 没有实现 `SpeechRecognition`;iOS WKWebView 同样不可用(只有 Safari 有)。
|
||||
- 桌面 Chrome 的识别走 Google 服务,国内基本报 `network`。
|
||||
- 三端行为不一致、错误码各异,最后还是要留一条服务端通路——不如只留一条。
|
||||
+65
-8
@@ -36,6 +36,11 @@ const IDLE_MS = Number(process.env.OFFICE_IDLE_MS || 6 * 3600 * 1000);
|
||||
const RESPOND_TIMEOUT_MS = 15000; // 数据请求(资源/API/SSE)等待 watch 首响应包的时限;拒连/超时走自愈重试
|
||||
const NAV_TIMEOUT_MS = 5000; // 导航请求等就绪页面首响应:更短——失败立刻转中转页,不白屏干等
|
||||
const COOLDOWN_MS = 8000; // 启动失败拉黑该文件一段时间:中转页自动刷新不成 spawn 风暴(二进制缺失/坏文件早退)
|
||||
// 探针(HEAD)等「首帧可用」的上限:watch 冷启动 = 起进程 + 首次渲染,实测 2~6s(大文档更久)。
|
||||
// 等不到就照回 200,把后续交给导航侧的中转页 + 自愈,探针不无限挂着。
|
||||
const HEAD_WAIT_MS = 15000;
|
||||
// 导航请求连续超时多少次才判死重启:单次超时只是「还没渲染完」(见 route 里的注释)。
|
||||
const NAV_TIMEOUT_RETRIES = 3;
|
||||
const BIN = process.env.OFFICECLI_BIN || '/workspace/skills/office/bin/officecli';
|
||||
const TOKEN = String(process.env.EXECUTOR_TOKEN || '');
|
||||
const EXTS = new Set(['.docx', '.xlsx', '.pptx']);
|
||||
@@ -164,10 +169,13 @@ async function spawnWatch(abs) {
|
||||
const page = pageOf(abs);
|
||||
let slot = -1;
|
||||
for (let i = 0; i < SLOTS; i++) if (!slots.has(i)) { slot = i; break; }
|
||||
if (slot < 0) { // 槽满:优先踢「无活动连接且最久未用」;全开着页面才抢最冷的
|
||||
if (slot < 0) { // 槽满:优先踢「就绪且无活动连接」里最久未用的;正在启动的槽最后才动
|
||||
// (踢掉一个还没就绪的槽 = 白白浪费数秒冷启动,且会让等它的那一路请求失败并进冷却)
|
||||
const ent = [...slots.entries()];
|
||||
const idleReady = ent.filter(([, s]) => !s.conns && s.ready);
|
||||
const idle = ent.filter(([, s]) => !s.conns);
|
||||
slot = (idle.length ? idle : ent).sort((a, b) => a[1].lastUsed - b[1].lastUsed)[0][0];
|
||||
slot = (idleReady.length ? idleReady : (idle.length ? idle : ent))
|
||||
.sort((a, b) => a[1].lastUsed - b[1].lastUsed)[0][0];
|
||||
stop(slot); // 端口按代轮转,新 watch 不会落在被踢者的旧口上,无需等其回收
|
||||
}
|
||||
const wp = await allocPort();
|
||||
@@ -293,6 +301,18 @@ function matchPage(rel) {
|
||||
return best;
|
||||
}
|
||||
|
||||
// 探针预热:把槽位等到「就绪」(= 首帧渲染完成,portUp 探的就是页面本身),复用时直接返回。
|
||||
// 与并发请求共用同一次 startWatch(starting 去重);任何失败/超时都静默——就绪与否不影响 200。
|
||||
async function warmSlot(rr) {
|
||||
let timer;
|
||||
try {
|
||||
await Promise.race([
|
||||
waitReady(rr),
|
||||
new Promise((r) => { timer = setTimeout(r, HEAD_WAIT_MS); }),
|
||||
]);
|
||||
} catch (_) {} finally { clearTimeout(timer); }
|
||||
}
|
||||
|
||||
// 未命中兜底:请求路径本身就是存在的会话文档 → 回收/挤掉后就地复活;不行 = 维持 404。
|
||||
async function revive(rel) {
|
||||
const rr = (rel || '').replace(/\/+$/, '');
|
||||
@@ -354,10 +374,12 @@ function tryForward(entry, req, res, innerPath, respondTimeoutMs) {
|
||||
res.once('close', release);
|
||||
res.once('finish', release);
|
||||
let started = false;
|
||||
let upRes = null;
|
||||
const upstream = http.request(
|
||||
{ host: '127.0.0.1', port: entry.port, path: encodeURI(inner) + search, method: req.method, headers: fwd },
|
||||
(up) => {
|
||||
started = true;
|
||||
upRes = up;
|
||||
clearTimeout(watchdog);
|
||||
const h = Object.assign({}, up.headers);
|
||||
const isSSE = String(h['content-type'] || '').includes('text/event-stream');
|
||||
@@ -385,8 +407,14 @@ function tryForward(entry, req, res, innerPath, respondTimeoutMs) {
|
||||
upstream.on('error', () => {
|
||||
clearTimeout(watchdog);
|
||||
release();
|
||||
if (!started && !res.headersSent) reject(new Error('watch 不可达'));
|
||||
else res.destroy();
|
||||
if (!started && !res.headersSent) { reject(new Error('watch 不可达')); return; }
|
||||
// 已开始回写后上游报错:不立刻销毁客户端响应。watch 会在响应其实已完整时断链
|
||||
//(对 HEAD 必现,见 route 里的注释),Node 客户端的报错早于 'end' 一拍到达,直接
|
||||
// destroy 会把马上要 flush 的响应打断。留一拍再判,真断裂照样收掉,不把半截响应当成功。
|
||||
setTimeout(() => {
|
||||
if (res.writableEnded) return;
|
||||
try { res.destroy(); } catch (_) {}
|
||||
}, upRes && upRes.complete ? 0 : 120);
|
||||
});
|
||||
req.pipe(upstream);
|
||||
});
|
||||
@@ -517,7 +545,8 @@ function fail502(res) {
|
||||
}
|
||||
|
||||
function notFound(res) {
|
||||
res.writeHead(404, { 'content-type': 'text/plain; charset=utf-8' })
|
||||
// no-store:前端拿 HEAD 当探针,404 一旦被浏览器缓存,文件补上后还会继续显示「暂无文档」
|
||||
res.writeHead(404, { 'content-type': 'text/plain; charset=utf-8', 'cache-control': 'no-store' })
|
||||
.end('无匹配预览(文档不存在/不在会话目录下,或不是 .docx/.xlsx/.pptx)');
|
||||
}
|
||||
|
||||
@@ -534,6 +563,24 @@ async function route(req, res) {
|
||||
if (u === PREFIX + '/' || u.startsWith(PREFIX + '/')) {
|
||||
const rel = u.slice(PREFIX.length) || '/';
|
||||
const rr = rel.replace(/\/+$/, '');
|
||||
// HEAD = 前端预览窗的「这份文档能不能预览」探针。文档在不在只在本地按文件系统回答,绝不上探 watch:
|
||||
// 实测 watch(Kestrel) 对 HEAD 只发响应头(带 content-length)不发 body 就断链,Node 客户端抛
|
||||
// HPE_CLOSED_CONNECTION;旧逻辑把它当上游故障 res.destroy(),同一文档十次 HEAD 里六七次变成
|
||||
// 空响应/502。
|
||||
// 但回 200 前要等槽位就绪(有上限):冷启动的首次渲染很贵,旧实现由 HEAD 自己把这段等掉,
|
||||
// iframe 一挂上就是渲染好的页面;若秒回 200,这段开销就落到随后 5s 时限的导航请求上,
|
||||
// 慢文档会被判超时,表现为冷启动转圈很久。等不到也照回 200(见 HEAD_WAIT_MS)。
|
||||
if (req.method === 'HEAD') {
|
||||
let abs = null;
|
||||
try { abs = guard(path.join(WORKSPACE, rr)); } catch (_) {}
|
||||
if (abs) {
|
||||
await warmSlot(rr);
|
||||
res.writeHead(200, { 'content-type': 'text/plain; charset=utf-8', 'cache-control': 'no-store', 'content-length': '0' }).end();
|
||||
return;
|
||||
}
|
||||
// 只有办公文档路径由本网关判定存在性;页面资源等其它 HEAD 仍走原转发逻辑
|
||||
if (EXTS.has(path.extname(rr).toLowerCase())) return notFound(res);
|
||||
}
|
||||
if (isNav(req)) {
|
||||
// 导航请求 = 用户正盯着这一刻:绝不为启动/自愈阻塞首字节。
|
||||
// 槽位不在或未就绪 → 中转页 + 后台热槽(startWatch 去重,在途就等它);就绪槽位转发失败 → 同样中转页。
|
||||
@@ -546,10 +593,20 @@ async function route(req, res) {
|
||||
if (fileChanged(m.entry)) { dropChanged(m.entry); startWatch(abs).catch(() => {}); return loading(res); }
|
||||
if (m.exact && !u.endsWith('/')) { res.writeHead(301, { location: (req.url || u) + '/' }).end(); return; } // 相对资源解析需要尾斜杠
|
||||
const cut = PREFIX.length + m.entry.pageDec.length; // 剥掉「前缀+页面路径」才是 watch 的内层路径
|
||||
try { await tryForward(m.entry, req, res, u.length > cut ? u.slice(cut) : '/', NAV_TIMEOUT_MS); return; }
|
||||
try { await tryForward(m.entry, req, res, u.length > cut ? u.slice(cut) : '/', NAV_TIMEOUT_MS); m.entry.navTimeouts = 0; return; }
|
||||
catch (e) {
|
||||
retire(m.entry, String(e && e.message || e));
|
||||
if (!res.headersSent) { startWatch(abs).catch(() => {}); return loading(res); }
|
||||
const msg = String(e && e.message || e);
|
||||
// 超时 ≠ 死:watch 可能只是首次渲染还没完(大文档 >5s)。一超时就摘掉会把在跑的渲染连坐杀掉,
|
||||
// 中转页刷新后从头再来——用户看到的就是「冷启动一直转圈」。单次超时保留槽位,
|
||||
// 连续 NAV_TIMEOUT_RETRIES 次(约 15s 无首字节)才判死重启,兼顶真卡死。
|
||||
m.entry.navTimeouts = /超时/.test(msg) ? (m.entry.navTimeouts || 0) + 1 : NAV_TIMEOUT_RETRIES;
|
||||
if (m.entry.navTimeouts < NAV_TIMEOUT_RETRIES) {
|
||||
log(`slot 保留:导航超时第 ${m.entry.navTimeouts} 次(${msg})`);
|
||||
} else {
|
||||
retire(m.entry, msg);
|
||||
startWatch(abs).catch(() => {});
|
||||
}
|
||||
if (!res.headersSent) return loading(res);
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
+12
-3
@@ -1,12 +1,13 @@
|
||||
# MCP 服务集合
|
||||
|
||||
三个 MCP(Model Context Protocol)服务,各占一个目录,由仓库根 `docker-compose.yml` 统一编排:
|
||||
四个 MCP(Model Context Protocol)服务,各占一个目录,由仓库根 `docker-compose.yml` 统一编排:
|
||||
|
||||
| 目录 | 容器名 | 作用 | 容器内端口 | MCP 端点 |
|
||||
|---|---|---|---|---|
|
||||
| `web_search/` | `mcp_web_search` | 免费网页搜索(Parallel / Exa,无需 key) | `3000` | `POST /` |
|
||||
| `image_task/` | `mcp_image_task` | 图片生成 / 编辑(智谱原生或任意 OpenAI 兼容 `/images/generations`,单工具 op=create/get) | `3000` | `POST /mcp` |
|
||||
| `video_task/` | `mcp_video_task` | 智谱 CogVideo 视频生成(异步,单工具 op=create/get) | `3000` | `POST /mcp` |
|
||||
| `asr/` | `mcp_asr` | 语音转文字(上游 OpenAI 兼容 `/audio/transcriptions`),**额外带一个给输入框用的 `POST /transcribe`** | `3000` | `POST /mcp` |
|
||||
|
||||
**这些服务一个宿主机端口都不占**——它们只挂在 `agent-mcp-net` 上,由 `agent-core` / `agent-executor`
|
||||
按容器名访问。所以整台服务器对外只需要开 agent-core 那一个端口(`AGENT_PORT`,默认 8090)。
|
||||
@@ -31,16 +32,23 @@ mcp_servers:
|
||||
url: "http://mcp_image_task:3000/mcp"
|
||||
- name: video_task
|
||||
url: "http://mcp_video_task:3000/mcp"
|
||||
# 可选:语音转写(低频,建议 skill 模式,不占每轮工具表)
|
||||
- name: asr
|
||||
url: "http://mcp_asr:3000/mcp"
|
||||
mode: skill
|
||||
```
|
||||
|
||||
服务没起来时,core 只在启动日志里告警,不影响 Web 主流程,但对应工具不会下发给模型。工具发现只在启动时做一次,失败后会自动按 2s / 20s / 1min 重试 3 次;三次都失败则需重启 core。三个服务都配了 `/health` 健康检查,`docker ps` 可直接看状态。
|
||||
服务没起来时,core 只在启动日志里告警,不影响 Web 主流程,但对应工具不会下发给模型。工具发现只在启动时做一次,失败后会自动按 2s / 20s / 1min 重试 3 次;三次都失败则需重启 core。四个服务都配了 `/health` 健康检查,`docker ps` 可直接看状态。
|
||||
|
||||
> 从旧的「agent + mcp 双仓库」部署升级:先在旧 mcp 仓库目录 `docker compose down`(只需一次,否则同名容器与网络会冲突),再 `up` 本栈。
|
||||
|
||||
`image_task` / `video_task` 的 token **不由服务端保存**——每个请求从 `Authorization` header 读取
|
||||
(agent 侧配在 `config.yml` 的 `mcp_servers[].headers`)。`video_task` 用智谱 token;
|
||||
`image_task` 用对应上游(智谱或 OpenAI 兼容网关)的 token,上游地址与模型分别由
|
||||
`X-Image-Task-Base-Url` / `X-Image-Task-Model` 指定。所以这里不用配任何密钥。
|
||||
`X-Image-Task-Base-Url` / `X-Image-Task-Model` 指定。所以这两个服务不用配任何密钥。
|
||||
|
||||
`asr` 是例外:输入框那条路(`agent-core` → `POST /transcribe`)不经过 MCP 调用,拿不到 header,
|
||||
所以它自己的上游密钥由容器 env `ASR_API_KEY` 提供(写在根 `.env`,已 gitignore,不挂给其他容器)。
|
||||
|
||||
### 调试端口
|
||||
|
||||
@@ -78,6 +86,7 @@ AI 把拿到的 URL 当 image_url 交给 video_task:3000/mcp
|
||||
| `mcp_web_search` | 无 | 服务端无鉴权,**永远不要映射出去**(会被白嫖搜索额度);agent 走内网访问 |
|
||||
| `mcp_image_task` | 无 | token 由调用方自带,暴露后可被拿去消耗你的上游图片额度 |
|
||||
| `mcp_video_task` | 无 | 它需要的公开临时 URL 由 agent 侧集成的工具提供,自己不占端口 |
|
||||
| `mcp_asr` | 无 | `/mcp` 与 `/transcribe` 都无鉴权,只能由 core 经内网调用;它持有上游 ASR 密钥 |
|
||||
|
||||
## 单独调试
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
node_modules
|
||||
*.md
|
||||
@@ -0,0 +1,33 @@
|
||||
FROM node:22-slim
|
||||
|
||||
# 国内构建慢:apt 默认源在国外,apt-get update 能卡十几分钟。默认换成腾讯云镜像。
|
||||
# 境外构建或想换源:docker compose build --build-arg DEBIAN_MIRROR=deb.debian.org mcp_asr
|
||||
# 腾讯云 CVM 上想更快可用内网镜像 mirrors.tencentyun.com(仅腾讯云内网可达)。
|
||||
ARG DEBIAN_MIRROR=mirrors.cloud.tencent.com
|
||||
|
||||
# 换 apt 源:bookworm/trixie 可能是 /etc/apt/sources.list,也可能是 deb822 的
|
||||
# /etc/apt/sources.list.d/*.sources,直接用 grep 扫一遍、只替换域名,两种格式都盖住
|
||||
#(路径 /debian、/debian-security 保持不变)。与 executor/Dockerfile 同一套做法。
|
||||
RUN set -eux; \
|
||||
files="$(grep -rl 'deb.debian.org' /etc/apt/ 2>/dev/null || true)"; \
|
||||
if [ -n "$files" ]; then sed -i "s|deb.debian.org|${DEBIAN_MIRROR}|g" $files; fi; \
|
||||
grep -rn "${DEBIAN_MIRROR}" /etc/apt/ | head -5 || true
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
# ffmpeg 只装在这里(core 镜像保持干净):WebView 录出来是 webm/opus,iOS 是 mp4/aac,
|
||||
# 云端只认 wav/mp3——统一在容器内转成 16k 单声道 wav,前端与引擎两头都不用关心格式。
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ffmpeg ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# 零依赖:不需要 npm install(也就没有 registry 慢/被墙的问题)
|
||||
COPY package.json ./
|
||||
COPY src ./src
|
||||
|
||||
ENV PORT=3000
|
||||
EXPOSE 3000
|
||||
|
||||
CMD ["node", "src/server.js"]
|
||||
@@ -0,0 +1,87 @@
|
||||
# mcp/asr — 语音识别服务
|
||||
|
||||
一个容器,两个入口,共用同一套转写实现:
|
||||
|
||||
| 入口 | 谁在用 | 说明 |
|
||||
|---|---|---|
|
||||
| `POST /transcribe` | `agent-core` 的 `/api/asr/transcribe`(H5 输入框) | 普通 HTTP,音频二进制进、`{text}` 出。**不经过模型、不消耗 token、1~3s** |
|
||||
| `POST /mcp` | 模型(MCP 工具 `transcribe_audio`) | 转写任务目录里已有的音频,参数是公网 URL(与 `video_task` 的 `image_url` 同构) |
|
||||
| `GET /health` | compose healthcheck | `{ok, model, configured}` |
|
||||
|
||||
端口 `3000` **只在 `agent-mcp-net` 内网**,不映射宿主机——本服务自身没有鉴权,暴露即被白嫖。
|
||||
对外唯一入口仍是 `agent-core`(`/api/asr` 走登录态鉴权)。
|
||||
|
||||
## 为什么不是「录音 → 上传附件 → 模型调工具」
|
||||
|
||||
那要额外一次完整模型往返(3~10s + token),识别结果落进聊天记录而不是输入框,用户还没法改错字。
|
||||
输入法级别的功能必须绕开模型;`/mcp` 那个口只留给「转写一段已上传的录音」这种低频需求。
|
||||
|
||||
## 部署
|
||||
|
||||
```bash
|
||||
# 1) 配上游密钥(写进仓库根 .env,已 gitignore)
|
||||
# ASR_API_KEY=你的智谱 token
|
||||
# 2) 起服务
|
||||
docker compose up -d --build mcp_asr
|
||||
docker compose restart agent-core # core 要在启动时读到 ASR_URL
|
||||
curl -s http://127.0.0.1:8090/api/asr/status # {"enabled":true,...}
|
||||
```
|
||||
|
||||
`agent-core` 的环境变量 `ASR_URL`(compose 里固定为 `http://mcp_asr:3000`)是开关:置空即
|
||||
不启用,`/api/asr/*` 回 503,前端不显示麦克风按钮。
|
||||
|
||||
## 上游
|
||||
|
||||
任何 OpenAI 兼容的 `/audio/transcriptions` 都能用;默认智谱:
|
||||
|
||||
| 变量 | 默认 |
|
||||
|---|---|
|
||||
| `ASR_API_BASE` | `https://open.bigmodel.cn/api/paas/v4` |
|
||||
| `ASR_API_KEY` | 空(空则 `/transcribe` 回 503) |
|
||||
| `ASR_MODEL` | `glm-asr` |
|
||||
|
||||
常驻内存约 35~40MB,CPU 基本为 0。没有本地模型可选:2 vCPU / 2GB 的机器跑不动能用的
|
||||
whisper(固定 30 秒窗口,`small` 单次要 15~30 秒),需要私有化就把这个容器放到另一台机器,
|
||||
只改 core 的 `ASR_URL` 指向它。
|
||||
|
||||
## HTTP 接口
|
||||
|
||||
```bash
|
||||
# 任意格式(webm / mp4 / mp3 / wav / ogg / aac / flac),服务端用 ffmpeg 归一化
|
||||
curl -s -X POST http://mcp_asr:3000/transcribe \
|
||||
-H 'Content-Type: audio/webm' \
|
||||
--data-binary @voice.webm
|
||||
# → {"text":"打开今天的日报","ms":842,"model":"glm-asr"}
|
||||
```
|
||||
|
||||
- 上限 20MB;超过 `ASR_MAX_CHUNK_SEC`(默认 25s)的音频自动切片后逐段识别再拼接
|
||||
(多数云端 ASR 单次只吃 ~30s)。
|
||||
- 同时最多 `ASR_MAX_CONCURRENT`(默认 2)个转写,超了回 429。
|
||||
|
||||
## MCP 工具
|
||||
|
||||
```yaml
|
||||
# config.yml
|
||||
mcp_servers:
|
||||
- name: asr
|
||||
url: "http://mcp_asr:3000/mcp"
|
||||
mode: skill # 低频能力,别每轮都发 schema
|
||||
```
|
||||
|
||||
`transcribe_audio({ url })`:本地文件先用内置工具 `mint_public_link` 换成公网短链,
|
||||
再把链接传进来(和 `video_task` 要 `image_url` 是同一套路)。
|
||||
|
||||
## 环境变量速查
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
|---|---|---|
|
||||
| `ASR_API_BASE` / `ASR_API_KEY` / `ASR_MODEL` | 见上 | 上游语音识别 |
|
||||
| `ASR_MAX_BYTES` | 20971520 | 单次上传上限 |
|
||||
| `ASR_MAX_CHUNK_SEC` | `25` | 超长音频切片长度,`0` = 不切 |
|
||||
| `ASR_MAX_CONCURRENT` | `2` | 并发转写数 |
|
||||
| `ASR_LOG_LEVEL` | `info` | debug / info / warn / error / none |
|
||||
|
||||
## 客户端怎么录音
|
||||
|
||||
浏览器侧只有 HTTPS 才允许开麦;**Android 套壳 App 用原生录音再把 base64 交给页面**,
|
||||
http 页面也能用。桥的契约与 Kotlin 示例见 [docs/voice-input.md](../../docs/voice-input.md)。
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"name": "mcp_asr",
|
||||
"version": "1.0.0",
|
||||
"description": "ASR service: POST /transcribe for the H5 composer, POST /mcp for the model (tool transcribe_audio). Zero dependencies.",
|
||||
"type": "module",
|
||||
"main": "src/server.js",
|
||||
"scripts": {
|
||||
"start": "node src/server.js"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,433 @@
|
||||
#!/usr/bin/env node
|
||||
// ASR(语音转文字)服务。一个进程两个入口,共用同一套转写实现:
|
||||
//
|
||||
// POST /transcribe 普通 HTTP —— 给 agent-core 的输入框用:录完音直接拿文字,
|
||||
// 不经过模型、不花 token、延迟 1~3s。
|
||||
// POST /mcp Streamable HTTP MCP —— 工具 transcribe_audio({url}),给模型转写
|
||||
// 会话目录里的音频(本地文件先 mint_public_link 换公网短链,与 video_task 同构)。
|
||||
//
|
||||
// 为什么不直接用 MCP 工具做语音输入:MCP 是模型协议,走一次工具调用要多一轮模型往返
|
||||
//(3~10s + token),识别结果还会落进聊天记录而不是输入框,用户没法改错字。输入法级别的
|
||||
// 功能必须绕开模型。
|
||||
//
|
||||
// 为什么零依赖(不用 express / MCP SDK):core 的 MCP 客户端只发 initialize / tools/list /
|
||||
// tools/call 三种请求,手写这几十行 JSON-RPC 就够;本机内存只有 2G,省掉 SDK 与 express
|
||||
// 的常驻开销(本服务空载约 35MB)。
|
||||
//
|
||||
// 转写走上游的 OpenAI 兼容 /audio/transcriptions(默认智谱 glm-asr,换平台改 env 即可)。
|
||||
// 音频一律先经 ffmpeg 归一化成 16kHz 单声道 wav:WebView 录出来是 webm/opus,iOS 是 mp4/aac,
|
||||
// 上游只认 wav/mp3,归一化后两端都自由。
|
||||
|
||||
import http from 'node:http';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import fs from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
// ── 配置(全部走 env,不读 config.yml:本容器按仓库惯例只拿 env) ──────────────
|
||||
const PORT = Number(process.env.PORT || 3000);
|
||||
|
||||
// 上游:任何 OpenAI 兼容的 /audio/transcriptions(智谱 glm-asr、OpenAI whisper-1 等)
|
||||
const API_BASE = String(process.env.ASR_API_BASE || 'https://open.bigmodel.cn/api/paas/v4').trim().replace(/\/+$/, '');
|
||||
const API_KEY = String(process.env.ASR_API_KEY || '').trim();
|
||||
const API_MODEL = String(process.env.ASR_MODEL || 'glm-asr').trim();
|
||||
// 单次上传上限。60s 的 AAC 约 240KB、16k wav 约 1.9MB,20MB 是给长音频留的余量。
|
||||
const MAX_BYTES = Math.max(1024 * 1024, Number(process.env.ASR_MAX_BYTES || 20 * 1024 * 1024));
|
||||
// 云端多数 ASR 单次只吃 ~30s:超长音频按这个秒数切片后逐段识别再拼接(0 = 不切)。
|
||||
// 16k/16bit 单声道 wav 恒为 32000 B/s,所以用文件大小就能估时长,不用再起 ffprobe。
|
||||
const CHUNK_SEC = Math.max(0, Number(process.env.ASR_MAX_CHUNK_SEC || 25));
|
||||
// 同时转写数:2 vCPU 上并发再高只会互相拖慢。
|
||||
const MAX_CONCURRENT = Math.max(1, Number(process.env.ASR_MAX_CONCURRENT || 2));
|
||||
|
||||
const FFMPEG = String(process.env.FFMPEG_BIN || 'ffmpeg').trim();
|
||||
const LOG_LEVEL = String(process.env.ASR_LOG_LEVEL || 'info').toLowerCase();
|
||||
const LEVELS = { debug: 10, info: 20, warn: 30, error: 40, none: 99 };
|
||||
|
||||
function log(level, msg) {
|
||||
if ((LEVELS[level] || 20) < (LEVELS[LOG_LEVEL] == null ? 20 : LEVELS[LOG_LEVEL])) return;
|
||||
console.log('[asr ' + level + '] ' + msg);
|
||||
}
|
||||
|
||||
// ── 子进程:统一超时 + 收 stdout/stderr ────────────────────────────────────
|
||||
function run(cmd, args, timeoutMs) {
|
||||
return new Promise((resolve, reject) => {
|
||||
let child;
|
||||
try {
|
||||
child = spawn(cmd, args, { stdio: ['ignore', 'pipe', 'pipe'] });
|
||||
} catch (e) {
|
||||
return reject(e);
|
||||
}
|
||||
let out = '';
|
||||
let err = '';
|
||||
let settled = false;
|
||||
const timer = setTimeout(() => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
try { child.kill('SIGKILL'); } catch (_) { /* 已退出 */ }
|
||||
reject(new Error(cmd + ' 超时(' + timeoutMs + 'ms)'));
|
||||
}, timeoutMs);
|
||||
timer.unref();
|
||||
child.stdout.on('data', (d) => { out += d; });
|
||||
child.stderr.on('data', (d) => { err += d; });
|
||||
child.on('error', (e) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
clearTimeout(timer);
|
||||
reject(e); // ENOENT:二进制没装(如没装 ffmpeg)
|
||||
});
|
||||
child.on('close', (code) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
clearTimeout(timer);
|
||||
if (code === 0) resolve({ stdout: out, stderr: err });
|
||||
else reject(new Error(cmd + ' 退出码 ' + code + ':' + ((err || out).slice(-400) || '无输出')));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// ── 音频:归一化成 16k 单声道 wav(再把超长音频切片) ───────────────────────
|
||||
async function toWav(buf, ext) {
|
||||
const id = randomUUID();
|
||||
const inPath = path.join(os.tmpdir(), 'asr-in-' + id + (ext ? '.' + ext : ''));
|
||||
const wavPath = path.join(os.tmpdir(), 'asr-out-' + id + '.wav');
|
||||
try {
|
||||
await fs.writeFile(inPath, buf);
|
||||
await run(FFMPEG, [
|
||||
'-hide_banner', '-loglevel', 'error', '-y',
|
||||
'-i', inPath,
|
||||
'-vn', '-ac', '1', '-ar', '16000', '-c:a', 'pcm_s16le', '-f', 'wav',
|
||||
wavPath,
|
||||
], 60000);
|
||||
return wavPath;
|
||||
} catch (e) {
|
||||
await fs.rm(inPath, { force: true }).catch(() => {});
|
||||
await fs.rm(wavPath, { force: true }).catch(() => {});
|
||||
// ffmpeg 的 stderr 很有用(定位是封装不对还是文件截断),但只进日志——
|
||||
// 回给前端的长英文堆栈对用户毫无意义,toast 里只会显示第一行。
|
||||
log('warn', '音频解码失败:' + String(e && e.message || e));
|
||||
const err = new Error('音频解码失败(格式不支持或文件损坏)');
|
||||
err.status = 400;
|
||||
throw err;
|
||||
} finally {
|
||||
await fs.rm(inPath, { force: true }).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
// 按秒切片:只在超过 CHUNK_SEC 时做。切片用 -c copy,纯 PCM 无损。
|
||||
async function splitWav(wavPath, chunkSec) {
|
||||
const dir = path.dirname(wavPath);
|
||||
const base = path.basename(wavPath, '.wav');
|
||||
const pattern = path.join(dir, base + '-part-%03d.wav');
|
||||
await run(FFMPEG, [
|
||||
'-hide_banner', '-loglevel', 'error', '-y',
|
||||
'-i', wavPath,
|
||||
'-f', 'segment', '-segment_time', String(chunkSec), '-c', 'copy',
|
||||
pattern,
|
||||
], 60000);
|
||||
const files = (await fs.readdir(dir))
|
||||
.filter((f) => f.startsWith(base + '-part-') && f.endsWith('.wav'))
|
||||
.sort();
|
||||
return files.map((f) => path.join(dir, f));
|
||||
}
|
||||
|
||||
// ── 引擎 ──────────────────────────────────────────────────────────────────
|
||||
function extractText(obj) {
|
||||
if (!obj) return '';
|
||||
if (typeof obj === 'string') return obj;
|
||||
if (typeof obj.text === 'string') return obj.text;
|
||||
if (obj.result && typeof obj.result.text === 'string') return obj.result.text;
|
||||
return '';
|
||||
}
|
||||
|
||||
async function runCloud(wavPath) {
|
||||
if (!API_KEY) {
|
||||
const e = new Error('语音识别未配置:容器环境变量 ASR_API_KEY 为空');
|
||||
e.status = 503;
|
||||
throw e;
|
||||
}
|
||||
const buf = await fs.readFile(wavPath);
|
||||
const form = new FormData();
|
||||
form.append('model', API_MODEL);
|
||||
form.append('file', new Blob([buf], { type: 'audio/wav' }), 'audio.wav');
|
||||
let res;
|
||||
try {
|
||||
res = await fetch(API_BASE + '/audio/transcriptions', {
|
||||
method: 'POST',
|
||||
headers: { Authorization: 'Bearer ' + API_KEY },
|
||||
body: form,
|
||||
signal: AbortSignal.timeout(60000),
|
||||
});
|
||||
} catch (e) {
|
||||
const err = new Error('调用上游语音识别失败:' + String(e && e.message || e));
|
||||
err.status = 502;
|
||||
throw err;
|
||||
}
|
||||
const raw = await res.text();
|
||||
if (!res.ok) {
|
||||
// 智谱对“解码后没有可识别语音”的音频回 400/1210 no audio segment found。
|
||||
// 这通常不是服务故障(录音是静音、或浏览器只录到空白),给一句人话,别把上游 JSON 丢给用户。
|
||||
let upstreamMsg = raw.slice(0, 300);
|
||||
let upstreamCode = '';
|
||||
try {
|
||||
const obj = JSON.parse(raw);
|
||||
if (obj && obj.error) {
|
||||
upstreamMsg = String(obj.error.message || upstreamMsg);
|
||||
upstreamCode = String(obj.error.code || '');
|
||||
}
|
||||
} catch (_) { /* 非 JSON,保留原文 */ }
|
||||
const noSpeech = /no audio segment/i.test(upstreamMsg) || upstreamCode === '1210';
|
||||
const err = new Error(noSpeech
|
||||
? '没有识别到语音内容(录到的是静音,或音频解码后为空)'
|
||||
: '上游识别失败 HTTP ' + res.status + ':' + upstreamMsg);
|
||||
err.status = noSpeech ? 400 : 502;
|
||||
err.upstream = upstreamMsg;
|
||||
throw err;
|
||||
}
|
||||
let obj = null;
|
||||
try { obj = JSON.parse(raw); } catch (_) { return raw.trim(); }
|
||||
const text = extractText(obj);
|
||||
if (!text) {
|
||||
const err = new Error('上游返回为空:' + raw.slice(0, 300));
|
||||
err.status = 502;
|
||||
throw err;
|
||||
}
|
||||
return text;
|
||||
}
|
||||
|
||||
// 归一化后 wav 的实测音量(仅 debug 级别调用):用于区分“文件坏了”和“录到的是静音”。
|
||||
// volumedetect 跟解码共用一次遍历,开销很小,但只在排查时跑。
|
||||
async function wavLevel(wavPath) {
|
||||
const r = await run(FFMPEG, [
|
||||
'-hide_banner', '-nostats', '-i', wavPath, '-af', 'volumedetect', '-f', 'null', '-',
|
||||
], 30000);
|
||||
const mean = /mean_volume:\s*(-?[\d.]+) dB/.exec(r.stderr);
|
||||
const max = /max_volume:\s*(-?[\d.]+) dB/.exec(r.stderr);
|
||||
return (mean ? mean[1] : '?') + ' / ' + (max ? max[1] : '?') + ' dB';
|
||||
}
|
||||
|
||||
// 转写一段音频缓冲:归一化 →(超长则切片)→ 逐段识别 → 拼接。
|
||||
async function transcribeBuffer(buf, opts) {
|
||||
const wavPath = await toWav(buf, opts && opts.ext);
|
||||
const parts = [];
|
||||
try {
|
||||
const st = await fs.stat(wavPath);
|
||||
const seconds = st.size / 32000; // 16k/单声道/16bit
|
||||
// 每次请求都打一行:原始字节 → 归一化时长。排查“上游说没声音”时第一眼看这个。
|
||||
log('info', '归一化:' + buf.length + 'B (' + ((opts && opts.ext) || '?') + ') → '
|
||||
+ seconds.toFixed(1) + 's wav');
|
||||
if (CHUNK_SEC > 0 && seconds > CHUNK_SEC * 1.2) {
|
||||
parts.push(...await splitWav(wavPath, CHUNK_SEC));
|
||||
log('info', '超长音频切成 ' + parts.length + ' 段');
|
||||
}
|
||||
if (!parts.length) {
|
||||
try {
|
||||
return (await runCloud(wavPath)).trim();
|
||||
} catch (e) {
|
||||
// 只在上游判定“没有语音”时量一遍音量(正常路径零开销):
|
||||
// 静音是 -91dB,正常语音 -20dB 上下,一眼就能分清是设备没收到声还是文件坏了。
|
||||
if (e && e.status === 400) {
|
||||
log('warn', '音量(mean/max): ' + await wavLevel(wavPath).catch(() => '?'));
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
const out = [];
|
||||
for (const p of parts) {
|
||||
// 每段独立识别;单段失败不连累其他段(长录音里偶发静音段常见)
|
||||
try {
|
||||
out.push((await runCloud(p)).trim());
|
||||
} catch (e) {
|
||||
log('warn', '切片识别失败(跳过):' + String(e && e.message || e).slice(0, 200));
|
||||
}
|
||||
}
|
||||
return out.filter(Boolean).join('');
|
||||
} finally {
|
||||
await fs.rm(wavPath, { force: true }).catch(() => {});
|
||||
for (const p of parts) await fs.rm(p, { force: true }).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
// ── HTTP 小工具 ───────────────────────────────────────────────────────────
|
||||
function json(res, code, obj) {
|
||||
if (res.headersSent) return;
|
||||
const body = JSON.stringify(obj);
|
||||
res.writeHead(code, {
|
||||
'Content-Type': 'application/json; charset=utf-8',
|
||||
'Content-Length': Buffer.byteLength(body),
|
||||
'Cache-Control': 'no-store',
|
||||
});
|
||||
res.end(body);
|
||||
}
|
||||
|
||||
function readBody(req, limit) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const chunks = [];
|
||||
let n = 0;
|
||||
let failed = false;
|
||||
req.on('data', (c) => {
|
||||
if (failed) return;
|
||||
n += c.length;
|
||||
if (n > limit) {
|
||||
failed = true;
|
||||
const e = new Error('音频过大(上限 ' + Math.round(limit / 1048576) + 'MB)');
|
||||
e.status = 413;
|
||||
reject(e);
|
||||
return;
|
||||
}
|
||||
chunks.push(c);
|
||||
});
|
||||
req.on('end', () => { if (!failed) resolve(Buffer.concat(chunks)); });
|
||||
req.on('error', (e) => { if (!failed) reject(e); });
|
||||
});
|
||||
}
|
||||
|
||||
const EXT_BY_MIME = {
|
||||
'audio/webm': 'webm', 'audio/ogg': 'ogg', 'audio/mp4': 'm4a', 'audio/x-m4a': 'm4a',
|
||||
'audio/mpeg': 'mp3', 'audio/mp3': 'mp3', 'audio/wav': 'wav', 'audio/x-wav': 'wav',
|
||||
'audio/aac': 'aac', 'audio/flac': 'flac', 'audio/amr': 'amr',
|
||||
};
|
||||
|
||||
function extOf(contentType) {
|
||||
const k = String(contentType || '').split(';')[0].trim().toLowerCase();
|
||||
return EXT_BY_MIME[k] || '';
|
||||
}
|
||||
|
||||
// ── 入口 1:POST /transcribe(普通 HTTP,给输入框) ────────────────────────
|
||||
let active = 0;
|
||||
|
||||
async function handleTranscribe(req, res, url) {
|
||||
if (active >= MAX_CONCURRENT) {
|
||||
return json(res, 429, { error: '语音识别繁忙,请稍后再试' });
|
||||
}
|
||||
const buf = await readBody(req, MAX_BYTES);
|
||||
if (!buf.length) return json(res, 400, { error: '未收到音频数据' });
|
||||
|
||||
active++;
|
||||
const t0 = Date.now();
|
||||
try {
|
||||
const text = await transcribeBuffer(buf, { ext: extOf(req.headers['content-type']) });
|
||||
log('info', 'transcribed ' + buf.length + 'B → ' + text.length + ' 字,' + (Date.now() - t0) + 'ms');
|
||||
json(res, 200, { text, ms: Date.now() - t0, model: API_MODEL });
|
||||
} finally {
|
||||
active--;
|
||||
}
|
||||
}
|
||||
|
||||
// ── 入口 2:POST /mcp(Streamable HTTP,给模型) ──────────────────────────
|
||||
const TOOL = {
|
||||
name: 'transcribe_audio',
|
||||
description:
|
||||
'Transcribe an audio file to text. Provide a publicly reachable http(s) URL '
|
||||
+ '(mp3 / wav / m4a / webm / ogg / aac / flac, <= 20MB). '
|
||||
+ 'For a file inside the current task directory, first call the built-in mint_public_link tool '
|
||||
+ 'to get a temporary public URL, then pass that URL here.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: {
|
||||
url: { type: 'string', description: 'Publicly reachable http(s) URL of the audio file.' },
|
||||
},
|
||||
required: ['url'],
|
||||
},
|
||||
};
|
||||
|
||||
async function fetchAudio(urlStr) {
|
||||
let u;
|
||||
try { u = new URL(urlStr); } catch (_) { throw new Error('url 不是合法链接'); }
|
||||
if (u.protocol !== 'http:' && u.protocol !== 'https:') throw new Error('只支持 http(s) 链接');
|
||||
const res = await fetch(u, { redirect: 'follow', signal: AbortSignal.timeout(30000) });
|
||||
if (!res.ok) throw new Error('下载音频失败 HTTP ' + res.status);
|
||||
const buf = Buffer.from(await res.arrayBuffer());
|
||||
if (!buf.length) throw new Error('下载到的音频为空');
|
||||
if (buf.length > MAX_BYTES) throw new Error('音频过大(上限 ' + Math.round(MAX_BYTES / 1048576) + 'MB)');
|
||||
return { buf, ext: extOf(res.headers.get('content-type')) };
|
||||
}
|
||||
|
||||
async function handleMcp(req, res) {
|
||||
const raw = await readBody(req, 1024 * 1024);
|
||||
let msg;
|
||||
try { msg = JSON.parse(raw.toString('utf8')); } catch (_) {
|
||||
return json(res, 400, { jsonrpc: '2.0', id: null, error: { code: -32700, message: 'Parse error' } });
|
||||
}
|
||||
const id = msg && msg.id !== undefined ? msg.id : null;
|
||||
const method = msg && msg.method;
|
||||
const params = (msg && msg.params) || {};
|
||||
const ok = (result) => json(res, 200, { jsonrpc: '2.0', id, result });
|
||||
const fail = (code, message) => json(res, 200, { jsonrpc: '2.0', id, error: { code, message } });
|
||||
|
||||
if (method === 'initialize') {
|
||||
return ok({
|
||||
protocolVersion: '2024-11-05',
|
||||
capabilities: { tools: {} },
|
||||
serverInfo: { name: 'asr', version: '1.0.0' },
|
||||
});
|
||||
}
|
||||
// 无状态实现:不返回 Mcp-Session-Id,core 也就不会带;通知直接回空结果(core 不校验)。
|
||||
if (method === 'notifications/initialized' || method === 'notifications/cancelled') return ok({});
|
||||
if (method === 'ping') return ok({});
|
||||
if (method === 'tools/list') return ok({ tools: [TOOL] });
|
||||
|
||||
if (method === 'tools/call') {
|
||||
if (params.name !== TOOL.name) return fail(-32602, 'Unknown tool: ' + params.name);
|
||||
const args = params.arguments || {};
|
||||
const urlStr = String(args.url || '').trim();
|
||||
if (!urlStr) return ok({ isError: true, content: [{ type: 'text', text: '缺少 url 参数' }] });
|
||||
try {
|
||||
const { buf, ext } = await fetchAudio(urlStr);
|
||||
const text = await transcribeBuffer(buf, { ext });
|
||||
return ok({ content: [{ type: 'text', text }] });
|
||||
} catch (e) {
|
||||
return ok({ isError: true, content: [{ type: 'text', text: '转写失败:' + String(e && e.message || e) }] });
|
||||
}
|
||||
}
|
||||
|
||||
return fail(-32601, 'Method not found: ' + method);
|
||||
}
|
||||
|
||||
// ── 服务 ──────────────────────────────────────────────────────────────────
|
||||
const server = http.createServer((req, res) => {
|
||||
let url;
|
||||
try { url = new URL(req.url, 'http://localhost'); } catch (_) {
|
||||
return json(res, 400, { error: 'bad url' });
|
||||
}
|
||||
const done = (p) => {
|
||||
Promise.resolve(p).catch((e) => {
|
||||
const code = (e && e.status) || 500;
|
||||
// 4xx 是可预期的输入问题(坏音频、没配 key),只记一行;5xx 才带堆栈
|
||||
log(code >= 500 ? 'error' : 'warn', String(e && e.message || e) + (code >= 500 && e && e.stack ? '\n' + e.stack : ''));
|
||||
if (!res.headersSent) json(res, code, { error: String(e && e.message || e) });
|
||||
else try { res.end(); } catch (_) { /* 已断 */ }
|
||||
});
|
||||
};
|
||||
|
||||
if (req.method === 'GET' && url.pathname === '/health') {
|
||||
return json(res, 200, {
|
||||
ok: true,
|
||||
model: API_MODEL,
|
||||
configured: !!API_KEY,
|
||||
});
|
||||
}
|
||||
if (req.method === 'POST' && url.pathname === '/transcribe') return done(handleTranscribe(req, res, url));
|
||||
if (url.pathname === '/mcp') {
|
||||
if (req.method !== 'POST') return json(res, 405, { error: 'only POST /mcp' });
|
||||
return done(handleMcp(req, res));
|
||||
}
|
||||
return json(res, 404, { error: 'not found' });
|
||||
});
|
||||
|
||||
let shuttingDown = false;
|
||||
function shutdown(sig) {
|
||||
if (shuttingDown) return;
|
||||
shuttingDown = true;
|
||||
log('info', '收到 ' + sig + ',退出');
|
||||
server.close(() => process.exit(0));
|
||||
setTimeout(() => process.exit(0), 3000).unref();
|
||||
}
|
||||
process.on('SIGTERM', () => shutdown('SIGTERM'));
|
||||
process.on('SIGINT', () => shutdown('SIGINT'));
|
||||
|
||||
server.listen(PORT, '0.0.0.0', () => {
|
||||
log('info', 'listening on ' + PORT + ',model=' + API_MODEL + ' @ ' + API_BASE);
|
||||
if (!API_KEY) log('warn', 'ASR_API_KEY 为空:/transcribe 会返回 503');
|
||||
});
|
||||
+29
-4
@@ -10,12 +10,14 @@
|
||||
#
|
||||
# 输入(都可选):
|
||||
# BIGMODEL_KEY / DEEPSEEK_KEY / QWEN_KEY 模型平台 key,填几个启用几个(0~3 个)
|
||||
# image_task / video_task 复用 BIGMODEL_KEY(智谱即 BigModel);
|
||||
# 未配 BIGMODEL_KEY 时这两个服务写为 mode: disable
|
||||
# image_task / video_task / asr 复用 BIGMODEL_KEY(智谱即 BigModel);
|
||||
# 未配 BIGMODEL_KEY 时这三个服务写为 mode: disable
|
||||
# AUTH_USER / AUTH_PASSWORD 登录账号(默认 admin / admin,请自行改掉)
|
||||
# PUBLIC_IP / AGENT_PORT 对外地址与端口,写进 public_link.base_url;
|
||||
# PUBLIC_IP 不配置则自动探测公网 IP
|
||||
# AUTH_SECRET / EXECUTOR_TOKEN / WXBOT_TOKEN 一般不用给:不给就自动生成并复用(见下)
|
||||
# ASR_API_KEY 语音识别(mcp_asr)的上游 key;不给则复用 BIGMODEL_KEY,
|
||||
# 已有 .env 里的值优先(语音输入只是可选能力,留空不影响主流程)
|
||||
#
|
||||
# 微信接入(wxbot 已合入根 compose,作为栈内服务):本脚本只负责生成/复用根 .env 的 WXBOT_TOKEN;
|
||||
# wxbot 自身零配置文件(compose 注 env + 内置默认,同 executor 模式),无需任何同步动作。
|
||||
@@ -51,6 +53,8 @@ EXECUTOR_TOKEN="${EXECUTOR_TOKEN:-}"
|
||||
AUTH_SECRET="${AUTH_SECRET:-}"
|
||||
# 微信接入服务 token(core ↔ wxbot 双向共用,compose 两侧同值注入;wxbot 已并入仓库)
|
||||
WXBOT_TOKEN="${WXBOT_TOKEN:-}"
|
||||
# 语音识别(mcp_asr)的上游 key:独立于模型平台 key,但默认复用 BIGMODEL_KEY(智谱 glm-asr)
|
||||
ASR_API_KEY="${ASR_API_KEY:-}"
|
||||
|
||||
log() { echo -e "\n\033[1;32m===== $* =====\033[0m"; }
|
||||
info() { echo -e "\033[1;36m[INFO]\033[0m $*"; }
|
||||
@@ -74,6 +78,8 @@ if [ -f "$ENV_FILE" ]; then
|
||||
EXECUTOR_TOKEN="${EXECUTOR_TOKEN:-$(sed -n 's/^EXECUTOR_TOKEN=//p' "$ENV_FILE" | head -1)}"
|
||||
AUTH_SECRET="${AUTH_SECRET:-$(sed -n 's/^AUTH_SECRET=//p' "$ENV_FILE" | head -1)}"
|
||||
WXBOT_TOKEN="${WXBOT_TOKEN:-$(sed -n 's/^WXBOT_TOKEN=//p' "$ENV_FILE" | head -1)}"
|
||||
# ASR_API_KEY 同理复用:手填的 key 不会被重跑抹掉
|
||||
ASR_API_KEY="${ASR_API_KEY:-$(sed -n 's/^ASR_API_KEY=//p' "$ENV_FILE" | head -1)}"
|
||||
# AGENT_PORT 同理复用:避免不带参数重跑时把部署机自定义端口重置回 8090(显式传参优先)
|
||||
AGENT_PORT="${AGENT_PORT_INPUT:-$(sed -n 's/^AGENT_PORT=//p' "$ENV_FILE" | head -1)}"
|
||||
AGENT_PORT="${AGENT_PORT:-8090}"
|
||||
@@ -81,19 +87,22 @@ fi
|
||||
EXECUTOR_TOKEN="${EXECUTOR_TOKEN:-$(gen_hex 32)}"
|
||||
AUTH_SECRET="${AUTH_SECRET:-$(gen_hex 32)}"
|
||||
WXBOT_TOKEN="${WXBOT_TOKEN:-$(gen_hex 32)}"
|
||||
ASR_API_KEY="${ASR_API_KEY:-$BIGMODEL_KEY}"
|
||||
|
||||
cat > "$ENV_FILE" <<EOF
|
||||
# 本文件由 scripts/init.sh 生成,勿手改(改配置请重新运行该脚本,或用环境变量覆盖)
|
||||
# EXECUTOR_TOKEN: core 与 executor 之间的调用鉴权(两侧同值,compose 注入)
|
||||
# AUTH_SECRET: 登录态签名密钥(写进 config.yml 的 auth.secret)
|
||||
# WXBOT_TOKEN: core ↔ wxbot 微信接入服务 token(compose 注入两侧;wxbot 零配置文件,env 即全部接线)
|
||||
# ASR_API_KEY: 语音识别(mcp_asr)的上游 key,空 = 语音输入关闭;默认复用 BIGMODEL_KEY
|
||||
# AGENT_PORT: agent-core 映射到宿主机的端口
|
||||
EXECUTOR_TOKEN=$EXECUTOR_TOKEN
|
||||
AUTH_SECRET=$AUTH_SECRET
|
||||
WXBOT_TOKEN=$WXBOT_TOKEN
|
||||
ASR_API_KEY=$ASR_API_KEY
|
||||
AGENT_PORT=$AGENT_PORT
|
||||
EOF
|
||||
info "EXECUTOR_TOKEN / AUTH_SECRET / WXBOT_TOKEN 已就位(沿用已有 .env 的值,没有则新生成)"
|
||||
info "EXECUTOR_TOKEN / AUTH_SECRET / WXBOT_TOKEN 已就位(沿用已有 .env 的值,没有则新生成);ASR_API_KEY $([ -n "$ASR_API_KEY" ] && echo 已配置 || echo 为空(语音输入关闭))"
|
||||
|
||||
# --env-only:只刷密钥,绝不碰 config.yml(平台 key 不在 .env 里,重跑全量会清空已部署配置)
|
||||
if [ "${ENV_ONLY:-}" = "1" ]; then
|
||||
@@ -229,6 +238,21 @@ EOF
|
||||
EOF
|
||||
fi
|
||||
|
||||
# 语音转写:key 在容器 env(ASR_API_KEY),不经 headers,所以单独按 ASR_API_KEY 判断
|
||||
if [ -n "$ASR_API_KEY" ]; then
|
||||
cat <<'EOF'
|
||||
- name: asr
|
||||
url: "http://mcp_asr:3000/mcp"
|
||||
mode: skill # 语音转写是低频能力,不下发到每轮工具表
|
||||
EOF
|
||||
else
|
||||
cat <<'EOF'
|
||||
- name: asr
|
||||
url: "http://mcp_asr:3000/mcp"
|
||||
mode: disable # 未配 ASR_API_KEY:不连接、不发现
|
||||
EOF
|
||||
fi
|
||||
|
||||
echo
|
||||
cat <<EOF
|
||||
# 给外部服务(如智谱)现签公网临时链接用;改地址后重跑本脚本
|
||||
@@ -249,7 +273,8 @@ echo " 账号 : $AUTH_USER"
|
||||
echo " 密码 : $AUTH_PASSWORD"
|
||||
echo " 平台 : ${enabled_plats[*]:-(无)}"
|
||||
echo " 图像/视频: $([ -n "$BIGMODEL_KEY" ] && echo "image_task / video_task 已启用" || echo "未配置(mode: disable)")"
|
||||
echo " MCP 服务 : web_search / image_task / video_task(同栈,随 compose 启动)"
|
||||
echo " 语音输入 : $([ -n "$ASR_API_KEY" ] && echo "mcp_asr 已启用" || echo "未配置(ASR_API_KEY 为空)")"
|
||||
echo " MCP 服务 : web_search / image_task / video_task / asr(同栈,随 compose 启动)"
|
||||
echo " 微信接入 : WXBOT_TOKEN 已就位(compose 注入 core 与 wxbot 两侧;wxbot 无需配置文件)"
|
||||
echo "---------------------------------------------"
|
||||
log "完成 ✅"
|
||||
|
||||
@@ -53,6 +53,7 @@ input_schema:
|
||||
output_schema:
|
||||
# 输出描述
|
||||
visit_url: /app/{task_id}/ # 可选:页面类技能的访问路径模板
|
||||
required: true # 可选:清单行首标 **required**,提示该域不要手写等价实现
|
||||
thinking: true # 可选:get 时自动开启本回合的深度思考
|
||||
---
|
||||
```
|
||||
@@ -61,6 +62,7 @@ thinking: true # 可选:get 时自动开启本回合的深度思
|
||||
- `description`:一句话描述技能功能,用于 `op=list` 的清单展示。
|
||||
- `thinking`:(可选)为 `true` 时,模型调用 `get` 会自动开启深度思考(合并模型的
|
||||
`thinking_params`),开启后保持,直到用户手动关闭;模型未配置思考参数时不生效。只影响模型回合,不影响脚本本身。
|
||||
- `required`:(可选)为 `true` 时,模型可见的技能清单里该行行首标 `**required**`,表示这个领域不要手写等价脚本(如 `office` / `x2pdf`)。纯提示,不做强制校验,也不会自动调用。
|
||||
- `input_schema`:输入参数的 JSON Schema 定义,用于参数校验和文档生成。
|
||||
- `output_schema`:输出结果的描述(可选)。
|
||||
- `visit_url`:(可选)页面类技能声明的访问路径模板,`{task_id}` 占位执行时替换为会话 id。`run` 成功后由 core(tools/executor.js)拼上 `public_link.base_url`,在输出末尾追加一行「访问地址:<完整 URL>」——技能进程跑在 executor 隔离视图里不知道公网地址,完整链接只能 core 补;AI 拿到相对路径会原样转给用户,而相对路径打不开。
|
||||
|
||||
@@ -31,14 +31,14 @@ visit_url: /app/{task_id}/
|
||||
|
||||
## 初始化
|
||||
|
||||
调用 `skill {"op":"run","name":"data-page"}`,无需参数。它会把以下文件复制到 `.app/`:
|
||||
调用 `skill {"op":"run","name":"data-page"}`,无需参数。它会把以下文件发布到 `.app/`:
|
||||
|
||||
- `table.html` / `dashboard.html` / `kanban.html` —— 三个数据页模板
|
||||
- `nav.html` —— 多页导航首页模板(静态卡片,无 config)
|
||||
- `base.css` —— 共用样式
|
||||
- `config-table.js` / `config-dashboard.js` / `config-kanban.js` —— 配置骨架
|
||||
|
||||
重跑行为按资产所有权分流:`table/dashboard/kanban/nav.html` 与 `base.css` 每次**覆盖刷新**(模板修复自动到达旧会话;这些文件契约禁止修改,覆盖无副作用);`config-*.js` 仅缺失时补默认骨架,不覆盖已写好的配置;`.app/index.html` 永不被碰。
|
||||
重跑行为按资产所有权分流:`table/dashboard/kanban/nav.html` 与 `base.css` 是指向技能的符号链接,技能更新立即生效(这些文件契约禁止修改);`config-*.js` 仅缺失时补默认骨架,不覆盖已写好的配置;`.app/index.html` 永不被碰。
|
||||
|
||||
## 数据 API
|
||||
|
||||
|
||||
+27
-8
@@ -1,13 +1,13 @@
|
||||
#!/usr/bin/env python3
|
||||
# data-page 技能入口:把技能自带 templates/ 下的文件复制到会话任务目录的 .app/。
|
||||
# data-page 技能入口:把技能自带 templates/ 下的文件发布到会话任务目录的 .app/。
|
||||
# 目标目录固定为 <任务目录>/.app/(不带 dest 参数:manager 对绝对路径参数不做沙箱
|
||||
# 校验,开放 dest 等于开放任意目录写入;路径也已写死在文档流程里,可配置只会造成漂移)。
|
||||
# 复制内容:
|
||||
# *.html / base.css → .app/ 页面模板与共用样式(每次覆盖刷新)
|
||||
# config-*.js → .app/ 配置骨架(仅缺失时补,AI 只改这些)
|
||||
# 复制语义按资产所有权分流:
|
||||
# *.html / *.css → .app/ 模板与共用样式——每次运行覆盖刷新(SKILL 契约禁止改这些文件,
|
||||
# 覆盖无副作用;不覆盖则旧会话永远吃不到模板修复)。index.html 不在复制清单,永不碰。
|
||||
# 发布内容:
|
||||
# *.html / base.css → .app/ 页面模板与共用样式(符号链接,技能是唯一真源)
|
||||
# config-*.js → .app/ 配置骨架(实体文件,仅缺失时补,AI 只改这些)
|
||||
# 发布语义按资产所有权分流:
|
||||
# *.html / *.css → .app/ 模板与共用样式——技能只读,链接后永远是最新版本;
|
||||
# SKILL 契约本来就禁止改这些文件,所以不必留副本。
|
||||
# config-*.js → .app/ 配置骨架——仅缺失时补默认,不覆盖 AI 已写好的配置。
|
||||
# 纯 stdlib,无 requirements。
|
||||
# cwd 契约:生产模式下 cwd 即会话任务目录;本地直跑时 cwd 是技能目录——
|
||||
@@ -19,6 +19,25 @@ import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def link_asset(src, dst):
|
||||
"""模板/样式是技能的只读资产:用符号链接发布,永远跟随技能最新版本。
|
||||
|
||||
core 的 /app 静态托管只放行指向技能根目录的链接(其余 404),
|
||||
所以这里必须链到技能的绝对路径。config-*.js 是每会话的数据,仍然复制。
|
||||
"""
|
||||
src = str(Path(src).resolve())
|
||||
if dst.is_symlink():
|
||||
try:
|
||||
if os.path.realpath(dst) == os.path.realpath(src):
|
||||
return
|
||||
except OSError:
|
||||
pass
|
||||
dst.unlink()
|
||||
elif dst.exists():
|
||||
dst.unlink() # 旧会话的实体副本:删掉换成链接
|
||||
os.symlink(src, str(dst))
|
||||
|
||||
|
||||
def main():
|
||||
# ── 参数校验:本技能无参数 ─────────────────────────────
|
||||
if len(sys.argv) >= 2 and sys.argv[1].strip():
|
||||
@@ -60,7 +79,7 @@ def main():
|
||||
# ── 模板/样式:无条件覆盖刷新;config:仅补缺 ────────────
|
||||
refreshed = []
|
||||
for f in assets:
|
||||
shutil.copy2(f, app_dir / f.name)
|
||||
link_asset(f, app_dir / f.name)
|
||||
refreshed.append(f.name)
|
||||
created, kept = [], []
|
||||
for f in cfgs:
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
# 审核页内部协议(维护者文档)
|
||||
|
||||
> 本文件刻意不在 SKILL.md 里引用,避免模型顺着指针来读;只有维护者需要看。
|
||||
|
||||
`SKILL.md` 是给 AI 的调用契约;本文件记录审核页/重生成/数据表的实现细节。**AI 日常调用
|
||||
不需要读这里**,`skill get` 也只返回 SKILL.md。
|
||||
|
||||
## 文件与目录
|
||||
|
||||
```
|
||||
skills/document-photo-to-pdf/
|
||||
SKILL.md ← AI 契约(只有 images/output)
|
||||
REVIEW.md ← 本文件
|
||||
run.py ← 入口
|
||||
doc_deskew/ ← 检测/几何/出页/检验
|
||||
review_store.py ← data.db 的 doc_review_* 表
|
||||
review_page.py ← 模板链接 + 缩略图发布
|
||||
web/index.html ← 审核页模板的单一事实源
|
||||
|
||||
tasks/<session-id>/
|
||||
out/ ← 原始校正页图(PNG,自然尺寸,文件名与 PDF 同名)
|
||||
.app/doc-review/
|
||||
index.html ← 指向 web/index.html 的符号链接(不含数据注入)
|
||||
<batch_id>/p1.jpg ... ← 本批缩略图(长边 1200,JPEG)
|
||||
data.db ← doc_review_batch / doc_review_page / doc_review_job
|
||||
<output>.pdf
|
||||
<output>.state.json ← state v2,供恢复自动/调试
|
||||
```
|
||||
|
||||
core 只托管会话 `.app/`,不托管 skill 目录,所以会话里要有一个入口文件;用符号链接而不是
|
||||
副本——源仍只有 `web/index.html` 一份,改模板即刻对所有会话生效。`/app` 静态托管会
|
||||
realpath,只放行落在会话目录或技能根目录内的链接(其余 404,防止链接把 config.yml 带出来)。
|
||||
|
||||
## 对 AI 隐藏的参数
|
||||
|
||||
`run.py` 仍接受以下参数,但不出现在 SKILL.md / input_schema 里;审核页和 apply 接口使用:
|
||||
|
||||
| 参数 | 用途 |
|
||||
|---|---|
|
||||
| `paper` | a4/letter/16k/b5/数字比例/auto |
|
||||
| `orientation` | auto/none/portrait/landscape |
|
||||
| `enhance` | original/normal/strong/bw |
|
||||
| `dpi` | standard/high/72~600 |
|
||||
| `review` | `always`(默认)/`auto`(仅有 needs 时发布)/`off`(自动化用) |
|
||||
| `batch_id` | 重生成:从 data.db 读源图/输出/编辑,覆盖同名 PDF |
|
||||
| `edits` | `review/1` 对象或 JSON 路径;优先级高于显式参数 |
|
||||
| `state` | 上一份 state.json;仅用于恢复自动/调试 |
|
||||
|
||||
优先级:`edits.batch` > 显式参数 > 默认值。
|
||||
|
||||
## data.db 表
|
||||
|
||||
所有表以 `doc_review_` 前缀,避免和用户业务表混淆。技能自己 `CREATE TABLE IF NOT EXISTS`;
|
||||
`busy_timeout=5000`,WAL 由 core 建库时设置,技能侧 best-effort。
|
||||
|
||||
### doc_review_batch
|
||||
|
||||
```sql
|
||||
id TEXT PRIMARY KEY, -- drYYYYMMDD-HHMMSS-xxxx,不含点/斜杠
|
||||
output TEXT, -- 相对会话目录的 PDF 路径
|
||||
paper TEXT, enhance TEXT, dpi TEXT, orientation TEXT,
|
||||
status TEXT, -- running/ready/review/error
|
||||
summary TEXT, warnings TEXT,-- warnings 是 JSON 数组
|
||||
schema_version INTEGER,
|
||||
created_at TEXT, updated_at TEXT
|
||||
```
|
||||
|
||||
### doc_review_page
|
||||
|
||||
```sql
|
||||
batch_id TEXT, idx INTEGER, -- idx 永远是原始 images 的 1-based 下标
|
||||
source TEXT, -- 相对会话目录的原图路径
|
||||
source_w INTEGER, source_h INTEGER,
|
||||
w INTEGER, h INTEGER, -- 缩略图尺寸(页面画布用)
|
||||
auto_corners TEXT, -- 原图像素,自动基线
|
||||
auto_corners_norm TEXT, -- 归一化,页面“恢复自动”用
|
||||
corners TEXT, -- 归一化,当前角点(未旋转)
|
||||
auto_rotate INTEGER, -- 自动方向基线(0/90)
|
||||
rotate INTEGER, -- 当前应用旋转(用户覆盖后是绝对值)
|
||||
deleted INTEGER, sort INTEGER,
|
||||
status TEXT, reason TEXT, thumb TEXT, error TEXT,
|
||||
PRIMARY KEY(batch_id, idx)
|
||||
```
|
||||
|
||||
### doc_review_job
|
||||
|
||||
```sql
|
||||
batch_id TEXT PRIMARY KEY,
|
||||
edits TEXT, -- review/1 完整快照
|
||||
status TEXT, -- queued/running/done/error
|
||||
message TEXT,
|
||||
requested_at TEXT, started_at TEXT, finished_at TEXT
|
||||
```
|
||||
|
||||
## 审核页如何工作
|
||||
|
||||
1. 页面从 `/app/<task_id>/doc-review/` 加载;无 `?b=` 时查 `doc_review_batch` 最新一行。
|
||||
2. 通过 `POST /api/db/<task_id>/query` 读 `doc_review_batch` / `doc_review_page`。
|
||||
3. 用户拖角/旋转/删页/排序/改四项参数,只改浏览器内状态并写 localStorage 草稿。
|
||||
4. 点「重新生成 PDF」:
|
||||
- `INSERT ... ON CONFLICT` 写 `doc_review_job`(`status=queued`,`edits` 是完整快照);
|
||||
- 调通用执行接口
|
||||
`POST /api/sessions/<task_id>/skills/document-photo-to-pdf/run`,body `{args:{batch_id}}`;
|
||||
- 技能读 job → 应用 edits → 覆盖同名 PDF → 更新 batch/page/job;
|
||||
- 页面重新读 data.db,刷新警告、状态、PDF 链接。
|
||||
5. 全程不经过模型,也不需要用户复制 JSON。
|
||||
|
||||
通用接口实现在 `app/server/routes/skill-run.js`:core 只做会话/并发/错误映射,执行仍走
|
||||
`tools/skills.runSkill`,不感知技能名或参数语义。
|
||||
|
||||
## 页面交互约定
|
||||
|
||||
- 顶部标题条(白底,参考 H5 全屏预览):左侧 PDF 名,右侧下载 PDF,最右 X 返回会话。
|
||||
- 加载失败(无批次 / `data.db` 缺 `doc_review_batch` 表 / data.db 被删)只保留标题条:
|
||||
X 仍可返回会话、下载置灰,编辑区与底栏隐藏。所以 X 的绑定在 `start()` 里,不能放 `boot()`。
|
||||
- 重新生成 PDF 不在顶栏:放在照片区底部(stage 与预览各一个,按当前视图显示;桌面只用
|
||||
stage),仅在存在未保存修改时出现。
|
||||
- 底部只保留翻页和当前页名,不再放操作按钮。
|
||||
- 页码不占工具栏:`1/n` 浮在图片右上角(裁切区、预览区各一个,按当前视图显示);
|
||||
左上角常驻当前模式标签(裁切区「手动定位」、预览区「校正预览」)。
|
||||
- 桌面端左右分栏常显(左裁切、右预览),翻页按钮浮在裁切图左右两侧(48px 圆形),
|
||||
顶栏只留旋转 / 重置 / 设置;移动端没有分栏,翻页靠滑动 + 底栏箭头。
|
||||
- 移动端顶栏按钮带文字而非只靠图标:切换 / 旋转 / 重置 / 设置;视图按钮固定叫「切换」,
|
||||
当前模式看图片左上角标签。
|
||||
- 裁切区/预览区支持左右滑动翻页:左滑下一页、右滑上一页;仅触摸/笔生效,拖角优先,
|
||||
按钮点击不触发。
|
||||
- 是否 dirty 用 `collect()` 与加载时的基线 `S.savedEdits` 比较;拖角、旋转、删页/排序、
|
||||
改四项参数都会触发。
|
||||
- 重新生成成功后重读 DB、清本地草稿、把新状态设为基线,按钮自动消失;失败则保留以便重试。
|
||||
- X 返回 `/?s=<task_id>`;独立页面里也适用。
|
||||
- 手机默认进校正预览,点左上角切到裁切。
|
||||
|
||||
## edits / 旋转语义
|
||||
|
||||
```json
|
||||
{"schema":"review/1",
|
||||
"batch":{"paper":"a4","enhance":"normal","dpi":"standard","orientation":"auto"},
|
||||
"pages":[{"i":3,"c":[[0.03,0.05],[0.97,0.04],[0.96,0.95],[0.04,0.96]],"r":90}],
|
||||
"order":[1,2,3],"deleted":[]}
|
||||
```
|
||||
|
||||
- `i` 是原始 images 下标,不随页序变化。
|
||||
- `c` 是 exif_transpose 后原图的归一化坐标,顺序左上/右上/右下/左下。
|
||||
- `r` 是该页最终顺时针旋转(0/90/180/270),覆盖自动方向;页面总会为“人工页”带上
|
||||
`r`(包括 0),服务端用 `user_r is not None` 判断是否覆盖。
|
||||
- `pages` 只放与自动基线不同的页;未改的页由服务端重新检测/评级,避免把 fallback 页
|
||||
误升成 ready。
|
||||
- `order` 缺省按 images 原序;`deleted` 的页不进出件。
|
||||
- 人工旋转 90/270 时,`orientation=auto` 下该页画布按旋转后的有效方向走,避免把横向内容
|
||||
压进竖向画布;`portrait`/`landscape` 强制方向时仍按强制值。
|
||||
|
||||
## state v2
|
||||
|
||||
每次成功出件写 `<output>.state.json`:
|
||||
|
||||
```json
|
||||
{"schema":"document-photo-to-pdf/2",
|
||||
"batch":{"doc_type":null,"paper":"auto","enhance":"normal","dpi":150,"orientation":"auto"},
|
||||
"pages":[{"index":1,"source":"p1.jpg","source_size":[1400,1700],
|
||||
"auto_corners":[[...]],"corners_norm":[[...]],
|
||||
"rotate":0,"rotate_source":"auto","status":"ready","reason":null}]}
|
||||
```
|
||||
|
||||
`rotate` 是自动基线(不是用户覆盖值);用户覆盖值在 data.db 的 `rotate` 列。state 只用于
|
||||
恢复自动/调试,审核页不读它。
|
||||
|
||||
## 状态与失败
|
||||
|
||||
- `ready`:可直接用。
|
||||
- `review`:建议看一眼(掩膜回退、覆盖率低、纸型存疑)。
|
||||
- `needs_corners`:检测为空或校验不过,已用“画面内缩 3%”兜底框出件,需人工补角。
|
||||
- `needs_orientation`:接近正方形,横竖判不出来,需人在页面旋转。
|
||||
- `render_failed` / `failed`:出页失败/原图打不开;后者会被跳过。
|
||||
- `deleted`:用户删除,不进出件。
|
||||
|
||||
单页检测/校验失败不再让整批失败;只有一页都出不了才 `ok:false`。检测到的无效四边形会
|
||||
退回兜底框,并把 `auto_corners` 一起换成兜底框,避免“恢复自动”把无效角点拖回来。
|
||||
|
||||
## 清理
|
||||
|
||||
审核页发布只保留最近 5 个批次的缩略图目录;新批次完成后删除更早的目录和 DB 记录。
|
||||
`batch_id` 重生成(apply)不会触发清理,避免把正在编辑的旧批次裁掉。
|
||||
@@ -1,159 +1,57 @@
|
||||
---
|
||||
name: document-photo-to-pdf
|
||||
description: 把手机拍摄的纸质文档照片拉正、去背景,按顺序合并为一个 PDF
|
||||
thinking: true
|
||||
description: 纸质文档(合同/发票/证件/书页)拍照/扫描件校正后,导出 PDF 和 out/ 下的单页 PNG 原图(未调色未缩放);自动找纸边、透视拉正、多页合并,附审核页微调
|
||||
visit_url: /app/{task_id}/doc-review/
|
||||
input_schema:
|
||||
type: object
|
||||
properties:
|
||||
images:
|
||||
type: array
|
||||
items: {type: string}
|
||||
description: 图片路径,按合并顺序;可只传一张
|
||||
output:
|
||||
type: string
|
||||
description: 输出 PDF 文件名,默认 文档照片_校正.pdf
|
||||
---
|
||||
|
||||
# 文档照片转 PDF
|
||||
|
||||
手机拍的纸质材料 → 一个版正字清的 PDF。适用于合同、票据、证照、手写稿等多页纸质材料,纸张尺寸不限。
|
||||
|
||||
## 怎么跑
|
||||
|
||||
本技能没有 `run` 入口,用 `terminal` 直接执行命令。脚本装在技能目录,而 terminal 的
|
||||
cwd 是会话任务目录——两者不是同一个目录,所以**脚本一律用绝对路径调用**,图片和
|
||||
`out/` 用任务目录里的相对路径:
|
||||
|
||||
```sh
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py <子命令> ...
|
||||
```json
|
||||
{"images": ["图1.jpg", "图2.jpg"], "output": "合同_校正.pdf"}
|
||||
```
|
||||
|
||||
## 铁律
|
||||
`images` 必填,按顺序合并;`output` 可选,默认 `文档照片_校正.pdf`。纸张、方向、调色、
|
||||
清晰度不要传,交给审核页。
|
||||
|
||||
> 以下 8 条为硬约束,违反任一条即视为任务失败:立即停手、向用户说明,不得"先改完再说"。
|
||||
## 返回值
|
||||
|
||||
1. **先说再做。** 每次调脚本前用一句话说明这次干什么(要标注步骤编号)。
|
||||
2. **坐标只能靠眼睛读,禁止一切自建脚本。** 不另造任何纸张处理的辅助的代码,一次性脚本也不行。确实无法达成目的需要新能力=先停下问用户,不许自己造。
|
||||
3. **不裁切、不放大、不逐角看。** 四个角在同一张网格图上一次读全,直接报 4 组数字。不为了"读准一点"造局部图、加刻度、逐个角反复看。
|
||||
4. **每张图读 2 次,且每次都要有目的。** 网格 1 次 + 角标 1 次,看完就交付。
|
||||
- **读之前先说清这次要确认什么。** 说不出目的就不读。"再看一眼确认一下"不算目的。
|
||||
- **不为读而读。** 允许重读,但必须是被动的、有明确对象的:图没送达/像素没收到、要核对某个具体数字或某条边。没有新对象就直接用手上已有的数字。
|
||||
- **只读这两类图。** 网格、角标。原图、局部放大图、成品图、自己另造的辅助图都不读。
|
||||
- 没把握就按"宁可外扩"给保守值,交给角标图验证,不在读图阶段磨。
|
||||
5. **一次成件,禁止试错。** 一条 `build` 出件 + 检验。一轮里每页最多跑 1 次 `build`;不调参、不重取角点、不换方法、不"先出一版看看"。不通过就报诊断,停下等确认。
|
||||
6. **只有用户明确说"再改"才允许修复。** 完整一轮 = `prepare` → `set`(每页一次)→ `build`(一次)→ 看角标 → 交付。这一轮内发现的任何偏差(框偏了、裁到内容、比例不对),一律只报诊断和坐标建议,**不自行改、不重跑、不补做**。用户在本轮交付后明确说"再改/重做"时才能动手,且还是只跑一轮,改完重新过角标关。
|
||||
7. **指标 ≠ 成品。** 校验只查"框内"像素,框漏掉的内容查不出来。角标图必须亲眼看。
|
||||
8. **违规即停。** 一旦意识到自己正在写脚本、造辅助图、二次 build 或擅自修复,立即停手,向用户说明,不带病交付。
|
||||
- `ok:true`:`summary` 一句话摘要,`pdf` 文件名,`images` 是 `out/` 下的原图路径;
|
||||
末尾附完整「访问地址」。把 PDF、地址和图片路径一起给用户。
|
||||
- `ok:false`:转述 `error.message` / `error.hint`,忽略访问地址。
|
||||
|
||||
### 交付前自检(有一项为"是"就停手报告)
|
||||
每次运行还会把校正后的单页原图(未调色、未缩放、无白边)导出到 `out/`,文件名与 PDF
|
||||
同名:`合同_校正.pdf` → `out/合同_校正-01.png`、`out/合同_校正-02.png`。不需要图片时
|
||||
忽略 `images` 即可。
|
||||
|
||||
- 本轮是否新建/修改过任何脚本或辅助图?
|
||||
- 本轮 `build` 是否跑超过 1 次?
|
||||
- 是否在用户没说过"再改"时自行调整过角点或重出件?
|
||||
- 每张图是否只读了网格、角标各 1 次?
|
||||
- 校验未过的项是否只报告、没自行修?
|
||||
|
||||
## 流程
|
||||
|
||||
### 1 · 出网格图
|
||||
|
||||
```sh
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py prepare 图1.jpg 图2.jpg
|
||||
```json
|
||||
{"ok":true,"summary":"3 页已生成;1 页建议在审核页确认;原始页图见 out/","pdf":"合同_校正.pdf","images":["out/合同_校正-01.png","out/合同_校正-02.png","out/合同_校正-03.png"]}
|
||||
访问地址:https://agent.ai.foresh.com/app/task-20261023-001/doc-review/
|
||||
```
|
||||
|
||||
不传图片名 = 取当前目录所有图片。输出 `out/N-网格-<名称>.png`(100px 红色主线带坐标刻度、50px 蓝色辅助线,两者同宽)。
|
||||
审核页可拖角、旋转、删页/排序、改纸张与清晰度;改动后点「重新生成 PDF」覆盖同名文件。
|
||||
|
||||
### 2 · 从网格图中读出角点,一张图一条命令
|
||||
## 失败与拍摄
|
||||
|
||||
顺序固定:左上 → 右上 → 右下 → 左下(屏幕坐标,原点左上)。
|
||||
- 图片打不开:确认路径与格式(jpg/png/webp/bmp/tif)。
|
||||
- 检测不到纸张:整张纸入画,四边留 3%~5% 背景,换反差大的桌面。
|
||||
- 只有全部页都出不了才 `ok:false`;单页失败会用兜底框出件,审核页会标出来。
|
||||
|
||||
**要找的是纸张本身真实的四个角**——纸面这个四边形的 4 个顶点,**坐标的质量直接决定最终成品的质量!**。
|
||||
## 边界
|
||||
|
||||
读数方法:
|
||||
1. 先找离目标点最近的红线(整百、带数字刻度)读基准值;
|
||||
2. 再识别目标点相对相邻蓝线(50px)的偏移;
|
||||
3. 把目的坐标复述成一句话,例如“右下:x 在 1000 红线右 52px,y 在 1700 红线下 22px → 1052,1722”。
|
||||
|
||||
读数要求:
|
||||
- 读之前先定好这一张要报的 4 组数字,看完立刻 `set` 落盘到 `out/corners.json`。
|
||||
- 角被遮挡:沿两条可见纸边延长交会出顶点。
|
||||
- 角出画面:顺两条边的趋势延长,坐标可为负或超界(脚本接受)。
|
||||
|
||||
写入坐标:
|
||||
```sh
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py set 图1.jpg \
|
||||
--quad "左上x,左上y;右上x,右上y;右下x,右下y;左下x,左下y"
|
||||
```
|
||||
- 一次只写一张图,脚本已强制拦截多图(防串页)。
|
||||
- 落盘 `out/corners.json`,同名键覆盖并打印旧值对比;`show` 可随时查看。
|
||||
- 报给用户的只能是 `--quad` 串或 4 组坐标数字,禁止"约 (140,390)、纸略倾斜"这类描述。
|
||||
|
||||
### 3 · 一条命令出件
|
||||
|
||||
```sh
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py build \
|
||||
--out 文档照片_校正.pdf --paper a4 --force
|
||||
```
|
||||
|
||||
内部依次:回画角标 → 角点几何校验 → 透视校正+按纸比例拉正+提亮 → 合并 PDF → PDF 渲染检验 → 打印汇总。
|
||||
|
||||
参数:
|
||||
- `--paper`:标准纸必须传 —— `a4`(0.707)、`letter`(0.773)、`16k`(0.712)、`b5`(0.704),或数字比例。票据、异形纸不确定就省略,保留原始比例。
|
||||
- `--force`:角点校验不过也出件(默认)。
|
||||
- 位置参数传图名:只出指定页,如 `build 图2.jpg --out 结果.pdf`(输出 PDF 只含这一页)。
|
||||
- `--dpi`:PDF 检验渲染 dpi,默认 100。
|
||||
|
||||
### 4 · 看角标图(强制,一次)
|
||||
|
||||
**角标图是决定图像处理质量的唯一标准!**
|
||||
|
||||
读 `out/N-角标-*.png`(原图 + 同一套网格 + 四角标注),四个角在同一张图上核对:
|
||||
1. 绿线是否压在纸边交界?
|
||||
2. 纸张上内容是否全在绿线框内?
|
||||
3. 4个角标是否都在纸张实际的角上!
|
||||
|
||||
**只看不改:** 发现问题只记进报告,记录修正前后角标坐标的表格、不重跑 `build`、不补造辅助图。
|
||||
|
||||
| 页码 | 实际使用的角点坐标 | 修正后的角点坐标 |
|
||||
|---|---|---|
|
||||
| 1 | x1,y1;x2,y2;x3,y3;x4,y4 | x1',y1';x2',y2';x3',y3';x4',y4' |
|
||||
|
||||
## 判定与报告
|
||||
|
||||
| 检查项 | 判据 | 不达标含义 |
|
||||
|---|---|---|
|
||||
| 页数 | 与源图数一致 | 合并出错 |
|
||||
| 纸面短边 | ≥ 533px @100dpi | 角点错或分辨率过低 |
|
||||
| 纸张比例 | 仅提示 | 传了 `--paper` 应≈1.00,偏差大说明角点有误 |
|
||||
| 边缘背景残留 | 边带亮度 ≥ 纸面 80% | 角点落在背景上,内收 3~5px |
|
||||
| 内容贴边 | 页边无成片文字级暗像素 | 内容被切,外扩 5px |
|
||||
|
||||
- **全部通过**:给产物路径、大小、页数、指标,加一句角标图上确认的结果。
|
||||
- **"外侧纸面占比 100%"一律当角点错误**,误差是几十到上百 px,回第 2 步重读,不是微调。
|
||||
- **不达标**:只报诊断+ 建议,**不自行改、不重跑、不换方法**。用户明确说"再改"后,才改动后重跑一轮,然后交用户决策。
|
||||
- **用户没发话就交付**:即使只差 1px、即使一眼看出能更好,也不许动。报告里可写建议,手不能动。
|
||||
|
||||
## 内容安全
|
||||
|
||||
**禁止用生成式模型处理这类图片。** 生成模型是重绘而非变换,会改写文档内容,且结果看起来完全正常。本流程只做纯几何变换 + 全局等比提亮,文字、金额、印章保持原始像素。
|
||||
|
||||
## 命令速查
|
||||
|
||||
下表省略公共前缀 `python3 /workspace/skills/document-photo-to-pdf/doc_deskew/`。
|
||||
|
||||
| 场景 | 命令 |
|
||||
|---|---|
|
||||
| 出网格图 | `pipeline.py prepare 图1.jpg 图2.jpg` |
|
||||
| 写角点(一张一次) | `pipeline.py set 图1.jpg --quad "x1,y1;x2,y2;x3,y3;x4,y4"` |
|
||||
| 查看已落盘 | `pipeline.py show` |
|
||||
| 出件 | `pipeline.py build --out 结果.pdf --paper a4` |
|
||||
| 只出指定页(PDF 只含该页) | `pipeline.py build 图2.jpg --out 结果.pdf` |
|
||||
| 复检已有 PDF(不重新出件) | `pipeline.py check 结果.pdf --expect-pages 2` |
|
||||
|
||||
底层模块(与 `pipeline.py` 同目录,由它调用,日常不用直接执行):
|
||||
|
||||
| 模块 | 职责 |
|
||||
|---|---|
|
||||
| `grid.py` | 画带坐标的网格图 |
|
||||
| `preview_quad.py` | 画角标图(叠网格) |
|
||||
| `verify_quad.py` | 角点几何校验 |
|
||||
| `homography.py` | 透视变换 + 提亮 |
|
||||
| `run.py` | 拉正 + 拼页(纯模块,无 CLI) |
|
||||
| `make_pdf.py` | 多页合成 PDF |
|
||||
| `check.py` | 成品 PDF 检验 |
|
||||
|
||||
## 已知边界
|
||||
|
||||
- 纸张出画面的部分无法还原,角点可外扩到画面外。
|
||||
- 不为凑标准比例拉坐标;比例偏离通常是取点不准或纸张非该规格。
|
||||
- 拍摄时四边各留 3%~5% 背景余量。
|
||||
- 纸出画面无法还原;角点可外扩。
|
||||
- 自动方向只判 90°,180° 在审核页旋转。
|
||||
- 默认按同批中位比例;严格 A4/16K/B5 在审核页选。
|
||||
- 支持 jpg/png/webp/bmp/tif;HEIC 不支持。
|
||||
- 一图一纸;同框多纸只取最大。
|
||||
- 建议 ≤20 页;更多或 300dpi 可能超 120s,可分批。
|
||||
- `out/` 里的原图是自然尺寸(不缩放、不调色),页与页尺寸可能不同;PDF 仍按纸张/批次统一。
|
||||
|
||||
@@ -0,0 +1,608 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""纸张边线检测:从一张已转正的 PIL 图里估四个纸角。
|
||||
|
||||
对外只有 detect(img, P) -> (corners, info):
|
||||
- img 是已做 exif_transpose 的 RGB 图;
|
||||
- corners 是原图像素坐标 [(x, y)],顺序 左上、右上、右下、左下;
|
||||
- info 里带覆盖率、纸面占比、回退标记,供上层做置信度告警。
|
||||
|
||||
设计要点:
|
||||
- 不打印、不落盘、不画图。检测失败返回 ([], info),由调用方决定是否终止。
|
||||
- 纸面掩膜保留两份:腐蚀过的只用于限定边缘搜索环带;回退几何用未腐蚀的
|
||||
原始掩膜,避免角点系统性内收(旧实现两次 erode 会让角点切进纸内)。
|
||||
- 回退时不再只用 4 个极值点连线,而是用整条掩膜边界按边拟合,再求交点。
|
||||
"""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 基础工具
|
||||
|
||||
def _odd(size):
|
||||
size = int(size)
|
||||
if size < 3:
|
||||
size = 3
|
||||
if size % 2 == 0:
|
||||
size += 1
|
||||
return size
|
||||
|
||||
|
||||
def _slide_reduce(a, k, op, axis):
|
||||
"""沿 axis 做窗口 k 的滑动 max/min,越界按中性值填充(与 PIL 的边缘复制等价)。
|
||||
|
||||
van Herk / Gil-Werman:块内前缀极值 + 块内后缀极值,O(1)/像素。
|
||||
二维 max/min 可分离,故行列两遍与二维滤波逐像素等价,
|
||||
但避开了 PIL rankfilter 的 O(k^2)/像素(k=35 时慢 30 倍以上)。
|
||||
"""
|
||||
a = np.moveaxis(np.asarray(a, dtype=bool).astype(np.uint8) * 255,
|
||||
axis, -1)
|
||||
M, N = a.shape
|
||||
r = k // 2
|
||||
n = N + 2 * r
|
||||
nb = (n + k - 1) // k
|
||||
m = nb * k
|
||||
p = np.full((M, m), 0 if op == "max" else 255, dtype=np.uint8)
|
||||
p[:, r:r + N] = a
|
||||
blk = p.reshape(M, nb, k)
|
||||
if op == "max":
|
||||
fwd = np.maximum.accumulate(blk, axis=-1).reshape(M, m)
|
||||
bwd = np.maximum.accumulate(blk[:, :, ::-1], axis=-1)[:, :, ::-1]
|
||||
bwd = bwd.reshape(M, m)
|
||||
out = np.maximum(bwd[:, :N], fwd[:, k - 1:k - 1 + N])
|
||||
else:
|
||||
fwd = np.minimum.accumulate(blk, axis=-1).reshape(M, m)
|
||||
bwd = np.minimum.accumulate(blk[:, :, ::-1], axis=-1)[:, :, ::-1]
|
||||
bwd = bwd.reshape(M, m)
|
||||
out = np.minimum(bwd[:, :N], fwd[:, k - 1:k - 1 + N])
|
||||
return np.moveaxis(out, -1, axis) > 127
|
||||
|
||||
|
||||
def dilate(mask, size):
|
||||
k = _odd(size)
|
||||
return _slide_reduce(_slide_reduce(mask, k, "max", 1), k, "max", 0)
|
||||
|
||||
|
||||
def erode(mask, size):
|
||||
k = _odd(size)
|
||||
return _slide_reduce(_slide_reduce(mask, k, "min", 1), k, "min", 0)
|
||||
|
||||
|
||||
def closing(mask, size):
|
||||
return erode(dilate(mask, size), size)
|
||||
|
||||
|
||||
def opening(mask, size):
|
||||
return dilate(erode(mask, size), size)
|
||||
|
||||
|
||||
def rgb_to_sv(arr):
|
||||
maxc = arr.max(axis=-1)
|
||||
minc = arr.min(axis=-1)
|
||||
delta = maxc - minc
|
||||
S = np.zeros_like(maxc, dtype=np.float32)
|
||||
np.divide(delta, maxc, out=S, where=maxc > 1e-6)
|
||||
return S.astype(np.float32), maxc.astype(np.float32)
|
||||
|
||||
|
||||
def luminance(arr):
|
||||
return (0.299 * arr[..., 0] + 0.587 * arr[..., 1]
|
||||
+ 0.114 * arr[..., 2]).astype(np.float32)
|
||||
|
||||
|
||||
def box_mean(img, r):
|
||||
"""O(1)/像素的方框均值(积分图),r 为半径。"""
|
||||
img = np.asarray(img, dtype=np.float32)
|
||||
H, W = img.shape
|
||||
pad = np.zeros((H + 1, W + 1), dtype=np.float32)
|
||||
pad[1:, 1:] = img
|
||||
ii = pad.cumsum(0).cumsum(1)
|
||||
ys = np.arange(H)
|
||||
xs = np.arange(W)
|
||||
y0 = np.clip(ys - r, 0, H)
|
||||
y1 = np.clip(ys + r + 1, 0, H)
|
||||
x0 = np.clip(xs - r, 0, W)
|
||||
x1 = np.clip(xs + r + 1, 0, W)
|
||||
# 边界裁剪等价于把积分图向外按边复制 r 行/列,这样四个角全部可用切片取,
|
||||
# 避开 np.ix_ 花式索引的 4 次全图 gather(快 3 倍以上)。
|
||||
k = 2 * r + 1
|
||||
pe = np.pad(ii, ((r, r + 1), (r, r + 1)), mode="edge")
|
||||
S = (pe[k:k + H, k:k + W] - pe[0:H, k:k + W]
|
||||
- pe[k:k + H, 0:W] + pe[0:H, 0:W])
|
||||
cnt = (y1 - y0)[:, None] * (x1 - x0)[None, :]
|
||||
return S / np.maximum(cnt, 1).astype(np.float32)
|
||||
|
||||
|
||||
def gaussian_blur_np(img, sigma):
|
||||
radius = int(np.ceil(sigma * 3))
|
||||
if radius < 1:
|
||||
return img.astype(np.float32)
|
||||
x = np.arange(-radius, radius + 1, dtype=np.float32)
|
||||
k = np.exp(-x * x / (2 * sigma * sigma))
|
||||
k /= k.sum()
|
||||
H, W = img.shape
|
||||
p = np.pad(img, radius, mode="edge")
|
||||
tmp = np.zeros((H, W), dtype=np.float32)
|
||||
for i, kv in enumerate(k):
|
||||
tmp += kv * p[radius:radius + H, i:i + W]
|
||||
p = np.pad(tmp, ((radius, radius), (0, 0)), mode="edge")
|
||||
out = np.zeros((H, W), dtype=np.float32)
|
||||
for i, kv in enumerate(k):
|
||||
out += kv * p[i:i + H, :]
|
||||
return out
|
||||
|
||||
|
||||
def convolve3x3(img, kernel):
|
||||
H, W = img.shape
|
||||
p = np.pad(img, 1, mode="edge")
|
||||
out = np.zeros((H, W), dtype=np.float32)
|
||||
for i in range(3):
|
||||
for j in range(3):
|
||||
kv = kernel[i, j]
|
||||
if kv != 0:
|
||||
out += kv * p[i:i + H, j:j + W]
|
||||
return out
|
||||
|
||||
|
||||
def sobel_magnitude(gray):
|
||||
kx = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], dtype=np.float32)
|
||||
ky = np.array([[-1, -2, -1], [0, 0, 0], [1, 2, 1]], dtype=np.float32)
|
||||
gx = convolve3x3(gray, kx)
|
||||
gy = convolve3x3(gray, ky)
|
||||
return np.sqrt(gx * gx + gy * gy)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 连通域 & 填孔
|
||||
|
||||
def downsample_any(mask, factor):
|
||||
if factor <= 1:
|
||||
return mask
|
||||
H, W = mask.shape
|
||||
H2, W2 = H // factor, W // factor
|
||||
return mask[:H2 * factor, :W2 * factor].reshape(
|
||||
H2, factor, W2, factor).any(axis=(1, 3))
|
||||
|
||||
|
||||
def upsample_nearest(mask_small, target_h, target_w):
|
||||
h, w = mask_small.shape
|
||||
yi = np.linspace(0, h - 1, target_h).astype(np.int32)
|
||||
xi = np.linspace(0, w - 1, target_w).astype(np.int32)
|
||||
return mask_small[np.ix_(yi, xi)]
|
||||
|
||||
|
||||
def _row_runs(mask):
|
||||
"""把二值掩膜拆成每行的连续段,返回 (row, start, end),end 不含。"""
|
||||
H, W = mask.shape
|
||||
pm = np.zeros((H, W + 2), dtype=np.int8)
|
||||
pm[:, 1:-1] = mask
|
||||
d = np.diff(pm, axis=1)
|
||||
ry, rs = np.nonzero(d == 1)
|
||||
_, re_ = np.nonzero(d == -1)
|
||||
return ry, rs, re_
|
||||
|
||||
|
||||
def _run_labels(ry, rs, re_, H, eight):
|
||||
"""行程级并查集:4/8 邻接,返回每个行程的连通分量标号。"""
|
||||
n = len(ry)
|
||||
parent = np.arange(n)
|
||||
|
||||
def find(a):
|
||||
while parent[a] != a:
|
||||
parent[a] = parent[parent[a]]
|
||||
a = parent[a]
|
||||
return a
|
||||
|
||||
row_start = np.searchsorted(ry, np.arange(H + 1))
|
||||
for y in range(H - 1):
|
||||
a0, a1 = row_start[y], row_start[y + 1]
|
||||
b0, b1 = row_start[y + 1], row_start[y + 2]
|
||||
i, j = a0, b0
|
||||
while i < a1 and j < b1:
|
||||
s1, e1 = rs[i], re_[i]
|
||||
s2, e2 = rs[j], re_[j]
|
||||
hit = (s2 <= e1 and s1 <= e2) if eight else (s2 < e1 and s1 < e2)
|
||||
if hit:
|
||||
ra, rb = find(i), find(j)
|
||||
if ra != rb:
|
||||
parent[rb] = ra
|
||||
if e1 <= e2:
|
||||
i += 1
|
||||
else:
|
||||
j += 1
|
||||
return np.array([find(i) for i in range(n)], dtype=np.int64)
|
||||
|
||||
|
||||
def fill_holes(mask_small):
|
||||
"""填掉不与画面边界连通的背景(4 邻接),行程 + 并查集实现。"""
|
||||
m = np.asarray(mask_small, dtype=bool)
|
||||
H, W = m.shape
|
||||
bg = ~m
|
||||
if not bg.any():
|
||||
return m.copy()
|
||||
ry, rs, re_ = _row_runs(bg)
|
||||
if len(ry) == 0:
|
||||
return m.copy()
|
||||
lab = _run_labels(ry, rs, re_, H, eight=False)
|
||||
touch = (ry == 0) | (ry == H - 1) | (rs == 0) | (re_ >= W)
|
||||
nlab = int(lab.max()) + 1
|
||||
edge = np.zeros(nlab, dtype=bool)
|
||||
np.logical_or.at(edge, lab, touch)
|
||||
sel = ~edge[lab]
|
||||
out = m.copy()
|
||||
for y, s, e in zip(ry[sel], rs[sel], re_[sel]):
|
||||
out[y, s:e] = True
|
||||
return out
|
||||
|
||||
|
||||
def largest_connected(mask_small):
|
||||
"""取最大 8 邻接连通域(行程 + 并查集,逐像素计数)。"""
|
||||
m = np.asarray(mask_small, dtype=bool)
|
||||
H, W = m.shape
|
||||
if not m.any():
|
||||
return np.zeros_like(m)
|
||||
ry, rs, re_ = _row_runs(m)
|
||||
lab = _run_labels(ry, rs, re_, H, eight=True)
|
||||
w = (re_ - rs).astype(np.int64)
|
||||
nlab = int(lab.max()) + 1
|
||||
size = np.bincount(lab, weights=w, minlength=nlab)
|
||||
best = int(np.argmax(size))
|
||||
sel = lab == best
|
||||
out = np.zeros((H, W), dtype=bool)
|
||||
for y, s, e in zip(ry[sel], rs[sel], re_[sel]):
|
||||
out[y, s:e] = True
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 纸面掩膜
|
||||
|
||||
def otsu_threshold(score):
|
||||
hist, bin_edges = np.histogram(score, bins=256, range=(0.0, 1.0))
|
||||
hist = hist.astype(np.float64)
|
||||
total = hist.sum()
|
||||
if total <= 0:
|
||||
return 0.5
|
||||
p = hist / total
|
||||
centers = (bin_edges[:-1] + bin_edges[1:]) * 0.5
|
||||
w0 = np.cumsum(p)
|
||||
w1 = 1.0 - w0
|
||||
m0 = np.cumsum(p * centers)
|
||||
mt = m0[-1]
|
||||
with np.errstate(divide="ignore", invalid="ignore"):
|
||||
mu0 = m0 / np.maximum(w0, 1e-12)
|
||||
mu1 = (mt - m0) / np.maximum(w1, 1e-12)
|
||||
between = w0 * w1 * (mu0 - mu1) ** 2
|
||||
between = np.nan_to_num(between, nan=-1.0)
|
||||
return float(centers[int(np.argmax(between))])
|
||||
|
||||
|
||||
def paper_masks(arr, P):
|
||||
"""返回 (raw, eroded)。
|
||||
|
||||
raw:未腐蚀的纸面掩膜,用于回退几何,避免角点内收。
|
||||
eroded:腐蚀后的掩膜,只用于限定边缘搜索环带。
|
||||
"""
|
||||
d = P["detect"]
|
||||
H, W = arr.shape[:2]
|
||||
S, V = rgb_to_sv(arr)
|
||||
score = (V * (1.0 - S)).astype(np.float32)
|
||||
th = max(otsu_threshold(score), d["paper_otsu_floor"])
|
||||
bright = (score > th) & (V > d["paper_v_min"]) & (S < d["paper_s_max"])
|
||||
|
||||
factor = max(1, min(H, W) // d["paper_mask_factor_div"])
|
||||
bs = downsample_any(bright, factor)
|
||||
|
||||
m = closing(bs, _odd(max(d["paper_close_min"],
|
||||
min(bs.shape) // d["paper_close_div"])))
|
||||
m = fill_holes(m)
|
||||
m = largest_connected(m)
|
||||
m = closing(m, _odd(max(d["paper_close_min"],
|
||||
min(bs.shape) // d["paper_close2_div"])))
|
||||
m = fill_holes(m)
|
||||
|
||||
raw_small = m
|
||||
eroded_small = erode(m, _odd(d["mask_erode_kernel"]))
|
||||
|
||||
if factor > 1:
|
||||
raw = upsample_nearest(raw_small, H, W)
|
||||
eroded = upsample_nearest(eroded_small, H, W)
|
||||
else:
|
||||
raw, eroded = raw_small, eroded_small
|
||||
return raw, eroded
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 边缘图
|
||||
|
||||
def boundary_band(paper, k):
|
||||
k = _odd(k)
|
||||
return dilate(paper, k) & ~erode(paper, k)
|
||||
|
||||
|
||||
def edge_map(arr, P):
|
||||
d = P["detect"]
|
||||
H = arr.shape[0]
|
||||
gray = luminance(arr)
|
||||
sigma = max(1.0, H / float(d["edge_sigma_div"]))
|
||||
blurred = gaussian_blur_np(gray, sigma)
|
||||
mag = sobel_magnitude(blurred)
|
||||
local = box_mean(mag, max(3, H // d["edge_box_div"]))
|
||||
th = np.maximum(d["edge_low"], local * d["edge_local_gain"])
|
||||
return mag > th
|
||||
|
||||
|
||||
def boundary_step_filter(arr, paper, P):
|
||||
"""跨纸边亮度阶跃:比较掩膜内侧/外侧小窗的 V 均值。
|
||||
|
||||
纸边是真实的明暗台阶,木纹/阴影纹理阶跃小。两级窗口:掩膜准确处用
|
||||
r_small,掩膜略有偏移处依靠 r_big 补上。
|
||||
"""
|
||||
d = P["detect"]
|
||||
_, V = rgb_to_sv(arr)
|
||||
p = paper.astype(np.float32)
|
||||
q = 1.0 - p
|
||||
|
||||
def side_means(r):
|
||||
din = box_mean(p, r)
|
||||
dout = box_mean(q, r)
|
||||
vin = box_mean(V * p, r) / np.maximum(din, 1e-6)
|
||||
vout = box_mean(V * q, r) / np.maximum(dout, 1e-6)
|
||||
ok = (din > 0.08) & (dout > 0.08)
|
||||
return np.where(ok, np.abs(vin - vout), 0.0)
|
||||
|
||||
step = np.maximum(side_means(d["step_r_small"]),
|
||||
side_means(d["step_r_big"]))
|
||||
return step > d["step_th"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 四角提取
|
||||
|
||||
def mask_boundary_points(mask, P):
|
||||
"""掩膜边界像素坐标 (N, 2),顺序 (x, y)。"""
|
||||
inner = erode(mask, _odd(P["detect"]["mask_erode_kernel"]))
|
||||
ys, xs = np.where(mask & ~inner)
|
||||
if len(xs) == 0:
|
||||
return np.zeros((0, 2), dtype=np.float32)
|
||||
return np.stack([xs, ys], axis=1).astype(np.float32)
|
||||
|
||||
|
||||
def _fit_line(pts):
|
||||
"""总体最小二乘拟直线,返回法向量 n 与 n·p = c。"""
|
||||
c0 = pts.mean(axis=0)
|
||||
q = pts - c0
|
||||
cov = (q.T @ q) / max(len(pts) - 1, 1)
|
||||
w, v = np.linalg.eigh(cov)
|
||||
direction = v[:, int(np.argmax(w))]
|
||||
n = np.array([-direction[1], direction[0]], dtype=np.float64)
|
||||
nn = np.linalg.norm(n)
|
||||
if nn < 1e-9:
|
||||
return np.array([0.0, 1.0]), float(c0[1])
|
||||
n = n / nn
|
||||
return n, float(n @ c0)
|
||||
|
||||
|
||||
def _robust_fit(pts, robust_px):
|
||||
n, c = _fit_line(pts)
|
||||
sel = pts
|
||||
for _ in range(2):
|
||||
res = np.abs(sel @ n - c)
|
||||
keep = sel[res <= max(robust_px, 1.0) * 2.5]
|
||||
if len(keep) < 40:
|
||||
break
|
||||
n, c = _fit_line(keep)
|
||||
sel = keep
|
||||
return n, c
|
||||
|
||||
|
||||
def _intersect(n1, c1, n2, c2):
|
||||
A = np.vstack([n1, n2])
|
||||
b = np.array([c1, c2], dtype=np.float64)
|
||||
try:
|
||||
return np.linalg.solve(A, b)
|
||||
except np.linalg.LinAlgError:
|
||||
return None
|
||||
|
||||
|
||||
def _side_line(p0, p1):
|
||||
dv = np.asarray(p1, dtype=np.float64) - np.asarray(p0, dtype=np.float64)
|
||||
L = float(np.linalg.norm(dv))
|
||||
if L < 1e-6:
|
||||
return None
|
||||
u = dv / L
|
||||
n = np.array([-u[1], u[0]], dtype=np.float64)
|
||||
return n, float(n @ np.asarray(p0, dtype=np.float64)), u, L
|
||||
|
||||
|
||||
def detect_quad_corners(paper_raw, points, P):
|
||||
"""从掩膜边界 + 边缘点估四角,返回 (corners, fallback_sides)。"""
|
||||
d = P["detect"]
|
||||
base = mask_boundary_points(paper_raw, P)
|
||||
if len(base) < 40:
|
||||
return [], [0, 1, 2, 3]
|
||||
|
||||
bx, by = base[:, 0], base[:, 1]
|
||||
anchors = np.array([
|
||||
base[int(np.argmin(bx + by))], # 左上
|
||||
base[int(np.argmax(bx - by))], # 右上
|
||||
base[int(np.argmax(bx + by))], # 右下
|
||||
base[int(np.argmin(bx - by))], # 左下
|
||||
], dtype=np.float64)
|
||||
|
||||
init = [_side_line(anchors[i], anchors[(i + 1) % 4]) for i in range(4)]
|
||||
|
||||
# 用整条掩膜边界重拟合每一边(排除两端,避开角点圆角)
|
||||
base_lines = []
|
||||
for i in range(4):
|
||||
sl = init[i]
|
||||
if sl is None:
|
||||
base_lines.append(None)
|
||||
continue
|
||||
n, c, u, L = sl
|
||||
p0 = anchors[i]
|
||||
dist = np.abs(base @ n - c)
|
||||
proj = (base - p0) @ u
|
||||
m = d["boundary_side_margin"]
|
||||
sel = base[(dist <= d["group_tol_px"]) &
|
||||
(proj >= L * m) & (proj <= L * (1.0 - m))]
|
||||
if len(sel) >= 40:
|
||||
n2, c2 = _robust_fit(sel, d["robust_px"])
|
||||
base_lines.append((n2, c2))
|
||||
else:
|
||||
base_lines.append((n, c))
|
||||
|
||||
final_lines = list(base_lines)
|
||||
fallback_sides = [0, 1, 2, 3]
|
||||
|
||||
if points is not None and len(points) >= d["min_edge_points"]:
|
||||
pts = np.asarray(points, dtype=np.float64)
|
||||
dists = []
|
||||
for line in base_lines:
|
||||
if line is None:
|
||||
dists.append(np.full(len(pts), 1e9))
|
||||
else:
|
||||
dists.append(np.abs(pts @ line[0] - line[1]))
|
||||
D = np.stack(dists, axis=1)
|
||||
idx = np.argmin(D, axis=1)
|
||||
dmin = D[np.arange(len(pts)), idx]
|
||||
|
||||
fallback_sides = []
|
||||
for i in range(4):
|
||||
line = base_lines[i]
|
||||
if line is None:
|
||||
fallback_sides.append(i)
|
||||
continue
|
||||
sel = pts[(idx == i) & (dmin <= d["group_tol_px"])]
|
||||
sl = init[i]
|
||||
if sl is not None:
|
||||
u, L = sl[2], sl[3]
|
||||
p0 = anchors[i]
|
||||
m = d["boundary_side_margin"]
|
||||
proj = (sel - p0) @ u
|
||||
sel = sel[(proj >= L * m) & (proj <= L * (1.0 - m))]
|
||||
if len(sel) >= d["min_edge_points"]:
|
||||
n2, c2 = _robust_fit(sel, d["robust_px"])
|
||||
final_lines[i] = (n2, c2)
|
||||
else:
|
||||
fallback_sides.append(i)
|
||||
|
||||
H, W = paper_raw.shape
|
||||
oob = d["corner_oob_frac"]
|
||||
out = []
|
||||
for i in range(4):
|
||||
chosen = None
|
||||
for lines in (final_lines, base_lines):
|
||||
l1, l2 = lines[i - 1], lines[i]
|
||||
if l1 is None or l2 is None:
|
||||
continue
|
||||
det = l1[0][0] * l2[0][1] - l2[0][0] * l1[0][1]
|
||||
if abs(det) < 1e-3:
|
||||
continue
|
||||
p = _intersect(l1[0], l1[1], l2[0], l2[1])
|
||||
if p is None:
|
||||
continue
|
||||
if (-oob * W <= p[0] <= (1.0 + oob) * W and
|
||||
-oob * H <= p[1] <= (1.0 + oob) * H):
|
||||
chosen = (float(p[0]), float(p[1]))
|
||||
break
|
||||
if chosen is None:
|
||||
chosen = (float(anchors[i][0]), float(anchors[i][1]))
|
||||
if i not in fallback_sides:
|
||||
fallback_sides.append(i)
|
||||
out.append(chosen)
|
||||
|
||||
return out, sorted(set(fallback_sides))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 覆盖率
|
||||
|
||||
def border_coverage_exact(edges, paper, tol):
|
||||
"""沿掩膜逐行/逐列的真实边界测边缘覆盖率。"""
|
||||
H, W = paper.shape
|
||||
col_has = paper.any(axis=0)
|
||||
row_has = paper.any(axis=1)
|
||||
top_hit = top_n = 0
|
||||
bot_hit = bot_n = 0
|
||||
for x in np.where(col_has)[0]:
|
||||
ys = np.where(paper[:, x])[0]
|
||||
y0, y1 = ys.min(), ys.max()
|
||||
if edges[max(0, y0 - tol):min(H, y0 + tol + 1), x].any():
|
||||
top_hit += 1
|
||||
top_n += 1
|
||||
if edges[max(0, y1 - tol):min(H, y1 + tol + 1), x].any():
|
||||
bot_hit += 1
|
||||
bot_n += 1
|
||||
left_hit = left_n = 0
|
||||
right_hit = right_n = 0
|
||||
for y in np.where(row_has)[0]:
|
||||
xs = np.where(paper[y, :])[0]
|
||||
x0, x1 = xs.min(), xs.max()
|
||||
if edges[y, max(0, x0 - tol):min(W, x0 + tol + 1)].any():
|
||||
left_hit += 1
|
||||
left_n += 1
|
||||
if edges[y, max(0, x1 - tol):min(W, x1 + tol + 1)].any():
|
||||
right_hit += 1
|
||||
right_n += 1
|
||||
return {
|
||||
"top": top_hit / max(top_n, 1),
|
||||
"bottom": bot_hit / max(bot_n, 1),
|
||||
"left": left_hit / max(left_n, 1),
|
||||
"right": right_hit / max(right_n, 1),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 对外接口
|
||||
|
||||
def detect(img, P):
|
||||
"""检测纸张四角,返回 (corners_原图坐标, info)。失败返回 ([], info)。"""
|
||||
d = P["detect"]
|
||||
src_w, src_h = img.size
|
||||
scale = min(1.0, d["max_side"] / float(max(src_w, src_h)))
|
||||
if scale < 1.0:
|
||||
pil = img.resize((int(round(src_w * scale)), int(round(src_h * scale))),
|
||||
Image.LANCZOS)
|
||||
else:
|
||||
pil = img
|
||||
arr = np.asarray(pil).astype(np.float32) / 255.0
|
||||
H, W = arr.shape[:2]
|
||||
|
||||
paper_raw, paper_eroded = paper_masks(arr, P)
|
||||
frac = float(paper_raw.sum()) / float(H * W)
|
||||
|
||||
info = {
|
||||
"source_size": [int(src_w), int(src_h)],
|
||||
"processed_size": [int(W), int(H)],
|
||||
"scale": round(scale, 6),
|
||||
"paper_frac": round(frac, 4),
|
||||
"edge_pixels": 0,
|
||||
"coverage": {},
|
||||
"fallback": True,
|
||||
"fallback_sides": [0, 1, 2, 3],
|
||||
}
|
||||
|
||||
if frac < 0.005 or not paper_raw.any():
|
||||
return [], info
|
||||
|
||||
band = boundary_band(paper_eroded,
|
||||
_odd(max(d["band_min"], H // d["band_div"])))
|
||||
edges = edge_map(arr, P)
|
||||
step = boundary_step_filter(arr, paper_eroded, P)
|
||||
cand = closing(edges & band & step,
|
||||
_odd(max(d["cand_close_min"], H // d["cand_close_div"])))
|
||||
cand = opening(cand,
|
||||
_odd(max(d["cand_open_min"], H // d["cand_open_div"])))
|
||||
|
||||
ey, ex = np.nonzero(cand)
|
||||
pts = (np.stack([ex, ey], axis=1).astype(np.float32)
|
||||
if len(ex) >= d["min_edge_points"] else None)
|
||||
|
||||
corners, fallback_sides = detect_quad_corners(paper_raw, pts, P)
|
||||
info["edge_pixels"] = int(cand.sum())
|
||||
info["fallback_sides"] = fallback_sides
|
||||
info["fallback"] = bool(fallback_sides)
|
||||
if not corners:
|
||||
return [], info
|
||||
|
||||
info["coverage"] = {
|
||||
k: float(v) for k, v in border_coverage_exact(
|
||||
cand, paper_eroded,
|
||||
tol=max(d["coverage_tol_min"], H // d["coverage_tol_div"])).items()
|
||||
}
|
||||
src_pts = [(float(x / scale), float(y / scale)) for x, y in corners]
|
||||
return src_pts, info
|
||||
@@ -1,20 +1,11 @@
|
||||
"""PDF 产出的通用质量检验(渲染真实页面后评估)。
|
||||
# -*- coding: utf-8 -*-
|
||||
"""PDF 产出的数值检验:渲染真实页面后评估,不画图、不修改文件。
|
||||
|
||||
检验对象是最终交付的 PDF,不是中间图:把每页按固定 DPI 渲染成位图,
|
||||
在渲染结果上做统计判定,所见即交付物。
|
||||
|
||||
检验项:
|
||||
1. 页数与源图数一致
|
||||
2. 有效内容分辨率(页面渲染后纸面区域的像素尺寸)
|
||||
3. 纸张比例失真(纸面外接框的宽高比,对照页面书写比例)
|
||||
4. 纸面外背景残留(边带亮度显著低于纸面)
|
||||
5. 内容贴边被切(页边存在成片文字级暗像素)
|
||||
|
||||
判据全部基于该页自身像素分布自适应(Otsu / 分位数),不含针对特定文档的常量。
|
||||
|
||||
用法:
|
||||
python3 doc_deskew/check.py 结果.pdf [--expect-pages 2] [--dpi 100]
|
||||
python3 doc_deskew/check.py 结果.pdf --render # 导出渲染图供目视
|
||||
关键修正(相对旧版):
|
||||
- “边缘背景残留 / 内容贴边”改为在 paper_box 裁出的纸面区域上量。
|
||||
旧版量的是 PDF 页面最外圈,而正常出件四周是白边,这两项恒为通过。
|
||||
- 渲染页按文件名数字自然排序,≥10 页不会出现 1,10,2 的顺序。
|
||||
- 渲染后端增加 PyMuPDF(executor 镜像已装 pymupdf),pdftoppm 不在时也能跑。
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
@@ -25,23 +16,26 @@ import sys
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
from params import P # noqa: E402
|
||||
|
||||
def render(pdf, dpi=100, outdir='.tmp/pdfcheck'):
|
||||
os.makedirs(outdir, exist_ok=True)
|
||||
for f in os.listdir(outdir):
|
||||
if f.startswith('page'):
|
||||
os.remove(os.path.join(outdir, f))
|
||||
base = os.path.join(outdir, 'page')
|
||||
if shutil.which('pdftoppm'):
|
||||
subprocess.run(['pdftoppm', '-r', str(dpi), '-png', pdf, base], check=True)
|
||||
else: # 退路:pypdfium2
|
||||
import pypdfium2 as pdfium
|
||||
doc = pdfium.PdfDocument(pdf)
|
||||
scale = dpi / 72
|
||||
for i in range(len(doc)):
|
||||
doc[i].render(scale=scale).to_pil().save('%s-%d.png' % (base, i + 1))
|
||||
files = sorted(f for f in os.listdir(outdir) if f.startswith('page') and f.endswith('.png'))
|
||||
return [os.path.join(outdir, f) for f in files]
|
||||
|
||||
def _natural_key(path):
|
||||
name = os.path.basename(path)
|
||||
out = []
|
||||
cur = ""
|
||||
for ch in name:
|
||||
if ch.isdigit():
|
||||
cur += ch
|
||||
else:
|
||||
if cur:
|
||||
out.append((1, int(cur)))
|
||||
cur = ""
|
||||
out.append((0, ch.lower()))
|
||||
if cur:
|
||||
out.append((1, int(cur)))
|
||||
return out
|
||||
|
||||
|
||||
def otsu(vals):
|
||||
@@ -55,148 +49,238 @@ def otsu(vals):
|
||||
w1 = total - w0
|
||||
m0 = np.cumsum(hist * centers)
|
||||
mt = m0[-1]
|
||||
with np.errstate(divide='ignore', invalid='ignore'):
|
||||
with np.errstate(divide="ignore", invalid="ignore"):
|
||||
score = (mt * w0 / total - m0) ** 2 / (w0 * w1 / total ** 2)
|
||||
score = np.nan_to_num(score)
|
||||
return float(centers[int(np.argmax(score))])
|
||||
|
||||
|
||||
def analyse(img):
|
||||
g = np.asarray(img.convert('L')).astype(np.float32)
|
||||
def render(pdf, dpi=100, outdir=None):
|
||||
"""把 PDF 逐页渲染成图片,返回按页序排好的文件列表。
|
||||
|
||||
pdftoppm 默认输出 PPM(无损,且不压缩)。之前用 -png 时 zlib 压缩占了
|
||||
整个自检 2 秒里的 2 秒;改 PPM 后像素完全一致,耗时降到 0.1 秒量级。
|
||||
"""
|
||||
outdir = outdir or P["check"]["tmp_dir"]
|
||||
os.makedirs(outdir, exist_ok=True)
|
||||
for f in os.listdir(outdir):
|
||||
if f.startswith("page") and (f.endswith(".png") or f.endswith(".ppm")):
|
||||
try:
|
||||
os.remove(os.path.join(outdir, f))
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
base = os.path.join(outdir, "page")
|
||||
done = False
|
||||
if shutil.which("pdftoppm"):
|
||||
try:
|
||||
subprocess.run(["pdftoppm", "-r", str(dpi), pdf, base],
|
||||
check=True, stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL)
|
||||
done = True
|
||||
except Exception:
|
||||
done = False
|
||||
|
||||
if not done:
|
||||
try:
|
||||
import fitz # PyMuPDF
|
||||
doc = fitz.open(pdf)
|
||||
scale = float(dpi) / 72.0
|
||||
for i, page in enumerate(doc):
|
||||
pix = page.get_pixmap(matrix=fitz.Matrix(scale, scale))
|
||||
pix.save("%s-%d.png" % (base, i + 1))
|
||||
doc.close()
|
||||
except Exception:
|
||||
import pypdfium2 as pdfium
|
||||
doc = pdfium.PdfDocument(pdf)
|
||||
scale = float(dpi) / 72.0
|
||||
for i in range(len(doc)):
|
||||
doc[i].render(scale=scale).to_pil().save(
|
||||
"%s-%d.png" % (base, i + 1))
|
||||
|
||||
files = [os.path.join(outdir, f) for f in os.listdir(outdir)
|
||||
if f.startswith("page") and (f.endswith(".png")
|
||||
or f.endswith(".ppm"))]
|
||||
files.sort(key=_natural_key)
|
||||
return files
|
||||
|
||||
|
||||
def analyse(img, params=None):
|
||||
"""分析一张渲染页,返回纸面框、亮度、边缘与贴边指标。"""
|
||||
c = (params or P)["check"]
|
||||
g = np.asarray(img.convert("L")).astype(np.float32)
|
||||
h, w = g.shape
|
||||
res = {'w': w, 'h': h, 'ratio': w / h}
|
||||
res = {"w": int(w), "h": int(h), "ratio": float(w) / max(1, h)}
|
||||
|
||||
# 纸面亮度:取全图高分位(纸面是主体亮区)
|
||||
res['paper'] = float(np.percentile(g, 80))
|
||||
# 背景亮度:低分位
|
||||
res['back'] = float(np.percentile(g, 2))
|
||||
# 内容区阈值
|
||||
res['ink'] = min(otsu(g), res['paper'] * 0.8)
|
||||
|
||||
# 纸面内容框:A4 页面上纸面之外是纯白留白,故"非纯白"区域即实际纸面
|
||||
# 用相对亮度判定,避免把扫描白纸的轻微灰底误判成内容
|
||||
flat = float(np.percentile(g, 99.5))
|
||||
nonwhite = g < (flat - 8)
|
||||
# 先用“非纯白”找纸面框:PDF 页面四周是白边,纸面被提亮到 ~245
|
||||
flat = float(np.percentile(g, c["flat_pct"]))
|
||||
nonwhite = g < (flat - c["nonwhite_delta"])
|
||||
row_frac = nonwhite.mean(axis=1)
|
||||
col_frac = nonwhite.mean(axis=0)
|
||||
# 取占比超过 5% 的连续范围作为纸面主体
|
||||
rows = np.where(row_frac > 0.05)[0]
|
||||
cols = np.where(col_frac > 0.05)[0]
|
||||
rows = np.where(row_frac > c["paper_row_frac"])[0]
|
||||
cols = np.where(col_frac > c["paper_row_frac"])[0]
|
||||
box = None
|
||||
if rows.size > 4 and cols.size > 4:
|
||||
res['paper_box'] = (int(cols[0]), int(rows[0]), int(cols[-1]), int(rows[-1]))
|
||||
pw = cols[-1] - cols[0] + 1
|
||||
ph = rows[-1] - rows[0] + 1
|
||||
res['paper_ratio'] = pw / ph
|
||||
res['paper_px'] = (int(pw), int(ph))
|
||||
box = (int(cols[0]), int(rows[0]), int(cols[-1]), int(rows[-1]))
|
||||
res["paper_box"] = box
|
||||
res["box_fallback"] = box is None
|
||||
|
||||
if box:
|
||||
x0, y0, x1, y1 = box
|
||||
gp = g[y0:y1 + 1, x0:x1 + 1]
|
||||
else:
|
||||
res['paper_box'] = None
|
||||
res['paper_ratio'] = res['ratio']
|
||||
res['paper_px'] = (w, h)
|
||||
gp = g
|
||||
ph, pw = gp.shape
|
||||
|
||||
# 边缘背景残留:四周窄带的平均亮度明显低于纸面亮度
|
||||
# 边带宽度按比例取,过窄会漏判
|
||||
b = max(4, int(min(w, h) * 0.012))
|
||||
# 透视采样在纸边常带 1~2px 背景毛边,先内缩再量边缘;
|
||||
# 否则会被误判成“边缘背景残留”或“内容贴边”。
|
||||
inset = max(2, int(round(min(pw, ph) * c["edge_inset_frac"])))
|
||||
if pw > 2 * inset + 4 and ph > 2 * inset + 4:
|
||||
gc = gp[inset:-inset, inset:-inset]
|
||||
else:
|
||||
gc = gp
|
||||
gh, gw = gc.shape
|
||||
|
||||
res["paper"] = float(np.percentile(gc, c["paper_pct"]))
|
||||
res["back"] = float(np.percentile(gc, c["back_pct"]))
|
||||
res["ink"] = min(otsu(gc), res["paper"] * c["ink_ratio"])
|
||||
res["paper_ratio"] = float(pw) / max(1, ph)
|
||||
res["paper_px"] = (int(pw), int(ph))
|
||||
|
||||
# 边缘背景残留:在纸面区域的最外圈量,而不是 PDF 白边
|
||||
b = max(4, int(min(gw, gh) * c["border_band_frac"]))
|
||||
border = {
|
||||
'top': float(g[:b, :].mean()), 'bottom': float(g[-b:, :].mean()),
|
||||
'left': float(g[:, :b].mean()), 'right': float(g[:, -b:].mean()),
|
||||
"top": float(gc[:b, :].mean()),
|
||||
"bottom": float(gc[-b:, :].mean()),
|
||||
"left": float(gc[:, :b].mean()),
|
||||
"right": float(gc[:, -b:].mean()),
|
||||
}
|
||||
res['border'] = border
|
||||
res['border_bad'] = [k for k, v in border.items() if v < res['paper'] * 0.80]
|
||||
# 另一路判据:边带内是否存在"整行整列偏暗"的条带(纸外背景)
|
||||
edge_dark = {}
|
||||
for k, band in (('top', g[:b, :]), ('bottom', g[-b:, :]),
|
||||
('left', g[:, :b]), ('right', g[:, -b:])):
|
||||
edge_dark[k] = float((band < res['paper'] * 0.6).mean())
|
||||
res['edge_dark'] = edge_dark
|
||||
res["border"] = border
|
||||
res["border_bad"] = [k for k, v in border.items()
|
||||
if v < res["paper"] * c["border_ratio"]]
|
||||
|
||||
# 内容贴边:最外窄带内成片文字级暗像素 <=> 内容被切
|
||||
m = max(2, int(min(w, h) * 0.003))
|
||||
edge_dark = {}
|
||||
for k, band in (("top", gc[:b, :]), ("bottom", gc[-b:, :]),
|
||||
("left", gc[:, :b]), ("right", gc[:, -b:])):
|
||||
edge_dark[k] = float(
|
||||
(band < res["paper"] * c["edge_dark_ratio"]).mean())
|
||||
res["edge_dark"] = edge_dark
|
||||
|
||||
# 内容贴边:纸面最外窄带内成片文字级暗像素
|
||||
m = max(2, int(min(gw, gh) * c["cut_band_frac"]))
|
||||
cut = []
|
||||
for k, band in (('top', g[:m, :]), ('bottom', g[-m:, :]),
|
||||
('left', g[:, :m]), ('right', g[:, -m:])):
|
||||
frac = float((band < res['ink']).mean())
|
||||
if frac > 0.05:
|
||||
for k, band in (("top", gc[:m, :]), ("bottom", gc[-m:, :]),
|
||||
("left", gc[:, :m]), ("right", gc[:, -m:])):
|
||||
frac = float((band < res["ink"]).mean())
|
||||
if (frac > c["cut_frac"] and
|
||||
float(band.mean()) < res["paper"] * c["cut_mean_ratio"]):
|
||||
cut.append((k, frac))
|
||||
res['cut'] = cut
|
||||
res["cut"] = cut
|
||||
return res
|
||||
|
||||
|
||||
def inspect_pdf(pdf, expect_pages=None, dpi=None, params=None):
|
||||
"""渲染 + 逐页检验,返回结构化结果;渲染失败时 ok=None。"""
|
||||
p = params or P
|
||||
c = p["check"]
|
||||
dpi = int(dpi or c["dpi"])
|
||||
outdir = os.path.join(os.path.dirname(os.path.abspath(pdf)), c["tmp_dir"])
|
||||
try:
|
||||
files = render(pdf, dpi, outdir)
|
||||
except Exception as e:
|
||||
return {"ok": None, "error": str(e), "pages": [], "page_count": None}
|
||||
|
||||
actual = len(files)
|
||||
count_ok = (expect_pages is None) or (actual == expect_pages)
|
||||
out = {
|
||||
"ok": bool(count_ok),
|
||||
"page_count": {"expected": expect_pages, "actual": actual,
|
||||
"ok": bool(count_ok)},
|
||||
"pages": [],
|
||||
}
|
||||
|
||||
# 阈值以 150dpi 出件为基准;check 用的是较低的渲染 dpi,要同时按出件 dpi
|
||||
# 折算,否则选 300dpi 会白白放宽、选 72dpi 会误报“分辨率不足”。
|
||||
render_dpi = float(p["render"]["dpi"])
|
||||
min_short = c["min_short"] * dpi / render_dpi
|
||||
try:
|
||||
for i, path in enumerate(files, 1):
|
||||
with Image.open(path) as im:
|
||||
r = analyse(im, p)
|
||||
pw, ph = r["paper_px"]
|
||||
short = min(pw, ph)
|
||||
short_ok = short >= min_short
|
||||
border_ok = ((not r["border_bad"]) and
|
||||
all(v <= c["edge_dark_frac"]
|
||||
for v in r["edge_dark"].values()))
|
||||
cut_ok = not r["cut"]
|
||||
dev = (abs(r["paper_ratio"] - r["ratio"]) / r["ratio"]
|
||||
if r["ratio"] else 0.0)
|
||||
|
||||
item = {
|
||||
"index": i,
|
||||
"paper_px": [int(pw), int(ph)],
|
||||
"short_side_px": int(short),
|
||||
"short_side_ok": bool(short_ok),
|
||||
"paper_ratio": round(float(r["paper_ratio"]), 4),
|
||||
"page_ratio": round(float(r["ratio"]), 4),
|
||||
"ratio_dev_pct": round(float(dev * 100), 1),
|
||||
"border_ok": bool(border_ok),
|
||||
"cut_ok": bool(cut_ok),
|
||||
"border": {k: round(float(v), 1) for k, v in r["border"].items()},
|
||||
"edge_dark": {k: round(float(v), 3)
|
||||
for k, v in r["edge_dark"].items()},
|
||||
"cut": [[k, round(float(v), 3)] for k, v in r["cut"]],
|
||||
"box_fallback": bool(r["box_fallback"]),
|
||||
"paper_luma": round(float(r["paper"]), 1),
|
||||
"back_luma": round(float(r["back"]), 1),
|
||||
}
|
||||
out["pages"].append(item)
|
||||
if not (short_ok and border_ok and cut_ok):
|
||||
out["ok"] = False
|
||||
finally:
|
||||
# 渲染图只是自检的中间产物:PPM 未压缩,100dpi 的 A4 一页约 3MB,
|
||||
# 检验完立刻删(量已经记录在返回值里),不要留在会话目录占空间。
|
||||
shutil.rmtree(outdir, ignore_errors=True)
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('pdf')
|
||||
ap.add_argument('--expect-pages', type=int, default=None)
|
||||
ap.add_argument('--dpi', type=int, default=100)
|
||||
ap.add_argument('--min-short', type=int, default=800)
|
||||
ap.add_argument('--ratio-tol', type=float, default=0.08)
|
||||
ap.add_argument('--render', action='store_true')
|
||||
ap = argparse.ArgumentParser(description="复检已有 PDF(只渲染检验)")
|
||||
ap.add_argument("pdf")
|
||||
ap.add_argument("--expect-pages", type=int, default=None)
|
||||
ap.add_argument("--dpi", type=int, default=P["check"]["dpi"])
|
||||
args = ap.parse_args()
|
||||
|
||||
if not os.path.exists(args.pdf):
|
||||
raise SystemExit('找不到 PDF: %s' % args.pdf)
|
||||
raise SystemExit("找不到 PDF:%s" % args.pdf)
|
||||
|
||||
files = render(args.pdf, args.dpi)
|
||||
print('PDF: %s' % args.pdf)
|
||||
print('文件大小: %.0f KB' % (os.path.getsize(args.pdf) / 1024))
|
||||
print('页数: %d' % len(files))
|
||||
|
||||
ok = True
|
||||
if args.expect_pages is not None:
|
||||
good = len(files) == args.expect_pages
|
||||
ok &= good
|
||||
print(' [%s] 页数与源图一致(期望 %d,实际 %d)'
|
||||
% ('OK' if good else 'XX', args.expect_pages, len(files)))
|
||||
|
||||
for i, f in enumerate(files, 1):
|
||||
img = Image.open(f)
|
||||
r = analyse(img)
|
||||
print('\n第 %d 页 渲染 %dx%d px @%ddpi' % (i, r['w'], r['h'], args.dpi))
|
||||
|
||||
# 纸张实际像素尺寸(按渲染图里纸面外接框估算,换算回原分辨率)
|
||||
if r['paper_box']:
|
||||
x0, y0, x1, y1 = r['paper_box']
|
||||
scale = 1.0 # 渲染已是实际像素
|
||||
short = min(x1 - x0, y1 - y0)
|
||||
good = short >= args.min_short * args.dpi / 150.0
|
||||
ok &= good
|
||||
print(' [%s] 纸面短边 %d px(下限 %d)'
|
||||
% ('OK' if good else 'XX', short,
|
||||
int(args.min_short * args.dpi / 150.0)))
|
||||
|
||||
# 纸张比例本身不是硬性标准:非 A4 材料(Letter / 16K / 票据)
|
||||
# 塞进 A4 页面时必然有留白,纸张比例 != 页面比例是正常的。
|
||||
# 这里只做"提示",判错交给内容贴边与背景残留两项。
|
||||
dev = abs(r['paper_ratio'] - r['ratio']) / r['ratio'] * 100
|
||||
note = '' if dev <= args.ratio_tol else '(留白 %.1f%%,非 A4 纸张属正常)' % dev
|
||||
print(' [--] 纸张比例 %.3f(页面 %.3f)%s'
|
||||
% (r['paper_ratio'], r['ratio'], note))
|
||||
else:
|
||||
print(' [XX] 未能识别纸面区域')
|
||||
ok = False
|
||||
|
||||
bad_dark = [k for k, v in r.get('edge_dark', {}).items() if v > 0.35]
|
||||
good = (not r['border_bad']) and (not bad_dark)
|
||||
ok &= good
|
||||
detail = ', '.join('%s=%.0f' % (k, v) for k, v in r['border'].items())
|
||||
print(' [%s] 边缘背景残留 %s(纸面 %.0f / 背景 %.0f)'
|
||||
% ('OK' if good else 'XX',
|
||||
'无' if good else '、'.join(r['border_bad']), r['paper'], r['back']))
|
||||
if not good:
|
||||
print(' 边带亮度: %s' % detail)
|
||||
if bad_dark:
|
||||
print(' 偏高暗占比: %s'
|
||||
% '、'.join('%s %.0f%%' % (k, r['edge_dark'][k] * 100) for k in bad_dark))
|
||||
|
||||
good = not r['cut']
|
||||
ok &= good
|
||||
print(' [%s] 内容贴边 %s'
|
||||
% ('OK' if good else 'XX',
|
||||
'、'.join('%s %.1f%%' % (k, v * 100) for k, v in r['cut']) if r['cut'] else '无'))
|
||||
|
||||
print('\n总判定:%s' % ('全部通过' if ok else '存在不达标项'))
|
||||
if args.render:
|
||||
print('渲染图: .tmp/pdfcheck/')
|
||||
return 0 if ok else 1
|
||||
r = inspect_pdf(args.pdf, args.expect_pages, args.dpi)
|
||||
print("PDF: %s" % args.pdf)
|
||||
print("文件大小: %.0f KB" % (os.path.getsize(args.pdf) / 1024.0))
|
||||
if r.get("ok") is None:
|
||||
print("渲染失败:%s" % r.get("error"))
|
||||
return 1
|
||||
print("页数: %d" % r["page_count"]["actual"])
|
||||
for item in r["pages"]:
|
||||
print("\n第 %d 页 纸面 %dx%d" % (
|
||||
item["index"], item["paper_px"][0], item["paper_px"][1]))
|
||||
print(" [%s] 纸面短边 %d px"
|
||||
% ("OK" if item["short_side_ok"] else "XX",
|
||||
item["short_side_px"]))
|
||||
print(" [%s] 边缘背景残留 %s"
|
||||
% ("OK" if item["border_ok"] else "XX",
|
||||
"无" if item["border_ok"] else "有"))
|
||||
print(" [%s] 内容贴边 %s"
|
||||
% ("OK" if item["cut_ok"] else "XX",
|
||||
"无" if item["cut_ok"] else
|
||||
"、".join("%s %.1f%%" % (k, v * 100)
|
||||
for k, v in item["cut"])))
|
||||
print(" [--] 纸面比例 %.3f(页面 %.3f)"
|
||||
% (item["paper_ratio"], item["page_ratio"]))
|
||||
print("\n总判定:%s" % ("全部通过" if r["ok"] else "存在不达标项"))
|
||||
return 0 if r["ok"] else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""出页与合成:批次方向、目标尺寸、页面拼版、PDF 保存。
|
||||
|
||||
坐标与尺寸约定:
|
||||
- 所有 quad 都是“转正后原图”坐标;
|
||||
- 目标尺寸把 PDF 画布内容区(页面尺寸 - 四周留白)的一条边填满,可放大低分辨率照片,
|
||||
避免同一批页面因原图像素多少而大小不一;
|
||||
- raw 页图(裁剪 + 透视拉正、自然尺寸、不调色)由 warp_raw 产出,PDF 页从它派生。
|
||||
"""
|
||||
import os
|
||||
import statistics
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from homography import brighten, warp_perspective, warp_perspective_pil
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 纸张
|
||||
|
||||
def resolve_paper(paper, P):
|
||||
"""把 paper 参数解析成 {name, canvas_mm, ratio}。
|
||||
|
||||
auto 不指定标准纸面比例;画布用 A4,比例由调用方按批次中位数决定
|
||||
a4/letter… 画布和纸面比例都用该标准纸
|
||||
数字 画布用 A4,纸面比例用该数字(宽/高)
|
||||
"""
|
||||
r = P["render"]
|
||||
name = str(paper or r["default_paper"]).strip().lower()
|
||||
a4 = r["papers_mm"]["a4"]
|
||||
if name in ("", "auto"):
|
||||
return {"name": "auto", "canvas_mm": a4, "ratio": None}
|
||||
if name in r["papers_mm"]:
|
||||
w_mm, h_mm = r["papers_mm"][name]
|
||||
return {"name": name, "canvas_mm": (w_mm, h_mm),
|
||||
"ratio": float(w_mm) / float(h_mm)}
|
||||
try:
|
||||
ratio = float(name)
|
||||
except ValueError:
|
||||
raise ValueError("未知纸张:%s(可选 %s / 数字宽高比 / auto)"
|
||||
% (paper, "/".join(r["papers_mm"])))
|
||||
if not (0.02 < ratio < 50.0):
|
||||
raise ValueError("纸张比例超出合理范围:%s" % paper)
|
||||
return {"name": "custom", "canvas_mm": a4, "ratio": ratio}
|
||||
|
||||
|
||||
def canvas_px(paper, orientation, P):
|
||||
"""按 dpi 换算画布像素;orientation=landscape 时交换宽高。"""
|
||||
w_mm, h_mm = paper["canvas_mm"]
|
||||
dpi = float(P["render"]["dpi"])
|
||||
cw = int(round(w_mm / 25.4 * dpi))
|
||||
ch = int(round(h_mm / 25.4 * dpi))
|
||||
if orientation == "landscape":
|
||||
cw, ch = ch, cw
|
||||
return max(1, cw), max(1, ch)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 方向
|
||||
|
||||
def aspect_of(quad):
|
||||
"""四边形在屏幕坐标下的自然宽高(像素)。"""
|
||||
q = np.asarray(quad, dtype=np.float64)
|
||||
w = (np.linalg.norm(q[1] - q[0]) + np.linalg.norm(q[2] - q[3])) / 2.0
|
||||
h = (np.linalg.norm(q[3] - q[0]) + np.linalg.norm(q[2] - q[1])) / 2.0
|
||||
return max(float(w), 1e-6), max(float(h), 1e-6)
|
||||
|
||||
|
||||
def classify(aspect, deadband):
|
||||
if aspect > 1.0 + deadband:
|
||||
return "landscape"
|
||||
if aspect < 1.0 - deadband:
|
||||
return "portrait"
|
||||
return "square"
|
||||
|
||||
|
||||
def decide_orientation(aspects, mode, P):
|
||||
"""返回 (batch_orientation, source, median_aspect)。"""
|
||||
dead = P["gate"]["aspect_deadband"]
|
||||
median = float(statistics.median(aspects)) if aspects else 1.0
|
||||
if mode == "none":
|
||||
return "mixed", "none", median
|
||||
if mode in ("portrait", "landscape"):
|
||||
return mode, "forced", median
|
||||
classes = [classify(a, dead) for a in aspects]
|
||||
n_land = sum(1 for c in classes if c == "landscape")
|
||||
n_port = sum(1 for c in classes if c == "portrait")
|
||||
if n_land > n_port:
|
||||
return "landscape", "majority", median
|
||||
if n_port > n_land:
|
||||
return "portrait", "majority", median
|
||||
if n_land == 0 and n_port == 0:
|
||||
return "portrait", "default", median
|
||||
return "portrait", "tie", median
|
||||
|
||||
|
||||
def rotate_quad_cw(quad):
|
||||
"""把角点序列旋转一位,等价于把页面顺时针转 90°。"""
|
||||
q = list(quad)
|
||||
return [q[3], q[0], q[1], q[2]]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 目标尺寸
|
||||
|
||||
def target_size(quad, paper, page_orientation, P, out_ratio=None):
|
||||
"""返回 (target_w, target_h, (canvas_w, canvas_h))。
|
||||
|
||||
out_ratio 是 auto 模式下按批次中位比例算出的输出宽高比;None 表示按本页
|
||||
自然比例。两种分支都把内容区的一条边填满:低分辨率照片会被放大,从而
|
||||
避免同一批页面因原图尺寸/分辨率不同而大小不一。
|
||||
"""
|
||||
w_src, h_src = aspect_of(quad)
|
||||
cw, ch = canvas_px(paper, page_orientation, P)
|
||||
margin = int(P["render"]["margin_px"])
|
||||
cont_w = max(1, cw - 2 * margin)
|
||||
cont_h = max(1, ch - 2 * margin)
|
||||
|
||||
if paper["ratio"] is not None:
|
||||
# 强制纸面比例:宽高比纠正透视压缩。
|
||||
r = (paper["ratio"] if page_orientation == "portrait"
|
||||
else 1.0 / paper["ratio"])
|
||||
else:
|
||||
r = out_ratio
|
||||
|
||||
if r is not None:
|
||||
tw = float(cont_w)
|
||||
th = tw / r
|
||||
if th > cont_h:
|
||||
th = float(cont_h)
|
||||
tw = th * r
|
||||
else:
|
||||
# auto 且未形成批次比例:按四边形自然比例填满内容区(可放大)。
|
||||
s = min(float(cont_w) / w_src, float(cont_h) / h_src)
|
||||
tw, th = w_src * s, h_src * s
|
||||
|
||||
return max(1, int(round(tw))), max(1, int(round(th))), (cw, ch)
|
||||
|
||||
|
||||
def warp_raw(img, quad):
|
||||
"""只做裁剪 + 透视拉正:自然尺寸、不调色。raw 页图的唯一来源。"""
|
||||
return warp_perspective_pil(img, quad, None)
|
||||
|
||||
|
||||
def page_for_pdf(raw, size, P):
|
||||
"""从 raw 页图派生 PDF 页:先调色,再按目标尺寸缩放(LANCZOS)。
|
||||
|
||||
与旧 warp_page 的差别只在采样顺序:先在自然尺寸做透视(超采样),
|
||||
再缩到画布内容区,缩小时比直接按目标尺寸采样更少混叠。
|
||||
"""
|
||||
r = P["render"]
|
||||
spec = P["enhance"].get(r.get("enhance") or r["default_enhance"])
|
||||
page = tone(raw, spec, r["brighten_target"], r["brighten_pct"])
|
||||
if page.size != tuple(size):
|
||||
page = page.resize(tuple(size), Image.LANCZOS)
|
||||
return page
|
||||
|
||||
|
||||
def warp_page(img, quad, size, P):
|
||||
"""旧路径:直接按目标尺寸 warp + 调色。PIL 实现出问题时回退用。"""
|
||||
r = P["render"]
|
||||
page = warp_perspective(img, quad, size,
|
||||
chunk_rows=r["warp_chunk_rows"])
|
||||
spec = P["enhance"].get(r.get("enhance") or r["default_enhance"])
|
||||
return tone(page, spec, r["brighten_target"], r["brighten_pct"])
|
||||
|
||||
|
||||
def tone(img, spec, target=245, pct=97):
|
||||
"""按调色档做整体映射;spec=None 表示原样返回。
|
||||
|
||||
只做“分位数 -> 目标亮度”的等比缩放(加可选灰度),不碰像素间的
|
||||
局部关系,所以同一张图重跑结果一致,也不会重绘文字边缘。
|
||||
"""
|
||||
if not spec:
|
||||
return img
|
||||
if spec.get("gray"):
|
||||
img = img.convert("L").convert("RGB")
|
||||
return brighten(img, spec.get("target", target), spec.get("pct", pct))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- PDF
|
||||
|
||||
def build_pdf(pages, out_pdf, canvases, P):
|
||||
"""把已校正的页面图居中放进各自画布,按顺序写 PDF。
|
||||
|
||||
pages / canvases 一一对应;画布可能因 orientation=none 而不同。
|
||||
"""
|
||||
r = P["render"]
|
||||
margin = int(r["margin_px"])
|
||||
dpi = int(r["dpi"])
|
||||
composed = []
|
||||
meta = []
|
||||
|
||||
for img, (cw, ch) in zip(pages, canvases):
|
||||
cw, ch = max(1, int(cw)), max(1, int(ch))
|
||||
cont_w = max(1, cw - 2 * margin)
|
||||
cont_h = max(1, ch - 2 * margin)
|
||||
s = min(1.0, float(cont_w) / img.width, float(cont_h) / img.height)
|
||||
if s < 1.0:
|
||||
nw = max(1, int(round(img.width * s)))
|
||||
nh = max(1, int(round(img.height * s)))
|
||||
img = img.resize((nw, nh), Image.LANCZOS)
|
||||
else:
|
||||
nw, nh = img.width, img.height
|
||||
canvas = Image.new("RGB", (cw, ch), "white")
|
||||
canvas.paste(img, ((cw - nw) // 2, (ch - nh) // 2))
|
||||
composed.append(canvas)
|
||||
meta.append({"canvas": [cw, ch], "pasted": [nw, nh]})
|
||||
|
||||
if not composed:
|
||||
raise ValueError("没有可合成的页面")
|
||||
|
||||
out_dir = os.path.dirname(os.path.abspath(out_pdf))
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
tmp = out_pdf + ".part.pdf"
|
||||
try:
|
||||
composed[0].save(tmp, "PDF", resolution=dpi, save_all=True,
|
||||
append_images=composed[1:])
|
||||
os.replace(tmp, out_pdf)
|
||||
finally:
|
||||
if os.path.exists(tmp):
|
||||
try:
|
||||
os.remove(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
return meta
|
||||
@@ -1,45 +0,0 @@
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
import sys
|
||||
|
||||
FONT = '/usr/share/fonts/opentype/noto/NotoSerifCJK-Bold.ttc'
|
||||
|
||||
# 网格配色:100px 红线(主线,带坐标刻度)+ 50px 蓝线(辅助),两者同宽,
|
||||
# 只靠颜色分主次;线太细时缩略图上看不见,所以不靠粗细区分。
|
||||
COLOR_MAJOR = (255, 0, 0, 110)
|
||||
COLOR_MINOR = (0, 96, 255, 70)
|
||||
LINE_WIDTH = 2
|
||||
|
||||
|
||||
def draw_grid_on(im, step=50, major=100):
|
||||
"""在原图上叠加坐标网格,返回同一个 Image 对象(不落盘)。"""
|
||||
W, H = im.size
|
||||
d = ImageDraw.Draw(im, 'RGBA')
|
||||
try:
|
||||
f = ImageFont.truetype(FONT, 18)
|
||||
except Exception:
|
||||
f = ImageFont.load_default()
|
||||
for x in range(0, W, step):
|
||||
d.line([(x, 0), (x, H)],
|
||||
fill=COLOR_MAJOR if x % major == 0 else COLOR_MINOR,
|
||||
width=LINE_WIDTH)
|
||||
for y in range(0, H, step):
|
||||
d.line([(0, y), (W, y)],
|
||||
fill=COLOR_MAJOR if y % major == 0 else COLOR_MINOR,
|
||||
width=LINE_WIDTH)
|
||||
# 坐标数字:每 100px 在顶部和左侧标注
|
||||
for x in range(0, W, major):
|
||||
d.rectangle([x + 1, 1, x + 38, 21], fill=(255, 255, 255, 220))
|
||||
d.text((x + 3, 1), str(x), fill=(200, 0, 0), font=f)
|
||||
for y in range(0, H, major):
|
||||
d.rectangle([1, y + 1, 46, y + 21], fill=(255, 255, 255, 220))
|
||||
d.text((2, y + 1), str(y), fill=(0, 0, 200), font=f)
|
||||
return im
|
||||
|
||||
|
||||
def draw_grid(src, dst, step=50, major=100):
|
||||
im = draw_grid_on(Image.open(src).convert('RGB'), step, major)
|
||||
im.save(dst)
|
||||
print('written', dst, im.size)
|
||||
|
||||
if __name__ == '__main__':
|
||||
draw_grid(sys.argv[1], sys.argv[2])
|
||||
@@ -1,10 +1,11 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""纯 numpy 单应变换:等价于 cv2.warpPerspective,只依赖 Pillow + numpy。
|
||||
|
||||
用法:
|
||||
python homography.py in.jpg out.png --quad x1,y1 x2,y2 x3,y3 x4,y4
|
||||
四角顺序:左上、右上、右下、左下。不给 --quad 时按整幅图重采样。
|
||||
与旧实现的区别:
|
||||
- 按行分块采样,峰值内存从“整页 × 多份 float64”降到几十 MB;
|
||||
- 只接受 PIL 图,坐标/方向由调用方统一(exif_transpose 已在入口做过);
|
||||
- 不再提供 CLI,出件统一走技能入口。
|
||||
"""
|
||||
import argparse
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
@@ -18,64 +19,100 @@ def _solve_h(src, dst):
|
||||
return vt[-1].reshape(3, 3)
|
||||
|
||||
|
||||
def warp_perspective(img, quad, out_size=None, fill=(255, 255, 255)):
|
||||
arr = np.asarray(img.convert('RGB') if isinstance(img, Image.Image) else img)
|
||||
def natural_size(quad):
|
||||
"""四边形按自然边长对应的输出尺寸(与 warp_perspective 的 out_size=None 一致)。"""
|
||||
q = np.asarray(quad, dtype=np.float64)
|
||||
tl, tr, br, bl = q
|
||||
w = int(round(max(np.linalg.norm(tr - tl), np.linalg.norm(br - bl))))
|
||||
h = int(round(max(np.linalg.norm(bl - tl), np.linalg.norm(br - tr))))
|
||||
return max(1, w), max(1, h)
|
||||
|
||||
|
||||
def warp_perspective(img, quad, out_size=None, fill=(255, 255, 255),
|
||||
chunk_rows=256):
|
||||
"""把 quad 指定的四边形透视校正到 out_size。
|
||||
|
||||
quad 顺序:左上、右上、右下、左下(屏幕坐标)。
|
||||
out_size=None 时按四边形自然边长输出(保留照片原始比例)。
|
||||
"""
|
||||
arr = np.asarray(img if img.mode == "RGB" else img.convert("RGB"))
|
||||
src_h, src_w = arr.shape[:2]
|
||||
quad = np.asarray(quad, dtype=np.float64)
|
||||
tl, tr, br, bl = quad
|
||||
|
||||
if out_size is None:
|
||||
w = int(round(max(np.linalg.norm(tr - tl), np.linalg.norm(br - bl))))
|
||||
h = int(round(max(np.linalg.norm(bl - tl), np.linalg.norm(br - tr))))
|
||||
out_size = (w, h)
|
||||
out_w, out_h = out_size
|
||||
dst = np.array([[0, 0], [out_w - 1, 0], [out_w - 1, out_h - 1], [0, out_h - 1]],
|
||||
out_size = natural_size(quad)
|
||||
out_w = max(1, int(out_size[0]))
|
||||
out_h = max(1, int(out_size[1]))
|
||||
|
||||
dst = np.array([[0, 0], [out_w - 1, 0],
|
||||
[out_w - 1, out_h - 1], [0, out_h - 1]],
|
||||
dtype=np.float64)
|
||||
Hinv = np.linalg.inv(_solve_h(quad, dst))
|
||||
xs, ys = np.meshgrid(np.arange(out_w), np.arange(out_h))
|
||||
dst_h = np.stack([xs.ravel(), ys.ravel(), np.ones(xs.size)]).astype(np.float64)
|
||||
src_h = Hinv @ dst_h
|
||||
sx = src_h[0] / src_h[2]
|
||||
sy = src_h[1] / src_h[2]
|
||||
h, w = arr.shape[:2]
|
||||
valid = (sx >= 0) & (sx <= w - 1) & (sy >= 0) & (sy <= h - 1)
|
||||
sx = np.clip(sx, 0, w - 1); sy = np.clip(sy, 0, h - 1)
|
||||
x0 = np.floor(sx).astype(np.int32); x1 = np.minimum(x0 + 1, w - 1)
|
||||
y0 = np.floor(sy).astype(np.int32); y1 = np.minimum(y0 + 1, h - 1)
|
||||
fx = (sx - x0)[:, None]; fy = (sy - y0)[:, None]
|
||||
a = arr[y0, x0].astype(np.float64); b = arr[y0, x1].astype(np.float64)
|
||||
c = arr[y1, x0].astype(np.float64); d = arr[y1, x1].astype(np.float64)
|
||||
out = (a * (1 - fx) * (1 - fy) + b * fx * (1 - fy) +
|
||||
c * (1 - fx) * fy + d * fx * fy)
|
||||
out[~valid] = fill
|
||||
out = np.clip(out, 0, 255).astype(np.uint8).reshape(out_h, out_w, -1)
|
||||
|
||||
out = np.empty((out_h, out_w, 3), dtype=np.uint8)
|
||||
step = max(1, int(chunk_rows))
|
||||
for y0 in range(0, out_h, step):
|
||||
y1 = min(y0 + step, out_h)
|
||||
ys, xs = np.mgrid[y0:y1, 0:out_w]
|
||||
dst_h = np.stack([xs.ravel(), ys.ravel(),
|
||||
np.ones(xs.size)], axis=0).astype(np.float64)
|
||||
sp = Hinv @ dst_h
|
||||
den = sp[2]
|
||||
den = np.where(np.abs(den) < 1e-9, 1e-9, den)
|
||||
sx = sp[0] / den
|
||||
sy = sp[1] / den
|
||||
|
||||
valid = (sx >= 0) & (sx <= src_w - 1) & (sy >= 0) & (sy <= src_h - 1)
|
||||
sx = np.clip(sx, 0, src_w - 1)
|
||||
sy = np.clip(sy, 0, src_h - 1)
|
||||
|
||||
x0 = np.floor(sx).astype(np.int32)
|
||||
x1 = np.minimum(x0 + 1, src_w - 1)
|
||||
y0i = np.floor(sy).astype(np.int32)
|
||||
y1i = np.minimum(y0i + 1, src_h - 1)
|
||||
fx = (sx - x0)[:, None]
|
||||
fy = (sy - y0i)[:, None]
|
||||
|
||||
a = arr[y0i, x0].astype(np.float32)
|
||||
b = arr[y0i, x1].astype(np.float32)
|
||||
c = arr[y1i, x0].astype(np.float32)
|
||||
d = arr[y1i, x1].astype(np.float32)
|
||||
block = (a * (1 - fx) * (1 - fy) + b * fx * (1 - fy)
|
||||
+ c * (1 - fx) * fy + d * fx * fy)
|
||||
block[~valid] = fill
|
||||
out[y0:y1] = np.clip(block, 0, 255).astype(np.uint8).reshape(
|
||||
y1 - y0, out_w, 3)
|
||||
|
||||
return Image.fromarray(out)
|
||||
|
||||
|
||||
def trim(img, px=3):
|
||||
w, h = img.size
|
||||
px = max(0, min(px, (min(w, h) - 1) // 2))
|
||||
return img.crop((px, px, w - px, h - px))
|
||||
def warp_perspective_pil(img, quad, out_size=None, fill=(255, 255, 255)):
|
||||
"""PIL(C) 实现的透视校正,采样约定与 warp_perspective 一致。
|
||||
|
||||
out_size=None 时按四边形自然边长输出。10MP 约 0.3s,比上面的 numpy
|
||||
逐像素实现快约 10 倍;实测两者平均差 0.26 灰度级、89% 像素完全一致。
|
||||
只用于 raw 页图导出;PDF 出件仍走 numpy 版本,避免改变既有输出的像素。
|
||||
"""
|
||||
if out_size is None:
|
||||
out_size = natural_size(quad)
|
||||
ow = max(1, int(out_size[0]))
|
||||
oh = max(1, int(out_size[1]))
|
||||
dst = np.array([[0, 0], [ow - 1, 0], [ow - 1, oh - 1], [0, oh - 1]],
|
||||
dtype=np.float64)
|
||||
H = np.linalg.inv(_solve_h(np.asarray(quad, dtype=np.float64), dst))
|
||||
H = H / H[2, 2]
|
||||
coeffs = (H[0, 0], H[0, 1], H[0, 2],
|
||||
H[1, 0], H[1, 1], H[1, 2],
|
||||
H[2, 0], H[2, 1])
|
||||
src = img if img.mode == "RGB" else img.convert("RGB")
|
||||
return src.transform((ow, oh), Image.PERSPECTIVE, coeffs, Image.BILINEAR,
|
||||
fillcolor=fill)
|
||||
|
||||
|
||||
def brighten(img, target=245, pct=97):
|
||||
arr = np.asarray(img.convert('RGB')).astype(np.float32)
|
||||
hi = np.percentile(arr, pct)
|
||||
if hi > 1:
|
||||
arr = np.clip(arr * (target / hi), 0, 255)
|
||||
"""全局等比提亮:把 pct 分位映射到 target,保持相对对比度。"""
|
||||
arr = np.asarray(img if img.mode == "RGB" else img.convert("RGB")).astype(np.float32)
|
||||
hi = float(np.percentile(arr, pct))
|
||||
if hi > 1.0:
|
||||
arr = np.clip(arr * (float(target) / hi), 0, 255)
|
||||
return Image.fromarray(arr.astype(np.uint8))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('src'); ap.add_argument('dst')
|
||||
ap.add_argument('--quad', nargs=4, metavar='x,y')
|
||||
ap.add_argument('--trim', type=int, default=3)
|
||||
a = ap.parse_args()
|
||||
im = Image.open(a.src).convert('RGB')
|
||||
if a.quad:
|
||||
quad = [tuple(float(v) for v in p.split(',')) for p in a.quad]
|
||||
else:
|
||||
w, h = im.size
|
||||
quad = [(0, 0), (w - 1, 0), (w - 1, h - 1), (0, h - 1)]
|
||||
out = trim(brighten(warp_perspective(im, quad)), a.trim)
|
||||
out.save(a.dst)
|
||||
print('written %s %dx%d' % (a.dst, out.size[0], out.size[1]))
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
"""多张图等比缩放居中,合并为一个 PDF。默认 A4 纵向 @150dpi。"""
|
||||
import sys
|
||||
from PIL import Image
|
||||
|
||||
A4_W, A4_H = 1240, 1754
|
||||
MARGIN = 40
|
||||
DPI = 150
|
||||
|
||||
|
||||
def fit_page(path):
|
||||
im = Image.open(path).convert('RGB')
|
||||
bw, bh = A4_W - MARGIN * 2, A4_H - MARGIN * 2
|
||||
s = min(bw / im.width, bh / im.height)
|
||||
nw, nh = max(1, int(round(im.width * s))), max(1, int(round(im.height * s)))
|
||||
im = im.resize((nw, nh), Image.LANCZOS)
|
||||
page = Image.new('RGB', (A4_W, A4_H), 'white')
|
||||
page.paste(im, ((A4_W - nw) // 2, (A4_H - nh) // 2))
|
||||
return page
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
*srcs, out = sys.argv[1:]
|
||||
pages = [fit_page(p) for p in srcs]
|
||||
pages[0].save(out, 'PDF', resolution=DPI, save_all=True, append_images=pages[1:])
|
||||
print('written %s (%d pages)' % (out, len(pages)))
|
||||
@@ -0,0 +1,120 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""document-photo-to-pdf 的全部可调参数。
|
||||
|
||||
改这里之前先读这段:
|
||||
- 这些值大多沿用旧流程(人工读网格图)的经验值,**没有用真实样张重新标定**。
|
||||
没有样张时只改行为开关,不要凭感觉调阈值。
|
||||
- 像素类的值默认指“检测处理图”坐标(检测会先缩到 detect.max_side),不是原图像素;
|
||||
带 _frac / _pct / _div 的是比例。少数原图/画布坐标的值会在字段名里写明 _px。
|
||||
- 纸张尺寸用毫米,画布像素由 render.dpi 换算,避免 1240/1754 这类数字散落各处。
|
||||
"""
|
||||
P = {
|
||||
"schema": "document-photo-to-pdf/2",
|
||||
|
||||
# ---------------------------------------------------------------- 检测
|
||||
"detect": {
|
||||
"max_side": 1600, # 检测降采样长边;越大越准越慢
|
||||
"min_edge_points": 40, # 单边拟合所需最少点数,低于则回退掩膜线
|
||||
"group_tol_px": 22.0, # 边线点归属容差(处理图像素)
|
||||
"robust_px": 2.0, # 稳健拟合的残差基准(处理图像素)
|
||||
"mask_erode_kernel": 3, # 掩膜内收核;3 = 半径 1 像素
|
||||
"paper_otsu_floor": 0.50, # 纸面阈值下限,防止亮桌面被并入
|
||||
"paper_v_min": 0.50, # 纸面亮度硬下限
|
||||
"paper_s_max": 0.45, # 纸面饱和度上限
|
||||
"paper_mask_factor_div": 400, # 掩膜降采样倍数 = min(H,W)//该值
|
||||
"paper_close_div": 60, # 第一次闭运算核 = min(H,W)//该值
|
||||
"paper_close_min": 5, # 形态学核最小值
|
||||
"paper_close2_div": 80, # 第二次闭运算核 = min(H,W)//该值
|
||||
"band_div": 45, # 搜索环带宽度 = H//该值
|
||||
"band_min": 15, # 环带宽度下限
|
||||
"edge_local_gain": 1.55, # 边缘图局部阈值增益
|
||||
"edge_low": 0.035, # 边缘图绝对下限
|
||||
"edge_sigma_div": 800, # 高斯模糊 sigma = H//该值
|
||||
"edge_box_div": 50, # 局部均值窗口 = H//该值
|
||||
"step_r_small": 10, # 阶跃检测小窗半径
|
||||
"step_r_big": 24, # 阶跃检测大窗半径
|
||||
"step_th": 0.055, # 跨纸边亮度阶跃阈值
|
||||
"cand_close_div": 160, # 候选边线闭运算 = H//该值
|
||||
"cand_close_min": 5,
|
||||
"cand_open_div": 500, # 候选边线开运算 = H//该值
|
||||
"cand_open_min": 3,
|
||||
"coverage_tol_div": 110, # 覆盖率检测容差 = H//该值
|
||||
"coverage_tol_min": 10,
|
||||
"corner_oob_frac": 0.15, # 角点越界允许幅度(相对画面长边)
|
||||
"boundary_side_margin": 0.10, # 拟合单边时排除两端各 10%,避开角点
|
||||
},
|
||||
|
||||
# ---------------------------------------------------------------- 几何门槛
|
||||
"gate": {
|
||||
"oob_max_pct": 15.0, # 越界超过此值判失败
|
||||
"inner_paper_min": 0.60, # 框内纸面占比下限
|
||||
"inner_edge_min": 0.75, # 每边内侧纸面占比下限
|
||||
"outer_dark_min": 0.35, # 外侧暗像素占比低于此值只告警
|
||||
"coverage_warn": 0.40, # 四边覆盖率低于此值告警(未标定)
|
||||
"aspect_deadband": 0.10, # 0.9~1.1 视为接近正方形,不参与方向投票
|
||||
"paper_mismatch_pct": 20.0, # 与批次中位宽高比偏差超过此值告警
|
||||
"sample_n": 60, # 校验采样网格边长
|
||||
"sample_shrink": 0.88, # 框内采样内收比例
|
||||
"fallback_inset": 0.03, # 检测为空时的兜底矩形:画面短边向内缩的比例
|
||||
},
|
||||
|
||||
# ---------------------------------------------------------------- 调色档
|
||||
# 全局四档,对应审核页的“图像处理”。都是“整体等比映射 + 可选灰度”:
|
||||
# 不做二值化——二值化会吃拮抗锯齿,笔画变胖变瘦,等于重绘文字。
|
||||
"enhance": {
|
||||
"original": None, # 原样:不提亮
|
||||
"normal": {"target": 245, "pct": 97}, # 标准
|
||||
"strong": {"target": 252, "pct": 90}, # 加强:背景白得更彻底
|
||||
"bw": {"target": 252, "pct": 60, "gray": True}, # 黑白(保留灰阶)
|
||||
},
|
||||
|
||||
# ---------------------------------------------------------------- 出页
|
||||
"render": {
|
||||
"dpi": 150, # PDF 输出 dpi
|
||||
"margin_px": 40, # PDF 页面四周留白(150dpi 下的像素)
|
||||
"brighten_target": 245, # 提亮目标:97 分位映射到该亮度
|
||||
"brighten_pct": 97,
|
||||
"warp_chunk_rows": 256, # 透视变换分块行数,控制峰值内存
|
||||
"default_paper": "auto", # auto=按批次中位比例统一版式,A4 画布内留白
|
||||
"default_enhance": "normal", # 调色档默认值,可选 key 见 P["enhance"]
|
||||
"enhance": "normal", # 本次生效的调色档(run.py 按参数覆写副本)
|
||||
"dpi_options": {"standard": 150, "high": 300}, # 清晰度档名 -> dpi
|
||||
"papers_mm": {
|
||||
"a4": (210.0, 297.0),
|
||||
"letter": (215.9, 279.4),
|
||||
"16k": (185.0, 260.0),
|
||||
"b5": (176.0, 250.0),
|
||||
},
|
||||
},
|
||||
|
||||
# ---------------------------------------------------------------- PDF 检验
|
||||
"check": {
|
||||
"dpi": 100, # 检验渲染 dpi
|
||||
"min_short": 800, # 纸面短边下限(按 150dpi 基准,随 dpi 换算)
|
||||
"ratio_tol": 0.08, # 纸面比例偏差提示阈值(仅提示)
|
||||
"border_ratio": 0.80, # 边带亮度 / 纸面亮度 下限
|
||||
"edge_warnings": False, # 边缘背景残留/内容贴边是否升为 warning
|
||||
# (启发式、未标定,误报多;默认只留在 checks)
|
||||
"edge_dark_ratio": 0.60, # 边带“暗像素”亮度 / 纸面亮度
|
||||
"edge_dark_frac": 0.35, # 边带暗像素占比上限
|
||||
"cut_frac": 0.05, # 最外带内文字级暗像素占比上限
|
||||
"cut_mean_ratio": 0.70, # 贴边带均值低于纸面该比例才算成片暗
|
||||
"edge_inset_frac": 0.002, # 纸面框内缩比例,避开透视采样毛边
|
||||
"border_band_frac": 0.012, # 边带宽度 = 纸面短边 * 该值
|
||||
"cut_band_frac": 0.003, # 贴边检测带宽度 = 纸面短边 * 该值
|
||||
"paper_pct": 80, # 纸面亮度分位
|
||||
"back_pct": 2, # 背景亮度分位
|
||||
"ink_ratio": 0.80, # 墨色阈值上限 = 纸面亮度 * 该值
|
||||
"flat_pct": 99.5, # 页面纯白分位
|
||||
"nonwhite_delta": 8.0, # 低于纯白该值视为非白(用于找纸面框)
|
||||
"paper_row_frac": 0.05, # 行列非白占比阈值
|
||||
"tmp_dir": ".tmp/pdfcheck", # 检验渲染临时目录(相对输出目录,检验完即删)
|
||||
},
|
||||
|
||||
# ---------------------------------------------------------------- 输出
|
||||
"output": {
|
||||
"default_name": "文档照片_校正.pdf",
|
||||
"state_suffix": ".state.json",
|
||||
"max_logs": 20,
|
||||
},
|
||||
}
|
||||
@@ -1,416 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""文档照片 -> 校正 PDF:全流程串起来的一站式脚本。
|
||||
|
||||
把原来手工分步的活(网格图 -> 读角点 -> 落盘 -> 回画确认 -> 出 PDF -> 检验)
|
||||
压成几条命令,且**产物都落在 out/ 里按序号命名**:
|
||||
|
||||
out/1-网格-<原图名>.png 每页的坐标网格图(读角点用)
|
||||
out/1-角标-<原图名>.png 每页角点回画图(原图 + 网格 + 四角标注,核查是否贴纸边)
|
||||
out/1-成品-<原图名>.png 每页矫正后的页面图
|
||||
out/corners.json 角点坐标(落盘,跨会话可用)
|
||||
out/<名字>.pdf 最终 PDF
|
||||
|
||||
角点约定(顺序固定,别写反):
|
||||
左上 ; 右上 ; 右下 ; 左下 —— 屏幕坐标,原点在图片左上角,单位像素
|
||||
|
||||
子命令:
|
||||
prepare 出网格图 + 生成 corners.json 模板(坐标留空待填)
|
||||
set 把某一页的四角坐标写进 corners.json(可反复调用)
|
||||
build 回画角标图 + 校验 + 出 PDF + 出成品图 + PDF 检验,一次跑完
|
||||
all prepare + build 合体(角点已存在则直接 build)
|
||||
check 复检已有 PDF(只渲染检验,不重新出件)
|
||||
|
||||
典型用法(脚本用绝对路径调用:cwd 是会话任务目录,不是本目录):
|
||||
# 1) 出网格图
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py prepare 图1.jpg 图2.jpg
|
||||
# 2) 看 out/1-网格-图1.png,把坐标写进文件
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py set "图1.jpg" --quad "150,360;1020,355;1050,1690;90,1660"
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py set "图2.jpg" --quad "120,345;1010,345;1010,1710;120,1710"
|
||||
# 3) 一路出 PDF
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py build --out 合同_校正.pdf
|
||||
|
||||
# 想一步到位(角点已落盘):
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py all 图1.jpg 图2.jpg --out 合同_校正.pdf
|
||||
|
||||
# 列出 / 检查已落盘角点
|
||||
python3 /workspace/skills/document-photo-to-pdf/doc_deskew/pipeline.py show
|
||||
|
||||
退出码:0 全部通过;2 角点校验未通过(已出件但需人工判断);3 PDF 检验未通过。
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
# 提示里回显的命令:cwd 是会话任务目录,必须用脚本绝对路径,写相对的跑不起来
|
||||
SELF = 'python3 ' + os.path.join(HERE, 'pipeline.py')
|
||||
|
||||
OUT_DIR = 'out'
|
||||
CORNERS = os.path.join(OUT_DIR, 'corners.json')
|
||||
LABELS = ('左上', '右上', '右下', '左下')
|
||||
|
||||
IMG_EXT = ('.jpg', '.jpeg', '.png', '.webp', '.bmp', '.tif', '.tiff')
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 基础工具
|
||||
|
||||
def natural_key(name):
|
||||
return [int(t) if t.isdigit() else t.lower()
|
||||
for t in re.split(r'(\d+)', os.path.basename(name))]
|
||||
|
||||
|
||||
def quad_to_str(quad):
|
||||
return ';'.join('%g,%g' % (p[0], p[1]) for p in quad)
|
||||
|
||||
|
||||
def load_corners(path=CORNERS):
|
||||
if not os.path.exists(path):
|
||||
return {}
|
||||
with open(path, encoding='utf-8') as f:
|
||||
raw = json.load(f)
|
||||
out = {}
|
||||
for name, v in raw.items():
|
||||
if isinstance(v, dict):
|
||||
v = v.get('quad') or v.get('corners')
|
||||
if not v:
|
||||
continue
|
||||
out[name] = [(float(p[0]), float(p[1])) for p in v]
|
||||
return out
|
||||
|
||||
|
||||
def save_corners(data, path=CORNERS):
|
||||
os.makedirs(os.path.dirname(os.path.abspath(path)) or '.', exist_ok=True)
|
||||
tmp = path + '.tmp'
|
||||
with open(tmp, 'w', encoding='utf-8') as f:
|
||||
json.dump({k: [[int(c) if float(c).is_integer() else c for c in p]
|
||||
for p in v] for k, v in data.items()},
|
||||
f, ensure_ascii=False, indent=2)
|
||||
f.write('\n')
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def parse_quad(text):
|
||||
quad = []
|
||||
for pt in text.replace(';', ';').split(';'):
|
||||
pt = pt.strip()
|
||||
if not pt:
|
||||
continue
|
||||
xy = pt.replace(' ', '').split(',')
|
||||
if len(xy) != 2:
|
||||
raise SystemExit('角点格式不对:%r(应为 x,y)' % pt)
|
||||
try:
|
||||
quad.append((float(xy[0]), float(xy[1])))
|
||||
except ValueError:
|
||||
raise SystemExit('角点不是数字:%r' % pt)
|
||||
if len(quad) != 4:
|
||||
raise SystemExit('角点必须 4 个(左上;右上;右下;左下),实际 %d 个' % len(quad))
|
||||
return quad
|
||||
|
||||
|
||||
def collect_images(args):
|
||||
"""参数里的图片 + 当前目录全部图片(去重、按自然序)。"""
|
||||
names = list(args)
|
||||
if not names:
|
||||
names = [f for f in os.listdir('.')
|
||||
if f.lower().endswith(IMG_EXT) and os.path.isfile(f)]
|
||||
seen, out = set(), []
|
||||
for n in names:
|
||||
if n not in seen:
|
||||
seen.add(n)
|
||||
out.append(n)
|
||||
if not out:
|
||||
raise SystemExit('没有找到图片:可在命令后面直接列文件名')
|
||||
missing = [n for n in out if not os.path.exists(n)]
|
||||
if missing:
|
||||
raise SystemExit('图片不存在:%s' % ', '.join(missing))
|
||||
return sorted(out, key=natural_key)
|
||||
|
||||
|
||||
def page_tag(i):
|
||||
return '%d' % i
|
||||
|
||||
|
||||
def out_path(prefix, i, name, ext='.png'):
|
||||
base = os.path.splitext(os.path.basename(name))[0]
|
||||
return os.path.join(OUT_DIR, '%s-%s-%s%s' % (page_tag(i), prefix, base, ext))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- prepare
|
||||
|
||||
def cmd_prepare(images):
|
||||
os.makedirs(OUT_DIR, exist_ok=True)
|
||||
from grid import draw_grid
|
||||
data = load_corners()
|
||||
for i, name in enumerate(images, 1):
|
||||
dst = out_path('网格', i, name)
|
||||
draw_grid(name, dst)
|
||||
data.setdefault(name, None)
|
||||
save_corners({k: v for k, v in data.items() if v})
|
||||
print('\n网格图已写入 %s/' % OUT_DIR)
|
||||
print('下一步:照着网格图读数,把角点写进 %s:' % CORNERS)
|
||||
for i, name in enumerate(images, 1):
|
||||
print(' %s set "%s" --quad "左上x,左上y;右上x,右上y;'
|
||||
'右下x,右下y;左下x,左下y"' % (SELF, name))
|
||||
return 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- set / show
|
||||
|
||||
def cmd_set(args):
|
||||
"""一次只写一张图:参数是 图片 + 一个 --quad,不批配、不错位。"""
|
||||
if '|' in args.image or ':' in args.image:
|
||||
raise SystemExit('图片名里不要带 : 或 |:%s' % args.image)
|
||||
if not os.path.exists(args.image):
|
||||
raise SystemExit('图片不存在:%s' % args.image)
|
||||
if len(args.quad) != 1:
|
||||
raise SystemExit('set 只能给一个 --quad(实际 %d 个)。一张图一次。'
|
||||
% len(args.quad))
|
||||
|
||||
name = args.image
|
||||
quad_s = args.quad[0]
|
||||
|
||||
data = load_corners()
|
||||
old = data.get(name)
|
||||
data[name] = parse_quad(quad_s)
|
||||
save_corners(data)
|
||||
if old is not None:
|
||||
print('已更新 %-36s %s(原 %s)'
|
||||
% (name, quad_to_str(data[name]), quad_to_str(old)))
|
||||
else:
|
||||
print('已写入 %-36s %s' % (name, quad_to_str(data[name])))
|
||||
print('→ %s(已存:%s)'
|
||||
% (CORNERS, '、'.join(sorted(data, key=natural_key))))
|
||||
left = [n for n in data if not os.path.exists(n)]
|
||||
if left:
|
||||
print('[注意] 这些图在 corners.json 里但文件不存在:%s' % '、'.join(left))
|
||||
return 0
|
||||
|
||||
|
||||
def cmd_show():
|
||||
data = load_corners()
|
||||
if not data:
|
||||
print('%s 里还没有角点' % CORNERS)
|
||||
return 1
|
||||
for i, name in enumerate(sorted(data, key=natural_key), 1):
|
||||
print('%d. %-40s %s' % (i, name, quad_to_str(data[name])))
|
||||
return 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 角标图
|
||||
|
||||
def draw_marks(name, pts, dst):
|
||||
"""把四角画回原图(底图叠加与网格图相同的网格),供人工核查。"""
|
||||
from preview_quad import draw_quad
|
||||
im = draw_quad(name, pts, with_grid=True)
|
||||
im.save(dst)
|
||||
return dst
|
||||
|
||||
|
||||
def make_zooms(name, marked_png, idx, zoom):
|
||||
"""从角标图上裁出四侧放大块,命名同其它核查图的序号规则。"""
|
||||
from PIL import Image
|
||||
from preview_quad import zoom_boxes
|
||||
im = Image.open(marked_png)
|
||||
boxes = zoom_boxes(im, [(0, 0), (im.width, 0), (im.width, im.height), (0, im.height)])
|
||||
keys = list(boxes) if zoom == 'all' else [zoom]
|
||||
base = os.path.join(OUT_DIR, '%d-角标放大-%s' % (idx, os.path.basename(name)))
|
||||
base = os.path.splitext(base)[0]
|
||||
for k in keys:
|
||||
box = tuple(int(v) for v in boxes[k])
|
||||
if box[2] <= box[0] or box[3] <= box[1]:
|
||||
continue
|
||||
crop = im.crop(box)
|
||||
# 细长条按方向切成两段,否则 585x5760 没法看
|
||||
segs = []
|
||||
if k in ('left', 'right') and crop.height > crop.width * 2.2:
|
||||
h = crop.height // 2
|
||||
segs = [('上', crop.crop((0, 0, crop.width, h))),
|
||||
('下', crop.crop((0, h, crop.width, crop.height)))]
|
||||
elif k in ('top', 'bottom') and crop.width > crop.height * 2.2:
|
||||
w = crop.width // 2
|
||||
segs = [('左', crop.crop((0, 0, w, crop.height))),
|
||||
('右', crop.crop((w, 0, crop.width, crop.height)))]
|
||||
else:
|
||||
segs = [('', crop)]
|
||||
for tag, c in segs:
|
||||
f = min(3.0, max(1.0, 700.0 / max(1, c.width)))
|
||||
if f > 1.0:
|
||||
c = c.resize((int(c.width * f), int(c.height * f)), Image.LANCZOS)
|
||||
z = '%s-%s%s.png' % (base, k, tag)
|
||||
c.save(z)
|
||||
print(' 放大块 %s (%dx%d)' % (z, c.width, c.height))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- build
|
||||
|
||||
def cmd_build(images, out_pdf, paper, dpi, force, zoom):
|
||||
data = load_corners()
|
||||
if images:
|
||||
names = images
|
||||
else:
|
||||
names = sorted(data, key=natural_key)
|
||||
if not names:
|
||||
raise SystemExit('还没有角点。先跑 prepare 出网格图,再用 set 落盘角点。')
|
||||
|
||||
lacking = [n for n in names if n not in data]
|
||||
if lacking:
|
||||
raise SystemExit('这些图还没有角点:%s\n先跑 %s set ...'
|
||||
% (', '.join(lacking), SELF))
|
||||
|
||||
os.makedirs(OUT_DIR, exist_ok=True)
|
||||
from verify_quad import check as verify_quad
|
||||
from run import build, PAPERS
|
||||
|
||||
ratio = None
|
||||
if paper:
|
||||
key = paper.strip().lower()
|
||||
ratio = PAPERS.get(key)
|
||||
if ratio is None:
|
||||
try:
|
||||
ratio = float(paper)
|
||||
except ValueError:
|
||||
raise SystemExit('未知纸张:%s(可选 %s 或数字比例)'
|
||||
% (paper, '/'.join(PAPERS)))
|
||||
|
||||
# 1. 角标图 + 校验
|
||||
print('=== 步骤 1/4:回画角点(核查用) ===')
|
||||
verdicts = []
|
||||
for i, name in enumerate(names, 1):
|
||||
dst = out_path('角标', i, name)
|
||||
draw_marks(name, data[name], dst)
|
||||
print('\n第 %d 页 %s → %s' % (i, name, dst))
|
||||
ok = verify_quad(name, data[name])
|
||||
verdicts.append(ok)
|
||||
if zoom:
|
||||
make_zooms(name, dst, i, zoom)
|
||||
print('\n>>> 请打开上面这些 角标 图,逐个角确认是否压在纸角上。\n')
|
||||
|
||||
bad_pages = [i for i, ok in enumerate(verdicts, 1) if not ok]
|
||||
if bad_pages:
|
||||
print('!! 角点校验未通过:第 %s 页(上面标 [XX] 的项)。'
|
||||
% '、'.join(map(str, bad_pages)))
|
||||
if not force:
|
||||
print('已停止,未出件。按上面的提示调整角点,再重跑 set + build。')
|
||||
return 2
|
||||
print('已按 --force 继续出件:结果需人工确认,不能当作校验通过交付。\n')
|
||||
|
||||
# 2. 出 PDF
|
||||
print('=== 步骤 2/4:透视矫正 + 合成 PDF ===')
|
||||
workdir = os.path.join(OUT_DIR, '_pages')
|
||||
specs = ['%s:%s' % (n, quad_to_str(data[n])) for n in names]
|
||||
pages = build(specs, out_pdf, workdir, ratio)
|
||||
|
||||
# 成品图按页序号重命名进 out/
|
||||
for i, (name, page) in enumerate(zip(names, pages), 1):
|
||||
dst = out_path('成品', i, name)
|
||||
os.replace(page, dst)
|
||||
print(' 成品图 %s' % dst)
|
||||
|
||||
# 3. PDF 检验
|
||||
print('\n=== 步骤 3/4:最终 PDF 检验 ===')
|
||||
rc = subprocess.run([sys.executable, os.path.join(HERE, 'check.py'), out_pdf,
|
||||
'--dpi', str(dpi), '--expect-pages', str(len(names))]).returncode
|
||||
|
||||
# 4. 汇总
|
||||
bad = bad_pages
|
||||
print('\n=== 步骤 4/4:汇总 ===')
|
||||
print('PDF :%s(%.0f KB,%d 页)' % (out_pdf, os.path.getsize(out_pdf) / 1024, len(names)))
|
||||
print('角点校验 :%s' % ('全部通过' if not bad else '第 %s 页未通过' % '、'.join(map(str, bad))))
|
||||
print('PDF 检验 :%s' % ('通过' if rc == 0 else '未通过'))
|
||||
print('核查材料 :')
|
||||
for i, name in enumerate(names, 1):
|
||||
print(' %s %s %s' % (out_path('网格', i, name), out_path('角标', i, name),
|
||||
out_path('成品', i, name)))
|
||||
if bad or rc != 0:
|
||||
print('注意:以上为原样结果,未做任何自动修正。')
|
||||
return 2 if bad else 3
|
||||
return 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- check
|
||||
|
||||
def cmd_check(pdf, expect_pages, dpi):
|
||||
"""复检已有 PDF:只渲染检验,不重新校正、不改动文件。"""
|
||||
if not os.path.exists(pdf):
|
||||
raise SystemExit('找不到 PDF:%s' % pdf)
|
||||
cmd = [sys.executable, os.path.join(HERE, 'check.py'), pdf, '--dpi', str(dpi)]
|
||||
if expect_pages is not None:
|
||||
cmd += ['--expect-pages', str(expect_pages)]
|
||||
return subprocess.run(cmd).returncode
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- main
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(
|
||||
description='文档照片 -> 校正 PDF 全流程脚本',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__)
|
||||
sub = ap.add_subparsers(dest='cmd', required=True)
|
||||
|
||||
p = sub.add_parser('prepare', help='出网格图(out/1-网格-*.png)')
|
||||
p.add_argument('images', nargs='*', help='图片名,省略则取当前目录所有图片')
|
||||
|
||||
p = sub.add_parser('set', help='把一张图的角点写进 out/corners.json')
|
||||
p.add_argument('image', metavar='IMG', help='图片名(一次一张)')
|
||||
p.add_argument('--quad', action='append', default=[], required=True,
|
||||
metavar='左上x,左上y;右上x,右上y;右下x,右下y;左下x,左下y',
|
||||
help='这一张图的四个角点(顺序固定:左上 右上 右下 左下)')
|
||||
|
||||
p = sub.add_parser('show', help='查看已落盘角点')
|
||||
|
||||
p = sub.add_parser('check', help='复检已有 PDF(只渲染检验,不重新出件)')
|
||||
p.add_argument('pdf', metavar='PDF')
|
||||
p.add_argument('--expect-pages', type=int, default=None)
|
||||
p.add_argument('--dpi', type=int, default=100)
|
||||
|
||||
p = sub.add_parser('build', help='回画角标 + 校验 + 出 PDF + 检验')
|
||||
p.add_argument('images', nargs='*', help='图片名,省略则用 corners.json 里的全部')
|
||||
p.add_argument('--out', default='文档_校正.pdf')
|
||||
p.add_argument('--paper', default=None, help='a4/letter/16k/b5 或比例数字')
|
||||
p.add_argument('--dpi', type=int, default=100, help='PDF 检验渲染 dpi')
|
||||
p.add_argument('--force', action='store_true', help='角点校验不通过也继续')
|
||||
p.add_argument('--zoom', default=None,
|
||||
choices=['left', 'right', 'top', 'bottom', 'all'],
|
||||
help='另存边角放大块,便于细看')
|
||||
|
||||
p = sub.add_parser('all', help='prepare + build 一步到位')
|
||||
p.add_argument('images', nargs='*')
|
||||
p.add_argument('--out', default='文档_校正.pdf')
|
||||
p.add_argument('--paper', default=None)
|
||||
p.add_argument('--dpi', type=int, default=100)
|
||||
p.add_argument('--force', action='store_true')
|
||||
p.add_argument('--zoom', default=None)
|
||||
|
||||
args = ap.parse_args()
|
||||
|
||||
if args.cmd == 'prepare':
|
||||
return cmd_prepare(collect_images(args.images))
|
||||
if args.cmd == 'set':
|
||||
return cmd_set(args)
|
||||
if args.cmd == 'show':
|
||||
return cmd_show()
|
||||
if args.cmd == 'check':
|
||||
return cmd_check(args.pdf, args.expect_pages, args.dpi)
|
||||
if args.cmd == 'build':
|
||||
imgs = collect_images(args.images) if args.images else []
|
||||
return cmd_build(imgs, args.out, args.paper, args.dpi, args.force, args.zoom)
|
||||
if args.cmd == 'all':
|
||||
imgs = collect_images(args.images)
|
||||
cmd_prepare(imgs)
|
||||
data = load_corners()
|
||||
lacking = [n for n in imgs if n not in data]
|
||||
if lacking:
|
||||
print('\n这些图还没有角点,填好后重跑 build:%s' % ', '.join(lacking))
|
||||
return 2
|
||||
return cmd_build(imgs, args.out, args.paper, args.dpi, args.force, args.zoom)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -1,185 +0,0 @@
|
||||
"""角点回画预览:把已落盘的四角画回原图,供二次确认。
|
||||
|
||||
用途:把落盘的数字角点重新变成可看的画面。
|
||||
本脚本把四边形 + 四角标记 + 边标签画回原图,让"数字"重新变成可看的画面,
|
||||
在出件前确认四条边是否真的压在纸张亮面与桌面暗面的交界线上。
|
||||
|
||||
图上元素:
|
||||
绿色四边形 —— 当前角点连线,应紧贴纸边
|
||||
红色十字 + 序号 —— 四个角点(1左上 2右上 3右下 4左下)
|
||||
黄色短刻度线 —— 每条边中点的外法线,用于看边是否贴边
|
||||
右下角文字 —— 两行:文件名 / 角点串(避开顶部/左侧刻度)
|
||||
|
||||
用法:
|
||||
python3 doc_deskew/preview_quad.py 图1.jpg 图2.jpg
|
||||
python3 doc_deskew/preview_quad.py --corners .tmp/corners.json
|
||||
python3 doc_deskew/preview_quad.py --corners .tmp/corners.json --zoom 右
|
||||
|
||||
输出:.tmp/quad_<名称>.png
|
||||
--zoom 另存右侧/左侧/上边/下边放大块,.tmp/quad_<名称>_<区域>.png
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
DEFAULT_CORNERS = '.tmp/corners.json'
|
||||
OUT_DIR = '.tmp'
|
||||
|
||||
LABELS = ['1 左上', '2 右上', '3 右下', '4 左下']
|
||||
COLOR_QUAD = (0, 255, 0)
|
||||
COLOR_CORNER = (255, 40, 40)
|
||||
COLOR_EDGE = (255, 220, 0)
|
||||
|
||||
|
||||
def load_corners(path):
|
||||
with open(path, encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
out = {}
|
||||
for name, quad in data.items():
|
||||
if isinstance(quad, dict):
|
||||
quad = quad.get('quad') or quad.get('corners')
|
||||
out[name] = [(float(p[0]), float(p[1])) for p in quad]
|
||||
return out
|
||||
|
||||
|
||||
def get_font(size=26):
|
||||
# 先找中文字体(图上要画 "1 左上" 这类标签,西文字体会变成豆腐块)
|
||||
for p in ('/usr/share/fonts/opentype/noto/NotoSansCJK-Bold.ttc',
|
||||
'/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc',
|
||||
'/usr/share/fonts/truetype/noto/NotoSansCJK-Regular.ttc',
|
||||
'/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf',
|
||||
'/usr/share/fonts/truetype/liberation/LiberationSans-Bold.ttf'):
|
||||
if os.path.exists(p):
|
||||
try:
|
||||
return ImageFont.truetype(p, size)
|
||||
except Exception:
|
||||
pass
|
||||
return ImageFont.load_default()
|
||||
|
||||
|
||||
def draw_quad(img_path, pts, scale=1.0, with_grid=False):
|
||||
im = Image.open(img_path).convert('RGB')
|
||||
# 角标图默认叠上与网格图同一套网格:坐标读数与框边可以对着看
|
||||
if with_grid:
|
||||
from grid import draw_grid_on
|
||||
im = draw_grid_on(im)
|
||||
if scale != 1.0:
|
||||
im = im.resize((int(im.width * scale), int(im.height * scale)),
|
||||
Image.LANCZOS)
|
||||
d = ImageDraw.Draw(im)
|
||||
font = get_font(max(14, int(26 * min(1.0, scale + 0.15))))
|
||||
sml = get_font(max(11, int(18 * min(1.0, scale + 0.15))))
|
||||
w = max(2, int(3 * scale + 0.5))
|
||||
P = [(x * scale, y * scale) for x, y in pts]
|
||||
|
||||
# 四边形
|
||||
d.line(P + [P[0]], fill=COLOR_QUAD, width=w)
|
||||
|
||||
# 四角十字 + 序号
|
||||
r = max(10, int(22 * scale + 0.5))
|
||||
for i, (x, y) in enumerate(P):
|
||||
d.line([(x - r, y), (x + r, y)], fill=COLOR_CORNER, width=w)
|
||||
d.line([(x, y - r), (x, y + r)], fill=COLOR_CORNER, width=w)
|
||||
tx, ty = x + r + 4, y - r - 4
|
||||
d.text((tx, ty), LABELS[i], fill=COLOR_CORNER, font=sml,
|
||||
stroke_width=2, stroke_fill=(0, 0, 0))
|
||||
|
||||
# 每条边中点外法线刻度
|
||||
cx = sum(p[0] for p in P) / 4.0
|
||||
cy = sum(p[1] for p in P) / 4.0
|
||||
L = max(16, int(40 * scale + 0.5))
|
||||
for i in range(4):
|
||||
a, b = P[i], P[(i + 1) % 4]
|
||||
mx, my = (a[0] + b[0]) / 2.0, (a[1] + b[1]) / 2.0
|
||||
dx, dy = mx - cx, my - cy
|
||||
n = (dx * dx + dy * dy) ** 0.5 or 1.0
|
||||
dx, dy = dx / n, dy / n
|
||||
d.line([(mx, my), (mx + dx * L, my + dy * L)],
|
||||
fill=COLOR_EDGE, width=max(2, w - 1))
|
||||
|
||||
# 右下角信息条(避开顶部/左侧的网格刻度):第一行文件名,第二行角点串
|
||||
lines = [os.path.basename(img_path),
|
||||
';'.join('%g,%g' % (p[0], p[1]) for p in pts)]
|
||||
pad = 8
|
||||
gap = 6
|
||||
tw = int(max(d.textlength(s, font=font) for s in lines))
|
||||
ascent, descent = font.getmetrics()
|
||||
line_h = ascent + descent
|
||||
th = line_h * len(lines) + gap * (len(lines) - 1)
|
||||
tx = max(pad, im.width - tw - pad * 2)
|
||||
ty = max(pad, im.height - th - pad * 2)
|
||||
d.rectangle([tx - 4, ty - 4, tx + tw + 4, ty + th + 4], fill=(0, 0, 0))
|
||||
for i, s in enumerate(lines):
|
||||
d.text((tx, ty + i * (line_h + gap)), s, fill=(255, 255, 255), font=font)
|
||||
return im
|
||||
|
||||
|
||||
def zoom_boxes(im, pts):
|
||||
"""按角点外接框给出四侧放大区域。"""
|
||||
xs = [p[0] for p in pts]
|
||||
ys = [p[1] for p in pts]
|
||||
x0, x1, y0, y1 = min(xs), max(xs), min(ys), max(ys)
|
||||
W, H = im.size
|
||||
mx = (x1 - x0) * 0.18
|
||||
my = (y1 - y0) * 0.18
|
||||
boxes = {
|
||||
'left': (max(0, x0 - mx), max(0, y0 - my), min(W, x0 + mx), min(H, y1 + my)),
|
||||
'right': (max(0, x1 - mx), max(0, y0 - my), min(W, x1 + mx), min(H, y1 + my)),
|
||||
'top': (max(0, x0 - mx), max(0, y0 - my), min(W, x1 + mx), min(H, y0 + my)),
|
||||
'bottom': (max(0, x0 - mx), max(0, y1 - my), min(W, x1 + mx), min(H, y1 + my)),
|
||||
}
|
||||
return boxes
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description='把角点画回原图做二次确认')
|
||||
ap.add_argument('images', nargs='*', help='图片文件名(角点已在 corners.json 中)')
|
||||
ap.add_argument('--corners', default=DEFAULT_CORNERS)
|
||||
ap.add_argument('--zoom', choices=['left', 'right', 'top', 'bottom', 'all'],
|
||||
default=None, help='另存某侧放大块')
|
||||
ap.add_argument('--scale', type=float, default=1.0, help='整体缩放,>1 放大')
|
||||
ap.add_argument('--out', default=None, help='输出目录,默认 .tmp')
|
||||
args = ap.parse_args()
|
||||
|
||||
data = load_corners(args.corners)
|
||||
names = args.images or list(data.keys())
|
||||
missing = [n for n in names if n not in data]
|
||||
if missing:
|
||||
raise SystemExit('corners.json 中没有这些图的角点:%s' % ', '.join(missing))
|
||||
|
||||
outdir = args.out or OUT_DIR
|
||||
os.makedirs(outdir, exist_ok=True)
|
||||
|
||||
for name in names:
|
||||
pts = data[name]
|
||||
im = draw_quad(name, pts, args.scale)
|
||||
base = os.path.splitext(os.path.basename(name))[0]
|
||||
dst = os.path.join(outdir, 'quad_%s.png' % base)
|
||||
im.save(dst)
|
||||
print('written %s (%dx%d) %s' % (dst, im.width, im.height,
|
||||
';'.join('%g,%g' % p for p in pts)))
|
||||
if args.zoom:
|
||||
boxes = zoom_boxes(im, [(x * args.scale, y * args.scale) for x, y in pts])
|
||||
keys = list(boxes) if args.zoom == 'all' else [args.zoom]
|
||||
for k in keys:
|
||||
box = tuple(int(v) for v in boxes[k])
|
||||
if box[2] <= box[0] or box[3] <= box[1]:
|
||||
continue
|
||||
crop = im.crop(box)
|
||||
f = min(3.0, max(1.0, 700.0 / max(1, crop.width)))
|
||||
if f > 1.0:
|
||||
crop = crop.resize((int(crop.width * f), int(crop.height * f)),
|
||||
Image.LANCZOS)
|
||||
z = os.path.join(outdir, 'quad_%s_%s.png' % (base, k))
|
||||
crop.save(z)
|
||||
print('written %s (%dx%d)' % (z, crop.width, crop.height))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,85 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""审核页发布:模板链接 + 每批缩略图,落会话 .app/doc-review/。
|
||||
|
||||
模板的单一事实源在 skill 目录的 web/;会话里的 index.html 只是指向它的符号链接,
|
||||
不含任何数据注入。core 的 /app 静态托管放行指向技能根目录的链接(其余位置 404),
|
||||
所以改一次模板就对所有会话生效,不用重新生成。页面运行时通过 /api/db 读 doc_review_*
|
||||
表,因此同一份 HTML 可服务所有批次。缩略图是每会话的数据,仍写真目录。
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
WEB_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "web")
|
||||
APP_SUBDIR = os.path.join(".app", "doc-review")
|
||||
THUMB_LONG = 1200
|
||||
THUMB_Q = 80
|
||||
|
||||
|
||||
def _link(src, dst):
|
||||
"""把技能里的静态文件以符号链接发布到会话目录。
|
||||
|
||||
旧版本是复制副本,模板改动到不了已有会话;改链接后技能就是唯一真源。
|
||||
链接目标用绝对路径:executor 与 core 都把技能挂在同一位置,两边都能解析。
|
||||
"""
|
||||
src = os.path.abspath(src)
|
||||
if os.path.islink(dst):
|
||||
try:
|
||||
if os.path.realpath(dst) == os.path.realpath(src):
|
||||
return
|
||||
except OSError:
|
||||
pass
|
||||
os.unlink(dst)
|
||||
elif os.path.exists(dst):
|
||||
os.unlink(dst) # 旧会话的实体副本:删掉换成链接
|
||||
os.symlink(src, dst)
|
||||
|
||||
|
||||
def publish(session_root, batch_id, pages):
|
||||
"""发布模板与缩略图。
|
||||
|
||||
pages: [{idx, source_abs}];返回 (url, thumbs),thumbs[idx] = (相对路径, w, h)。
|
||||
"""
|
||||
app_dir = os.path.join(session_root, APP_SUBDIR)
|
||||
os.makedirs(app_dir, exist_ok=True)
|
||||
for name in sorted(os.listdir(WEB_DIR)):
|
||||
src = os.path.join(WEB_DIR, name)
|
||||
if os.path.isfile(src):
|
||||
_link(src, os.path.join(app_dir, name))
|
||||
|
||||
thumb_dir = os.path.join(app_dir, batch_id)
|
||||
os.makedirs(thumb_dir, exist_ok=True)
|
||||
thumbs = {}
|
||||
for p in pages:
|
||||
src = p.get("source_abs")
|
||||
idx = p.get("idx")
|
||||
if not src or idx is None or not os.path.isfile(src):
|
||||
continue
|
||||
try:
|
||||
with Image.open(src) as raw:
|
||||
im = ImageOps.exif_transpose(raw).convert("RGB")
|
||||
w, h = im.size
|
||||
s = min(1.0, float(THUMB_LONG) / max(w, h))
|
||||
if s < 1.0:
|
||||
im = im.resize((max(1, int(round(w * s))), max(1, int(round(h * s)))),
|
||||
Image.LANCZOS)
|
||||
name = "p%d.jpg" % int(idx)
|
||||
im.save(os.path.join(thumb_dir, name), "JPEG", quality=THUMB_Q, optimize=True)
|
||||
thumbs[int(idx)] = (batch_id + "/" + name, im.width, im.height)
|
||||
except Exception:
|
||||
continue
|
||||
task_id = os.path.basename(os.path.abspath(session_root))
|
||||
return ("/app/%s/doc-review/?b=%s" % (task_id, batch_id), thumbs)
|
||||
|
||||
|
||||
def prune_dirs(session_root, keep_ids):
|
||||
"""删除不在 keep_ids 里的旧批次缩略图目录(模板文件不动)。"""
|
||||
root = os.path.join(session_root, APP_SUBDIR)
|
||||
if not os.path.isdir(root):
|
||||
return
|
||||
keep = set(keep_ids)
|
||||
for name in os.listdir(root):
|
||||
path = os.path.join(root, name)
|
||||
if os.path.isdir(path) and name not in keep:
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
@@ -0,0 +1,223 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""审核页运行记录:会话 data.db 里的 doc_review_* 表。
|
||||
|
||||
为什么由技能自己写:
|
||||
- core 对 data.db 只有通用 SQL API(/api/db)和 db 工具,不认识任何业务表;
|
||||
- 技能自己拥有 schema、写入与清理,core 保持零感知(松耦合)。
|
||||
约定:
|
||||
- 时间一律 ISO8601 UTC 字符串;
|
||||
- 坐标/结构一律 JSON 文本,页面用 JSON.parse;
|
||||
- 表名统一 doc_review_ 前缀,避免和用户业务表混淆。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import secrets
|
||||
import sqlite3
|
||||
from datetime import datetime, timezone
|
||||
|
||||
SCHEMA_VERSION = 1
|
||||
DB_NAME = "data.db"
|
||||
|
||||
|
||||
def now_iso():
|
||||
return datetime.now(timezone.utc).isoformat(timespec="seconds")
|
||||
|
||||
|
||||
def new_batch_id():
|
||||
# 不含点/斜杠/空格:避免 skills.absolutizeArgs 把 batch_id 误判成路径。
|
||||
return "dr" + datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S") + "-" + secrets.token_hex(2)
|
||||
|
||||
|
||||
def _json(v):
|
||||
return None if v is None else json.dumps(v, ensure_ascii=False, separators=(",", ":"))
|
||||
|
||||
|
||||
def _loads(s, default=None):
|
||||
if not s:
|
||||
return default
|
||||
try:
|
||||
return json.loads(s)
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
|
||||
def connect(session_root):
|
||||
path = os.path.join(session_root, DB_NAME)
|
||||
conn = sqlite3.connect(path, timeout=5.0)
|
||||
conn.row_factory = sqlite3.Row
|
||||
conn.execute("PRAGMA busy_timeout=5000")
|
||||
# core 建库时已是 WAL;这里 best-effort,不强行切换(并发打开时切换会拿写锁)。
|
||||
try:
|
||||
conn.execute("PRAGMA journal_mode=WAL")
|
||||
except sqlite3.DatabaseError:
|
||||
pass
|
||||
ensure_schema(conn)
|
||||
return conn
|
||||
|
||||
|
||||
def ensure_schema(conn):
|
||||
conn.executescript("""
|
||||
CREATE TABLE IF NOT EXISTS doc_review_batch (
|
||||
id TEXT PRIMARY KEY,
|
||||
output TEXT NOT NULL,
|
||||
paper TEXT,
|
||||
enhance TEXT,
|
||||
dpi TEXT,
|
||||
orientation TEXT,
|
||||
status TEXT,
|
||||
summary TEXT,
|
||||
warnings TEXT,
|
||||
schema_version INTEGER,
|
||||
created_at TEXT,
|
||||
updated_at TEXT
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS doc_review_page (
|
||||
batch_id TEXT NOT NULL,
|
||||
idx INTEGER NOT NULL,
|
||||
source TEXT,
|
||||
source_w INTEGER,
|
||||
source_h INTEGER,
|
||||
w INTEGER,
|
||||
h INTEGER,
|
||||
auto_corners TEXT,
|
||||
auto_corners_norm TEXT,
|
||||
corners TEXT,
|
||||
auto_rotate INTEGER,
|
||||
rotate INTEGER,
|
||||
deleted INTEGER DEFAULT 0,
|
||||
sort INTEGER,
|
||||
status TEXT,
|
||||
reason TEXT,
|
||||
thumb TEXT,
|
||||
error TEXT,
|
||||
PRIMARY KEY (batch_id, idx)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS doc_review_job (
|
||||
batch_id TEXT PRIMARY KEY,
|
||||
edits TEXT,
|
||||
status TEXT,
|
||||
message TEXT,
|
||||
requested_at TEXT,
|
||||
started_at TEXT,
|
||||
finished_at TEXT
|
||||
);
|
||||
""")
|
||||
conn.commit()
|
||||
|
||||
|
||||
def upsert_batch(conn, batch_id, output, opts):
|
||||
ts = now_iso()
|
||||
conn.execute("""
|
||||
INSERT INTO doc_review_batch
|
||||
(id, output, paper, enhance, dpi, orientation, status, summary, warnings,
|
||||
schema_version, created_at, updated_at)
|
||||
VALUES (?,?,?,?,?,?,'running',NULL,NULL,?,?,?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
output=excluded.output, paper=excluded.paper, enhance=excluded.enhance,
|
||||
dpi=excluded.dpi, orientation=excluded.orientation,
|
||||
schema_version=excluded.schema_version, updated_at=excluded.updated_at
|
||||
""", (batch_id, output, opts.get("paper"), opts.get("enhance"),
|
||||
str(opts.get("dpi")), opts.get("orientation"),
|
||||
SCHEMA_VERSION, ts, ts))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def replace_pages(conn, batch_id, pages):
|
||||
conn.execute("DELETE FROM doc_review_page WHERE batch_id = ?", (batch_id,))
|
||||
for p in pages:
|
||||
conn.execute("""
|
||||
INSERT INTO doc_review_page
|
||||
(batch_id, idx, source, source_w, source_h, w, h,
|
||||
auto_corners, auto_corners_norm, corners, auto_rotate, rotate,
|
||||
deleted, sort, status, reason, thumb, error)
|
||||
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
|
||||
""", (batch_id, int(p.get("idx")), p.get("source"),
|
||||
p.get("source_w"), p.get("source_h"), p.get("w"), p.get("h"),
|
||||
_json(p.get("auto_corners")), _json(p.get("auto_corners_norm")),
|
||||
_json(p.get("corners")), int(p.get("auto_rotate") or 0),
|
||||
int(p.get("rotate") or 0), 1 if p.get("deleted") else 0,
|
||||
int(p.get("sort") or 0), p.get("status"), p.get("reason"),
|
||||
p.get("thumb"), p.get("error")))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def _row(r):
|
||||
return dict(r) if r is not None else None
|
||||
|
||||
|
||||
def get_batch(conn, batch_id):
|
||||
return _row(conn.execute("SELECT * FROM doc_review_batch WHERE id = ?", (batch_id,)).fetchone())
|
||||
|
||||
|
||||
def get_latest_batch(conn):
|
||||
return _row(conn.execute(
|
||||
"SELECT * FROM doc_review_batch ORDER BY created_at DESC, updated_at DESC LIMIT 1").fetchone())
|
||||
|
||||
|
||||
def get_pages(conn, batch_id):
|
||||
rows = [_row(r) for r in conn.execute(
|
||||
"SELECT * FROM doc_review_page WHERE batch_id = ? ORDER BY sort, idx", (batch_id,))]
|
||||
for r in rows:
|
||||
r["auto_corners"] = _loads(r.get("auto_corners"))
|
||||
r["auto_corners_norm"] = _loads(r.get("auto_corners_norm"))
|
||||
r["corners"] = _loads(r.get("corners"))
|
||||
return rows
|
||||
|
||||
|
||||
def get_job(conn, batch_id):
|
||||
r = _row(conn.execute("SELECT * FROM doc_review_job WHERE batch_id = ?", (batch_id,)).fetchone())
|
||||
if r:
|
||||
r["edits_obj"] = _loads(r.get("edits"))
|
||||
return r
|
||||
|
||||
|
||||
def upsert_job(conn, batch_id, edits, status="queued", message=None):
|
||||
ts = now_iso()
|
||||
conn.execute("""
|
||||
INSERT INTO doc_review_job(batch_id, edits, status, message, requested_at, started_at, finished_at)
|
||||
VALUES (?,?,?,?,?,NULL,NULL)
|
||||
ON CONFLICT(batch_id) DO UPDATE SET
|
||||
edits=excluded.edits, status=excluded.status, message=excluded.message,
|
||||
requested_at=excluded.requested_at, started_at=NULL, finished_at=NULL
|
||||
""", (batch_id, _json(edits), status, message, ts))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def set_job(conn, batch_id, status, message=None):
|
||||
ts = now_iso()
|
||||
if status == "running":
|
||||
conn.execute("UPDATE doc_review_job SET status=?, message=?, started_at=?, finished_at=NULL WHERE batch_id=?",
|
||||
(status, message, ts, batch_id))
|
||||
elif status in ("done", "error"):
|
||||
conn.execute("UPDATE doc_review_job SET status=?, message=?, finished_at=? WHERE batch_id=?",
|
||||
(status, message, ts, batch_id))
|
||||
else:
|
||||
conn.execute("UPDATE doc_review_job SET status=?, message=? WHERE batch_id=?",
|
||||
(status, message, batch_id))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def finish_batch(conn, batch_id, ok, summary, warnings, status=None):
|
||||
ts = now_iso()
|
||||
if status is None:
|
||||
status = "ready" if ok else "error"
|
||||
conn.execute("UPDATE doc_review_batch SET status=?, summary=?, warnings=?, updated_at=? WHERE id=?",
|
||||
(status, summary, _json(warnings or []), ts, batch_id))
|
||||
conn.commit()
|
||||
|
||||
|
||||
def prune(conn, keep=5, protect=None):
|
||||
"""只保留最近 keep 个批次;protect 指定的批次永不删(防止在旧批次上重生成时被裁掉)。"""
|
||||
rows = conn.execute(
|
||||
"SELECT id FROM doc_review_batch ORDER BY created_at DESC, updated_at DESC").fetchall()
|
||||
kept = [r["id"] for r in rows[:keep]]
|
||||
if protect and protect not in kept:
|
||||
kept.append(protect)
|
||||
old = [r["id"] for r in rows if r["id"] not in kept]
|
||||
for bid in old:
|
||||
conn.execute("DELETE FROM doc_review_page WHERE batch_id=?", (bid,))
|
||||
conn.execute("DELETE FROM doc_review_job WHERE batch_id=?", (bid,))
|
||||
conn.execute("DELETE FROM doc_review_batch WHERE id=?", (bid,))
|
||||
if old:
|
||||
conn.commit()
|
||||
return old, kept
|
||||
@@ -1,69 +0,0 @@
|
||||
"""拉正合成的核心模块:四边形透视变换 + 按页拼版。
|
||||
|
||||
由 pipeline.py 的 build 子命令调用,不单独提供命令行入口(原来那套
|
||||
--grid/--page/--check 已分别由 prepare / build / check 接管,免得两个入口各说各话)。
|
||||
|
||||
设计原则:
|
||||
1. 不做中间校正。角点只用来定义纸张四边,几何变换一次到位,
|
||||
校正结果直接进 PDF;所有质量判定都在最终 PDF 上做。
|
||||
2. 不在流程里"自动收紧/自动修补"。这类启发式规则容易误裁内容,
|
||||
只保留能明确判定对错的检验项。
|
||||
3. 判定阈值来自每页图像自身统计,不是针对某份文档调出来的常量。
|
||||
"""
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, HERE)
|
||||
|
||||
import numpy as np
|
||||
from homography import brighten, warp_perspective
|
||||
from PIL import Image
|
||||
|
||||
# 纸张宽高比(宽/高):pipeline 与 run 共用这一份,避免两套系数漂移
|
||||
PAPERS = {'a4': 210 / 297, 'letter': 8.5 / 11, '16k': 185 / 260, 'b5': 176 / 250}
|
||||
|
||||
|
||||
def parse_page(spec):
|
||||
src, _, quad_s = spec.partition(':')
|
||||
if not src or not quad_s:
|
||||
raise SystemExit('格式应为 "图片:左上;右上;右下;左下":%s' % spec)
|
||||
quad = []
|
||||
for pt in quad_s.split(';'):
|
||||
x, y = pt.split(',')
|
||||
quad.append((float(x), float(y)))
|
||||
if len(quad) != 4:
|
||||
raise SystemExit('角点必须 4 个:%s' % spec)
|
||||
return src, quad
|
||||
|
||||
|
||||
def build(specs, out_pdf, workdir='out', ratio=None):
|
||||
"""一次几何变换得到页面图,直接合并。不做二次裁切。
|
||||
|
||||
ratio=None 时按四边形自然尺寸输出(保留照片原始比例);
|
||||
给定 ratio(如 A4=0.707)时强制映射到该纸张比例,纠正透视压缩。
|
||||
"""
|
||||
os.makedirs(workdir, exist_ok=True)
|
||||
pages = []
|
||||
for i, spec in enumerate(specs, 1):
|
||||
src, quad = parse_page(spec)
|
||||
im = Image.open(src).convert('RGB')
|
||||
if ratio is None:
|
||||
page = brighten(warp_perspective(im, quad))
|
||||
else:
|
||||
q = np.array(quad, float)
|
||||
wpx = (np.linalg.norm(q[1] - q[0]) + np.linalg.norm(q[2] - q[3])) / 2
|
||||
hpx = wpx / ratio
|
||||
page = brighten(warp_perspective(
|
||||
im, quad, out_size=(int(round(wpx)), int(round(hpx)))))
|
||||
path = os.path.join(workdir, 'p%d.png' % i)
|
||||
page.save(path)
|
||||
pages.append(path)
|
||||
nw, nh = page.size
|
||||
print(' 第 %d 页 %-28s -> %s %dx%d 比例 %.3f'
|
||||
% (i, os.path.basename(src), path, nw, nh, nw / nh))
|
||||
|
||||
subprocess.run([sys.executable, os.path.join(HERE, 'make_pdf.py')]
|
||||
+ pages + [out_pdf], check=True)
|
||||
return pages
|
||||
@@ -0,0 +1,225 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""合成样张回归测试(不依赖真实照片)。
|
||||
|
||||
覆盖:
|
||||
1. 单页透视校正 + JSON 输出 + PDF 生成 + state 文件
|
||||
2. EXIF Orientation=6 的转正
|
||||
3. 批次方向不一致时的旋转告警
|
||||
4. paper=a4 的画布尺寸
|
||||
5. orientation=none 的混排
|
||||
6. orientation=landscape 的强制方向
|
||||
|
||||
运行:
|
||||
python3 skills/document-photo-to-pdf/doc_deskew/smoke_test.py
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
SKILL = os.path.dirname(HERE)
|
||||
sys.path.insert(0, HERE)
|
||||
from homography import _solve_h # noqa: E402
|
||||
import check # noqa: E402
|
||||
from params import P # noqa: E402
|
||||
|
||||
W_PAGE, H_PAGE = 900, 1272
|
||||
QUAD = [(200, 240), (1080, 190), (1120, 1420), (150, 1380)]
|
||||
PHOTO_SIZE = (1200, 1600)
|
||||
|
||||
|
||||
def make_page():
|
||||
im = Image.new("RGB", (W_PAGE, H_PAGE), "white")
|
||||
d = ImageDraw.Draw(im)
|
||||
for i in range(18):
|
||||
y = 90 + i * 58
|
||||
d.line([(90, y), (W_PAGE - 90, y)], fill=(30, 30, 30), width=8)
|
||||
if i % 4 == 0:
|
||||
d.line([(90, y + 18), (W_PAGE - 300, y + 18)],
|
||||
fill=(60, 60, 60), width=5)
|
||||
return im
|
||||
|
||||
|
||||
def make_photo(page, quad=QUAD, size=PHOTO_SIZE):
|
||||
bg = Image.new("RGB", size, (38, 42, 48))
|
||||
dst = [(0, 0), (W_PAGE - 1, 0), (W_PAGE - 1, H_PAGE - 1),
|
||||
(0, H_PAGE - 1)]
|
||||
H = _solve_h(quad, dst)
|
||||
H = H / H[2, 2]
|
||||
coeffs = (H[0, 0], H[0, 1], H[0, 2],
|
||||
H[1, 0], H[1, 1], H[1, 2],
|
||||
H[2, 0], H[2, 1])
|
||||
warped = page.transform(size, Image.PERSPECTIVE, coeffs, Image.BICUBIC)
|
||||
mask = Image.new("L", (W_PAGE, H_PAGE), 255)
|
||||
mask = mask.transform(size, Image.PERSPECTIVE, coeffs, Image.BICUBIC)
|
||||
mask = mask.point(lambda v: 255 if v > 127 else 0)
|
||||
photo = bg.copy()
|
||||
photo.paste(warped, mask=mask)
|
||||
return photo
|
||||
|
||||
|
||||
def run_skill(args, cwd):
|
||||
proc = subprocess.run(
|
||||
[sys.executable, os.path.join(SKILL, "run.py"),
|
||||
json.dumps(args, ensure_ascii=False)],
|
||||
cwd=cwd, capture_output=True, text=True)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
# stdout 必须是纯 JSON(不含进度日志)
|
||||
data = json.loads(proc.stdout)
|
||||
assert isinstance(data, dict)
|
||||
return data
|
||||
|
||||
|
||||
def assert_corners(got, want, tol=35):
|
||||
assert len(got) == 4, got
|
||||
for (gx, gy), (ex, ey) in zip(got, want):
|
||||
assert abs(gx - ex) <= tol and abs(gy - ey) <= tol, (
|
||||
"角点偏差过大:got=%s want=%s" % (got, want))
|
||||
|
||||
|
||||
def assert_checks(res):
|
||||
checks = res.get("checks") or {}
|
||||
if checks.get("ok") is None:
|
||||
print("[skip] PDF 渲染不可用:%s" % checks.get("error"))
|
||||
return
|
||||
assert checks.get("ok") is True, checks
|
||||
assert checks["page_count"]["ok"], checks
|
||||
|
||||
|
||||
def verbose(label, res):
|
||||
if not os.environ.get("SMOKE_VERBOSE"):
|
||||
return
|
||||
print("[%s] %s" % (label, json.dumps({
|
||||
"summary": res.get("summary"),
|
||||
"batch": res.get("batch"),
|
||||
"warnings": res.get("warnings"),
|
||||
"checks": res.get("checks"),
|
||||
}, ensure_ascii=False, indent=2)))
|
||||
|
||||
|
||||
def make_composed_page(strip=None, edge_text=False):
|
||||
"""造一张“PDF 渲染页”:白画布 + 居中纸面(~245 亮度)。"""
|
||||
canvas = Image.new("RGB", (1240, 1754), "white")
|
||||
paper = Image.new("RGB", (1160, 1640), (245, 245, 245))
|
||||
d = ImageDraw.Draw(paper)
|
||||
for i in range(12):
|
||||
y = 120 + i * 90
|
||||
d.line([(100, y), (1060, y)], fill=(30, 30, 30), width=8)
|
||||
if strip is not None:
|
||||
d.rectangle([0, 0, 1159, strip - 1], fill=(60, 60, 60))
|
||||
if edge_text:
|
||||
d.rectangle([60, 0, 1100, 4], fill=(0, 0, 0))
|
||||
canvas.paste(paper, (40, 57))
|
||||
return canvas
|
||||
|
||||
|
||||
def test_analyse():
|
||||
"""直接验证 check.analyse 量的是纸边,不是 PDF 白边。"""
|
||||
normal = check.analyse(make_composed_page(), P)
|
||||
assert not normal["border_bad"], normal["border_bad"]
|
||||
assert not normal["cut"], normal["cut"]
|
||||
|
||||
residue = check.analyse(make_composed_page(strip=12), P)
|
||||
assert "top" in residue["border_bad"], residue["border_bad"]
|
||||
assert residue["edge_dark"]["top"] > P["check"]["edge_dark_frac"], residue
|
||||
|
||||
cut = check.analyse(make_composed_page(edge_text=True), P)
|
||||
assert any(k == "top" for k, _ in cut["cut"]), cut["cut"]
|
||||
|
||||
|
||||
def main():
|
||||
test_analyse()
|
||||
page = make_page()
|
||||
photo = make_photo(page)
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
# 1. 单页 + auto
|
||||
src = os.path.join(td, "图1.jpg")
|
||||
photo.save(src, quality=92)
|
||||
res = run_skill({"images": ["图1.jpg"], "output": "结果.pdf",
|
||||
"paper": "auto"}, td)
|
||||
assert res["ok"], res
|
||||
assert res["output"]["pages"] == 1, res
|
||||
assert os.path.exists(res["output"]["path"])
|
||||
assert_corners(res["pages"][0]["corners"], QUAD)
|
||||
assert_checks(res)
|
||||
# 低分辨率纸面也要放大到内容区,避免各页大小不一
|
||||
assert res["pages"][0]["out_size"][0] == 1160, res["pages"][0]
|
||||
assert os.path.exists(res["output"]["state"])
|
||||
verbose("auto", res)
|
||||
|
||||
# check.py CLI 也跑一遍(executor 有渲染后端时)
|
||||
if (res.get("checks") or {}).get("ok") is not None:
|
||||
cli = subprocess.run(
|
||||
[sys.executable, os.path.join(HERE, "check.py"),
|
||||
res["output"]["path"], "--expect-pages", "1"],
|
||||
capture_output=True, text=True)
|
||||
assert cli.returncode == 0, cli.stdout + cli.stderr
|
||||
|
||||
# 2. EXIF Orientation=6:存 90° CCW + 标记 6,读入应转回原方向
|
||||
stored = photo.transpose(Image.ROTATE_90)
|
||||
exif = Image.Exif()
|
||||
exif[274] = 6
|
||||
exif_src = os.path.join(td, "exif.jpg")
|
||||
stored.save(exif_src, quality=92, exif=exif)
|
||||
res2 = run_skill({"images": ["exif.jpg"], "output": "exif.pdf"}, td)
|
||||
assert res2["ok"], res2
|
||||
assert_corners(res2["pages"][0]["corners"], QUAD)
|
||||
assert_checks(res2)
|
||||
verbose("exif", res2)
|
||||
|
||||
# 3. 批次方向:第二页横拍,应被旋转并告警
|
||||
sideways = photo.transpose(Image.ROTATE_270)
|
||||
side_src = os.path.join(td, "图2.jpg")
|
||||
sideways.save(side_src, quality=92)
|
||||
res3 = run_skill({"images": ["图1.jpg", "图2.jpg"],
|
||||
"output": "批次.pdf"}, td)
|
||||
assert res3["ok"], res3
|
||||
assert res3["output"]["pages"] == 2, res3
|
||||
assert res3["batch"]["rotated_pages"] == [2], res3["batch"]
|
||||
assert res3["pages"][1]["warnings"], res3["pages"][1]
|
||||
# 同批页面即使一竖一横,也应放大/缩放到同一版式宽度
|
||||
s1, s2 = res3["pages"][0]["out_size"], res3["pages"][1]["out_size"]
|
||||
assert s1[0] == s2[0] == 1160, (s1, s2)
|
||||
assert abs(s1[1] - s2[1]) <= 2, (s1, s2)
|
||||
assert_checks(res3)
|
||||
verbose("batch", res3)
|
||||
|
||||
# 4. paper=a4 画布
|
||||
res4 = run_skill({"images": ["图1.jpg"], "output": "a4.pdf",
|
||||
"paper": "a4"}, td)
|
||||
assert res4["ok"], res4
|
||||
assert res4["batch"]["canvas_px"] == [1240, 1754], res4["batch"]
|
||||
assert res4["batch"]["content_px"] == [1160, 1674], res4["batch"]
|
||||
assert_checks(res4)
|
||||
verbose("a4", res4)
|
||||
|
||||
# 5. orientation=none:不旋转,允许横竖混排
|
||||
res5 = run_skill({"images": ["图1.jpg", "图2.jpg"],
|
||||
"output": "none.pdf", "orientation": "none"}, td)
|
||||
assert res5["ok"], res5
|
||||
assert res5["batch"]["orientation"] == "mixed", res5["batch"]
|
||||
assert res5["batch"]["rotated_pages"] == [], res5["batch"]
|
||||
assert res5["batch"]["canvas_px"] is None, res5["batch"]
|
||||
assert_checks(res5)
|
||||
verbose("none", res5)
|
||||
|
||||
# 6. orientation=landscape:强制横版,第一页(竖版)被旋转
|
||||
res6 = run_skill({"images": ["图1.jpg", "图2.jpg"],
|
||||
"output": "land.pdf",
|
||||
"orientation": "landscape"}, td)
|
||||
assert res6["ok"], res6
|
||||
assert res6["batch"]["rotated_pages"] == [1], res6["batch"]
|
||||
assert_checks(res6)
|
||||
verbose("landscape", res6)
|
||||
|
||||
print("SMOKE OK")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,236 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""state.json v2 与 edits.json:归一化坐标、页面状态分级、编辑覆盖。
|
||||
|
||||
坐标约定
|
||||
--------
|
||||
corners_norm / edits 里的 "c" 一律是归一化 0~1,相对 exif_transpose 之后的
|
||||
**原图**:[[x_tl,y_tl],[x_tr,y_tr],[x_br,y_br],[x_bl,y_bl]]。
|
||||
原图像素坐标只在检测、渲染时用;进出 state 一律换成归一化,
|
||||
换预览分辨率、换 UI 画布都不会错位。
|
||||
|
||||
页面状态(取代"整批失败")
|
||||
--------------------------
|
||||
ready 已就绪,无需人看
|
||||
review 建议看一眼(掩膜回退角点、覆盖率低、纸型存疑)
|
||||
needs_orientation 宽高比接近正方形,横竖判不出来,需要人点旋转
|
||||
needs_corners 必须补角(检测为空、框内纸面占比过低)
|
||||
render_failed 出页失败,已用白页占位(极少见)
|
||||
failed 原图打不开,本页已跳过(传错文件)
|
||||
deleted 人主动删掉的页,不进出件
|
||||
|
||||
严重度只用来决定"一页命中多条规则时记哪一条",不参与任何几何计算。
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
|
||||
STATE_SCHEMA = "document-photo-to-pdf/2"
|
||||
EDITS_SCHEMA = "review/1"
|
||||
|
||||
# 顺序即严重度,越靠后越需要人介入
|
||||
STATUS_ORDER = ["ready", "review", "needs_orientation",
|
||||
"needs_corners", "render_failed", "failed", "deleted"]
|
||||
_SEV = {s: i for i, s in enumerate(STATUS_ORDER)}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 坐标
|
||||
|
||||
def norm_quad(quad, size):
|
||||
"""原图像素坐标 -> 归一化 0~1(4 位小数,300dpi 往返误差远小于 1px)。"""
|
||||
w = float(size[0]) or 1.0
|
||||
h = float(size[1]) or 1.0
|
||||
return [[round(float(p[0]) / w, 4), round(float(p[1]) / h, 4)]
|
||||
for p in quad]
|
||||
|
||||
|
||||
def denorm_quad(norm, size):
|
||||
"""归一化 0~1 -> 原图像素坐标。"""
|
||||
w, h = float(size[0]), float(size[1])
|
||||
return [[float(p[0]) * w, float(p[1]) * h] for p in norm]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 状态
|
||||
|
||||
def escalate(status, reason, cand, cand_reason):
|
||||
"""取更严重的一对 (status, reason);同级保留先到的。"""
|
||||
if _SEV.get(cand, 0) > _SEV.get(status, 0):
|
||||
return cand, cand_reason
|
||||
return status, reason
|
||||
|
||||
|
||||
def grade_page(meta, P, ref_ratio=None):
|
||||
"""按方案第 6 节的表,把一页的检测数字转成"要不要人看"。
|
||||
|
||||
纯路由:不重新检测、不改数字,只决定这一页在审核页上标不标 `!`。
|
||||
ref_ratio 是这张纸应有的宽高比(竖版为正);None 表示未知,跳过纸型判断。
|
||||
"""
|
||||
g = P["gate"]
|
||||
dead = g["aspect_deadband"]
|
||||
status, reason = "ready", None
|
||||
|
||||
if meta.get("fallback"):
|
||||
status, reason = escalate(status, reason, "review", "mask_fallback")
|
||||
|
||||
cov = meta.get("coverage") or {}
|
||||
if cov and min(float(v) for v in cov.values()) < g["coverage_warn"]:
|
||||
status, reason = escalate(status, reason, "review", "low_coverage")
|
||||
|
||||
metrics = (meta.get("validation") or {}).get("metrics") or {}
|
||||
frac = metrics.get("inner_paper_frac")
|
||||
if frac is not None and float(frac) <= g["inner_paper_min"]:
|
||||
status, reason = escalate(status, reason,
|
||||
"needs_corners", "inner_paper_low")
|
||||
|
||||
aspect = meta.get("aspect")
|
||||
if aspect:
|
||||
if abs(float(aspect) - 1.0) <= dead:
|
||||
# 横竖看不出来:不猜,交给人点旋转
|
||||
status, reason = escalate(status, reason,
|
||||
"needs_orientation", "aspect_ambiguous")
|
||||
elif ref_ratio:
|
||||
# 同向折算后再比:横放的 A4 不该被判成异形纸
|
||||
r = float(ref_ratio)
|
||||
if float(aspect) > 1.0:
|
||||
r = 1.0 / r if r > 0 else r
|
||||
dev = abs(float(aspect) - r) / r if r > 0 else 0.0
|
||||
if dev > g["paper_mismatch_pct"] / 100.0:
|
||||
status, reason = escalate(status, reason,
|
||||
"review", "paper_mismatch")
|
||||
return status, reason
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- edits
|
||||
|
||||
def _quad_ok(c):
|
||||
"""校验归一化四边形:4 点、每点 2 数、范围放宽到 -1~2(角点允许拖出画面)。"""
|
||||
if not isinstance(c, (list, tuple)) or len(c) != 4:
|
||||
return None
|
||||
out = []
|
||||
for p in c:
|
||||
if not isinstance(p, (list, tuple)) or len(p) != 2:
|
||||
return None
|
||||
try:
|
||||
x, y = float(p[0]), float(p[1])
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
# 方案第 11 节:纸被切掉一块时角点必须能拖到画面外,所以下限放到 -1
|
||||
if not (-1.0 <= x <= 2.0 and -1.0 <= y <= 2.0):
|
||||
return None
|
||||
out.append([x, y])
|
||||
return out
|
||||
|
||||
|
||||
def _rot_ok(v):
|
||||
"""旋转角归一到 0/90/180/270,认不出来按 0(不转)处理。"""
|
||||
try:
|
||||
r = int(round(float(v))) % 360
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
return r if r in (0, 90, 180, 270) else 0
|
||||
|
||||
|
||||
def load_edits(value):
|
||||
"""把 edits.json 读成内部结构;value 是路径、dict 或 None。
|
||||
|
||||
只校验结构与范围,不认识的多余字段一律忽略——前端的版本总比后端快,
|
||||
不该因为多了一个字段就拒绝出件。
|
||||
"""
|
||||
if value is None or value == "" or value == {}:
|
||||
return None
|
||||
if isinstance(value, str):
|
||||
path = os.path.abspath(os.path.expanduser(value.strip()))
|
||||
if not os.path.isfile(path):
|
||||
raise ValueError("edits 文件不存在:%s" % path)
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
value = json.load(f)
|
||||
if not isinstance(value, dict):
|
||||
raise ValueError("edits 必须是 JSON 对象或 edits.json 路径")
|
||||
|
||||
batch = value.get("batch")
|
||||
batch = dict(batch) if isinstance(batch, dict) else {}
|
||||
|
||||
pages = {}
|
||||
for item in (value.get("pages") or []):
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
try:
|
||||
idx = int(item.get("i"))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if idx < 1:
|
||||
continue
|
||||
rec = {}
|
||||
q = _quad_ok(item.get("c"))
|
||||
if q:
|
||||
rec["corners_norm"] = q
|
||||
if item.get("r") is not None:
|
||||
rec["rotate"] = _rot_ok(item.get("r"))
|
||||
if rec:
|
||||
pages[idx] = rec
|
||||
|
||||
order = []
|
||||
for v in (value.get("order") or []):
|
||||
try:
|
||||
order.append(int(v))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
# order 缺省/为空 = 按 images 传入顺序出件,调用方自己兜底
|
||||
|
||||
deleted = set()
|
||||
for v in (value.get("deleted") or []):
|
||||
try:
|
||||
deleted.add(int(v))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
|
||||
return {"schema": EDITS_SCHEMA, "batch": batch, "pages": pages,
|
||||
"order": order, "deleted": deleted}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- state v2
|
||||
|
||||
_STATE_PAGE_KEYS = ("index", "source", "source_size", "auto_corners",
|
||||
"corners_norm", "rotate", "rotate_source",
|
||||
"status", "reason")
|
||||
|
||||
|
||||
def build_state(batch, pages):
|
||||
"""state.json v2:只放重建这次结果需要的东西,不放 PDF 检验数据。
|
||||
|
||||
pages 项由调用方组装好,这里只裁字段、保顺序,不再做判断。
|
||||
"""
|
||||
return {
|
||||
"schema": STATE_SCHEMA,
|
||||
"batch": {
|
||||
"doc_type": batch.get("doc_type"),
|
||||
"paper": batch.get("paper"),
|
||||
"enhance": batch.get("enhance"),
|
||||
"dpi": batch.get("dpi"),
|
||||
"orientation": batch.get("orientation"),
|
||||
},
|
||||
"pages": [{k: p.get(k) for k in _STATE_PAGE_KEYS} for p in pages],
|
||||
}
|
||||
|
||||
|
||||
def load_state(value):
|
||||
"""读上一份 state v2(可选输入):按 index 取 auto_corners,供"恢复自动"。
|
||||
|
||||
缺文件、旧 schema、字段不全都当成没传——state 是参考,不是契约。
|
||||
"""
|
||||
if value in (None, ""):
|
||||
return {}
|
||||
try:
|
||||
if isinstance(value, str):
|
||||
path = os.path.abspath(os.path.expanduser(value.strip()))
|
||||
if not os.path.isfile(path):
|
||||
return {}
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
value = json.load(f)
|
||||
if not isinstance(value, dict) or value.get("schema") != STATE_SCHEMA:
|
||||
return {}
|
||||
out = {}
|
||||
for item in (value.get("pages") or []):
|
||||
if isinstance(item, dict) and item.get("index") is not None:
|
||||
out[int(item["index"])] = item
|
||||
return out
|
||||
except Exception:
|
||||
return {}
|
||||
@@ -0,0 +1,149 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""角点几何校验:纯函数,返回结论和指标,不打印、不落盘。
|
||||
|
||||
判定只看几何与亮度关系,不依赖固定像素常量:
|
||||
1. 四边形凸,且顺序为 左上→右上→右下→左下
|
||||
2. 越界幅度在允许范围内(角点出画面属正常)
|
||||
3. 框内以纸面(亮像素)为主
|
||||
4. 每边内侧都是纸面(混入桌面说明取点落在纸外)
|
||||
外侧偏亮只作提示:可能是取点偏内,也可能是纸边倾斜,亮度无法区分。
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
|
||||
def _sample_map(a, pts, n, shrink=1.0, outward=0.0):
|
||||
"""在四边形内部取网格点(shrink<1 向内收),或在外部环带取点(outward>0)。"""
|
||||
h, w = a.shape
|
||||
tl, tr, br, bl = [np.asarray(p, dtype=np.float64) for p in pts]
|
||||
c = (tl + tr + br + bl) / 4.0
|
||||
if shrink != 1.0:
|
||||
tl, tr, br, bl = [c + (p - c) * shrink for p in (tl, tr, br, bl)]
|
||||
if outward > 0:
|
||||
tl, tr, br, bl = [c + (p - c) * (1.0 + outward)
|
||||
for p in (tl, tr, br, bl)]
|
||||
vals = []
|
||||
for u in np.linspace(0.03, 0.97, n):
|
||||
p1 = tl + (tr - tl) * u
|
||||
p2 = bl + (br - bl) * u
|
||||
for v in np.linspace(0.03, 0.97, n):
|
||||
x, y = p1 + (p2 - p1) * v
|
||||
xi, yi = int(round(x)), int(round(y))
|
||||
if 0 <= xi < w and 0 <= yi < h:
|
||||
vals.append(a[yi, xi])
|
||||
return np.asarray(vals)
|
||||
|
||||
|
||||
def _cross(o, a, b):
|
||||
return ((a[0] - o[0]) * (b[1] - o[1])
|
||||
- (a[1] - o[1]) * (b[0] - o[0]))
|
||||
|
||||
|
||||
def validate(img, pts, P):
|
||||
"""返回 {ok, metrics, failures, warnings}。"""
|
||||
g = P["gate"]
|
||||
a = np.asarray(img.convert("L")).astype(np.float32)
|
||||
h, w = a.shape
|
||||
paper = float(np.percentile(a, 80))
|
||||
back = float(np.percentile(a, 2))
|
||||
thr = back + (paper - back) * 0.5
|
||||
|
||||
failures = []
|
||||
warnings = []
|
||||
metrics = {}
|
||||
|
||||
# 1. 凸性与顺序
|
||||
signs = [_cross(pts[i], pts[(i + 1) % 4], pts[(i + 2) % 4])
|
||||
for i in range(4)]
|
||||
convex = all(s > 0 for s in signs)
|
||||
metrics["convex"] = bool(convex)
|
||||
if not convex:
|
||||
failures.append("角点顺序/凸性不对,应为 左上→右上→右下→左下")
|
||||
|
||||
# 2. 越界幅度
|
||||
xs = [p[0] for p in pts]
|
||||
ys = [p[1] for p in pts]
|
||||
out = [max(0.0, -min(xs)), max(0.0, max(xs) - w),
|
||||
max(0.0, -min(ys)), max(0.0, max(ys) - h)]
|
||||
oob = max(out) / float(max(w, h)) * 100.0
|
||||
metrics["oob_pct"] = round(float(oob), 2)
|
||||
if oob >= g["oob_max_pct"]:
|
||||
failures.append("角点越界 %.1f%%,超过 %.0f%%"
|
||||
% (oob, g["oob_max_pct"]))
|
||||
|
||||
# 3. 框内纸面占比
|
||||
inner = _sample_map(a, pts, g["sample_n"], shrink=g["sample_shrink"])
|
||||
if inner.size == 0:
|
||||
frac_paper = 0.0
|
||||
else:
|
||||
frac_paper = float((inner > thr).mean())
|
||||
metrics["inner_paper_frac"] = round(frac_paper, 4)
|
||||
if frac_paper <= g["inner_paper_min"]:
|
||||
failures.append("框内纸面占比 %.0f%%,偏低(可能取到桌面)"
|
||||
% (frac_paper * 100))
|
||||
|
||||
# 4. 四边外侧:只提示
|
||||
names = ["上边", "右边", "下边", "左边"]
|
||||
outer_dark = {}
|
||||
span = float(max(w, h))
|
||||
for i in range(4):
|
||||
p1, p2 = np.asarray(pts[i], float), np.asarray(pts[(i + 1) % 4], float)
|
||||
mid = (p1 + p2) / 2.0
|
||||
c = sum(np.asarray(p, float) for p in pts) / 4.0
|
||||
d = mid - c
|
||||
norm = float(np.linalg.norm(d))
|
||||
if norm < 1e-6:
|
||||
continue
|
||||
d = d / norm
|
||||
vals = []
|
||||
for t in np.linspace(0.15, 0.85, 12):
|
||||
base = p1 + (p2 - p1) * t
|
||||
for k in (span * 0.02, span * 0.035, span * 0.05):
|
||||
x, y = base + d * k
|
||||
xi, yi = int(round(x)), int(round(y))
|
||||
if 0 <= xi < w and 0 <= yi < h:
|
||||
vals.append(a[yi, xi])
|
||||
if len(vals) < 8:
|
||||
continue
|
||||
vals = np.asarray(vals)
|
||||
dark_frac = float((vals < thr).mean())
|
||||
outer_dark[names[i]] = round(dark_frac, 3)
|
||||
if dark_frac < g["outer_dark_min"]:
|
||||
warnings.append("%s外侧仍偏亮(可能取点偏内,也可能纸边倾斜)"
|
||||
% names[i])
|
||||
metrics["outer_dark"] = outer_dark
|
||||
|
||||
# 5. 四边内侧:必须是纸面
|
||||
inner_paper = {}
|
||||
for i in range(4):
|
||||
p1, p2 = np.asarray(pts[i], float), np.asarray(pts[(i + 1) % 4], float)
|
||||
mid = (p1 + p2) / 2.0
|
||||
c = sum(np.asarray(p, float) for p in pts) / 4.0
|
||||
d = mid - c
|
||||
norm = float(np.linalg.norm(d))
|
||||
if norm < 1e-6:
|
||||
continue
|
||||
d = d / norm
|
||||
vals = []
|
||||
for t in np.linspace(0.15, 0.85, 12):
|
||||
base = p1 + (p2 - p1) * t
|
||||
for k in (span * 0.015, span * 0.03, span * 0.045):
|
||||
x, y = base - d * k
|
||||
xi, yi = int(round(x)), int(round(y))
|
||||
if 0 <= xi < w and 0 <= yi < h:
|
||||
vals.append(a[yi, xi])
|
||||
if len(vals) < 8:
|
||||
continue
|
||||
vals = np.asarray(vals)
|
||||
paper_frac = float((vals > thr).mean())
|
||||
inner_paper[names[i]] = round(paper_frac, 3)
|
||||
if paper_frac < g["inner_edge_min"]:
|
||||
failures.append("%s内侧混入桌面(纸面仅 %.0f%%),取点落在纸外"
|
||||
% (names[i], paper_frac * 100))
|
||||
metrics["inner_paper"] = inner_paper
|
||||
|
||||
return {
|
||||
"ok": not failures,
|
||||
"metrics": metrics,
|
||||
"failures": failures,
|
||||
"warnings": warnings,
|
||||
}
|
||||
@@ -1,185 +0,0 @@
|
||||
"""几何校验:四角坐标是否合理覆盖了纸张。
|
||||
|
||||
不依赖图像亮度,只看几何关系,因此不受白底干扰:
|
||||
1. 四边形必须凸,且顺序为 左上→右上→右下→左下
|
||||
2. 四边形必须落在源图范围内(允许少量越界,用于角点出画面的情况)
|
||||
3. 四边形内应基本是纸面(亮像素占多数),说明没有取到桌面
|
||||
4. 沿四条边外侧一圈应是桌面(暗像素占多数),说明没有少取纸张
|
||||
|
||||
任一项失败即说明该角点需调整,并指出是哪条边。
|
||||
|
||||
用法:
|
||||
python3 doc_deskew/verify_quad.py 图.jpg "左上;右上;右下;左下"
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def parse(qs):
|
||||
pts = []
|
||||
for pt in qs.split(';'):
|
||||
x, y = pt.split(',')
|
||||
pts.append((float(x), float(y)))
|
||||
if len(pts) != 4:
|
||||
raise SystemExit('需要 4 个角点')
|
||||
return pts
|
||||
|
||||
|
||||
def sample_map(a, pts, n=60, shrink=1.0, outward=0.0):
|
||||
"""在四边形内部取网格点(shrink<1 向内收),或在外部环带取点(outward>0)。"""
|
||||
|
||||
h, w = a.shape
|
||||
tl, tr, br, bl = [np.asarray(p, float) for p in pts]
|
||||
c = (tl + tr + br + bl) / 4.0
|
||||
if shrink != 1.0:
|
||||
tl, tr, br, bl = [c + (p - c) * shrink for p in (tl, tr, br, bl)]
|
||||
if outward > 0:
|
||||
tl, tr, br, bl = [c + (p - c) * (1 + outward) for p in (tl, tr, br, bl)]
|
||||
vals = []
|
||||
for u in np.linspace(0.03, 0.97, n):
|
||||
p1 = tl + (tr - tl) * u
|
||||
p2 = bl + (br - bl) * u
|
||||
for v in np.linspace(0.03, 0.97, n):
|
||||
x, y = p1 + (p2 - p1) * v
|
||||
xi, yi = int(round(x)), int(round(y))
|
||||
if 0 <= xi < w and 0 <= yi < h:
|
||||
vals.append(a[yi, xi])
|
||||
return np.asarray(vals)
|
||||
|
||||
|
||||
def cross(o, a, b):
|
||||
return (a[0] - o[0]) * (b[1] - o[1]) - (a[1] - o[1]) * (b[0] - o[0])
|
||||
|
||||
|
||||
def check(img, pts, verbose=True):
|
||||
a = np.asarray(Image.open(img).convert('L')).astype(np.float32)
|
||||
h, w = a.shape
|
||||
paper = float(np.percentile(a, 80))
|
||||
back = float(np.percentile(a, 2))
|
||||
thr = back + (paper - back) * 0.5
|
||||
ok = True
|
||||
|
||||
# 1. 凸性与顺序
|
||||
signs = [cross(pts[i], pts[(i + 1) % 4], pts[(i + 2) % 4]) for i in range(4)]
|
||||
convex = all(s > 0 for s in signs)
|
||||
if verbose:
|
||||
print(' [%s] 角点顺序/凸性' % ('OK' if convex else 'XX'))
|
||||
if not convex:
|
||||
print(' 方向不一致,顺序应为 左上→右上→右下→左下')
|
||||
ok = False
|
||||
|
||||
# 2. 越界幅度
|
||||
xs = [p[0] for p in pts]; ys = [p[1] for p in pts]
|
||||
out = [max(0, -min(xs)), max(0, max(xs) - w), max(0, -min(ys)), max(0, max(ys) - h)]
|
||||
oob = max(out) / max(w, h) * 100
|
||||
good = oob < 15
|
||||
if verbose:
|
||||
print(' [%s] 越界幅度 %.1f%%(角点出画面属正常,上限 15%%)'
|
||||
% ('OK' if good else 'XX', oob))
|
||||
ok &= good
|
||||
|
||||
# 3. 内部应是纸面(亮)
|
||||
inner = sample_map(a, pts, shrink=0.88)
|
||||
frac_paper = float((inner > thr).mean())
|
||||
good = frac_paper > 0.6
|
||||
if verbose:
|
||||
print(' [%s] 框内纸面占比 %.0f%%(应 >60%%,偏低说明取到桌面)'
|
||||
% ('OK' if good else 'XX', frac_paper * 100))
|
||||
ok &= good
|
||||
|
||||
# 4. 四条边外侧应是桌面(暗)——逐边判定,指出问题边
|
||||
names = ['上边', '右边', '下边', '左边']
|
||||
bad_edges = []
|
||||
for i in range(4):
|
||||
p1, p2 = pts[i], pts[(i + 1) % 4]
|
||||
mid = (np.asarray(p1) + np.asarray(p2)) / 2
|
||||
c = sum(np.asarray(p, float) for p in pts) / 4
|
||||
d = mid - c
|
||||
norm = np.linalg.norm(d)
|
||||
if norm < 1e-6:
|
||||
continue
|
||||
d = d / norm
|
||||
# 沿该边外侧取样
|
||||
tc = (np.asarray(p1) + np.asarray(p2)) / 2
|
||||
fracs = []
|
||||
for t in np.linspace(0.15, 0.85, 12):
|
||||
base = np.asarray(p1) + (np.asarray(p2) - np.asarray(p1)) * t
|
||||
span = max(w, h)
|
||||
for k in (span * 0.02, span * 0.035, span * 0.05): # 外移 2%%~5%% 图幅
|
||||
x, y = base + d * k
|
||||
xi, yi = int(round(x)), int(round(y))
|
||||
if 0 <= xi < w and 0 <= yi < h:
|
||||
fracs.append(a[yi, xi])
|
||||
if len(fracs) < 8:
|
||||
continue
|
||||
fracs = np.asarray(fracs)
|
||||
dark_frac = float((fracs < thr).mean())
|
||||
# 外侧大部分应是暗背景;若几乎全亮,说明该边取进了纸内,纸边被切
|
||||
if dark_frac < 0.35:
|
||||
bad_edges.append((names[i], dark_frac))
|
||||
# 5. 沿四条边内侧取样: 应全是纸面; 若混入桌面说明该边取到纸外
|
||||
outside_edges = []
|
||||
for i in range(4):
|
||||
p1, p2 = pts[i], pts[(i + 1) % 4]
|
||||
mid = (np.asarray(p1) + np.asarray(p2)) / 2
|
||||
c = sum(np.asarray(p, float) for p in pts) / 4
|
||||
d = mid - c
|
||||
norm = np.linalg.norm(d)
|
||||
if norm < 1e-6:
|
||||
continue
|
||||
d = d / norm
|
||||
vals = []
|
||||
span = max(w, h)
|
||||
for t in np.linspace(0.15, 0.85, 12):
|
||||
base = np.asarray(p1) + (np.asarray(p2) - np.asarray(p1)) * t
|
||||
for k in (span * 0.015, span * 0.03, span * 0.045):
|
||||
x, y = base - d * k # 向内
|
||||
xi, yi = int(round(x)), int(round(y))
|
||||
if 0 <= xi < w and 0 <= yi < h:
|
||||
vals.append(a[yi, xi])
|
||||
if len(vals) < 8:
|
||||
continue
|
||||
vals = np.asarray(vals)
|
||||
paper_frac = float((vals > thr).mean())
|
||||
if paper_frac < 0.75:
|
||||
outside_edges.append((names[i], paper_frac))
|
||||
good2 = not outside_edges
|
||||
|
||||
if verbose:
|
||||
if not bad_edges:
|
||||
print(' [OK] 四边外侧检测到桌面,纸边已被框住')
|
||||
else:
|
||||
# 外侧仍偏亮有两种可能:取点偏内(会切内容),或纸边本身倾斜导致
|
||||
# 固定外移采样仍落在纸内。二者无法用亮度区分,故只作提示不判错。
|
||||
print(' [--] 提示:下列边外侧仍偏亮(可能取点偏内,也可能是纸边倾斜):')
|
||||
for n, f in bad_edges:
|
||||
print(' %s 外侧纸面占比 %.0f%% → 可尝试再向外扩 3~5px 后目视确认'
|
||||
% (n, (1 - f) * 100))
|
||||
|
||||
if verbose:
|
||||
if good2:
|
||||
print(' [OK] 四边内侧均为纸面')
|
||||
else:
|
||||
print(' [XX] 下列边内侧混入桌面,说明取点落在纸外:')
|
||||
for n, f in outside_edges:
|
||||
print(' %s(内侧纸面仅 %.0f%%)→ 该边角点向内收 5px' % (n, f * 100))
|
||||
ok &= good2
|
||||
return ok
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('img'); ap.add_argument('quad')
|
||||
a = ap.parse_args()
|
||||
pts = parse(a.quad)
|
||||
print('校验 %s 角点 %s' % (a.img, a.quad))
|
||||
ok = check(a.img, pts)
|
||||
print('\n结论:%s' % ('角点可用' if ok else '需调整上面标 XX 的项'))
|
||||
return 0 if ok else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,951 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""document-photo-to-pdf 技能入口。
|
||||
|
||||
两种用法共用一条出件内核:
|
||||
全自动:{"images": [...], "output": "..."}
|
||||
出 PDF,并把校正后的单页原图导出到 out/(不碰 .app/、不写 data.db)。
|
||||
半自动:{"images": [...], "review": "always|auto"}
|
||||
出 PDF 和 out/ 页图,并把审核页发布到 .app/doc-review/、运行记录写进 data.db。
|
||||
重生成:{"batch_id": "dr..."}
|
||||
从 data.db 读上次的源图/输出/编辑,覆盖同名 PDF;args 带 edits 时以 edits
|
||||
为准,不带时读 doc_review_job.edits,再退化为当前页记录(按现在的角点再渲染)。
|
||||
|
||||
stdout 只输出一个 JSON 对象;进度、警告、异常都在字段里,不写 stderr。
|
||||
失败也返回 exit 0,用 ok=false 表达(平台非 0 会给输出追加 [exit code],破坏 JSON)。
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import statistics
|
||||
import sys
|
||||
import time
|
||||
import traceback
|
||||
|
||||
from PIL import Image, ImageOps
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.insert(0, os.path.join(HERE, "doc_deskew"))
|
||||
|
||||
from params import P # noqa: E402
|
||||
from auto_quad import detect # noqa: E402
|
||||
from validate import validate # noqa: E402
|
||||
import compose # noqa: E402
|
||||
import check # noqa: E402
|
||||
import state # noqa: E402
|
||||
import review_store # noqa: E402
|
||||
import review_page # noqa: E402
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 基础
|
||||
|
||||
def _err(code, message, page=None, hint=None):
|
||||
e = {"code": code, "message": message}
|
||||
if page is not None:
|
||||
e["page"] = int(page)
|
||||
if hint:
|
||||
e["hint"] = hint
|
||||
return e
|
||||
|
||||
|
||||
def _json_default(o):
|
||||
try:
|
||||
import numpy as np
|
||||
if isinstance(o, np.integer):
|
||||
return int(o)
|
||||
if isinstance(o, np.floating):
|
||||
return float(o)
|
||||
if isinstance(o, np.ndarray):
|
||||
return o.tolist()
|
||||
except Exception:
|
||||
pass
|
||||
return str(o)
|
||||
|
||||
|
||||
def load_image(path):
|
||||
"""唯一的读图入口:转正 EXIF、压白透明底、转 RGB。"""
|
||||
with Image.open(path) as im:
|
||||
im = ImageOps.exif_transpose(im)
|
||||
if (im.mode in ("RGBA", "LA")
|
||||
or (im.mode == "P" and "transparency" in im.info)):
|
||||
im = im.convert("RGBA")
|
||||
bg = Image.new("RGB", im.size, "white")
|
||||
bg.paste(im, mask=im.split()[-1])
|
||||
im = bg
|
||||
else:
|
||||
im = im.convert("RGB")
|
||||
im.load()
|
||||
return im
|
||||
|
||||
|
||||
def parse_images(args):
|
||||
raw = args.get("images")
|
||||
if raw is None:
|
||||
return []
|
||||
if isinstance(raw, str):
|
||||
return [raw.strip()] if raw.strip() else []
|
||||
if isinstance(raw, (list, tuple)):
|
||||
out, seen = [], set()
|
||||
for v in raw:
|
||||
if not isinstance(v, str) or not v.strip():
|
||||
raise ValueError("images 里只能放路径字符串")
|
||||
p = v.strip()
|
||||
if p not in seen:
|
||||
seen.add(p)
|
||||
out.append(p)
|
||||
return out
|
||||
raise ValueError("images 必须是字符串或字符串数组")
|
||||
|
||||
|
||||
def write_state(path, data):
|
||||
if not path:
|
||||
return False
|
||||
try:
|
||||
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
|
||||
tmp = path + ".tmp"
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||
os.replace(tmp, path)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 参数/路径
|
||||
|
||||
def _review_mode(args):
|
||||
if args.get("batch_id") and "review" not in args:
|
||||
return "always"
|
||||
v = args.get("review", "always")
|
||||
if isinstance(v, bool):
|
||||
return "always" if v else "off"
|
||||
s = str(v).strip().lower()
|
||||
if s in ("1", "true", "yes", "on"):
|
||||
s = "always"
|
||||
if s in ("0", "false", "no", ""):
|
||||
s = "off"
|
||||
if s not in ("off", "always", "auto"):
|
||||
raise ValueError("review 只能是 off/always/auto")
|
||||
return s
|
||||
|
||||
|
||||
def _resolve_dpi(v, P):
|
||||
if v is None or str(v).strip() == "":
|
||||
return int(P["render"]["dpi"])
|
||||
s = str(v).strip().lower()
|
||||
opts = P["render"]["dpi_options"]
|
||||
if s in opts:
|
||||
return int(opts[s])
|
||||
try:
|
||||
n = int(float(s))
|
||||
except ValueError:
|
||||
raise ValueError("dpi 只能是 standard/high 或 72~600 的数字")
|
||||
if not (72 <= n <= 600):
|
||||
raise ValueError("dpi 超出范围(72~600):%s" % v)
|
||||
return n
|
||||
|
||||
|
||||
def _dpi_label(v, dpi):
|
||||
# 150/300 归一到审核页的两个档名,页面分段控件才能正确高亮;其它数值原样返回。
|
||||
if int(dpi) == 150:
|
||||
return "standard"
|
||||
if int(dpi) == 300:
|
||||
return "high"
|
||||
return str(dpi)
|
||||
|
||||
|
||||
def _batch_opts(args, ed, P):
|
||||
b = (ed or {}).get("batch") or {}
|
||||
paper_input = b.get("paper", args.get("paper"))
|
||||
if paper_input is None or str(paper_input).strip() == "":
|
||||
paper_input = P["render"]["default_paper"]
|
||||
paper_input = str(paper_input).strip()
|
||||
paper = compose.resolve_paper(paper_input, P)
|
||||
|
||||
orientation = str(b.get("orientation", args.get("orientation") or "auto")).strip().lower()
|
||||
if orientation not in ("auto", "none", "portrait", "landscape"):
|
||||
raise ValueError("orientation 只能是 auto/none/portrait/landscape")
|
||||
|
||||
enhance = str(b.get("enhance", args.get("enhance") or P["render"]["default_enhance"])).strip().lower()
|
||||
if enhance not in P["enhance"]:
|
||||
raise ValueError("enhance 只能是 " + "/".join(P["enhance"].keys()))
|
||||
P["render"]["enhance"] = enhance
|
||||
|
||||
dpi_arg = b.get("dpi", args.get("dpi"))
|
||||
dpi = _resolve_dpi(dpi_arg, P)
|
||||
P["render"]["dpi"] = dpi
|
||||
return paper_input, paper, orientation, enhance, _dpi_label(dpi_arg, dpi), dpi
|
||||
|
||||
|
||||
def _inside(root, path):
|
||||
try:
|
||||
rel = os.path.relpath(os.path.abspath(path), os.path.abspath(root))
|
||||
return rel != ".." and not rel.startswith(".." + os.sep)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _rel(root, path):
|
||||
try:
|
||||
rel = os.path.relpath(os.path.abspath(path), os.path.abspath(root))
|
||||
if rel == ".." or rel.startswith(".." + os.sep):
|
||||
return os.path.abspath(path)
|
||||
return rel
|
||||
except Exception:
|
||||
return path
|
||||
|
||||
|
||||
def _abs(root, path):
|
||||
p = str(path or "")
|
||||
return p if os.path.isabs(p) else os.path.join(root, p)
|
||||
|
||||
|
||||
def _norm(quad, size):
|
||||
w = float(size[0]) or 1.0
|
||||
h = float(size[1]) or 1.0
|
||||
return [[round(float(p[0]) / w, 4), round(float(p[1]) / h, 4)]
|
||||
for p in quad]
|
||||
|
||||
|
||||
def _denorm(norm, size):
|
||||
w, h = float(size[0]), float(size[1])
|
||||
return [[float(p[0]) * w, float(p[1]) * h] for p in norm]
|
||||
|
||||
|
||||
def _aspect(quad):
|
||||
w, h = compose.aspect_of(quad)
|
||||
return round(float(w / h), 4)
|
||||
|
||||
|
||||
def _fallback_quad(size, P):
|
||||
inset = float(P["gate"]["fallback_inset"])
|
||||
w, h = float(size[0]), float(size[1])
|
||||
return [[inset * w, inset * h], [(1 - inset) * w, inset * h],
|
||||
[(1 - inset) * w, (1 - inset) * h], [inset * w, (1 - inset) * h]]
|
||||
|
||||
|
||||
def _prev_page(rows, idx):
|
||||
for r in rows or []:
|
||||
try:
|
||||
if int(r.get("idx")) == int(idx):
|
||||
return r
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _edits_from_pages(rows, batch):
|
||||
pages, deleted, order = [], [], []
|
||||
for r in rows or []:
|
||||
try:
|
||||
idx = int(r.get("idx"))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if r.get("deleted"):
|
||||
deleted.append(idx)
|
||||
continue
|
||||
c = r.get("corners")
|
||||
if c:
|
||||
pages.append({"i": idx, "c": c, "r": int(r.get("rotate") or 0)})
|
||||
order.append(idx)
|
||||
b = {}
|
||||
if batch:
|
||||
b = {"paper": batch.get("paper"), "enhance": batch.get("enhance"),
|
||||
"dpi": batch.get("dpi"), "orientation": batch.get("orientation")}
|
||||
return state.load_edits({"schema": "review/1", "batch": b, "pages": pages,
|
||||
"order": order, "deleted": deleted})
|
||||
|
||||
|
||||
def _targets(args, batch_id):
|
||||
output_arg = args.get("output")
|
||||
if output_arg is not None:
|
||||
output_arg = str(output_arg).strip()
|
||||
cwd = os.getcwd()
|
||||
if batch_id:
|
||||
if args.get("db"):
|
||||
root = os.path.dirname(os.path.abspath(os.path.expanduser(str(args["db"]))))
|
||||
elif output_arg:
|
||||
root = os.path.dirname(os.path.abspath(os.path.expanduser(output_arg)))
|
||||
else:
|
||||
root = cwd
|
||||
conn = review_store.connect(root)
|
||||
batch = review_store.get_batch(conn, batch_id)
|
||||
if not batch:
|
||||
conn.close()
|
||||
raise ValueError("batch_id 不存在:%s" % batch_id)
|
||||
output = _abs(root, batch["output"])
|
||||
if not output.lower().endswith(".pdf"):
|
||||
output += ".pdf"
|
||||
return output, root, conn, batch
|
||||
|
||||
output = os.path.abspath(os.path.expanduser(
|
||||
str(output_arg or P["output"]["default_name"])))
|
||||
if not output.lower().endswith(".pdf"):
|
||||
output += ".pdf"
|
||||
# 生产环境 cwd 就是会话目录;输出在会话内时以 cwd 为根,子目录输出也不会把
|
||||
# .app/data.db 建到子目录里。本地从别的目录跑时退化为输出目录。
|
||||
root = cwd if _inside(cwd, output) else os.path.dirname(output)
|
||||
return output, root, None, None
|
||||
|
||||
|
||||
def _png_dir(cwd):
|
||||
d = os.path.join(cwd, "out")
|
||||
if os.path.exists(d) and not os.path.isdir(d):
|
||||
raise ValueError("out 已存在且不是目录:%s" % d)
|
||||
os.makedirs(d, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def _clean_pngs(d, stem, keep):
|
||||
"""删掉本 PDF 前缀下编号大于本次出页数的旧页图;不动目录里其它文件。"""
|
||||
pat = re.compile(r"^" + re.escape(stem) + r"-(\d+)\.png$")
|
||||
for name in os.listdir(d):
|
||||
m = pat.match(name)
|
||||
if m and int(m.group(1)) > keep:
|
||||
try:
|
||||
os.remove(os.path.join(d, name))
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _mark_error(conn, batch_id, message):
|
||||
if not conn or not batch_id:
|
||||
return
|
||||
try:
|
||||
review_store.set_job(conn, batch_id, "error", str(message))
|
||||
review_store.finish_batch(conn, batch_id, False, str(message), [], status="error")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 主流程
|
||||
|
||||
def process(args, t0):
|
||||
review_mode = _review_mode(args)
|
||||
batch_id_arg = args.get("batch_id")
|
||||
batch_id_arg = str(batch_id_arg).strip() if batch_id_arg else None
|
||||
|
||||
output, session_root, conn, batch_row = _targets(args, batch_id_arg)
|
||||
# 审核页由 core 从会话 .app/ 托管;输出不在会话目录内时页面/下载都不可达。
|
||||
if review_mode != "off" and not _inside(os.getcwd(), output):
|
||||
raise ValueError("审核页要求 PDF 位于会话目录内(当前工作目录):%s" % output)
|
||||
|
||||
use_db = bool(batch_id_arg) or review_mode != "off"
|
||||
if use_db and conn is None:
|
||||
conn = review_store.connect(session_root)
|
||||
|
||||
batch_id = batch_id_arg
|
||||
is_new_batch = batch_id is None
|
||||
if use_db and not batch_id:
|
||||
batch_id = review_store.new_batch_id()
|
||||
|
||||
state_path = os.path.splitext(output)[0] + P["output"]["state_suffix"]
|
||||
|
||||
try:
|
||||
old_pages = []
|
||||
if batch_id_arg:
|
||||
if not batch_row:
|
||||
batch_row = review_store.get_batch(conn, batch_id)
|
||||
old_pages = review_store.get_pages(conn, batch_id)
|
||||
if not old_pages:
|
||||
raise ValueError("批次没有页面记录:%s" % batch_id)
|
||||
|
||||
ed = state.load_edits(args.get("edits"))
|
||||
job = None
|
||||
if ed is None and batch_id:
|
||||
job = review_store.get_job(conn, batch_id)
|
||||
if job and job.get("edits_obj"):
|
||||
ed = state.load_edits(job["edits_obj"])
|
||||
else:
|
||||
ed = _edits_from_pages(old_pages, batch_row)
|
||||
if batch_id:
|
||||
if not job:
|
||||
job = review_store.get_job(conn, batch_id)
|
||||
if job:
|
||||
review_store.set_job(conn, batch_id, "running")
|
||||
|
||||
paper_input, paper, orientation, enhance, dpi_label, dpi = \
|
||||
_batch_opts(args, ed, P)
|
||||
|
||||
# ---------------- 源图清单
|
||||
if batch_id_arg:
|
||||
source_map = {}
|
||||
for r in old_pages:
|
||||
try:
|
||||
source_map[int(r["idx"])] = _abs(session_root, r["source"])
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
all_indices = sorted(source_map)
|
||||
else:
|
||||
images = parse_images(args)
|
||||
if not images:
|
||||
raise ValueError("必须提供 images")
|
||||
all_indices = list(range(1, len(images) + 1))
|
||||
source_map = {i: os.path.abspath(os.path.expanduser(str(images[i - 1])))
|
||||
for i in all_indices}
|
||||
|
||||
deleted = set(ed.get("deleted") or []) if ed else set()
|
||||
run_indices = [i for i in all_indices if i not in deleted]
|
||||
if not run_indices:
|
||||
raise ValueError("所有页面都被删除,没有可处理的页")
|
||||
|
||||
pages = []
|
||||
plan = []
|
||||
unreadable = []
|
||||
|
||||
for idx in all_indices:
|
||||
src = source_map.get(idx)
|
||||
meta = {"index": idx, "source_abs": src, "warnings": [],
|
||||
"deleted": idx in deleted}
|
||||
if meta["deleted"]:
|
||||
meta.update(status="deleted", reason="user_deleted",
|
||||
corners=None, corners_norm=None, auto_corners=None,
|
||||
auto_corners_norm=None, auto_rotate=0, rotated_deg=0)
|
||||
pages.append(meta)
|
||||
continue
|
||||
|
||||
try:
|
||||
img = load_image(src)
|
||||
except Exception as e:
|
||||
err = _err(
|
||||
"IMAGE_UNREADABLE",
|
||||
"第 %d 页 %s 打不开:%s" % (idx, os.path.basename(src or ""), e),
|
||||
idx,
|
||||
"确认文件存在且是 jpg/png/webp/bmp/tif 等常见格式")
|
||||
meta.update(status="failed", reason="unreadable", error=err,
|
||||
corners=None, corners_norm=None,
|
||||
auto_corners=None, auto_corners_norm=None,
|
||||
auto_rotate=0, rotated_deg=0)
|
||||
meta["warnings"].append(err["message"])
|
||||
unreadable.append(err)
|
||||
pages.append(meta)
|
||||
continue
|
||||
|
||||
meta["source_size"] = [int(img.width), int(img.height)]
|
||||
edit = (ed or {}).get("pages", {}).get(idx) if ed else None
|
||||
manual = bool(edit and edit.get("corners_norm"))
|
||||
|
||||
if manual:
|
||||
meta["manual"] = True
|
||||
meta["corners"] = _denorm(edit["corners_norm"], meta["source_size"])
|
||||
meta["corners_norm"] = [[float(x), float(y)]
|
||||
for x, y in edit["corners_norm"]]
|
||||
meta["aspect"] = _aspect(meta["corners"])
|
||||
meta["coverage"] = {}
|
||||
meta["paper_frac"] = None
|
||||
meta["fallback"] = False
|
||||
meta["fallback_sides"] = []
|
||||
meta["validation"] = {"ok": True, "metrics": {},
|
||||
"warnings": [], "failures": []}
|
||||
prev = _prev_page(old_pages, idx)
|
||||
if prev and prev.get("auto_corners"):
|
||||
meta["auto_corners"] = prev["auto_corners"]
|
||||
meta["auto_corners_norm"] = prev.get("auto_corners_norm")
|
||||
meta["auto_rotate"] = int(prev.get("auto_rotate") or 0)
|
||||
else:
|
||||
meta["auto_corners"] = [list(map(float, q))
|
||||
for q in meta["corners"]]
|
||||
meta["auto_corners_norm"] = meta["corners_norm"]
|
||||
meta["auto_rotate"] = 0
|
||||
meta["status"] = "ready"
|
||||
meta["reason"] = None
|
||||
else:
|
||||
try:
|
||||
corners, info = detect(img, P)
|
||||
except Exception as e:
|
||||
corners, info = [], {"error": str(e)}
|
||||
|
||||
if not corners:
|
||||
corners = _fallback_quad(meta["source_size"], P)
|
||||
info = {"fallback": True, "fallback_sides": [0, 1, 2, 3],
|
||||
"coverage": {}, "paper_frac": None}
|
||||
meta["status"] = "needs_corners"
|
||||
meta["reason"] = "detect_failed"
|
||||
else:
|
||||
meta["status"] = "ready"
|
||||
meta["reason"] = None
|
||||
|
||||
meta["corners"] = [[float(x), float(y)] for x, y in corners]
|
||||
meta["corners_norm"] = _norm(meta["corners"], meta["source_size"])
|
||||
meta["auto_corners"] = [list(map(float, q))
|
||||
for q in meta["corners"]]
|
||||
meta["auto_corners_norm"] = meta["corners_norm"]
|
||||
meta["auto_rotate"] = 0
|
||||
meta["coverage"] = {k: round(float(v), 3)
|
||||
for k, v in (info.get("coverage") or {}).items()}
|
||||
pf = info.get("paper_frac")
|
||||
meta["paper_frac"] = None if pf is None else round(float(pf), 4)
|
||||
meta["fallback"] = bool(info.get("fallback", False))
|
||||
meta["fallback_sides"] = [int(s)
|
||||
for s in (info.get("fallback_sides") or [])]
|
||||
try:
|
||||
v = validate(img, meta["corners"], P)
|
||||
except Exception as e:
|
||||
v = {"ok": False, "metrics": {},
|
||||
"failures": ["校验异常:%s" % e], "warnings": []}
|
||||
meta["validation"] = v
|
||||
if not v["ok"]:
|
||||
# 检测到了但几何不可信:退回兜底框,交给人补角,而不是让整批失败。
|
||||
# auto_corners 也要跟着换成兜底框,否则“恢复自动”会把无效角点拖回来。
|
||||
meta["detected_corners"] = meta["corners"]
|
||||
meta["corners"] = _fallback_quad(meta["source_size"], P)
|
||||
meta["corners_norm"] = _norm(meta["corners"],
|
||||
meta["source_size"])
|
||||
meta["auto_corners"] = [list(map(float, q))
|
||||
for q in meta["corners"]]
|
||||
meta["auto_corners_norm"] = meta["corners_norm"]
|
||||
meta["status"] = "needs_corners"
|
||||
meta["reason"] = "invalid_quad"
|
||||
meta["aspect"] = _aspect(meta["corners"])
|
||||
|
||||
pages.append(meta)
|
||||
plan.append((idx, src, meta))
|
||||
img.close()
|
||||
|
||||
if not plan:
|
||||
failures = unreadable or [
|
||||
_err("NO_PAGES", "没有可处理的页面", None,
|
||||
"检查 images 是否都删除了/是否都打不开")]
|
||||
res = finish_failure(pages, failures, output, state_path, t0)
|
||||
_mark_error(conn, batch_id, res["summary"])
|
||||
return res
|
||||
|
||||
# 出件顺序先定下来:raw 页图按最终 PDF 页序编号(文件名与 PDF 同名)。
|
||||
plan_by_idx = {m["index"]: (i, s, m) for (i, s, m) in plan}
|
||||
ordered_idx = []
|
||||
if ed and ed.get("order"):
|
||||
for v in ed["order"]:
|
||||
if v in plan_by_idx and v not in ordered_idx:
|
||||
ordered_idx.append(v)
|
||||
for i in sorted(plan_by_idx):
|
||||
if i not in ordered_idx:
|
||||
ordered_idx.append(i)
|
||||
plan = [plan_by_idx[i] for i in ordered_idx]
|
||||
|
||||
# raw 页图固定落会话根下的 out/;重生成页数变少时,成功后清掉多余编号。
|
||||
png_dir = _png_dir(os.getcwd())
|
||||
pdf_stem = os.path.splitext(os.path.basename(output))[0]
|
||||
png_pad = max(2, len(str(len(plan))))
|
||||
png_n = 0
|
||||
png_paths = []
|
||||
|
||||
# ---------------- 批次方向
|
||||
aspects = [m["aspect"] for (_, _, m) in plan]
|
||||
batch_orientation, _, median_aspect = \
|
||||
compose.decide_orientation(aspects, orientation, P)
|
||||
dead = P["gate"]["aspect_deadband"]
|
||||
if batch_orientation in ("portrait", "landscape"):
|
||||
same = [m["aspect"] for (_, _, m) in plan
|
||||
if compose.classify(m["aspect"], dead) == batch_orientation]
|
||||
batch_median = float(statistics.median(same)) if same else float(median_aspect)
|
||||
else:
|
||||
batch_median = float(median_aspect)
|
||||
|
||||
batch_ratio = None
|
||||
if paper["ratio"] is None and orientation != "none":
|
||||
m_class = compose.classify(batch_median, dead)
|
||||
if m_class == batch_orientation:
|
||||
batch_ratio = batch_median
|
||||
elif m_class != "square":
|
||||
batch_ratio = 1.0 / batch_median
|
||||
|
||||
# 人工改过的页不再重新评级;自动页按方案表分级后与已有状态取较严重者
|
||||
# grade_page 的 ref_ratio 约定是「竖版为正」,横版批次要先折回竖版,否则
|
||||
# 所有横页都会被判成 paper_mismatch。
|
||||
ref_ratio = paper["ratio"]
|
||||
if ref_ratio is None:
|
||||
ref_ratio = batch_median
|
||||
if ref_ratio > 1.0:
|
||||
ref_ratio = 1.0 / ref_ratio
|
||||
for (_, _, m) in plan:
|
||||
if m.get("manual"):
|
||||
continue
|
||||
st2, r2 = state.grade_page(m, P, ref_ratio)
|
||||
m["status"], m["reason"] = state.escalate(
|
||||
m.get("status") or "ready", m.get("reason"), st2, r2)
|
||||
|
||||
# ---------------- 出页
|
||||
out_meta = {}
|
||||
for (idx, src, meta) in plan:
|
||||
page_class = compose.classify(meta["aspect"], dead)
|
||||
auto_rotate = 0
|
||||
if orientation == "auto":
|
||||
if page_class not in ("square", batch_orientation):
|
||||
auto_rotate = 90
|
||||
elif orientation in ("portrait", "landscape"):
|
||||
if page_class not in ("square", orientation):
|
||||
auto_rotate = 90
|
||||
|
||||
edit = (ed or {}).get("pages", {}).get(idx) if ed else None
|
||||
user_r = edit.get("rotate") if edit else None
|
||||
applied = int(user_r) if user_r is not None else auto_rotate
|
||||
applied = ((applied % 360) + 360) % 360
|
||||
if applied not in (0, 90, 180, 270):
|
||||
applied = 0
|
||||
|
||||
out_corners = [list(map(float, q)) for q in meta["corners"]]
|
||||
for _ in range(applied // 90):
|
||||
out_corners = compose.rotate_quad_cw(out_corners)
|
||||
|
||||
meta["rotated_deg"] = applied
|
||||
if not meta.get("manual"):
|
||||
meta["auto_rotate"] = auto_rotate
|
||||
|
||||
if orientation == "none":
|
||||
out_aspect = _aspect(out_corners)
|
||||
page_orient = compose.classify(out_aspect, dead)
|
||||
if page_orient == "square":
|
||||
page_orient = "portrait"
|
||||
elif orientation in ("portrait", "landscape"):
|
||||
page_orient = orientation
|
||||
else:
|
||||
# auto 模式下,用户显式旋转过就按旋转后的有效方向出画布;
|
||||
# 否则跟批次统一方向。否则 90° 旋转会把内容塞进反向画布压扁。
|
||||
if user_r is not None:
|
||||
page_orient = compose.classify(_aspect(out_corners), dead)
|
||||
if page_orient == "square":
|
||||
page_orient = batch_orientation \
|
||||
if batch_orientation in ("portrait", "landscape") \
|
||||
else "portrait"
|
||||
else:
|
||||
page_orient = batch_orientation
|
||||
|
||||
out_ratio = None
|
||||
if batch_ratio is not None:
|
||||
eff_w, eff_h = compose.aspect_of(out_corners)
|
||||
eff = eff_w / eff_h
|
||||
if abs(eff - batch_ratio) / batch_ratio \
|
||||
<= P["gate"]["paper_mismatch_pct"] / 100.0:
|
||||
out_ratio = batch_ratio
|
||||
|
||||
tw, th, canvas = compose.target_size(
|
||||
out_corners, paper, page_orient, P, out_ratio)
|
||||
|
||||
try:
|
||||
img = load_image(src)
|
||||
try:
|
||||
raw = compose.warp_raw(img, out_corners)
|
||||
finally:
|
||||
img.close()
|
||||
except Exception as e:
|
||||
err = _err("INTERNAL",
|
||||
"第 %d 页重新读取/透视变换失败:%s" % (idx, e), idx,
|
||||
"重试;若持续出现请反馈")
|
||||
meta["status"] = "render_failed"
|
||||
meta["reason"] = "render_failed"
|
||||
meta["error"] = err
|
||||
meta["warnings"].append(err["message"])
|
||||
out_meta[idx] = {"page": None, "canvas": None, "error": err}
|
||||
continue
|
||||
|
||||
try:
|
||||
page = compose.page_for_pdf(raw, (tw, th), P)
|
||||
except Exception as e:
|
||||
err = _err("INTERNAL",
|
||||
"第 %d 页生成 PDF 页面失败:%s" % (idx, e), idx,
|
||||
"重试;若持续出现请反馈")
|
||||
meta["status"] = "render_failed"
|
||||
meta["reason"] = "render_failed"
|
||||
meta["error"] = err
|
||||
meta["warnings"].append(err["message"])
|
||||
out_meta[idx] = {"page": None, "canvas": None, "error": err}
|
||||
continue
|
||||
|
||||
# raw 页图:自然尺寸、不调色、无白边。编号跟最终页序走。
|
||||
png_n += 1
|
||||
png_name = "%s-%0*d.png" % (pdf_stem, png_pad, png_n)
|
||||
png_path = os.path.join(png_dir, png_name)
|
||||
try:
|
||||
raw.save(png_path, "PNG", compress_level=6)
|
||||
png_paths.append(_rel(os.getcwd(), png_path))
|
||||
except Exception as e:
|
||||
meta["warnings"].append("第 %d 页原图导出失败:%s" % (idx, e))
|
||||
|
||||
meta["out_size"] = [int(tw), int(th)]
|
||||
meta["canvas_size"] = [int(canvas[0]), int(canvas[1])]
|
||||
meta["page_orientation"] = page_orient
|
||||
out_meta[idx] = {"page": page, "canvas": canvas, "error": None}
|
||||
|
||||
cov = meta.get("coverage") or {}
|
||||
if cov:
|
||||
cmin = min(float(v) for v in cov.values())
|
||||
if cmin < P["gate"]["coverage_warn"]:
|
||||
meta["warnings"].append(
|
||||
"四边覆盖率最低 %.2f(低于 %.2f),角点可能不准"
|
||||
% (cmin, P["gate"]["coverage_warn"]))
|
||||
if meta.get("reason") == "detect_failed":
|
||||
meta["warnings"].append(
|
||||
"第 %d 页未检测到纸张四角,已用画面内缩兜底框出件;需人工补角"
|
||||
% idx)
|
||||
elif meta.get("reason") == "invalid_quad":
|
||||
meta["warnings"].append(
|
||||
"第 %d 页角点校验未通过,已用兜底框出件;需人工补角" % idx)
|
||||
elif meta.get("fallback"):
|
||||
names = ["上", "右", "下", "左"]
|
||||
sides = "、".join(names[s] for s in meta.get("fallback_sides", [])
|
||||
if 0 <= s < 4)
|
||||
if sides:
|
||||
meta["warnings"].append(
|
||||
"第 %s 边使用了掩膜回退角点,精度可能略低" % sides)
|
||||
for w in (meta.get("validation") or {}).get("warnings", []):
|
||||
meta["warnings"].append(w)
|
||||
if (not meta.get("manual") and batch_orientation != "mixed"
|
||||
and batch_median > 0 and page_class == batch_orientation):
|
||||
dev = abs(meta["aspect"] - batch_median) / batch_median * 100.0
|
||||
if dev > P["gate"]["paper_mismatch_pct"]:
|
||||
meta["warnings"].append(
|
||||
"宽高比与批次中位相差 %.0f%%,可能不是同一种纸;"
|
||||
"可改 paper=auto 或单独处理" % dev)
|
||||
if applied != 0 and user_r is None:
|
||||
meta["warnings"].append(
|
||||
"方向与批次不一致,已旋转 %d°;文字方向需人工确认" % applied)
|
||||
|
||||
# ---------------- 页序 / PDF
|
||||
# plan 已按 ed.order + 源序号排好;只保留真正出页成功的。
|
||||
order = [i for (i, _s, _m) in plan
|
||||
if out_meta.get(i, {}).get("page") is not None]
|
||||
|
||||
ordered_pages = [out_meta[i]["page"] for i in order]
|
||||
ordered_canvases = [out_meta[i]["canvas"] for i in order]
|
||||
if not ordered_pages:
|
||||
failures = unreadable or [
|
||||
_err("RENDER_FAILED", "所有页面都未能出页", None, "重试或检查原图")]
|
||||
res = finish_failure(pages, failures, output, state_path, t0)
|
||||
_mark_error(conn, batch_id, res["summary"])
|
||||
return res
|
||||
|
||||
try:
|
||||
compose.build_pdf(ordered_pages, output, ordered_canvases, P)
|
||||
except Exception as e:
|
||||
err = _err("PDF_WRITE_FAILED", "写 PDF 失败:%s" % e, None,
|
||||
"确认输出目录可写、磁盘空间足够")
|
||||
res = finish_failure(pages, [err], output, state_path, t0)
|
||||
_mark_error(conn, batch_id, res["summary"])
|
||||
return res
|
||||
|
||||
# PDF 落盘成功后再清多余页图:失败时保留上一份,避免两边都不完整。
|
||||
_clean_pngs(png_dir, pdf_stem, png_n)
|
||||
|
||||
# ---------------- 检验
|
||||
try:
|
||||
checks = check.inspect_pdf(output, len(ordered_pages),
|
||||
P["check"]["dpi"], P)
|
||||
except Exception as e:
|
||||
checks = {"ok": None, "error": str(e), "pages": [],
|
||||
"page_count": None}
|
||||
|
||||
warnings = []
|
||||
for m in pages:
|
||||
warnings.extend(m.get("warnings", []))
|
||||
if checks.get("ok") is None:
|
||||
warnings.append("PDF 自检无法执行:%s" % checks.get("error"))
|
||||
else:
|
||||
for item in checks.get("pages", []):
|
||||
orig = order[item["index"] - 1] if 0 < item["index"] <= len(order) \
|
||||
else item["index"]
|
||||
if item.get("box_fallback"):
|
||||
warnings.append(
|
||||
"第 %d 页未能稳定识别纸面区域,边缘检验仅供参考" % orig)
|
||||
if not item["short_side_ok"]:
|
||||
warnings.append(
|
||||
"第 %d 页纸面短边 %dpx 偏低,可能分辨率不足"
|
||||
% (orig, item["short_side_px"]))
|
||||
if not item["border_ok"]:
|
||||
warnings.append(
|
||||
"第 %d 页边缘有背景残留,角点可能落在背景上" % orig)
|
||||
if not item["cut_ok"]:
|
||||
warnings.append(
|
||||
"第 %d 页内容贴边,可能裁到了内容" % orig)
|
||||
if (paper["ratio"] is not None
|
||||
and item["ratio_dev_pct"] > P["check"]["ratio_tol"] * 100):
|
||||
warnings.append(
|
||||
"第 %d 页纸面比例与页面相差 %.1f%%"
|
||||
% (orig, item["ratio_dev_pct"]))
|
||||
pc = checks.get("page_count") or {}
|
||||
if pc and not pc.get("ok"):
|
||||
warnings.append(
|
||||
"PDF 页数与出页数不一致(期望 %s,实际 %s)"
|
||||
% (pc.get("expected"), pc.get("actual")))
|
||||
|
||||
# ---------------- 记录顺序 / 审核状态
|
||||
sort_map = {idx: pos for pos, idx in enumerate(order)}
|
||||
next_sort = len(order)
|
||||
for idx in all_indices:
|
||||
if idx not in sort_map:
|
||||
sort_map[idx] = next_sort
|
||||
next_sort += 1
|
||||
|
||||
needs = []
|
||||
for m in pages:
|
||||
if m.get("status") in ("review", "needs_corners", "needs_orientation",
|
||||
"render_failed", "failed"):
|
||||
needs.append(m)
|
||||
|
||||
# 发布审核页:always 总是发布;auto 只在有页需要人看时发布
|
||||
publish = review_mode == "always" or (review_mode == "auto" and bool(needs))
|
||||
thumbs = {}
|
||||
if publish and batch_id:
|
||||
_, thumbs = review_page.publish(
|
||||
session_root, batch_id,
|
||||
[{"idx": m["index"], "source_abs": m.get("source_abs")}
|
||||
for m in pages
|
||||
if not m.get("deleted") and m.get("source_abs")])
|
||||
|
||||
# ---------------- state v2
|
||||
order_all = sorted(pages, key=lambda m: (sort_map.get(m["index"], m["index"]),
|
||||
m["index"]))
|
||||
state_pages = []
|
||||
for m in order_all:
|
||||
src_rel = _rel(session_root, m.get("source_abs") or "")
|
||||
auto_r = int(m.get("auto_rotate") or 0)
|
||||
state_pages.append({
|
||||
"index": m["index"],
|
||||
"source": src_rel,
|
||||
"source_size": m.get("source_size"),
|
||||
"auto_corners": m.get("auto_corners"),
|
||||
"corners_norm": m.get("corners_norm"),
|
||||
"rotate": auto_r,
|
||||
"rotate_source": "manual" if m.get("manual") else "auto",
|
||||
"status": m.get("status"),
|
||||
"reason": m.get("reason"),
|
||||
})
|
||||
|
||||
state_obj = state.build_state({
|
||||
"paper": paper_input,
|
||||
"enhance": enhance,
|
||||
"dpi": int(P["render"]["dpi"]),
|
||||
"orientation": orientation,
|
||||
}, state_pages)
|
||||
state_ok = write_state(state_path, state_obj)
|
||||
if not state_ok:
|
||||
warnings.append("state 文件写入失败:%s" % state_path)
|
||||
|
||||
if checks.get("ok") is False:
|
||||
summary = "%d 页已生成;自检有未通过项" % len(ordered_pages)
|
||||
elif needs:
|
||||
summary = "%d 页已生成;%d 页建议在审核页确认" % (
|
||||
len(ordered_pages), len(needs))
|
||||
elif warnings:
|
||||
summary = "%d 页已生成;%d 项提示,见审核页" % (
|
||||
len(ordered_pages), len(warnings))
|
||||
else:
|
||||
summary = "%d 页已生成" % len(ordered_pages)
|
||||
if png_paths:
|
||||
summary += ";原始页图见 out/"
|
||||
|
||||
# ---------------- 写 data.db
|
||||
if use_db:
|
||||
output_rel = _rel(session_root, output)
|
||||
review_store.upsert_batch(conn, batch_id, output_rel, {
|
||||
"paper": paper_input, "enhance": enhance,
|
||||
"dpi": dpi_label, "orientation": orientation,
|
||||
})
|
||||
recs = []
|
||||
for m in pages:
|
||||
ti = thumbs.get(m["index"])
|
||||
size = m.get("source_size") or [None, None]
|
||||
rec = {
|
||||
"idx": m["index"],
|
||||
"source": _rel(session_root, m.get("source_abs") or ""),
|
||||
"source_w": size[0], "source_h": size[1],
|
||||
"auto_corners": m.get("auto_corners"),
|
||||
"auto_corners_norm": m.get("auto_corners_norm"),
|
||||
"corners": m.get("corners_norm"),
|
||||
"auto_rotate": int(m.get("auto_rotate") or 0),
|
||||
"rotate": int(m.get("rotated_deg") or 0),
|
||||
"deleted": 1 if m.get("deleted") else 0,
|
||||
"sort": sort_map.get(m["index"], m["index"]),
|
||||
"status": m.get("status"),
|
||||
"reason": m.get("reason"),
|
||||
"thumb": ti[0] if ti else None,
|
||||
"w": ti[1] if ti else (size[0] or None),
|
||||
"h": ti[2] if ti else (size[1] or None),
|
||||
"error": ((m.get("error") or {}).get("message")
|
||||
if isinstance(m.get("error"), dict)
|
||||
else m.get("error")),
|
||||
}
|
||||
recs.append(rec)
|
||||
review_store.replace_pages(conn, batch_id, recs)
|
||||
batch_status = "ready" if not needs else "review"
|
||||
review_store.finish_batch(conn, batch_id, True, summary,
|
||||
warnings, status=batch_status)
|
||||
if review_store.get_job(conn, batch_id):
|
||||
review_store.set_job(conn, batch_id, "done", summary)
|
||||
if is_new_batch:
|
||||
old, kept = review_store.prune(conn, keep=5, protect=batch_id)
|
||||
if old:
|
||||
review_page.prune_dirs(session_root, kept)
|
||||
|
||||
# AI 侧只回三件事:是否成功、一句摘要、PDF 文件名。
|
||||
# 完整访问地址由 core 的 visit_url 追加;pages/checks/批次 ID 都在 data.db/state 里。
|
||||
return {
|
||||
"ok": True,
|
||||
"summary": summary,
|
||||
"pdf": _rel(session_root, output),
|
||||
"images": png_paths,
|
||||
}
|
||||
|
||||
except Exception as e:
|
||||
_mark_error(conn, batch_id, e)
|
||||
raise
|
||||
finally:
|
||||
if conn:
|
||||
try:
|
||||
conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def finish_failure(pages, failures, output, state_path, t0):
|
||||
err = failures[0] if failures else _err("INTERNAL", "未知错误")
|
||||
return {
|
||||
"ok": False,
|
||||
"summary": "失败:%s" % err.get("message", "未知错误"),
|
||||
"error": err,
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 入口
|
||||
|
||||
def main():
|
||||
t0 = time.time()
|
||||
buf = io.StringIO()
|
||||
old_stderr = sys.stderr
|
||||
sys.stderr = buf
|
||||
try:
|
||||
if len(sys.argv) != 2:
|
||||
raise ValueError(
|
||||
'用法:python3 run.py \'{"images": ["图1.jpg"]}\'')
|
||||
try:
|
||||
args = json.loads(sys.argv[1])
|
||||
except json.JSONDecodeError as e:
|
||||
raise ValueError("参数不是合法 JSON:%s" % e)
|
||||
if not isinstance(args, dict):
|
||||
raise ValueError("参数必须是 JSON 对象")
|
||||
res = process(args, t0)
|
||||
except Exception as e:
|
||||
code = "BAD_ARGS" if isinstance(e, ValueError) else "INTERNAL"
|
||||
hint = ("检查参数后重试" if code == "BAD_ARGS"
|
||||
else "可用 DOC_DESKEW_DEBUG=1 重跑查看堆栈")
|
||||
res = {
|
||||
"ok": False,
|
||||
"summary": "失败:%s" % e,
|
||||
"error": _err(code, str(e), None, hint),
|
||||
}
|
||||
if os.environ.get("DOC_DESKEW_DEBUG"):
|
||||
res["logs"] = traceback.format_exc().splitlines()
|
||||
finally:
|
||||
sys.stderr = old_stderr
|
||||
|
||||
logs = [line for line in buf.getvalue().splitlines() if line.strip()]
|
||||
if logs:
|
||||
res["logs"] = (res.get("logs", []) + logs)[:P["output"]["max_logs"]]
|
||||
|
||||
print(json.dumps(res, ensure_ascii=False, separators=(",", ":"),
|
||||
default=_json_default))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: image-process
|
||||
description: 图像处理(Pillow)。查看图片信息、格式转换、缩放、裁剪、压缩、文字水印、九宫格切片、多图拼图。仅做确定性图像处理,不做 AI 生成/修图
|
||||
description: 普通图片/截图的确定性编辑(Pillow):查看信息、转格式、缩放、裁剪、压缩、加水印、九宫格切片、多图拼图
|
||||
input_schema:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
Reference in New Issue
Block a user