video-gen: auto-expire uploaded files older than UPLOAD_TTL_HOURS
Adds cleanupUploads(): on startup and every hour, deletes files in UPLOAD_DIR whose mtime is older than UPLOAD_TTL_HOURS (default 24). Uses fs.promises + parallel stat/unlink, unref() the interval so it never blocks process shutdown, ignore ENOENT races silently. Storage is multer diskStorage so memory never holds a full file body, but disk could grow without bound if uploads pile up unused. This bounds the disk footprint without requiring a manual cron, and keeps the 'temp upload' semantics: the URL is meant to be consumed within the same session anyway. README: document UPLOAD_TTL_HOURS / UPLOAD_DIR env vars, add design note for the cleanup loop.
This commit is contained in:
@@ -81,6 +81,14 @@ environment:
|
||||
|
||||
未设置时 fallback 到 `http://localhost:3000`(本机调试)。
|
||||
|
||||
可选环境变量:
|
||||
|
||||
| 变量 | 默认 | 说明 |
|
||||
|---|---|---|
|
||||
| `PUBLIC_BASE_URL` | `http://localhost:3000` | 返回 `url` 字段里拼接的对外 base URL |
|
||||
| `UPLOAD_DIR` | `/tmp/video-gen-uploads` | 磁盘存储目录(容器内临时) |
|
||||
| `UPLOAD_TTL_HOURS` | `24` | 超过该小时数的文件会在启动时和每小时被自动删除 |
|
||||
|
||||
### 接口
|
||||
|
||||
```
|
||||
@@ -98,6 +106,7 @@ file: <二进制图片文件>
|
||||
| 白名单 content-type | `image/png`、`image/jpeg`、`image/jpg` |
|
||||
| 存储位置 | 容器内 `/tmp/video-gen-uploads/`(`UPLOAD_DIR` 可覆盖) |
|
||||
| 持久化 | **不持久化**。容器重启即丢,请尽快消费 URL |
|
||||
| 过期清理 | 启动时 + 每小时一次删除 `>UPLOAD_TTL_HOURS` 的文件,默认 24h |
|
||||
| 鉴权 | 无 |
|
||||
|
||||
### 返回
|
||||
@@ -355,6 +364,7 @@ package.json # @modelcontextprotocol/sdk + express + multer
|
||||
- **异步语义暴露给 AI**:服务端不做轮询,`generate_video` 只提交、`query_video` 只查一次。AI 自主决定节奏
|
||||
- **服务端自动 `request_id`**:每次提交服务端生成一个 UUID 作为智谱 `request_id`,便于日志关联,但不暴露给 AI
|
||||
- **上传接口与 MCP 走同一进程**:额外 `POST /upload` + `GET /uploads/*` 不走 MCP、不鉴权,靠内网隔离;`PUBLIC_BASE_URL` 环境变量控制返回 URL 的拼接 host,AI 看到的 tool description 启动时拼接进真实地址,避免 AI 自由发挥
|
||||
- **上传过期清理**:启动时 + 每小时一次删除 `mtime > UPLOAD_TTL_HOURS` 的文件,避免磁盘被不消费的上传堆满;清理 timer 用 `unref()` 不阻塞进程退出
|
||||
|
||||
### 默认模型
|
||||
|
||||
|
||||
@@ -42,6 +42,50 @@ const UPLOAD_ALLOWED_MIME = new Set([
|
||||
"image/jpg",
|
||||
]);
|
||||
|
||||
// 过期清理:UPLOAD_TTL_HOURS 小时之前的文件会被定时删除
|
||||
const UPLOAD_TTL_HOURS = (() => {
|
||||
const n = parseInt(process.env.UPLOAD_TTL_HOURS || "24", 10);
|
||||
return Number.isFinite(n) && n > 0 ? n : 24;
|
||||
})();
|
||||
const UPLOAD_TTL_MS = UPLOAD_TTL_HOURS * 60 * 60 * 1000;
|
||||
const UPLOAD_CLEANUP_INTERVAL_MS = 60 * 60 * 1000; // 每小时一次
|
||||
|
||||
// 清理 UPLOAD_DIR 下 mtime 早于 TTL 的文件。返回 {kept, removed, errors}
|
||||
async function cleanupUploads({ now = Date.now(), dir = UPLOAD_DIR } = {}) {
|
||||
let entries;
|
||||
try {
|
||||
entries = await fs.promises.readdir(dir);
|
||||
} catch (e) {
|
||||
if (e && e.code === "ENOENT") return { kept: 0, removed: 0, errors: 0 };
|
||||
console.error(`[video-gen] cleanup: readdir failed: ${e.message}`);
|
||||
return { kept: 0, removed: 0, errors: 1 };
|
||||
}
|
||||
|
||||
let kept = 0;
|
||||
let removed = 0;
|
||||
let errors = 0;
|
||||
await Promise.all(
|
||||
entries.map(async (name) => {
|
||||
const full = path.join(dir, name);
|
||||
try {
|
||||
const st = await fs.promises.stat(full);
|
||||
if (!st.isFile()) return;
|
||||
if (now - st.mtimeMs >= UPLOAD_TTL_MS) {
|
||||
await fs.promises.unlink(full);
|
||||
removed++;
|
||||
} else {
|
||||
kept++;
|
||||
}
|
||||
} catch (e) {
|
||||
if (e && e.code === "ENOENT") return;
|
||||
errors++;
|
||||
console.error(`[video-gen] cleanup: ${name}: ${e.message}`);
|
||||
}
|
||||
})
|
||||
);
|
||||
return { kept, removed, errors };
|
||||
}
|
||||
|
||||
function parseArgs(argv) {
|
||||
const args = {
|
||||
http: false,
|
||||
@@ -415,8 +459,29 @@ async function startHttp({ port, host }) {
|
||||
console.log(`[video-gen] PUBLIC_BASE_URL = ${publicBaseUrl}`);
|
||||
|
||||
// ===== /upload:接收图片,返回 URL(不走 MCP)=====
|
||||
// 磁盘存储:保证 UPLOAD_DIR 存在(容器内临时,重启即丢)
|
||||
fs.mkdirSync(UPLOAD_DIR, { recursive: true });
|
||||
// 磁盘存储:保证 UPLOAD_DIR 存在(容器内临时,重启即丢)
|
||||
fs.mkdirSync(UPLOAD_DIR, { recursive: true });
|
||||
|
||||
// 过期清理:启动时跑一次,之后每小时一次
|
||||
cleanupUploads()
|
||||
.then((r) =>
|
||||
console.log(
|
||||
`[video-gen] startup cleanup: kept=${r.kept} removed=${r.removed} errors=${r.errors} ttl=${UPLOAD_TTL_HOURS}h`
|
||||
)
|
||||
)
|
||||
.catch((e) => console.error(`[video-gen] startup cleanup failed: ${e.message}`));
|
||||
const cleanupTimer = setInterval(() => {
|
||||
cleanupUploads()
|
||||
.then((r) => {
|
||||
if (r.removed > 0 || r.errors > 0) {
|
||||
console.log(
|
||||
`[video-gen] periodic cleanup: kept=${r.kept} removed=${r.removed} errors=${r.errors}`
|
||||
);
|
||||
}
|
||||
})
|
||||
.catch((e) => console.error(`[video-gen] periodic cleanup failed: ${e.message}`));
|
||||
}, UPLOAD_CLEANUP_INTERVAL_MS);
|
||||
cleanupTimer.unref(); // 不阻止进程退出
|
||||
|
||||
const storage = multer.diskStorage({
|
||||
destination: (_req, _file, cb) => cb(null, UPLOAD_DIR),
|
||||
|
||||
Reference in New Issue
Block a user