Compare commits

106 Commits
Author SHA1 Message Date
lxy 0317873a5b 文档: 上下文管理设计/对标u-claw差距分析与看板登记
- 专项设计:M1-M4四批修复蓝图+对标u-claw六能力画像与P0-P2补差
- 看板:M1销账(验收证据)/收尾令/对标P0-P2条目登记
2026-10-19 00:25:52 +08:00
lxy eec5978fa7 修复: 上下文管理第二批(E5注入产物/E13超时收敛/E2降级透出)
- E5:[补充指令]与[系统恢复]同治三处——载入滤出/恢复回放剔除/
  首条任务消息排除(434c8c40占位实证),is_guide_artifact统一判据
- E13:同工具超时连败≥3跨签名聚合,执行前软收敛警告(治换档位调参
  逃逸,434c8c40 run_command 8连败实证)
- E2:压缩失败走关键词摘要兜底时新发AiContextDegraded,前端toast
  「上下文已降级」不再静默失忆(76730ce8 seq166实证)
- vite watch忽略target/(cargo编译写.o触发EBUSY杀dev进程)
- provider-list模板根div缺闭合补齐(拆分半成品断链)

验收:tool_failure 5/keyword_fallback 3/recovery 6 passed;vue-tsc EXIT 0
2026-10-19 00:25:51 +08:00
lxy 25ec9cf69a 优化: 前端u-agent令牌对齐与组件拆分续批
- 全局样式类(df-sec-h/df-toast族)替换散装实现,主题token去硬编码兜底
- 拆分子组件:ai-chat/queue-panel(队列时钟与编辑随迁)、
  conversation-sidebar/ConversationList、settings/provider-panel
- 父组件死副本清理(队列编辑/时钟/键位表,TS6133清零)并补
  fill-draft父侧转发(重构半成品断链修复)
- 验收:vue-tsc EXIT 0
2026-10-18 23:15:57 +08:00
lxy 302c4bcf52 修复: 飞书审批决议终态闭环与决议词容错
- F5-1:决议词归一(trim/剥尾标点/同义:批准|同意|拒绝|不同意)
- F5-2:审批终态卡原地更新(桌面决议/超时拒后不再恒挂可点);
  迟到决议30min窗内提示已处理,不再注入运行中任务
- APPROVAL_CARDS/RESOLVED_CHATS内存句柄,重启后走幂等回执无损失
2026-10-18 23:15:51 +08:00
lxy 05d0f89c16 修复: aichat上下文管理与信息组织P1批
逐件(证据:深查-aichat质量/信息UX-2026-10-17):
- 恢复链套娃:RECOVERY_ARTIFACT_PREFIX双侧过滤+单点拦截(E1)
- message_repo全量重写保留原created_at(E11,治1750条时间线污染)
- search_files/grep 0命中附阴性存疑提示(E6,共享negative_hit_hint)
- 项目清单三源活跃度排序+system prompt指纹对齐(E4跑题)
- 会话列表活跃/归档offset分页+侧栏加载更多(F4-1,治50硬上限)
- inject消息source标记(轮次分桶不误计发起,V52同源)
- 提示词回复风格条款中英双语(反口水话,含测试断言)
- i18n补键:复制/折叠量/发送title/历史定位提示(F1-1/F2-1/F6-1)

验收:cargo test --workspace 2281 passed 0 failed;vue-tsc EXIT 0
2026-10-18 23:15:45 +08:00
lxy a273a86a7d 优化: 参考 u-agent 管理台范式升级 UI 设计令牌与公共组件 2026-10-18 15:35:15 +08:00
lxy 45f0560f85 修复: 飞书长连接假活自愈——探活持续断流后主动断开重建
看门狗连续 3 个巡检周期(断流约13分钟)判定假活后 Notify 唤醒
run_loop 主动退出,复用外层 backoff 重连循环重建全新 WS 握手
只告警不自愈的盲区已补齐,比 SDK 内部重连更彻底(全新握手)
2026-10-17 22:19:57 +08:00
lxy f74d51b905 优化: 会话侧栏字体层级收敛+v0.1.3发布收尾脚本入库,归档飞书WS假活排查文档 2026-10-17 21:54:25 +08:00
lxy e08b0f4311 优化: 升版 0.1.2 -> 0.1.3,启动 B2/B3 端到端升级验证 2026-10-17 21:18:03 +08:00
lxy 3bb662f092 修复: 工具外置集成测试夹具失效
- 夹具误用错误行信号(.rs:)做普通大结果,错误行全量保留致压缩断言先天不成立,改中性行
- 生产行为缺陷(错误行洪泛击穿压缩,grep/rustc 类输出为主要受害者)登记看板待修
2026-10-15 14:27:33 +08:00
lxy a644e93604 修复: 项目详情页三态分离与反馈一致性
- notFound/已归档/加载失败三态分离:单查兜底+归档恢复入口+加载失败重试钮
- 反馈一致性六项:同步守卫与完成提示/首屏并行加载/筛选空态文案/计数改过滤口径/路径勾选 seq 守卫/删除确认危险态
- 文件列表滚动条对齐全局范式(scrollbar-width:thin 致 Chromium 忽略定制样式)
- 删除失败不再跳转(store 返布尔)/幽灵勾选清理/折叠阈值与 CSS 对齐
2026-10-15 14:27:32 +08:00
lxy 795935bf6a 新增: 飞书文件消息入站与补充注入链完善
- 文件消息入站:20MB 上限落盘引用,模型经 read_file 按需读盘;下载带 3 次退避+120s 超时防线,复用媒体域 hash 去重/OSS 双写
- 群聊门控对齐 u-claw 模式:前移至类型分流前,@自己/@所有人响应,未艾特静默旁观(原门控后置致未支持类型误回提示卡)
- 修复补充注入丢失:pending_injects 未消化计数+收敛出口续轮+桌面注入 IPC+前端本地气泡
- df-ai-core 测试构造器补 source 字段(溯源字段遗漏,全量门禁暴露)
2026-10-15 14:27:22 +08:00
lxy 7548896f6d 文档: 登记目标污染与停滞判定实证待办 2026-10-13 13:29:17 +08:00
lxy 5faaa88302 新增: 知识路标投影进系统提示词
- published 且验证或高复用条目按项目优先进投影稳定段
- 段尾固定引导工具故障先检索路标,治有经验不想查
- 与规则文件投影共用会话级缓存,空库零回归
2026-10-13 13:29:16 +08:00
lxy 8c4aa5dc5b 新增: 工具失败连败升级与熔断停靠
- 同工具连败三级提示:再败软提示,三败强制换路并引导检索经验
- 连拦五次强制安全停靠,不给无视熔断的余地
- 连击按会话计数防参数抖动绕过,成功即清零
2026-10-13 13:29:15 +08:00
lxy 9142e30714 新增: 项目规则文件投影进系统提示词
- 会话启动读 devflow.md 纪律与领单两分区,CLAUDE.md 同目录自动并入
- system prompt 稳定尾纯追加,主会话与子代理双路径同源
- 分区字符上限裁剪,会话级缓存同会话只读一次盘
- 文件缺失零回归,缓存前缀零击穿
2026-10-12 18:12:36 +08:00
lxy 9e26d6b19c 优化: 脉络卡归属置顶与源会话跳转
- 当前会话归属项目组置顶,focused 标记高亮
- source_ref 解析出可跳转会话 id,消息级引用反查补全
- 条目相对时间/空态引导/标题点击跳源,失败静默
2026-10-12 18:12:26 +08:00
lxy cb23061088 新增: 跨会话召回四层架构与脉络卡
- 会话终态提炼: 删除前+停靠双挂点,快照与清水位同临界区
- 目标快照: 恢复引导按最终有效意图生成,旧目标不复活
- 知识检索项目维度软排序,WorkingContext 持久化迁移
- 脉络卡: published 知识按项目聚合纯读侧,带入上下文经输入框显式注入
- 压缩与缓存能力在主循环/子代理/提示词三面接线
- 两份专项设计文档与看板销账
2026-10-11 22:27:20 +08:00
lxy 6f5c6b7532 优化: 上下文压缩缓存友好化与召回句柄
- 块状裁剪: 水位目标+轮边界吸附+病态回退,贪心路径保留为开关回退
- 存储态 System 锚点出请求时视图转 user,治前缀缓存整体折叠失配
- BP3 断点机制+缓存命中率观测日志,调参有据
- 压缩摘要可召回: JSONL 档案落盘+句柄行注入,take 语义一次性消费
- 字符估算修正: CJK 分档计权+改图地板,估算对象对齐实际消耗
- namespace 引用跨重启内容哈希懒回填
2026-10-11 22:27:08 +08:00
lxy c7d0ed939c 新增: 连发消息防抖与长命令后台化
- 飞书连发消息 0.5s 窗(拍板值)合并为一次任务:文本空行拼接+图片到达序合并
- 命令/审批/注入即时路径不进窗;追加消息并入即静默返回;窗满统一受理,守卫拒绝落待发箱
- run_command 超 grace 窗(默认 90s,可配)转后台:LLM 收非终态结果,不再整轮阻塞
- 后台任务自持心跳+删除自中止+失败记忆+完成通知+回填续跑;占用闸单点保会话态自洽
- 开关默认关=同步路径逐字节不变;新增 13 测
2026-10-11 12:37:23 +08:00
lxy fda991cbe1 优化: 上下文压缩域补偿与召回设计
- 裁剪淘汰补偿摘要上限 300→1000 字:原值对几十轮历史形同虚设,放大后相对裁剪收益仍是零头
- 测试断言改「不超上限」(结构性产出远低于天花板,精确等于过脆)
- 压缩可召回(压缩不丢/可回读)七条实施要点入档,待小批实施
2026-10-11 12:37:22 +08:00
lxy f9d3b4c76b 修复: namespace 引用按真实体量入账估算
- ref 尾段 ?len= 携带原文字节数, 估算器解析入账(原 ~15 token 低估致预算/裁剪/压缩三门失效)
- 走查五线复核通过: 入口唯一/展开在发送链/估算无旁路/压缩扣减闭环/子代理同效
- 归一方案否决留档: 近度豁免失效等四破坏, 重consider 条件三条
- 已知取舍: 字节直入字符口径中文 3x 高估, 方向安全
2026-10-11 11:14:23 +08:00
lxy 3a1b8ccda9 清理: 撤销 namespace 引用体量入账估算
方案重审: ref 尾段携带长度牵动协议格式/解析/测试多处,
收益与复杂度不成比例; 退回估算器原口径, 另寻不改协议的入账路径
2026-10-11 10:52:27 +08:00
lxy 10cd9d2465 修复: namespace引用按真实体量入账估算
ref 尾段携带原文字节数, 估算器解析入账(原 ~15 token 低估致
预算/裁剪/压缩三门失效); 旧格式向后兼容, 淘汰内容不虚占预算
2026-10-11 10:08:20 +08:00
lxy 96bab8cd49 修复: 子代理上下文预算恒128K不随模型窗口
子代理预算改继承父会话(同模型同窗口单一事实来源), 939-③ 收口;
939-⑪⑫ 核实已由前批落地, 看板销账更新
2026-10-11 09:35:49 +08:00
lxy 423c1df398 优化: 执行中折叠卡头补生成间隙即时摘要
工具间隙(LLM 写正文)显「生成回复中」, 折叠卡头任意时刻可见当前动作
2026-10-11 08:55:37 +08:00
lxy c2809bd927 新增: 压缩摘要折叠进所属轮次轨迹卡
- 压缩摘要进轮次桶随轨迹卡收起(旧口径原样展开作废)
- 轨迹卡内渲染为摘要 pill 单行, 点击展开全文(复用思考展开态)
- 压缩摘要不竞选最终答复气泡, 过程记录语义
2026-10-11 08:53:36 +08:00
lxy b4b18e351b 新增: 轮次按发起边界分桶与稳定编号
- 轮号=用户发起消息计数(单一真值来源), 单消息桶不吞号, 编号不再漂移
- 补充注入标 source=inject, 归属当前轮不开新轮(961 轮次语义)
- 压缩摘要/错误气泡归属当前轮, 压缩后不再另起新轮次
- reload 消息透传 source 字段, 轮次导航条自动同口径
2026-10-11 08:50:35 +08:00
lxy 461354d7f4 新增: 执行中轨迹卡展开与占用期消息必达
- 执行中 step 卡支持点击展开(复用轨迹展开态, 实时过程工具卡回看)
- 占用期消息改落待发箱自动续投, 不再拒绝让用户重发(959)
- 待发项派发后完成结果回推来源飞书会话(35min 弃守)
- 961 轮次语义/960 聚合防抖设计要点登记
2026-10-11 07:49:34 +08:00
lxy 3dcb3c6fea 新增: 截断标记事件契约字段(953 后端配套)
AiChatEvent 补 truncated_by_max_tokens, 与前端提示条消费配套
2026-10-11 07:38:04 +08:00
lxy f894196924 优化: 换装脚本本地模式适配与版本号
- 未设签名私钥时自动关 updater 产物, 免 tauri build 中断
- 发版(需签名)分流 publish-release.ps1, 本地换装/正式发布分离
2026-10-11 07:38:03 +08:00
lxy 78dd079344 文档: 签名发布SOP与两份走查报告落档
- 签名与发布 SOP: 签名→打包→发布清单→自动升级全链可重复执行
- 上下文管理五维走查全文(瓶颈清单已登记 939)
- 版本升级链走查报告
2026-10-11 07:38:02 +08:00
lxy cd77c1d6c7 优化: Anthropic prompt 缓存默认开启
- 默认开(原默认关): 长会话显式缓存断点省 token, 按端点类型自动判定
- 仅支持缓存的端点生效, 其余零影响; 网关异常可手动关闭回退
2026-10-11 07:38:01 +08:00
lxy 3818a63522 新增: 待发箱抽屉与对话体验前端批
- 待发箱抽屉: ChatInput ⏰ 入口 + OutboxDrawer + useAiOutbox IPC 封装
- U2 进度语义三层: 内容层(最近动作) + 静默三档降速脉冲, 永不自动判死
- 流式心跳自愈: 竞态窗口旧任务迟到完成事件误收敛的自愈判定(949 前端兜底)
- 953 截断提示: 末条气泡渲染截断标记 + 继续输出按钮, 事件契约透传
2026-10-11 07:38:00 +08:00
lxy 31510d3555 新增: 工具结果落盘瘦身与空轮自愈
- 超阈值工具结果正文落盘+句柄可读回(治 read_file 29.6% 体量裸奔)
- read_file 改小文件豁免(8KB), 超阈值压缩+落盘句柄, 无损可找回
- 空轮判据自愈: 连续空轮先 nudge 续轮, 达阈值停靠止损(开关可回退)
- 零 token 收敛信号透传(非截断语义分离)
- 上下文清单描述按字符截断, 防清单体量无限膨胀
2026-10-11 07:33:27 +08:00
lxy 18e50c5af7 新增: 媒体引用化存储与消息溯源
- media_store 模块: sha256 去重落盘 + 阿里云 OSS 双写(签名手写), OSS 失败降级本机引用
- 飞书图片入站: image 消息下载装配 vision 片, 3 次退避+120s 超时+残件清理防线
- 出站回传: 报告 [交付] 路径行自动上传飞书原生附件, 凭据/密钥名拦截兜底
- 溯源 V52: ai_messages.source 列, 飞书 msg_id/chat_id/发送者随消息落库
- spawn offload: 桌面粘贴图 base64 统一转落盘引用, SQLite 不进大字符串
- 引用统一 url 字段(https/file://)不自造 scheme, image_ref 发送前按需展开
2026-10-11 07:33:18 +08:00
lxy 8aefe3254c 修复: df-im-gateway Hello 握手补 proto 字段
- Wave A 7b 给 df-tunnel 的 Hello 结构体新增必填 proto 字段,但 df-im-gateway(commander.rs:84)的构造点漏同步 → cargo check --workspace 挂 E0063
- 补 proto: 1(serde 缺省 v1,与旧服务端兼容);教训:跨 crate 结构体改动自证必须 cargo check --workspace(-p 单 crate 不覆盖下游 bin)
2026-10-10 22:32:48 +08:00
lxy 39e0eac39d 优化: AI Chat 渲染与无障碍 + 文件查看 + 同步 IO 收口
- 渲染: 结果/过程分离(气泡只放最终结果,过程轮叙述+工具卡收进轨迹卡默认收起可展开——治『流式后正文消失』且不再 N 气泡打扰)
- 渲染 P0: v-memo 定向 dep(只有真正渲染流式块的 item 失效,其余取占位) + renderItems 桶 memo(签名覆盖全消费输入,item 引用稳定) + 指纹 watch 降为 O(1) toolEpoch
- TopBar 三处 O(n) messages computed 合并单次遍历;md 缓存改有界 LRU(除满即全清抖动);useTurnTrace 补 never 穷尽 + default;MessageItem 模板去重
- 无障碍(四批累计): 可点容器键盘化(含 role/tabindex/keydown/aria-expanded) + icon-only aria-label 约 60 处 + 装饰图标 aria-hidden + 52 个滚动容器 overscroll-behavior:contain + 设置开关聚焦可见环 + outline 零补偿站点补 :focus-visible
- UX 走查项: 灵感 IDEA-10/11、知识 KNOW-6/7/9、项目详情 O8、审计 SA-4、设置 SA-6/26/27/28(脏表单/冗余 save/clamp/删除二次确认)
- 文件查看专项: 切工程状态残留 F5/F6、FileTree 去重与展开反馈 F7/F15/F17、切文件头部残留 F18、R 状态着色 F19;FilePreview diff 文件头(---/+++)误当增删渲染修复 + 大纲行高亮死样式修复 + 图片失败文案 i18n 化
- 同步 IO 收口(异步上下文内阻塞): unattended 看板读写改 tokio::fs、fetch_url 探活改 spawn_blocking、http/generate_image 清理改 tokio::fs
- 子代理 P3 批量: 任务书钉扎上限/截断文案区分父子语境/环境画像日期/temperature 可覆盖/model 回退上报/lang 透传
- crate 清理: df-execute·df-types 注释与格式、df-ideas 过时注释、df-nodes subflow 夸大描述、df-project 死变量
2026-10-10 08:09:45 +08:00
lxy 8df4e33656 文档: 看板销账与登记 + 设计档六份 + 索引补录
- 看板: 907 补注/909~941 登记与销账(含 929 模型停用根因与 4 处次生缺口、931 队列丢失、933/935/937 新需求、939 上下文瓶颈清单、941 C-r0 行为变更待拍板)
- 归档: 全量完成态条目/段落 + 只读核验销账 4 段迁 todo归档/2026-10-10.md(机械搬运零措辞改动,未完成项零丢失)
- 设计档: 输入框列表预览与自动编号 / 定时消息与待发箱持久化(统一 ai_outbox 底座) / 对话内定时延时任务 / 消息渲染管线重构(P0-P2 分期) / 上下文分层组装器落地方案(记账基点单一真值,与并行写者职责划清) / FileTree 大目录方案(推荐后端分页,否决虚拟滚动) / u-relay 运维手册
- 索引与事实修正: docs/INDEX crate 数 8→13 + 四子树补录 25 条 + 专项设计补录 27 条;纠正「项目无独立 ARCHITECTURE.md」等过时声明;统一 bun 口径
- CLAUDE.md: 新增「命名与代码风格」节(变量/方法名精简+准确;DRY·简洁·扩展性·高内聚低耦合·单一真值来源)
2026-10-10 08:09:25 +08:00
lxy b37279f3a2 新增: 待发箱/唤醒原语/预算随窗口 + 治理门禁与场景护栏
- 待发箱地基: V51 ai_outbox 表 + 手写 OutboxRepo(claim CAS/崩溃残留复位/瞬时失败保留) + outbox.rs 调度器(KV 默认关/Notify 唤醒/会话忙跳过不丢)
- Wake 异步唤醒原语: 三条件(delay/file-appear/process-exit)+纯判定+注入走正典 spawn_agentic_session,默认关
- 上下文预算随模型窗口: model_context.rs 三级优先(池显式>预设表>回退 128K);上界取 1M 而非 128K——max_tokens 仅裁剪预算,卡 128K 等于不治根因
- 文件租约补纯函数与单测(未另造双轨:LEASE 早随 cb260ab 落地) / 跨会话召回补纯函数单测(主体早已落地)
- 子代理: 完整卡已消化 + 子会话压缩层 + P3 批量
- 隧道/中继: 指令离线不丢 + Hello proto 版本 + 心跳与 runbook
- df-mcp: 写工具后落 KV 变更标记(供 GUI 轮询兜底,补独立 stdio 进程无感知)
- 迁移链顶点改单一真值常量 LATEST_SCHEMA_VERSION(修 2 处因 V51 失效的断言)
- 治理: i18n 特殊字符提交门禁(含反引号覆盖 + 10 自测) / 根目录卫生门禁 / 看板归档器 / 单一 lockfile(bun) / swap-install 落 temp/ 且兼容无 Get-FileHash
- 场景护栏: 流式正文可见性 e2e(playwright 免真机事件回放) + 列表续写/外置句柄 node 测试
2026-10-10 08:09:17 +08:00
lxy 9c68f98d89 修复: 模型停用无效根因 + 上下文归位/压缩守卫
- 模型停用无效根因: get_active_provider 两条路径均不校验 enabled(显式活跃直返/默认只挑 is_default) → 收敛为启用集合解析,无启用 provider 明确报错
- 压缩/标题兜底模型不校验 enabled → 新增 resolve_fallback_model 纯函数(title 复用同一实现,单一源无双轨)
- 上下文多夹层归位修复: gather_tool_responses 聚拢该头全部响应(此前第二个夹层之后的响应仍被判未闭合剥除)
- 压缩后台任务加内容版本哨兵(取首条消息 id,否掉 len:压缩期间 loop 正常 append 会致守卫永不生效) + panic 兜底复位标志
- 902 超时类失败与其它 Fatal 区分停靠文案(点继续可续跑)
- 子代理事件按 conv 改挂(route_scope) + 运行期事件推父会话视图
- 飞书 13a 真根因: run_stream_refresher 死代码(提前 drop)+静默钟被心跳重置 → 帧通道+仅内容事件重置
- 项目详情文件查看: F22 目录不存在误显非 Git 仓库 / F24 模块为仓库子目录时 git 徽标全失效(scope_git_path 按 show-prefix 剥前缀)
- 更新徽章通道错配(Tauri 事件 vs window CustomEvent 同名不同通道,徽章此前永不触发)
- ProjectCard 卡片根 keydown 顶替内嵌按钮(补 e.target 守卫)
- i18n 坏键 ai.aiTool.rejectedHint → aiTool.rejectedHint(双缺会渲染裸 key)
- miniapp 三处: 会话列表刷新漏网/看门狗节流致新一轮无兜底/newConversation 漏清看门狗
2026-10-10 08:08:48 +08:00
lxy 2a3a6cd25b 优化: 无障碍第二批(icon-only aria-label + 容器键盘化 20 处)
- TopBar 10 处 icon-only 按钮补 aria-label(复用同 :title 表达式,含三态 title)
- ConversationSidebar: 4 icon-only aria-label + 2 分组标题键盘化(role/tabindex/aria-expanded/keydown)
- MessageList 2 icon-only; KnowledgeDetail 3 章节折叠键盘化; dashboard 两列表项键盘化
- FileExplorer 面包屑/模块下拉项键盘化; ConflictResolver 冲突文件折叠键盘化
- 约定: 遮罩/点外关闭类(@click.self)非键盘缺口,跳过;零新增 i18n 键
- 验收: bunx vue-tsc --noEmit EXIT 0;看板 P3-2 更新(两批共 23 处)
2026-10-10 01:04:09 +08:00
lxy 9d83a06300 优化: 无障碍批次(可点容器键盘化 + icon-only aria-label)
- P2-2 命名 4 处可点容器键盘可达: ToolCard 卡头 / MessageList 轨迹卡头 / AuditLog audit-row(tr) / conv-list-item —— tabindex+keydown 复用原 handler, div 类加 role=button+aria-expanded; tr 因 role=button 会抹掉行语义故不加 role
- P3-2 3 处 icon-only 按钮补 aria-label(EnrichmentPanel 关闭 / ProjectDetail 编辑·删除),复用同 :title 的 i18n key,零新增键
- 硬约束执行: 4 个并行子代理按文件分区,零跨文件改动;主代理独立核查 diff 后修正一处语义缺陷(tr role)
- 看板同步: P2-2 销账 / P3-2 部分完成(~12 处待续) / 残留诊断代码 authz_debug 已清销账 / G6 标题泄漏(title.rs sanitize_title_source 双路径)销账
- 验收: bunx vue-tsc --noEmit EXIT 0 + node scripts/check-i18n.cjs 0 失败
2026-10-10 01:00:28 +08:00
lxy bcb0b82f33 修复: 流式正文被轮次折叠吞掉(真根因)
- 根因: renderItems 完成轮分桶把整轮含正文折进轨迹卡且默认收起,thinkPlainText 又 reasoning_content||content 把正文当思考塞单行 pill → 流式期间可见、轮末随卡消失(现象=气泡清空回到等待)
- 修复: 过程轮有正文的各成常显气泡(剥 toolCalls,工具卡仍归轨迹卡不重复);思考 pill 只认 reasoning_content
- 历次修补(40c4a21/4652112/524120b/BUG-260624-01/flush guard)都在状态清空层,未碰渲染折叠层 → 故无效;看板 911 留痕教训
- 验收: vue-tsc --noEmit EXIT 0
2026-10-10 00:51:50 +08:00
lxy e61fcdea79 新增: 看板归档器 + 已完成内容再分离
- scripts/archive-todo.cjs: 把看板已完成内容分离到归档(条目块 [x] 自动移出;历史/导航段走标题白名单整段移出,不靠启发式猜--本板存在大量无勾选的行内待办,启发式会误移)
- 安全闸: 未完成项零丢失断言(失败即中止不写);默认 dry-run;--list 从 UTF-8 文件读白名单(规避 Windows CLI 中文编码);容器标题(下挂更深子节)不当作空标题删
- 本次移出 6 段(统一规划 2026-06-26 全景/交接状态 2026-06-14/编排推进总览 2026-06-18)+ 909 条目;主板 847→736 行
- 核对: 未完成项 109 项零丢失,主板 [x] 清零,归档收 909 及其验收块
2026-10-10 00:47:10 +08:00
lxy 73dec866a4 修复: 压缩重入标志复位与兜底提交收口单锁
- keyword_fallback_commit 内单锁一并 set_compressing(false) + 兜底提交,消「锁外复位→再取锁提交」并发二次压缩窗口(放大楔入概率)
- 复位置于 KEYWORD_FALLBACK_ENABLED 门控之前(关开关同样复位,防标志永真致压缩停摆)
- 两处调用点(provider 重建失败/LLM 失败)独立复位块删除
- 验收: cargo check -p devflow EXIT 0 + cargo test -p devflow --lib 749 passed/0 failed;看板 909 收口
2026-10-10 00:42:48 +08:00
lxy bdd723f62c 修复: swap-install 兼容无 Get-FileHash 的环境
- 本机 Windows PowerShell 5.1 实测缺 Get-FileHash(v3 脚本 3 处调用直接报错中断,换装不执行)
- 新增 Get-Sha256 辅助:优先 Get-FileHash,异常兜底系统 certutil;两法实测同值
- 收口 3 处调用(baseline/before/after)
2026-10-10 00:42:47 +08:00
lxy da036db1f2 清理: 根目录散乱文件归位
- 删根 9 个构建/换装日志(*.log 本已 gitignore,属磁盘残留)+ 根 todo.md(陈旧副本,真板为 docs/todo.md)+ .dockerignore(无 Dockerfile)
- 根 swap-install.ps1(v3 加固版)迁 scripts/,替换其中陈旧 43 行版(句柄探测/SHA256 校验/重试/自清理)
- dev-tools.bat / dev-server.bat 迁 scripts/
- Batch.md / PROGRESS.md 迁 docs/07-项目管理/,同步修正 5 处指向根的文档链接
- .gitignore 补 /swap-install.lock(v3 运行期锁)
2026-10-10 00:37:19 +08:00
lxy 58834c9443 新增: i18n 消息特殊字符提交门禁
- 字面 @ / | / 裸 {} 未转义只在 prod runtime 抛 Invalid linked format(dev 宽松回退、vue-tsc 捕获不到),已两次致生产回归(白屏/输入框消失)
- scripts/check-i18n.cjs:用 @intlify/message-compiler 编译全部含特殊字符的 i18n 值,npm run check:i18n 可手动跑
- commit-msg 规则 i18n-special-chars.sh:暂存区含 src/i18n/ 即跑预检,失败拒提交;双向测试过(干净放行/坏值 exit1)
2026-10-10 00:28:40 +08:00
lxy a95e205eb6 修复: 三元组相邻性归位保住工具轮
- df-ai sanitize 新增 step 0.5 reorder_interlopers_after_tool_run: 楔入 head 与 tool 响应间的异类消息(压缩摘要/用户插话/警示)移到该组响应之后,只重排不删内容,工具轮不再被剥除
- 幂等;仅处理单头+夹层+紧随连续 tool 段,其余畸形仍走剥除兜底
- 单测: 旧剥除用例改为归位保轮 + 新增幂等用例;cargo test -p df-ai --lib context 156 passed
- 实证更正: 2026-08-04 报告「实证4」被推翻——已定位原始会话 b4d6b4e0,首轮裸问候即发两个 tool_calls;同步更新看板 909 状态
2026-10-10 00:27:47 +08:00
lxy d918237327 修复: i18n 裸 @ 致生产环境对话输入框消失
- 905 将 @ 改为裸 @,而 vue-i18n 的 @ 是 linked-message 语法字符,未转义只在 prod runtime 抛 Invalid linked format(dev 宽松回退/vue-tsc 捕获不到)
- zh/en aiChat 8 处(placeholderDefault/mentionHint/searchFilesBlocked/projectLabel)按规范转义为 {'@'}
- 输入框 placeholder 消息编译抛错→ChatInput 渲染失败→输入框消失,即本次生产回归根因
- 验证: 全库 462 个含 @/{/| 的 i18n 值经 message-compiler 编译 0 失败 + vue-tsc EXIT 0
2026-10-10 00:27:43 +08:00
lxy 1f5e6cce90 修复: system 锚点避让悬空 tool 调用头防 400
- df-ai 新增 ContextManager::push_system_anchor 统一出口:末条为悬空 ToolCallHead 时插其前,否则尾部追加(不位移前缀)
- 5 处 system 尾部 push 全量收口(压缩成功/兜底提交、警示单点、主题标记、续轮 nudge),从源头杜绝后台压缩摘要/警示楔入工具三元组致上游 400 insufficient tool messages
- 单测 push_system_anchor 两态锁定;cargo test -p df-ai --lib context 155 passed / cargo check -p devflow EXIT 0
- 看板归档:全量完成态条目与段落迁 docs/07-项目管理/todo归档/2026-10-10.md(零措辞改动机械搬运,未完成项110条零丢失/完成项37条全入归档/行数守恒)
2026-10-10 00:21:13 +08:00
lxy c71d0c4657 修复: AI上下文邻接性收口 + 设置页i18n缺失键
- df-ai 三元组邻接性收口: sanitize/openai_compat 闭合判定由「id 全局存在」收紧为「紧后 tool 连续段覆盖」,夹层序列(后台压缩/用户插话楔入)不再直送 provider 致 400 insufficient tool messages
- 设置页补 settings.loading 缺失键(zh/en),修版本信息返回前渲染字面串
- 看板: 907 接手核查补注(实现口径=剥除非归位,有损)+ 遗留登记 909 + 908 销账
2026-10-10 00:08:35 +08:00
lxy f4ca01f2f2 优化: df-ai/df-storage/df-nodes 底层加固: 上下文/模型/节点/仓储
df-ai:
- context/mod.rs + manager_tests.rs: 上下文管理调整 + 测试同步(48 行)
- openai_compat.rs: OpenAI 兼容层增强(+51)
- presets/models.json: 模型预设 +18

df-storage:
- crud/conversation_repo.rs: 会话仓储(loop_state 相关)
- crud/task_repo.rs: 任务仓储
- crud/mod.rs: 导出调整

df-nodes:
- task_advance_node.rs: 任务推进节点

验收: cargo check -p devflow EXIT 0。
2026-10-09 22:54:25 +08:00
lxy d7e626a813 修复: 飞书走查六项 + 停靠状态写侧根因解 + agentic/audit 加固
飞书集成走查六项(feishu_channel.rs +420):
1. 审批卡按钮化: on_card_action + handle_card_action(解析 {approval_id,approved}
   → ai_approve) + send_approval_button_card(Card JSON 1.0 双按钮, 失败降级文本卡)
2. 首帧流式提速: AiTextDelta 首个 delta 立即刷占位卡, 不等 3s ticker
3. 断线重连通知: 按 run_loop 时长判据(≥30s 推断开 / <30s 累计 3 次推不稳定),
   新增 notify_monitor()
4. 占位卡带任务原文回显(truncate 60)
5. 白名单配置引导: 后端分步自助引导 + FeishuPanel hintFeishuWhitelist + i18n
6. 审批决议后追加执行结果: spawn_approval_result_watcher 订阅 AiToolCallCompleted
   (匹配 tool_call_id, 60s 超时兜底); 订阅须在 ai_approve 前完成(无订阅者跳过序列化)

停靠(parked)写侧根因解(904):
emit_task_parked(ai/mod.rs) 升级为「写 parked + 发事件 + 通知」三件套——
新增 set_loop_state(conv_id, Some("parked")) fire-and-forget; 8 个业务调用点
(H2 三闸/熔断两路/挂起取消/max_rounds/midstream/fatal + 审批超时) 全经单点。
治「停靠卡继续必报 AI 未在暂停态」(全仓原无写 parked 路径)。

agentic/audit 加固: context_lifecycle/stream_relay/mod 调整 + audit record 完善 +
tool_failure_memory/tool_result_offload/tools/task/settings 同步。

验收: cargo check -p devflow EXIT 0。
2026-10-09 22:54:19 +08:00
lxy b283622c64 文档: 看板销账: 🔴 清零(187 路径误弹窗/214 401/229 架构扫描/457 AI自审/提交后跟进)
G2 门禁前置清理——5 条历史 🔴 逐条实证后销账:
1. 187 BUG-260620-05 工程内路径误弹窗: 方案已演进为动态推导(优于原「无条件插入
   workspace_root」), state.rs:579 实证「不再硬塞编译期 workspace_root」, 工程根
   授权改由 bind_directory + Settings.allowed_dirs 两途径, 黑名单兜底。已修销账。
2. 214 aichat API Key 401: 排查结论登记(非 devflow bug, 401 来自服务端), 非代码
   缺陷, 降级销账。
3. 229 架构坏味道扫描: 6 路 Explore agent 报告类归档(非缺陷), P1 项已各自销账,
   降级非 🔴。
4. 457 AI 自审/工作流推进链路: 标题即「P0×3 已修, 2026-10-05 核验销账」。
5. 提交后跟进: 10/11/12 三项已修销账。

标题 🔴 → ✅, 复核 `### 🔴` 余量 = 0。.gitignore 补规则。

验收: Select-String '^### 🔴' docs/todo.md → (无), G2 门禁通过。
2026-10-09 22:54:07 +08:00
lxy 40c4a21a7e 优化: AI Chat 消息视图: 闪烁修复+骨架统一+过程卡降级+任务关联 chip
MessageList.vue:
1. 流式闪烁修复: _mdRenderKey 稳定化 + key 对齐, 消除流式重渲染抖动
2. 骨架统一: 过程步骤卡改用 trace 卡骨架(头部一行摘要 + 正文增量)
3. 过程卡视觉降级: 缩进收窄 + 紧贴结果, 弱化过程突出结论
4. 消息任务关联: TASK_REF_TOOLS 映射(advance_task/run_workflow) 从
   toolCalls[].args 派生任务 chip, 零数据模型改动(单一事实源), 无任务工具
   的消息不渲染 chip; 新增 .ai-trace-task-chip 样式 + go-to-task emit

AiChat.vue: goToTask(taskId) 接参, router.push('/tasks?taskId=xxx')
Tasks.vue: applyRouteTaskId 消费 query.taskId 自动开抽屉(onMounted 跨路由 +
  watch 同路由), 消费后 router.replace 清 query 保持 URL 干净
i18n zh/en: aiChat.goToTaskHint + ai.trace.taskRef 双语补键

验收: vue-tsc --noEmit EXIT 0 + build EXIT 0。
2026-10-09 22:53:22 +08:00
lxy a8c92e9619 新增: 应用内版本更新与自动升级全链路
后端 update.rs: app_check_update(拉 latest.json+ed25519 签名校验) /
app_download_install_update / prefs 读写 / mark_checked / mark_notified 五命令,
lib.rs 注册;tauri.conf updater 段(endpoints=app.1216.top/download/devflow/latest.json
+ pubkey + createUpdaterArtifacts) + tauri-plugin-updater 依赖 + capabilities 权限。

前端 UpdateBadge.vue(顶栏徽章+下拉) / useUpdateCheck.ts(检查/下载/安装状态机) /
AboutPanel.vue(设置页关于面板) + AppLayout 接线 + SettingsNav/searchIndex 索引 +
i18n zh/en settings 键。

运维 scripts/publish-release.ps1(发版) + swap-install.ps1(换装校验) +
设计文档 版本更新与自动升级-设计-2026-10-09.md。

验收: cargo check -p devflow EXIT 0 + vue-tsc EXIT 0 + build EXIT 0。
2026-10-09 22:53:14 +08:00
lxy c70faa7bb8 优化: 版本 0.1.0 升 0.1.1,app.1216 发版配套 2026-10-08 00:41:09 +08:00
lxy 46521124de 修复: 会话迟到收尾仲裁误杀新任务流式态
- 症状: 上条处理中→发新消息入队→点「立即发送」→新气泡丢失+红按钮状态丢失
- 根因: force_send 打断旧任务后旧任务 AiCompleted 迟到到达, cleanup 无差别 setStreaming(false) 误杀新任务 streaming
- 仲裁: 四终态收尾前 IPC 查后端真实 generating(aiApi.isGenerating), 仍在生成=迟到伪影跳过全部下游收敛; 查态失败按正常收尾兜底
- 附: 901 看板验收附言补录(任务书⑤闭环)
2026-10-08 00:32:44 +08:00
lxy 0aac14ed9a 文档: 看板销账: 挂机模式 P1 总装与换装校验修复入库,901 销账
- 901 提交工作区改动转交实战完成: G4 前置走查过(风格/残留零问题)+分组三笔(挂机总装/换装修复/本笔销账)+vue-tsc 0+cargo check 0
- 看板顶部提交冻结解除标注(2026-10-07 用户推送令)与攒批状态更新一并入库
2026-10-07 22:06:47 +08:00
lxy 6bfcf01641 修复: 换装校验同版本重装误判不通过
- retryHash 与磁盘基线相同(同版本重装)时原判据走不进 pass 分支误报 mismatch
- 现区分两种通过情形: 确定性新装(installer deterministic output)与同版本重装(already up to date)各留日志
2026-10-07 22:04:58 +08:00
lxy 2441101978 新增: 挂机模式 P1 总装: 设置面板+顶栏徽章+边界卡精简+监控告警推送
- 新增 UnattendedPanel.vue 挂机配置面板(引擎自动认领看板任务,Settings KV 热读即改即生效)
- Settings.vue 接入挂机面板;searchIndex.ts 补挂机搜索词条
- TopBar.vue 新增挂机进行中徽章(脉冲动效+透明度过渡)
- TaskBoundaryCard.vue 精简边界状态判定(wallState 合并比对,删冗余 watch)
- i18n zh-CN/en: settings/ai/aiChat 六文件补挂机词条双语
- unattended.rs 新增 push_monitor_alert 监控告警推送(异常快照+告警头直发 monitor_chat)
- feishu_notify.rs push/push_card 出口重构收敛
- lib.rs 注册 unattended_status/unattended_set_enabled 两 IPC
- aiShared.ts/useAiStreamingEvents.ts 挂机状态流接入
2026-10-07 22:03:41 +08:00
lxy e60bd38444 文档: 看板销账与引擎实测记录 2026-10-07 18:33:42 +08:00
lxy a2c69be4a2 修复: 换装校验改安装器确定性判据 2026-10-07 18:33:42 +08:00
lxy c2382325cb 新增: 弱网韧性: send 网络感知重试与基准面探针
- connect 类失败重试: 10 分钟窗口+基准面感知(IP 直连与 DNS 双测判本机断网), 断网进长等待自适应探测(5s 快探到 30s 封顶省电), 恢复即重试; 单点故障指数短退避 1s→8s
- 两协议(anthropic/openai)同构, 隧道级分钟级弱网不再直接吐中断
2026-10-07 18:33:41 +08:00
lxy 76ace953be 修复: 飞书细项批: 取消审批通路+多chat路由+命令容错 2026-10-07 18:33:41 +08:00
lxy 8c06076873 新增: B5 续作引擎完整批: 自领+停靠恢复+监控推送+提交冻结
- B5 引擎: 看板轮询自动领单(G2 门禁 fail-closed, 领单原子写), parked 自动恢复优先, 收敛回读续作, 3 次熔断自停, KV 配置默认关
- 引擎配套: 提交冻结同步(df-commit-frozen), 工具结果外载过期清理, 工具失败快速记忆, 自监控快照飞书推送(df-monitor-chat-id)
- 领单判据收紧: 正文以 🔴 开头才判高危(正文提及字样不误判)
- 28 注入末轮竞态: 补充指令落末轮时顺延轮次预算(封顶 8)
- 停靠卡继续伪失败根修: ai_continue_loop 非 active 时分流 parked 恢复内核(recover_parked_inner 单一事实源)
- 实测: 写板 70 秒自动领单, 领单→执行→销账→回读→领下一单 全链闭环, 2050 测试过
2026-10-07 18:32:43 +08:00
lxy e36b55a2f6 文档: 看板销账与提交冻结令: 交互批八项+namespace治理销账, 后续攒批等通知 2026-10-06 23:49:09 +08:00
lxy 19ecadf3f6 新增: 编号唯一机械门禁G8+换装脚本v3加固+自主修复纪律
- commit-msg 规则外置: 主hook 遍历 rules/, G3 中文类型+G8 看板编号全域唯一(主板+归档恰好一次, 双向测试过), core.hooksPath 分发
- 换装脚本 v3: 句柄释放独占探测+构建产物 SHA256 校验+失败重试+SYSTEM 转投用户会话可见重启(上线验证过: SHA256 一致+H1 实测)
- CLAUDE.md 自主修复纪律: 明确缺陷禁反问直接修, 本机脚本行为变更=report_after 级
2026-10-06 23:48:45 +08:00
lxy ece7cd3fa8 新增: namespace淘汰治理与可恢复会话分组
- namespace 淘汰治理: pin 机制(关键产出 LRU 永不淘汰, 超池 50% 最旧降级) + EvictionLog 留档富提示(原工具名+内容摘要+重调指引) + owner 软隔离(无主最旧先淘); 子代理报告出口 pin 收编
- 错误路径广播 AiAgentFinished(status=error): 错误任务有终态, watcher/任务卡可收敛
- 桌面侧栏可恢复分组: parked 会话独立分组+点击恢复, 断点续跑读侧闭环
- 验收: cargo check 零 error + vue-tsc EXIT 0
2026-10-06 23:48:45 +08:00
lxy 1ce1360a0b 修复: 飞书交互批: 推送汇总卡/伪受理/回查/watcher弃守/status/md净化
- 34 push_to_chat 滚动汇总卡: 同 chat 在途事务卡原地追加, ✅/❌ 终态另起新卡, 更新失败回退重绑; L3 过程不再逐条新开卡片刷屏
- 20 伪受理断层: 发票前过 can_accept_request 守卫, 不可启动回实况话术; inflight 移交 watcher 持有, 运行期计数不随 handler 返回归零
- 22 兜底回查限定受理时间戳边界(纯函数+单测), 缺时间戳视为边界外, 禁投旧回答冒充
- 23 watcher 消费本会话 AiError 视同终局投失败卡, 不再空挂 70min
- 26 话术改实况: 未受理(暂不入队)可稍后重发, 删假排队承诺
- 27 快任务兜底改受理卡发出后 8s 宽限窗判定, 替换入场恒假死分支
- 21 /status 命令桌面入口: ai_chat_send 拦截, 零 LLM 零 spawn 返回进度摘要
- 38 出站 md 净化: 标题转粗体/表格转列表/围栏保留, feishu_channel 与 df-im-gateway 同规则互引
- 受理卡符号对齐规范(任务开始用🚀, ✅仅完成)
- 验收: cargo check 零 error + feishu 13/df-im-gateway 19/chat 17 passed + vue-tsc 0
2026-10-06 23:48:29 +08:00
lxy 9520c198a2 修复: MCP list_tasks 默认排除终态 + read_skill 隔离围栏
- list_tasks 无 status 过滤时默认排除 done/cancelled, 终态存量(260+条)挤占分页窗口致活跃任务被挤出视图; 显式传 status 不干预
- df-storage TaskQuery 新增 status_not_in 字段 + SQL NOT IN 下推(占位符先拼后绑, 防序号/值错位)
- read_skill 返回内容外包 skill_documentation XML 围栏+隔离声明, 防技能文档内容与系统指令混淆
- df-mcp 64 passed(含新增终态排除测试)
2026-10-06 23:48:11 +08:00
lxy 9ff26f0c8f 修复: 子代理报告双缺陷——熔断补收尾轮 + 报告新近度豁免
① 熔断无收尾轮: max_rounds 打满(达 effective_max 上界且未收敛)时, 旧逻辑直接 teardown 截断, 父任务/消费方拿到的是末轮思考独白而非任务报告。现补一次无工具收尾轮: system 指令「输出最终任务总结报告」+ tools=None(stream_round 空工具表), 成功则 push 报告文本+落库再走原收尾出口; 失败(全候选耗尽/MidStream/Stale)不重试不停靠, 静默落回原截断路径——报告是增益非闸门, 预算仅 1 轮 max_retries=0。

② 报告被视图压缩吞掉: summarize_tool_results 加新近度豁免(RECENCY_EXEMPT_TOOL_RESULTS=2), 末尾 2 条 tool_result 恒不压缩——豁免判定在冻结决策之前且不进决策表, 无前缀缓存漂移; 委托完成报告/末轮工具结果全文保留给 LLM 下一轮消费, 根治「>2KB envelope 下一轮即被 JSON 分支折叠 512 字符, 父 LLM 从未见过报告全文」。

测试: 新增 summarize_recency_exempt_keeps_last_two_tool_results 锁定豁免行为; cargo test -p devflow --lib 712 passed 0 failed; todo.md 两项销账。
2026-10-06 16:47:07 +08:00
lxy 438d47bc9f 新增: 无人值守安全系列(H1-H3) + P5 审计日志与部署运维 + #14 异步委托
- H3 外部干预感知: run_agentic_loop 增 h3_watch_dir 参数, loop 启动取 git HEAD 基线, 每 5 轮探测, HEAD 被外部改写(切分支/reset/他话提交)即 emit_task_parked(stop_kind=external_change) + teardown 停靠; 4 调用方(发送/编辑/强制发送/审批续跑)从 allowed_dirs.persistent 排序取首目录, 修复初版 current_dir 恒空缺陷
- P5 审计日志: df-im-gateway audit.rs append-only SQLite(tunnel_audit 表, payload 4KB 截断, recent/by_request 查询), main.rs 8 打点点位(inbound/command/outbound), 写失败不阻断主链路, 17 tests passed
- P5 部署运维: publish-im-gateway.ps1(musl 交叉编译→scp .new→systemd 原子切换→9191 验证→自动回滚) + systemd unit + nginx im.1216.top 反代 + 运维手册
- #14 异步委托: 受理/完成两段式 + 桌面任务卡(后台进行中徽章/轮次/切换) + 飞书双端同构, 关联缺陷 #8-#12 速清

验收: cargo check workspace EXIT 0, cargo test workspace 全绿(711+560+33+12...), vue-tsc EXIT 0
2026-10-06 15:59:10 +08:00
lxy 7c252cff96 新增: ai_list_recoverable IPC+桌面可恢复入口双轨数据源(B3) 2026-10-06 12:54:11 +08:00
lxy 40392ebded 新增: 飞书 push_to_chat outbound 统一出口+ACTIVE_CLIENT 全局句柄(19) 2026-10-06 12:49:48 +08:00
lxy d2f586fc52 新增: 完成汇报附 git 实测附录 branch/HEAD/未提交计数(B1-H1 单点) 2026-10-06 12:47:18 +08:00
lxy 71e548aafc 修复: 飞书渠道错误话术分类透传+探活 inflight 口径前移(18) 2026-10-06 12:40:27 +08:00
lxy 0dfb50be9b 文档: 无人值守专项设计档+todo 销账(#10/#11/#12), Web 规范合规批(App/视图/组件样式) 2026-10-06 12:13:39 +08:00
lxy 04db8c0d3b 新增: 后台任务卡徽章+轮次投影(convRounds), 可恢复分区(parked), reattachToMain i18n 补键(#12) 2026-10-06 12:13:33 +08:00
lxy 70d6e98f82 修复: loop_state 懒创建继承+入口早退清理(#10), update_full 剥离列封 lost-update(#11), MidStream 保文不回填半截 reasoning, 标题剥标签 sanitize(#9), ANSI 源头抑制+strip 兜底(#8) 2026-10-06 12:13:25 +08:00
lxy 03dc27bc14 文档: todo 销账同步 2026-10-05 23:58:21 +08:00
lxy 28e6acbf3f 优化: 聊天体验批与Web规范合规 2026-10-05 23:41:32 +08:00
lxy 9e81182b4f 优化: 知识检索同项目加权与拒绝池 2026-10-05 23:41:27 +08:00
lxy 0f3c3ce1aa 优化: 飞书渠道文本链路与资源释放 2026-10-05 23:41:20 +08:00
lxy cb260aba4b 新增: 断点续跑与日志租约底座 2026-10-05 23:41:18 +08:00
lxy 8233ba095e 新增: 飞书直连渠道接线(文本链路第一批)
- 桌面端出站WS长连接,复用spawn_agentic_session通用核心与完成通知桥
- 配置读feishu-config KV,白名单空=放行并自助提示open_id
- todo并入下午分析产出(存储架构/知识库/规范治理/基建走查)
2026-10-05 14:59:55 +08:00
lxy d5766c2142 文档: 设计文档与问题记录归档 2026-10-05 14:59:12 +08:00
lxy 2eea0118db 重构: 审批交互重设计 2026-10-05 14:59:12 +08:00
lxy 24d6803ecc 新增: 项目归档 2026-10-05 14:59:12 +08:00
lxy 63c5c3d4f0 新增: WebChat 端 2026-10-05 14:59:11 +08:00
lxy 7c76ee6a60 新增: u-claw 集成 2026-10-05 14:59:11 +08:00
lxy d4d3a8deaf 新增: 飞书设置面板 2026-10-05 14:59:11 +08:00
lxy 8e1e56b3e5 优化: 打包提速与换装脚本
- Cargo.toml release profile 开 incremental(小改动重打包显著提速,正式发版在意体积可临时注释)
- 新增 scripts/package-devflow.ps1 双通道打包脚本:verify 快速编译验证/full 全量发版换装,含磁盘预检与启动验证
2026-10-05 00:35:05 +08:00
lxy 26cc3c3df9 文档: 能力批设计收口与走查状态同步
- DRY技术债走查报告:状态表同步 12/12 全收口,头部注记状态表为走查快照
- 并行化与异步唤醒设计:补 §13.4 有界失败记忆(窗口折叠/失败分册按类去重/监控轮转,三界类目口径同源)
- todo:登记 GitHub 直连受限替代方案(失败学习环首条)+F-m 实施细节补充
2026-10-05 00:35:04 +08:00
lxy 753de25696 重构: DRY截断正典改名收尾
- audit/utils.rs truncate_chars→truncate_chars_ellipsis 正典改名(补…语义),re-export 链同步
- record.rs/generate_image.rs/knowledge_inject.rs/tools/knowledge.rs 调用点全链更新
- df-ai context_helpers.rs 私有版跨 crate 不引依赖保留原名,加正典标注注释
- DRY #11 收口,同名不同义陷阱消除
2026-10-05 00:35:04 +08:00
lxy 6a50193b34 优化: 狗粮短板收口与patch_file增强
- patch_file old_text 精确未中时空白归一化回退(行首缩进归一+多空白折叠,唯一命中按整行区间替换,warning 注明)
- patch_file anchor end 唯一性校验:多处命中报错并列前两处位置,治实测×2 锚点错位;三模式同传按优先级仲裁(anchor>old_text>replace_lines),warning 注明
- tool_result 压缩 memoization:PerConvState 加 tool_compress_cache(内容哈希→压缩结果),主 loop+子代理双调用点接线,治 369 组同内容反复重压
- 标题采样改两段式:user 首条必进锚定主题+其余按序补足 6 条,治 assistant 首句被当主题
- prompt 注入文件读取纪律段(父会话+子代理同源):read_file 分页/search 直读,禁 run_command 切片读文件;read_file 工具描述与 truncated 提示补大 limit 建议
- 顺带 DRY#1:agentic/mod.rs 三处裸 emit 换正典双写(心跳/破坏性回环熔断/stall 熔断),miniapp 跨端盲区补齐
2026-10-05 00:35:04 +08:00
lxy c83ea0c315 新增: advance_task auto_path跳态自动推进
- 跳态目标按 shortest_path 自动逐跳推进(advance_task_auto_path),每跳独立校验+review_rounds 回归计数照旧
- 直达单跳与严格路径行为一致;workflow.rs 回调等机器生成单跳场景仍直用 advance_task_atomic 保持严格语义
- update_task 工具描述与 status 拦截错误文案同步 auto_path 语义
- 治 LLM 反复请求跳态推进被状态机拒绝的空耗往返
2026-10-05 00:35:03 +08:00
lxy c10f0e46f0 新增: aichat前端体验四件与队列残留收口
- R4 代码块语言角标:MessageList decorateCodeHtml 捕获 language-xxx 输出 data-lang+CSS ::before 左上角标,hover 显,无语言裸块不渲染
- R6 工具结果路径可复制:ToolResultBody 五处 file-bar 的 path span→button(read_file/list_directory/search_files/grep/patch_file),点击复制完整路径+✓/✕ 1.2s 反馈+execCommand 降级链
- I6 重新生成 UI 入口:MessageItem user 气泡 hover 补重新生成按钮(canRegenerate=isLastUser&&非流式),复用 handleRetry→store.regenerate 链路
- V4 浮窗审批 High 风险二次确认:useToolApproval 导出 isHighRiskTool/highRiskConfirmText,ApprovalPopup.onApprove 前置 ConfirmDialog,数字键守卫不绕确认
- 9-28#5 两处全局清队列残留:useAiEvents/useAiLifecycleEvents 无 convId 的 AiError 分支改保留队列+console.warn(防误伤其他会话排队消息)
- i18n zh/en 补 clickToCopy 键;裸 @ 改 &#64; 转义(vue-i18n linked-message 前缀吞噬)
2026-10-05 00:35:03 +08:00
lxy 83e86cd6b7 新增: 灵感归档箱与任务归档动线
- ideasStore/taskStore: archiveIdea/archiveTask IPC 封装(本地移除/本地更新,对齐 delete 语义)
- project.ts 组合 store 显式透传 archiveIdea/archiveTask
- Ideas.vue: 筛选栏归档箱 Tab(独立数据源 ideaApi.listArchived,主查询不受扰);archiveCurrentIdea 处理器(归档确认+归档箱内取消归档自动清选中);IdeaDetail 传 in-archive-box prop
- IdeaDetail.vue: inArchiveBox prop + archive emit;操作区归档/取消归档按钮,归档箱模式隐藏删除(防误删归档记忆)
- i18n zh-CN/en ideas 补 filterArchived/archiveBoxEmpty/quickArchive/quickUnarchive/confirmArchive/toast 键(前批遗留)
验收: vue-tsc --noEmit EXIT 0
2026-10-05 00:35:02 +08:00
lxy c6ebf15407 文档: 能力建设批设计归档 2026-10-05 00:35:01 +08:00
lxy f4d31e884a 新增: 任务灵感归档后端 2026-10-05 00:35:01 +08:00
lxy c8c13d302a 新增: LLM目标列表主动修正 2026-10-05 00:35:01 +08:00
lxy 7dea2b86c4 优化: Batch.md 状态同步 2026-10-05 00:35:00 +08:00
lxy e492f03191 新增: df-im-gateway 网关骨架 2026-10-05 00:35:00 +08:00
371 changed files with 47762 additions and 8290 deletions
-11
View File
@@ -1,11 +0,0 @@
# docker build 排除(docker context = repo 根,仅 df-relay 编译需 workspace 源码)
target
node_modules
dist
dist-ssr
.git
*.db
*.log
apps/df-miniapp/node_modules
apps/df-miniapp/dist
src-tauri/target
+17
View File
@@ -3,6 +3,9 @@ logs
*.log *.log
npm-debug.log* npm-debug.log*
# 换装脚本(scripts/swap-install.ps1)运行期锁文件
/swap-install.lock
# Dependencies # Dependencies
node_modules/ node_modules/
@@ -32,6 +35,10 @@ target-musl/ # df-relay 交叉编译产物隔离目录(见 .cargo/config.toml)
src-tauri/target/ src-tauri/target/
src-tauri/Cargo.lock src-tauri/Cargo.lock
# 签名私钥(应用内升级,设计 §七 高风险项:私钥绝不入仓)
*.key
*.key.pub
# Tauri 自动生成的 schema(每次构建变,可重建) # Tauri 自动生成的 schema(每次构建变,可重建)
src-tauri/gen/schemas/ src-tauri/gen/schemas/
@@ -94,3 +101,13 @@ devflow-dev.db-wal
apps/df-miniapp/src/manifest.json apps/df-miniapp/src/manifest.json
# 微信小程序 appid(真实 appid 本地保留,不提交仓库) # 微信小程序 appid(真实 appid 本地保留,不提交仓库)
temp/ temp/
# 本地工具链缓存(protoc 等,不入库)
.tools/
# df-im-gateway 运行时数据(冒烟测试残留)
crates/df-im-gateway/data/
# Playwright 运行产物
test-results/
playwright-report/
+25
View File
@@ -0,0 +1,25 @@
# DevFlow 工作规范
## 任务看板自动领单(最高优先级纪律)
1. **会话启动即读板**:任何会话开工前,先读 `docs/todo.md`——按优先级领任务:🔴 高危项 > B 批次/编号项 > C 类候选。
2. **批收敛即回读**:每完成一个任务批(提交后),重读看板领下一项——挂机/连续工作模式下每批必回读,禁凭记忆连续开工。
3. **领单动作**:领单后将条目改为 `[→]`(进行中),完成后改 `[x]` 并附验收命令输出(G4);无法完成的改 `[!]` 附卡点,禁静默跳过。
4. **🔴 门禁(G2)**:看板存在未清零 🔴 条目时,只允许领 🔴 销账任务,新功能任务禁领(除非用户明示)。
5. **提交纪律(G3/G4)**:提交描述与 diff 机械对应;一笔一事或逐件列全;分组提交按看板标注的主题分组。
6. **汇报实测**:任何进度/完成汇报,先跑 `git log -1 --format="%h %s"` 与 `git status --short | wc -l` 实测再报,禁凭记忆报状态。
7. **看板顶部「挂机必读」警示**:启动时若存在,视为最高优先级上下文覆盖。
## 自主修复纪律
- 明确 bug(实证在案/用户指出/唯一正确解或用户已给修法方向):直接修并报告,禁反问「要不要修」(2026-10-06 用户纠正:发现换装脚本缺陷后反问=自主性差)。
- 边界对齐意志卡 forbidden:发版/部署/重启线上、git push、生产数据操作、对外发消息,攒齐等拍板。
- 本机工具/脚本的行为变更属 report_after 级(干完通报、可回滚),不属确认级。
## 命名与代码风格(2026-10-10 用户重申)
- **变量/方法名:精简 + 准确**——一眼看出「是什么/做什么」且不超必要长度。忌把实现细节写进名字(`..._after_tool_run`)、忌无信息缩写(`tmp2`/`h`)、忌同概念多叫法。名字过长通常说明该拆函数或把说明下沉到注释(注释承载「为什么」,名字只承载「是什么」)。
- **DRY · 简洁 · 扩展性 · 高内聚低耦合**(同期重申):新增能力先找共用底座(如定时消息与定时任务共用一张 `ai_outbox` + 一个调度器),不造双轨;同一事实只留**单一真值来源**(如 token 估值口径)。
## 其他规范
- 架构设计文档在 `docs/02-架构设计/`;当前活跃专项:无人值守可靠性(B1-B4)+ 异步任务会话(13-17),设计见 `temp/` 与 docs。
Generated
+749 -18
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -26,3 +26,11 @@ keyring = { version = "3", features = ["windows-native", "apple-native", "linux-
# ConvState 提到 ConvStateStore(DashMap),guard.reset/ai_is_generating 不依赖 session lock, # ConvState 提到 ConvStateStore(DashMap),guard.reset/ai_is_generating 不依赖 session lock,
# 治卡死连环(AiCompleted 延迟/工具后中断/第二条进队列同源根因)。详见 workflow session-lock-redesign。 # 治卡死连环(AiCompleted 延迟/工具后中断/第二条进队列同源根因)。详见 workflow session-lock-redesign。
dashmap = "6" dashmap = "6"
# release 增量编译:小改动重打包显著提速(2026-10-04 打包提速项);
# ⚠️ 2026-10-09 回退:release + incremental 触发链接期匿名符号丢失
# (anon.*.llvm.* undefined symbol,rust-lld 与 MSVC link.exe 均复现),
# 根因是增量编译 CGU 划分与跨 crate 内联冲突。cargo check 不链接故不暴露,
# cargo build/tauri build 链接期必挂。release 禁用 incremental 为 Rust 官方推荐。
# [profile.release]
# incremental = true
File diff suppressed because it is too large Load Diff
@@ -348,6 +348,9 @@ function clearAllApprovalTimers(): void {
function clearWatchdog(): void { function clearWatchdog(): void {
if (streamTimer) clearTimeout(streamTimer) if (streamTimer) clearTimeout(streamTimer)
streamTimer = null streamTimer = null
// 复位节流时间戳:resetWatchdog 的 5s 节流假定「已有 timer 在跑」,而此处刚清空 timer。
// 不复位则「停止/报错后 5s 内重发」会因节流跳过 arm → 新一轮无看门狗(中途断则 generating 永卡)。
lastWatchdogReset = 0
} }
/** 已注册事件监听标志(防多次订阅 wsClient) */ /** 已注册事件监听标志(防多次订阅 wsClient) */
@@ -1374,6 +1377,7 @@ function newConversation(): void {
tokenUsage.value = null // 新会话清 token(对齐 switchConversation) tokenUsage.value = null // 新会话清 token(对齐 switchConversation)
activeModel.value = null // 新会话复位模型选择(新会话无 per_conv 模型,走默认路由) activeModel.value = null // 新会话复位模型选择(新会话无 per_conv 模型,走默认路由)
pendingApprovals.value.splice(0, pendingApprovals.value.length); clearAllApprovalTimers() pendingApprovals.value.splice(0, pendingApprovals.value.length); clearAllApprovalTimers()
clearWatchdog() // 与 switchConversation 对齐:清残留看门狗,防新一轮 send 沿用旧 timer(超时点偏移)
} }
/** /**
+5 -2
View File
@@ -7,6 +7,9 @@ import { useAiChat } from '@/composables/useAiChat'
import type { MentionSpan, MiniContentPart } from '@/types/relay' import type { MentionSpan, MiniContentPart } from '@/types/relay'
import type { ProjectRecord, TaskRecord, IdeaRecord, AiToolCallInfo, ChatMessage, TokenUsage } from '@/types/events' import type { ProjectRecord, TaskRecord, IdeaRecord, AiToolCallInfo, ChatMessage, TokenUsage } from '@/types/events'
/** 调试日志门控(对齐 useAiChat/ws.ts:import.meta.env.DEV,生产零 bridge 成本) */
const DEBUG = import.meta.env.DEV
// 组件内联(绕过微信工具 3.15.2 组件解析缓存 bug):独立组件恒报 Component not found, // 组件内联(绕过微信工具 3.15.2 组件解析缓存 bug):独立组件恒报 Component not found,
// inline 进页面后 chat/index.json usingComponents 空,工具无需解析 components/,问题消失。 // inline 进页面后 chat/index.json usingComponents 空,工具无需解析 components/,问题消失。
// MdView/MentionInput 源保留(src/components/ 下,工具 bug 修复后可拆回)。 // MdView/MentionInput 源保留(src/components/ 下,工具 bug 修复后可拆回)。
@@ -512,11 +515,11 @@ function onMentionClose(): void {
* 保留原有守卫:空文本忽略 / generating 中忽略 / 关闭联想浮层。 * 保留原有守卫:空文本忽略 / generating 中忽略 / 关闭联想浮层。
*/ */
function handleSend(): void { function handleSend(): void {
console.log('[v2:send] enter text=', JSON.stringify(inputText.value), 'gen=', generating.value, 'imgs=', pendingImages.value.length) if (DEBUG) console.log('[v2:send] enter text=', JSON.stringify(inputText.value), 'gen=', generating.value, 'imgs=', pendingImages.value.length)
const text = inputText.value.trim() const text = inputText.value.trim()
const hasImage = pendingImages.value.length > 0 const hasImage = pendingImages.value.length > 0
if (!text && !hasImage) { if (!text && !hasImage) {
console.log('[v2:send] 空文本且无图片,忽略') if (DEBUG) console.log('[v2:send] 空文本且无图片,忽略')
return return
} }
if (generating.value) { if (generating.value) {
@@ -12,11 +12,15 @@ const ai = useAiChat()
*/ */
const conversations = ref<Conversation[]>([]) const conversations = ref<Conversation[]>([])
/** /**
* 监听 ai.conversations.length 变化(比 deep watch 更可靠)。 * 监听 ai.conversations 整体替换(比 deep watch 更可靠,也比只盯 length 完整)。
* 用 `[...ai.conversations]` 创建普通数组赋值给 ref,确保每次变化都触发模板更新。 * 用 `[...ai.conversations]` 创建普通数组赋值给 ref,确保每次变化都触发模板更新。
*
* 为什么监听 ref 而非 length:device 回推 AiConversationList 是「整体替换」且条数常不变
* (改名后条数不变),只盯 length 时 watch 回调不触发 → 本地副本滞留旧标题。
* 监听 ref 则在赋值(含同长度替换)时即触发刷新。
*/ */
watch(() => ai.conversations.value.length, () => { watch(ai.conversations, (list) => {
conversations.value = [...ai.conversations.value] conversations.value = [...(list || [])]
}, { immediate: true }) }, { immediate: true })
/** 点击会话项切换 */ /** 点击会话项切换 */
+9 -6
View File
@@ -42,25 +42,25 @@ impl ContentPart {
impl ChatMessage { impl ChatMessage {
pub fn system(content: impl Into<String>) -> Self { pub fn system(content: impl Into<String>) -> Self {
Self { id: Some(new_message_id()), role: MessageRole::System, content: content.into(), parts: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) } Self { id: Some(new_message_id()), role: MessageRole::System, content: content.into(), parts: None, source: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) }
} }
pub fn user(content: impl Into<String>) -> Self { pub fn user(content: impl Into<String>) -> Self {
Self { id: Some(new_message_id()), role: MessageRole::User, content: content.into(), parts: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) } Self { id: Some(new_message_id()), role: MessageRole::User, content: content.into(), parts: None, source: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) }
} }
pub fn assistant(content: impl Into<String>) -> Self { pub fn assistant(content: impl Into<String>) -> Self {
Self { id: Some(new_message_id()), role: MessageRole::Assistant, content: content.into(), parts: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) } Self { id: Some(new_message_id()), role: MessageRole::Assistant, content: content.into(), parts: None, source: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) }
} }
pub fn assistant_with_tools(content: impl Into<String>, tool_calls: Vec<ToolCall>) -> Self { pub fn assistant_with_tools(content: impl Into<String>, tool_calls: Vec<ToolCall>) -> Self {
Self { id: Some(new_message_id()), role: MessageRole::Assistant, content: content.into(), parts: None, tool_call_id: None, tool_calls: Some(tool_calls), model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) } Self { id: Some(new_message_id()), role: MessageRole::Assistant, content: content.into(), parts: None, source: None, tool_call_id: None, tool_calls: Some(tool_calls), model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) }
} }
pub fn tool_result(call_id: impl Into<String>, content: impl Into<String>) -> Self { pub fn tool_result(call_id: impl Into<String>, content: impl Into<String>) -> Self {
Self { id: Some(new_message_id()), role: MessageRole::Tool, content: content.into(), parts: None, tool_call_id: Some(call_id.into()), tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) } Self { id: Some(new_message_id()), role: MessageRole::Tool, content: content.into(), source: None, parts: None, tool_call_id: Some(call_id.into()), tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) }
} }
/// 多模态 user 消息:content 文本 + parts(含 Image 片)。 /// 多模态 user 消息:content 文本 + parts(含 Image 片)。
/// content 作为人类可读文本(也作非 vision 端点降级载荷);parts 透传给 vision 端点。 /// content 作为人类可读文本(也作非 vision 端点降级载荷);parts 透传给 vision 端点。
pub fn user_parts(content: impl Into<String>, parts: Vec<ContentPart>) -> Self { pub fn user_parts(content: impl Into<String>, parts: Vec<ContentPart>) -> Self {
Self { id: Some(new_message_id()), role: MessageRole::User, content: content.into(), parts: Some(parts), tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) } Self { id: Some(new_message_id()), role: MessageRole::User, content: content.into(), parts: Some(parts), source: None, tool_call_id: None, tool_calls: None, model: None, status: None, reasoning_content: None, prompt_tokens: None, completion_tokens: None, prompt_cache_hit_tokens: None, prompt_cache_miss_tokens: None, reasoning_tokens: None, is_estimated: None, timestamp: Some(now_millis_i64()) }
} }
/// 是否含图片片(供 provider 判定走多模态分支)。 /// 是否含图片片(供 provider 判定走多模态分支)。
@@ -310,6 +310,7 @@ mod tests {
role: MessageRole::User, role: MessageRole::User,
content: "字面量构造".into(), content: "字面量构造".into(),
parts: None, parts: None,
source: None,
tool_call_id: None, tool_call_id: None,
tool_calls: None, tool_calls: None,
model: None, model: None,
@@ -368,6 +369,7 @@ mod tests {
role: MessageRole::Assistant, role: MessageRole::Assistant,
content: "answer".to_string(), content: "answer".to_string(),
parts: None, parts: None,
source: None,
tool_call_id: None, tool_call_id: None,
tool_calls: None, tool_calls: None,
model: None, model: None,
@@ -429,6 +431,7 @@ mod tests {
role: MessageRole::Assistant, role: MessageRole::Assistant,
content: "answer".to_string(), content: "answer".to_string(),
parts: None, parts: None,
source: None,
tool_call_id: None, tool_call_id: None,
tool_calls: None, tool_calls: None,
model: None, model: None,
+21
View File
@@ -68,6 +68,22 @@ pub enum ContentPart {
}, },
} }
/// 消息来源(#958 溯源):外部渠道进入会话的原始消息定位。序列化为 JSON 随消息
/// 落库(ai_messages.source 列);桌面本端消息恒 None 不落
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct MessageSource {
/// 渠道标识:"feishu"(后续可扩 openapi/webhook 等)
pub channel: String,
/// 渠道侧原始消息 ID(飞书 om_/oc_ 前缀,可跳原消息追溯)
pub message_id: String,
/// 渠道侧会话/群 ID(可选,群聊定位)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub chat_id: Option<String>,
/// 发送者标识(飞书 open_id,可选)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub sender: Option<String>,
}
/// 聊天消息 /// 聊天消息
/// ///
/// ⚠️ **迁移耦合点**:下方各字段名(`role`/`content`/`parts`/`tool_call_id`/ /// ⚠️ **迁移耦合点**:下方各字段名(`role`/`content`/`parts`/`tool_call_id`/
@@ -85,6 +101,11 @@ pub struct ChatMessage {
pub id: Option<String>, pub id: Option<String>,
pub role: MessageRole, pub role: MessageRole,
pub content: String, pub content: String,
/// 消息来源(#958 溯源):消息从哪个渠道/端点进入会话。None=桌面本端(既有语义,
/// 老数据反序列化为 None 零回归);Some=外部渠道(飞书 message_id 可跳原消息追溯,
/// 多端混发可区分来源端)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source: Option<MessageSource>,
/// 多模态内容片。 /// 多模态内容片。
/// ///
/// `None`/空 → 纯文本消息(绝大多数场景,content 即全部载荷)。 /// `None`/空 → 纯文本消息(绝大多数场景,content 即全部载荷)。
+1
View File
@@ -12,6 +12,7 @@ tokio = { workspace = true, features = ["sync", "time"] }
async-trait = { workspace = true } async-trait = { workspace = true }
anyhow = { workspace = true } anyhow = { workspace = true }
tracing = { workspace = true } tracing = { workspace = true }
base64 = "0.22"
# HTTP + 流式 # HTTP + 流式
reqwest = { version = "0.12", features = ["stream", "json", "rustls-tls"] } reqwest = { version = "0.12", features = ["stream", "json", "rustls-tls"] }
+18
View File
@@ -136,6 +136,24 @@
"context_window": 262144, "context_window": 262144,
"temperature_supported": false "temperature_supported": false
}, },
{
"model_id": "sensenova-6.8-flash-lite",
"enabled": true,
"modalities": ["text", "vision"],
"capabilities": ["tool_use"],
"weight": 85,
"context_window": 262144,
"temperature_supported": false
},
{
"model_id": "sn2-kimi",
"enabled": true,
"modalities": ["text"],
"capabilities": ["tool_use"],
"weight": 80,
"context_window": 131072,
"temperature_supported": false
},
{ {
"model_id": "deepseek-v4-flash", "model_id": "deepseek-v4-flash",
"enabled": true, "enabled": true,
+400 -60
View File
@@ -62,6 +62,46 @@ pub fn anthropic_cache_enabled() -> bool {
ANTHROPIC_CACHE_ENABLED.load(std::sync::atomic::Ordering::SeqCst) ANTHROPIC_CACHE_ENABLED.load(std::sync::atomic::Ordering::SeqCst)
} }
/// BP3 摘要链尾断点开关(缓存友好上下文总开关在本模块的投影,默认开)。
///
/// 本模块不读 src-tauri KV(跨 crate),上层读 settings `df-ai-cache-friendly-context`
/// 后经 [`set_bp3_enabled`] 同步(同 [`set_anthropic_cache_enabled`] 先例)。
/// 关时消息级只打 BP2,断点布局与历史逐字节一致。BP3 仍内嵌于 anthropic 缓存总开关
/// 之下(它发的也是 cache_control),两层正交。
static BP3_ENABLED: std::sync::atomic::AtomicBool = std::sync::atomic::AtomicBool::new(true);
/// 设置 BP3 摘要链尾断点开关(上层读 settings 后同步)。
pub fn set_bp3_enabled(enabled: bool) {
BP3_ENABLED.store(enabled, std::sync::atomic::Ordering::SeqCst);
}
/// 读取 BP3 开关(`convert_request` 组装消息级断点用)。
pub fn bp3_enabled() -> bool {
BP3_ENABLED.load(std::sync::atomic::Ordering::SeqCst)
}
/// 缓存命中率观测日志(usage 就绪点每轮一条,供后续水位调参取数)。
///
/// anthropic 语义:input_tokens 为未缓存输入,cache_read/creation 独立计数,
/// 故命中率分母 = read + creation + input;三者全零(端点未回 usage)跳过不报。
fn log_cache_usage(usage: &TokenUsage) {
let total_input = usage.prompt_cache_hit_tokens as u64
+ usage.prompt_cache_miss_tokens as u64
+ usage.prompt_tokens as u64;
if total_input == 0 {
return;
}
let hit_rate = usage.prompt_cache_hit_tokens as f64 / total_input as f64;
let hit_pct = format!("{:.1}%", hit_rate * 100.0);
tracing::info!(
input_tokens = usage.prompt_tokens,
cache_read_input_tokens = usage.prompt_cache_hit_tokens,
cache_creation_input_tokens = usage.prompt_cache_miss_tokens,
hit_rate = %hit_pct,
"Anthropic prompt cache 命中率观测"
);
}
// ============================================================ // ============================================================
// Provider 实现 // Provider 实现
// ============================================================ // ============================================================
@@ -147,10 +187,21 @@ impl AnthropicCompatProvider {
// 构建非 system 消息(保留顺序,合并连续 tool_result) // 构建非 system 消息(保留顺序,合并连续 tool_result)
let mut messages: Vec<serde_json::Value> = Vec::new(); let mut messages: Vec<serde_json::Value> = Vec::new();
let mut pending_tool_results: Vec<serde_json::Value> = Vec::new(); let mut pending_tool_results: Vec<serde_json::Value> = Vec::new();
// 摘要锚点 = index>0 的 System(存储态:压缩摘要/话题标记/催办;首条是组装层
// system_prompt,归 BP1 顶层 system 非锚点)。记录「末锚点之后的消息条数」而非
// 下标:取用时按 len 反推目标,此前的 merge/补首条等数组增删天然自校正。
// 锚点以存储态 System 识别——view 层 System→user 转换落地后锚点以 user 形态
// 到达,此处识别不到,BP3 自然旁路(不加断点),无副作用。
let mut msgs_after_anchor: Option<usize> = None;
for m in req.messages.iter() { for (src_idx, m) in req.messages.iter().enumerate() {
match m.role { match m.role {
MessageRole::System => continue, MessageRole::System => {
if src_idx > 0 {
msgs_after_anchor = Some(0);
}
continue;
}
MessageRole::Tool => { MessageRole::Tool => {
// 累积 tool_result 块,遇到非 Tool 消息时 flush。 // 累积 tool_result 块,遇到非 Tool 消息时 flush。
// tool_use_id 为 None 时绝不能发 null(GLM anthropic 端点会 500 // tool_use_id 为 None 时绝不能发 null(GLM anthropic 端点会 500
@@ -173,7 +224,7 @@ impl AnthropicCompatProvider {
} }
} }
MessageRole::User => { MessageRole::User => {
Self::flush_tool_results(&mut messages, &mut pending_tool_results); Self::flush_tool_results(&mut messages, &mut pending_tool_results, &mut msgs_after_anchor);
// 多模态 user 消息 → content blocks 数组(text/image)。 // 多模态 user 消息 → content blocks 数组(text/image)。
// 含图时把 content + parts 拍平成 blocks:Text 片 → {type:text}, // 含图时把 content + parts 拍平成 blocks:Text 片 → {type:text},
// Image 片 → {type:image, source:{type:base64, media_type, data}}。 // Image 片 → {type:image, source:{type:base64, media_type, data}}。
@@ -192,9 +243,12 @@ impl AnthropicCompatProvider {
} else { } else {
messages.push(serde_json::json!({ "role": "user", "content": m.content })); messages.push(serde_json::json!({ "role": "user", "content": m.content }));
} }
if let Some(k) = &mut msgs_after_anchor {
*k += 1;
}
} }
MessageRole::Assistant => { MessageRole::Assistant => {
Self::flush_tool_results(&mut messages, &mut pending_tool_results); Self::flush_tool_results(&mut messages, &mut pending_tool_results, &mut msgs_after_anchor);
let mut content: Vec<serde_json::Value> = Vec::new(); let mut content: Vec<serde_json::Value> = Vec::new();
if !m.content.is_empty() { if !m.content.is_empty() {
content.push(serde_json::json!({ "type": "text", "text": m.content })); content.push(serde_json::json!({ "type": "text", "text": m.content }));
@@ -218,10 +272,13 @@ impl AnthropicCompatProvider {
content.push(serde_json::json!({ "type": "text", "text": "" })); content.push(serde_json::json!({ "type": "text", "text": "" }));
} }
messages.push(serde_json::json!({ "role": "assistant", "content": content })); messages.push(serde_json::json!({ "role": "assistant", "content": content }));
if let Some(k) = &mut msgs_after_anchor {
*k += 1;
}
} }
} }
} }
Self::flush_tool_results(&mut messages, &mut pending_tool_results); Self::flush_tool_results(&mut messages, &mut pending_tool_results, &mut msgs_after_anchor);
// 合并相邻 user 块。Anthropic 协议要求 user/assistant 严格交替,连续 user // 合并相邻 user 块。Anthropic 协议要求 user/assistant 严格交替,连续 user
// 触发 GLM 1214。场景:drainQueue 续发(前一轮以 tool_result 结尾 + 新 user)→ flush 把 // 触发 GLM 1214。场景:drainQueue 续发(前一轮以 tool_result 结尾 + 新 user)→ flush 把
@@ -240,10 +297,12 @@ impl AnthropicCompatProvider {
// (正常会话首条本就是 user),远优于丢弃上下文。 // (正常会话首条本就是 user),远优于丢弃上下文。
Self::ensure_leading_user(&mut messages); Self::ensure_leading_user(&mut messages);
// 消息级缓存断点(开关开):除 system 断点外在历史段尾部再打一个 cache_control, // 消息级缓存断点(开关开):BP2 打在历史段稳定边界,使 system 之后的对话历史
// 使 system 之后的对话历史也可命中(否则只缓存 system,长会话历史段全价重算)。 // 也可命中(否则只缓存 system,长会话历史段全价重算);BP3(缓存友好开关,默认开)
// 加打摘要锚点链之后第一条,BP2 因裁剪边界/压缩失效时 [system+摘要链] 小前缀仍命中。
if anthropic_cache_enabled() { if anthropic_cache_enabled() {
Self::apply_message_cache_breakpoint(&mut messages); let after_anchor = if bp3_enabled() { msgs_after_anchor } else { None };
Self::apply_message_cache_breakpoint(&mut messages, after_anchor);
} }
let tools = req.tools.map(|defs| { let tools = req.tools.map(|defs| {
@@ -288,9 +347,11 @@ impl AnthropicCompatProvider {
/// 单个 ContentPart → Anthropic content block(text/image)。 /// 单个 ContentPart → Anthropic content block(text/image)。
/// - Text 片 → {type:text, text} /// - Text 片 → {type:text, text}
/// - Image 片 → {type:image, source:{type:base64, media_type, data}} /// - Image 片 → {type:image, source:{type:base64, media_type, data}}
/// base64 内嵌;url 模式当前不可达,兜底发空 data + warn(保留原行为)。 /// base64 内嵌;本地 file:// 引用(#958)先展开为 base64,http(s) url 模式
/// 当前不可达,兜底发空 data + warn(保留原行为)。
/// match 取得 p 所有权后直接 move media_type/base64,避免大 base64 clone。 /// match 取得 p 所有权后直接 move media_type/base64,避免大 base64 clone。
fn content_part_to_block(p: crate::provider::ContentPart) -> serde_json::Value { fn content_part_to_block(p: crate::provider::ContentPart) -> serde_json::Value {
let p = crate::image_ref::inline_local_image_part(p);
match p { match p {
crate::provider::ContentPart::Text { text } => serde_json::json!({ crate::provider::ContentPart::Text { text } => serde_json::json!({
"type": "text", "type": "text",
@@ -326,34 +387,61 @@ impl AnthropicCompatProvider {
fn flush_tool_results( fn flush_tool_results(
messages: &mut Vec<serde_json::Value>, messages: &mut Vec<serde_json::Value>,
pending: &mut Vec<serde_json::Value>, pending: &mut Vec<serde_json::Value>,
msgs_after_anchor: &mut Option<usize>,
) { ) {
if pending.is_empty() { if pending.is_empty() {
return; return;
} }
let blocks: Vec<serde_json::Value> = pending.drain(..).collect(); let blocks: Vec<serde_json::Value> = pending.drain(..).collect();
messages.push(serde_json::json!({ "role": "user", "content": blocks })); messages.push(serde_json::json!({ "role": "user", "content": blocks }));
if let Some(k) = msgs_after_anchor {
*k += 1;
}
} }
/// 消息级缓存断点:在「历史段稳定边界」消息的末个 content block 上打 cache_control。 /// 消息级缓存断点:BP2 打在「历史段稳定边界」的末个 content block 上,BP3(可选)
/// 加打「摘要锚点链之后第一条」——`after_anchor` 为锚点链后的消息条数
/// (Some(0)=锚点收尾无后继,不打;None=无锚点,只打 BP2)。
/// ///
/// 断点落点取**末条 user 消息之前的相邻前一条**(末条非 user 收尾的异常序列则取末条): /// BP2 落点取**末条 user 消息之前的相邻前一条**(末条非 user 收尾的异常序列则取末条):
/// 末条 user 是每轮请求的漂移边界——缓存拆分模式下它是请求末尾追加的易变尾段 /// 末条 user 是每轮请求的漂移边界——缓存拆分模式下它是请求末尾追加的易变尾段
/// (日期/清单/augmentation,每次重组),非拆分模式下是本轮新问题,二者都不属于 /// (日期/清单/augmentation,每次重组),非拆分模式下是本轮新问题,二者都不属于
/// 只增历史;打在它身上断点位置随请求漂移,永远无法命中。标其前一条(assistant / /// 只增历史;打在它身上断点位置随请求漂移,永远无法命中。标其前一条(assistant /
/// tool_result 归并的 user)则断点始终落在只增历史的末尾,历史只增不改位: /// tool_result 归并的 user)则断点始终落在只增历史的末尾,历史只增不改位:
/// 上一轮写入的缓存前缀是本轮请求前缀的逐字节前缀 → 命中后增量续写,滚动前进。 /// 上一轮写入的缓存前缀是本轮请求前缀的逐字节前缀 → 命中后增量续写,滚动前进。
/// ///
/// BP3 服务于 BP2 频繁失效的场景:BP2 随裁剪边界移动/压缩事件/新近度豁免改写而
/// 迁位,前缀反复全价重算;BP3 前缀只含 [system+摘要链+1 条],失效源仅压缩事件,
/// 为该小前缀保底命中。与 BP2 同条(短序列)时跳过,BP2 优先。
///
/// content 形态:string(简写)→ 升级为单 text block 数组再标;array(blocks)→ 标末块。 /// content 形态:string(简写)→ 升级为单 text block 数组再标;array(blocks)→ 标末块。
/// 断点预算:Anthropic 单请求最多 4 个 cache_control,system 1 个 + 消息级 1 个,余量充足。 /// 断点预算:Anthropic 单请求最多 4 个 cache_control——system 1 + BP2 1 + BP3 1 = 3,
fn apply_message_cache_breakpoint(messages: &mut [serde_json::Value]) { /// 余 1 备用。
fn apply_message_cache_breakpoint(
messages: &mut [serde_json::Value],
after_anchor: Option<usize>,
) {
let n = messages.len(); let n = messages.len();
if n == 0 { if n == 0 {
return; return;
} }
let last_is_user = let last_is_user =
messages[n - 1].get("role").and_then(|r| r.as_str()) == Some("user"); messages[n - 1].get("role").and_then(|r| r.as_str()) == Some("user");
let idx = if last_is_user && n >= 2 { n - 2 } else { n - 1 }; let bp2_idx = if last_is_user && n >= 2 { n - 2 } else { n - 1 };
let msg = &mut messages[idx]; if let Some(k) = after_anchor {
if k > 0 && k <= n {
let bp3_idx = n - k;
if bp3_idx != bp2_idx {
Self::mark_cache_control(&mut messages[bp3_idx]);
}
}
}
Self::mark_cache_control(&mut messages[bp2_idx]);
}
/// 单条消息打 cache_control:string content 升级为单 text block 数组再标;
/// array(blocks) 标末块;其余形态(异常)不动。
fn mark_cache_control(msg: &mut serde_json::Value) {
let cache_control = serde_json::json!({ "type": "ephemeral" }); let cache_control = serde_json::json!({ "type": "ephemeral" });
match msg.get("content").cloned() { match msg.get("content").cloned() {
Some(serde_json::Value::Array(mut blocks)) => { Some(serde_json::Value::Array(mut blocks)) => {
@@ -427,6 +515,22 @@ impl AnthropicCompatProvider {
.and_then(|m| m.get("role").and_then(|r| r.as_str())) .and_then(|m| m.get("role").and_then(|r| r.as_str()))
.unwrap_or(""); .unwrap_or("");
if first_role == "user" { if first_role == "user" {
// 首条 user 但 content 为空串(会话压缩/续发残留,典型 text(0B)):原样发出会被
// precheck "user content 空" 拦下而整请求失败。与上方补占位同思路——就地填
// 占位文案(空串无信息可丢),上游实测接受非空占位,precheck 亦放行。
let empty_str = messages[0]
.get("content")
.and_then(|c| c.as_str())
.map(|s| s.is_empty())
.unwrap_or(false);
if empty_str {
warn!(
msg_count = messages.len(),
"ensure_leading_user: 首条 user content 为空串,填占位(防 precheck 拦截)"
);
messages[0]["content"] =
serde_json::json!("(continued from previous context)");
}
return; return;
} }
// 空 Vec(异常会话经 sanitize 清空)或首条非 user → 补 user 占位: // 空 Vec(异常会话经 sanitize 清空)或首条非 user → 补 user 占位:
@@ -711,6 +815,7 @@ impl LlmProvider for AnthropicCompatProvider {
prompt_cache_miss_tokens: resp.usage.cache_creation_input_tokens, prompt_cache_miss_tokens: resp.usage.cache_creation_input_tokens,
reasoning_tokens: 0, reasoning_tokens: 0,
}; };
log_cache_usage(&usage);
AttemptOutcome::Ok(CompletionResponse { AttemptOutcome::Ok(CompletionResponse {
text, text,
model: resp.model, model: resp.model,
@@ -743,47 +848,94 @@ impl LlmProvider for AnthropicCompatProvider {
// 流式长生成任务会被误砍。改用 tokio::time::timeout // 流式长生成任务会被误砍。改用 tokio::time::timeout
// 包裹 send().await,只管建连+首响应头,不管后续 body 读取(后续由 stream_llm idle timeout 兜底)。 // 包裹 send().await,只管建连+首响应头,不管后续 body 读取(后续由 stream_llm idle timeout 兜底)。
// 60s 选型:正常 send(建连+收 200 headers)<5s,60s 足够宽容。 // 60s 选型:正常 send(建连+收 200 headers)<5s,60s 足够宽容。
let send_future = self // send 阶段弱网退避: connect 类失败(隧道断网/基站切换/DNS 抖动)请求未发出, 重试零副作用。
.auth_headers(self.client.post(self.messages_url())) // 网络感知分速(net_probe 基准面): 本机断网进长等待按自适应间隔探测恢复(隧道级分钟级波动),
.json(&body) // 单点故障指数短退避 1s→8s; 10 分钟窗口耗尽才放弃。timeout/decode/HTTP 状态错不重试。
.version(reqwest::Version::HTTP_11) // 与 openai_compat 同构。
.send(); let resp = {
let resp = match tokio::time::timeout(Duration::from_secs(60), send_future).await { let start = std::time::Instant::now();
Ok(Ok(r)) => r, let mut offline_logged = false;
Ok(Err(e)) => { let mut offline_polls = 0usize;
// 谓词结构化分类(对齐 openai 路径,共用 crate::send_error_class)。原 bail 串 let mut attempt = 0usize;
// [timeout={} connect={}=...] 的字面词让一切错误(含 builder 错)侥幸可重试。 loop {
// 类词改由谓词决定置串首;谓词转储缩写为 t/c/b/r 防误触分类词表,source 链 attempt += 1;
// 保留真实传输词(reset/broken pipe 等)作串兜底。 let send_future = self
let class = crate::send_error_class(crate::SendErrPreds::of(&e)); .auth_headers(self.client.post(self.messages_url()))
tracing::error!( .json(&body)
is_timeout = e.is_timeout(), .version(reqwest::Version::HTTP_11)
is_connect = e.is_connect(), .send();
is_body = e.is_body(), let outcome = tokio::time::timeout(Duration::from_secs(60), send_future).await;
is_request = e.is_request(), match outcome {
class = class, Ok(Ok(r)) => break r,
url = %self.messages_url(), Ok(Err(e)) => {
source = ?std::error::Error::source(&e), // 谓词结构化分类(对齐 openai 路径,共用 crate::send_error_class)。原 bail 串
"Anthropic 流式 send 失败" // [timeout={} connect={}=...] 的字面词让一切错误(含 builder 错)侥幸可重试。
); // 类词改由谓词决定置串首;谓词转储缩写为 t/c/b/r 防误触分类词表,source 链
anyhow::bail!( // 保留真实传输词(reset/broken pipe 等)作串兜底。
"send 失败({}): {} [t={} c={} b={} r={} src={:?}]", let preds = crate::SendErrPreds::of(&e);
class, let retryable = preds.is_connect || (preds.is_request && !preds.is_builder);
e, if retryable && start.elapsed() < crate::net_probe::SEND_RETRY_WINDOW {
e.is_timeout(), // 本机断网: 等基准面恢复(自适应间隔), 不消耗退避档位
e.is_connect(), if !crate::net_probe::baseline_alive().await {
e.is_body(), offline_polls += 1;
e.is_request(), let wait = crate::net_probe::offline_probe_wait(offline_polls);
std::error::Error::source(&e) if !offline_logged {
); warn!(
} url = %self.messages_url(),
Err(_elapsed) => { window_secs = crate::net_probe::SEND_RETRY_WINDOW.as_secs(),
// send 阶段超时(60s 未返回 HTTP 响应头):GLM 端点可能不可达或极慢 "本机网络不可达(疑似隧道/电梯弱网), 按自适应间隔探测恢复, 恢复后立即重试"
tracing::error!( );
url = %self.messages_url(), offline_logged = true;
"Anthropic 流式 send 超时(60s 未返回响应头)" }
); tokio::time::sleep(wait).await;
anyhow::bail!("流式请求超时(60秒未收到 HTTP 响应,可能服务不可达或被防火墙拦截)"); continue;
}
offline_polls = 0;
offline_logged = false;
let delay = Duration::from_millis(
1000u64.saturating_mul(1 << (attempt - 1).min(3)).min(8000),
);
warn!(
url = %self.messages_url(),
attempt,
delay_ms = delay.as_millis() as u64,
error = %e,
"流式 send 连接失败, 退避重试"
);
tokio::time::sleep(delay).await;
continue;
}
let class = crate::send_error_class(preds);
tracing::error!(
is_timeout = e.is_timeout(),
is_connect = e.is_connect(),
is_body = e.is_body(),
is_request = e.is_request(),
class = class,
url = %self.messages_url(),
source = ?std::error::Error::source(&e),
"Anthropic 流式 send 失败"
);
anyhow::bail!(
"send 失败({}): {} [t={} c={} b={} r={} src={:?}]",
class,
e,
e.is_timeout(),
e.is_connect(),
e.is_body(),
e.is_request(),
std::error::Error::source(&e)
);
}
Err(_elapsed) => {
// send 阶段超时(60s 未返回 HTTP 响应头):GLM 端点可能不可达或极慢
tracing::error!(
url = %self.messages_url(),
"Anthropic 流式 send 超时(60s 未返回响应头)"
);
anyhow::bail!("流式请求超时(60秒未收到 HTTP 响应,可能服务不可达或被防火墙拦截)");
}
}
} }
}; };
@@ -809,6 +961,12 @@ impl LlmProvider for AnthropicCompatProvider {
if let Some(err) = chunk.error.as_mut() { if let Some(err) = chunk.error.as_mut() {
*err = format!("{} | messages 摘要: {}", err, messages_summary); *err = format!("{} | messages 摘要: {}", err, messages_summary);
} }
// 流式收尾(message_stop 终态帧)即 usage 就绪点,输出命中率观测日志
if chunk.finished {
if let Some(u) = &chunk.usage {
log_cache_usage(u);
}
}
chunks.push(Ok(chunk)); chunks.push(Ok(chunk));
} }
} }
@@ -1116,6 +1274,46 @@ mod tests {
assert_eq!(content[1]["source"]["data"], "iVBOR"); assert_eq!(content[1]["source"]["data"], "iVBOR");
} }
/// 本地 file:// 引用(#958)在转换层展开为 base64 source;缺失降级 text 块不炸请求
#[test]
fn anthropic_convert_expands_local_file_ref() {
use base64::Engine as _;
let _convert = convert_lock();
let dir = std::env::temp_dir().join("df-ai-anthropic-ref-test");
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("ref.png");
std::fs::write(&path, b"\x89PNG ok").unwrap();
let provider = AnthropicCompatProvider::new("https://api.anthropic.com", "k", "claude-3-5-sonnet");
let req = CompletionRequest {
model: "claude-3-5-sonnet".into(),
messages: vec![ChatMessage::user_parts(
"看图",
vec![crate::provider::ContentPart::Image {
url: Some(format!("file://{}", path.display())),
base64: None,
media_type: Some("image/png".into()),
alt: None,
}],
)],
temperature: None,
max_tokens: None,
stream: false,
tools: None,
tool_choice: None,
reasoning_content: None,
};
let body = provider.convert_request(req);
let user_msg = body
.messages
.iter()
.find(|m| m.get("role").and_then(|r| r.as_str()) == Some("user"))
.expect("应有 user 消息");
let content = user_msg.get("content").and_then(|c| c.as_array()).expect("user content 应为数组");
assert_eq!(content[1]["source"]["data"], base64::engine::general_purpose::STANDARD.encode(b"\x89PNG ok"));
std::fs::remove_file(&path).ok();
}
/// 纯文本 user 消息 → content 仍是字符串简写(无图不数组化) /// 纯文本 user 消息 → content 仍是字符串简写(无图不数组化)
#[test] #[test]
fn anthropic_convert_text_only_user_remains_string() { fn anthropic_convert_text_only_user_remains_string() {
@@ -1153,7 +1351,7 @@ mod tests {
serde_json::json!({ "role": "assistant", "content": [{"type": "text", "text": "a1"}] }), serde_json::json!({ "role": "assistant", "content": [{"type": "text", "text": "a1"}] }),
serde_json::json!({ "role": "user", "content": "q2" }), serde_json::json!({ "role": "user", "content": "q2" }),
]; ];
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages); AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, None);
// 前一条 assistant 的末块带 cache_control // 前一条 assistant 的末块带 cache_control
let a1 = &messages[1]; let a1 = &messages[1];
let blocks = a1["content"].as_array().unwrap(); let blocks = a1["content"].as_array().unwrap();
@@ -1170,7 +1368,7 @@ mod tests {
serde_json::json!({ "role": "assistant", "content": "a1" }), serde_json::json!({ "role": "assistant", "content": "a1" }),
serde_json::json!({ "role": "user", "content": "q2" }), serde_json::json!({ "role": "user", "content": "q2" }),
]; ];
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages); AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, None);
let blocks = messages[1]["content"].as_array().expect("string 应升级为 blocks 数组"); let blocks = messages[1]["content"].as_array().expect("string 应升级为 blocks 数组");
assert_eq!(blocks.len(), 1); assert_eq!(blocks.len(), 1);
assert_eq!(blocks[0]["type"], "text"); assert_eq!(blocks[0]["type"], "text");
@@ -1184,7 +1382,7 @@ mod tests {
let mut messages = vec![ let mut messages = vec![
serde_json::json!({ "role": "user", "content": "only" }), serde_json::json!({ "role": "user", "content": "only" }),
]; ];
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages); AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, None);
let blocks = messages[0]["content"].as_array().expect("单条应升级为 blocks"); let blocks = messages[0]["content"].as_array().expect("单条应升级为 blocks");
assert_eq!(blocks[0]["cache_control"]["type"], "ephemeral"); assert_eq!(blocks[0]["cache_control"]["type"], "ephemeral");
} }
@@ -1193,10 +1391,152 @@ mod tests {
#[test] #[test]
fn cache_breakpoint_empty_messages_noop() { fn cache_breakpoint_empty_messages_noop() {
let mut messages: Vec<serde_json::Value> = vec![]; let mut messages: Vec<serde_json::Value> = vec![];
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages); AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, None);
assert!(messages.is_empty()); assert!(messages.is_empty());
} }
/// BP3 落「锚点链后首条」(按锚点后条数反推下标),与 BP2 各占一条互不覆盖。
#[test]
fn cache_breakpoint_bp3_marks_first_message_after_anchor_chain() {
// 模拟 [锚点链][q0][a0][q1][a1][q2] 转换后的数组:锚点后 5 条
let mut messages = vec![
serde_json::json!({ "role": "user", "content": "q0" }),
serde_json::json!({ "role": "assistant", "content": "a0" }),
serde_json::json!({ "role": "user", "content": "q1" }),
serde_json::json!({ "role": "assistant", "content": "a1" }),
serde_json::json!({ "role": "user", "content": "q2" }),
];
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, Some(5));
// BP3 → idx0(q0);BP2(末条 user,取 n-2)→ idx3(a1)
for idx in [0usize, 3] {
let blocks = messages[idx]["content"].as_array().unwrap();
assert_eq!(
blocks.last().unwrap()["cache_control"]["type"],
"ephemeral",
"idx{} 应带断点",
idx
);
}
assert_eq!(messages[4]["content"].as_str(), Some("q2"), "末条 user 不打点");
}
/// BP3 与 BP2 同条(短序列)→ 跳过 BP3,BP2 优先,断点预算不重复占。
#[test]
fn cache_breakpoint_bp3_skipped_when_colliding_with_bp2() {
let mut messages = vec![
serde_json::json!({ "role": "user", "content": "q0" }),
serde_json::json!({ "role": "assistant", "content": "a0" }),
serde_json::json!({ "role": "user", "content": "q1" }),
];
// 锚点后 2 条 → BP3 目标 idx1,与 BP2(n-2=1)同条 → 跳过
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, Some(2));
let blocks = messages[1]["content"].as_array().unwrap();
assert_eq!(blocks.last().unwrap()["cache_control"]["type"], "ephemeral");
assert!(messages[0]["content"].as_str().is_some(), "q0 应保持字符串简写");
}
/// 锚点收尾无后继(after_anchor=0)→ 不打 BP3,BP2 照常落点。
#[test]
fn cache_breakpoint_bp3_zero_after_anchor_noop() {
let mut messages = vec![
serde_json::json!({ "role": "user", "content": "q0" }),
serde_json::json!({ "role": "user", "content": "q1" }),
];
AnthropicCompatProvider::apply_message_cache_breakpoint(&mut messages, Some(0));
// BP2(末条 user → n-2=0)照常标 q0
let blocks = messages[0]["content"].as_array().expect("BP2 应照常落点");
assert_eq!(blocks.last().unwrap()["cache_control"]["type"], "ephemeral");
}
// ---------- BP3 开关门(convert 级,进程级开关用例须持 convert_lock) ----------
/// 构造 BP3 开关用例的请求:首条组装层 system_prompt + (可选)锚点链 + 历史段。
/// 转换后数组恒为 [q0, a0, q1](锚点折叠进顶层 system)。
fn bp3_req(with_anchor: bool) -> CompletionRequest {
let mut messages = vec![
ChatMessage::system("sys-prompt"),
ChatMessage::user("q0"),
ChatMessage::assistant("a0"),
ChatMessage::user("q1"),
];
if with_anchor {
messages.insert(1, ChatMessage::system("## 压缩摘要"));
messages.insert(2, ChatMessage::system("## 话题标记"));
}
CompletionRequest {
model: "claude-3-5-sonnet".into(),
messages,
temperature: None,
max_tokens: None,
stream: false,
tools: None,
tool_choice: None,
reasoning_content: None,
}
}
/// BP3 开 + 有锚点 → 断点落摘要链后第一条(q0),BP2 照常落历史尾(a0)。
#[test]
fn bp3_enabled_with_anchor_marks_first_message_after_chain() {
let _convert = convert_lock();
set_anthropic_cache_enabled(true);
set_bp3_enabled(true);
let provider = AnthropicCompatProvider::new("https://api.anthropic.com", "k", "claude-3-5-sonnet");
let body = provider.convert_request(bp3_req(true));
assert_eq!(body.messages.len(), 3);
// BP3:锚点链后首条 q0(原字符串升级为 blocks + cache_control)
let q0_blocks = body.messages[0]["content"].as_array().expect("BP3 应标在锚点链后首条 q0");
assert_eq!(q0_blocks[0]["text"], "q0");
assert_eq!(q0_blocks[0]["cache_control"]["type"], "ephemeral");
// BP2 照常:末条 user 前相邻一条 a0
let a0_blocks = body.messages[1]["content"].as_array().unwrap();
assert_eq!(a0_blocks.last().unwrap()["cache_control"]["type"], "ephemeral");
// 末条 user 保持字符串简写
assert_eq!(body.messages[2]["content"].as_str(), Some("q1"));
set_anthropic_cache_enabled(false);
set_bp3_enabled(true); // 复位到进程默认(默认开),防污染其他并行用例
}
/// BP3 关 → 断点布局与历史一致(仅 BP2,锚点链后首条不打点)。
#[test]
fn bp3_disabled_keeps_legacy_breakpoint_layout() {
let _convert = convert_lock();
set_anthropic_cache_enabled(true);
set_bp3_enabled(false);
let provider = AnthropicCompatProvider::new("https://api.anthropic.com", "k", "claude-3-5-sonnet");
let body = provider.convert_request(bp3_req(true));
assert_eq!(body.messages.len(), 3);
assert_eq!(
body.messages[0]["content"].as_str(),
Some("q0"),
"BP3 关,q0 保持字符串简写"
);
let a0_blocks = body.messages[1]["content"].as_array().unwrap();
assert_eq!(a0_blocks.last().unwrap()["cache_control"]["type"], "ephemeral", "BP2 照常");
set_anthropic_cache_enabled(false);
set_bp3_enabled(true);
}
/// 无锚点(仅组装层首条 system)→ 不加 BP3,仅 BP2。
#[test]
fn bp3_no_anchor_no_extra_breakpoint() {
let _convert = convert_lock();
set_anthropic_cache_enabled(true);
set_bp3_enabled(true);
let provider = AnthropicCompatProvider::new("https://api.anthropic.com", "k", "claude-3-5-sonnet");
let body = provider.convert_request(bp3_req(false));
assert_eq!(body.messages.len(), 3);
assert_eq!(
body.messages[0]["content"].as_str(),
Some("q0"),
"无锚点应不加 BP3"
);
let a0_blocks = body.messages[1]["content"].as_array().unwrap();
assert_eq!(a0_blocks.last().unwrap()["cache_control"]["type"], "ephemeral", "BP2 照常");
set_anthropic_cache_enabled(false);
set_bp3_enabled(true);
}
// ---------- ensure_leading_user(首条 assistant → 补 user 占位,保留上下文)---------- // ---------- ensure_leading_user(首条 assistant → 补 user 占位,保留上下文)----------
/// 辅助:构造 assistant(tool_use) 消息 /// 辅助:构造 assistant(tool_use) 消息
+180
View File
@@ -0,0 +1,180 @@
//! 压缩可召回存档:被压缩原文 JSONL 落盘 + 召回句柄拼装。
//!
//! 压缩把原文从 LLM 上下文隔离(is_active 白名单过滤),摘要必有损。本模块在压缩标记时
//! 把本次被压原文追加写入 `<data_dir>/context-archive/<conv_id>-archive.jsonl`
//! (JSON Lines,一行一条 {role, content}),调用方在压缩摘要尾部拼召回句柄行,
//! LLM 需要原文时经 read_file 回读。会话一文件、追加写,目录不存在则创建;
//! 会话删除级联清理留待后续挂点。存档只落盘不进上下文,token 估算零影响;
//! 不与 tool_offload 复用(那是 src-tauri 侧机制,跨 crate 不混用)。
use std::fs::OpenOptions;
use std::io::Write;
use std::path::{Path, PathBuf};
use crate::provider::ChatMessage;
/// 存档目录名(data_dir 下)
const ARCHIVE_DIR: &str = "context-archive";
/// 一次压缩事件的存档目标(会话维度)。调用方读到开关开(`df-ai-compress-recall`)后经
/// [`super::ContextManager::set_compress_recall`] 注入,压缩标记时一次性消费。
pub(super) struct RecallTarget {
data_dir: PathBuf,
conv_id: String,
}
impl RecallTarget {
pub(super) fn new(data_dir: &Path, conv_id: &str) -> Self {
Self {
data_dir: data_dir.to_path_buf(),
conv_id: conv_id.to_string(),
}
}
/// 存档文件路径:`<data_dir>/context-archive/<conv_id>-archive.jsonl`
fn file(&self) -> PathBuf {
self.data_dir
.join(ARCHIVE_DIR)
.join(format!("{}-archive.jsonl", self.conv_id))
}
}
/// 把本次被压原文追加写入会话存档(JSONL 追加语义,不覆盖既有行)。
/// 返回存档路径供句柄拼装;空消息/IO 失败返 None(调用方降级为无句柄,压缩本身不受影响)。
pub(super) fn append_archive(target: &RecallTarget, msgs: &[ChatMessage]) -> Option<PathBuf> {
if msgs.is_empty() {
return None;
}
let path = target.file();
let dir = path.parent()?;
if let Err(e) = std::fs::create_dir_all(dir) {
tracing::warn!(error = %e, dir = %dir.display(), "[archive] 存档目录创建失败,本轮无召回句柄");
return None;
}
let mut file = match OpenOptions::new().create(true).append(true).open(&path) {
Ok(f) => f,
Err(e) => {
tracing::warn!(error = %e, path = %path.display(), "[archive] 存档文件打开失败,本轮无召回句柄");
return None;
}
};
for m in msgs {
let line = serde_json::json!({ "role": &m.role, "content": m.content });
if let Err(e) = writeln!(file, "{line}") {
tracing::warn!(error = %e, path = %path.display(), "[archive] 存档行写入失败,本轮无召回句柄");
return None;
}
}
Some(path)
}
/// 召回句柄行:LLM 按此路径 read_file 回读被压原文。
pub fn recall_line(path: &Path) -> String {
format!("早期历史全文已存档: `{}`(read_file 可召回)", path.display())
}
/// 摘要尾部拼召回句柄行(开关开且存档落盘成功时用;开关关调用方直接用原摘要)。
pub fn with_recall_line(summary: &str, path: &Path) -> String {
format!("{}\n{}", summary, recall_line(path))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::context::ContextManager;
use crate::context_helpers::ContextConfig;
use tempfile::TempDir;
fn mgr() -> ContextManager {
ContextManager::new(ContextConfig {
max_tokens: 100_000,
..Default::default()
})
}
/// 灌 2 条可压 + 1 条保护区消息,注入召回目标后压缩 [0..2),返回存档路径。
fn compress_with_recall(tmp: &TempDir, conv_id: &str) -> PathBuf {
let mut m = mgr();
m.push(ChatMessage::user("早期问题"));
m.push(ChatMessage::assistant("早期回复"));
m.push(ChatMessage::user("近期问题"));
m.set_compress_recall(tmp.path(), conv_id);
let compressed = m.compress_old_messages(2);
assert_eq!(compressed.len(), 2, "前置:应压缩 2 条");
m.take_archive_path().expect("存档应落盘")
}
/// 压缩时存档文件创建,且每行含被压原文的 role+content。
#[test]
fn compress_with_recall_archives_original_text() {
let tmp = TempDir::new().unwrap();
let path = compress_with_recall(&tmp, "conv-a");
assert_eq!(
path,
tmp.path().join("context-archive").join("conv-a-archive.jsonl")
);
let text = std::fs::read_to_string(&path).unwrap();
let mut lines = text.lines();
let first: serde_json::Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(first["role"], "user");
assert_eq!(first["content"], "早期问题");
let second: serde_json::Value = serde_json::from_str(lines.next().unwrap()).unwrap();
assert_eq!(second["role"], "assistant");
assert_eq!(second["content"], "早期回复");
assert!(lines.next().is_none(), "不应有多余行");
}
/// 开关开:摘要尾部含召回句柄行(含存档路径与 read_file 指引);路径一次性消费。
#[test]
fn recall_enabled_summary_gets_handle_line() {
let tmp = TempDir::new().unwrap();
let path = compress_with_recall(&tmp, "conv-b");
let summary = "## 意图\n- 测试主题";
let with_handle = with_recall_line(summary, &path);
assert!(with_handle.starts_with(summary), "句柄拼在摘要尾部,原摘要在前");
let handle = recall_line(&path);
assert!(with_handle.ends_with(&handle));
assert!(handle.contains(&path.display().to_string()));
assert!(handle.contains("read_file"));
}
/// 开关关(不注入召回目标):不落盘、无句柄路径可拼。
#[test]
fn recall_off_no_archive_no_handle() {
let tmp = TempDir::new().unwrap();
let mut m = mgr();
m.push(ChatMessage::user("问题"));
m.push(ChatMessage::assistant("回复"));
m.push(ChatMessage::user("近期"));
let compressed = m.compress_old_messages(2);
assert_eq!(compressed.len(), 2);
assert!(m.take_archive_path().is_none(), "开关关不应有存档路径");
assert!(
!tmp.path().join("context-archive").exists(),
"开关关不应创建存档目录"
);
}
/// 二次压缩追加不覆盖(JSONL 追加语义):两批原文都在,行数累计。
#[test]
fn second_compress_appends_not_overwrites() {
let tmp = TempDir::new().unwrap();
let mut m = mgr();
m.push(ChatMessage::user("第一批"));
m.push(ChatMessage::assistant("第一批回复"));
m.push(ChatMessage::user("第二批"));
m.push(ChatMessage::assistant("第二批回复"));
m.push(ChatMessage::user("近期"));
m.set_compress_recall(tmp.path(), "conv-c");
let _ = m.compress_old_messages(2);
// 第二次压缩事件:调用方重新注入(开关可能已变),幂等跳过已压缩,只压新批
m.set_compress_recall(tmp.path(), "conv-c");
let second = m.compress_old_messages(4);
assert_eq!(second.len(), 2, "第二次只压新激活的 2 条");
let path = m.take_archive_path().unwrap();
let text = std::fs::read_to_string(&path).unwrap();
assert_eq!(text.lines().count(), 4, "两次压缩累计 4 行,不覆盖");
assert!(text.contains("第一批回复"), "第一批原文仍在");
assert!(text.contains("第二批回复"), "第二批原文已追加");
}
}
+308 -27
View File
@@ -44,10 +44,10 @@ fn estimate_message_counts_parts_tokens() {
mm_tokens, mm_tokens,
text_tokens text_tokens
); );
// base64 字符按 0.35 粗估,约 1.7k * 0.35 ≈ 595 tokens 量级 // base64 改按图地板计(~1600,与 provider 端按图计费同口径),不随字符串长度膨胀
assert!( assert!(
mm_tokens > 500, mm_tokens >= 1600,
"大 base64 应贡献可观 token,实际 {}", "大 base64 应按图地板计 token,实际 {}",
mm_tokens mm_tokens
); );
@@ -72,8 +72,10 @@ fn short_history_no_trim() {
#[test] #[test]
fn over_budget_trims_old() { fn over_budget_trims_old() {
// 小预算强制裁剪:20 条超预算,触发裁剪且保留保护区 // 小预算强制裁剪:20 条超预算,触发裁剪且保留保护区。
let mut mgr = ContextManager::new(cfg(200)); // 预算取值须容纳「补偿摘要+保护区 6 条」(中文按 1.2 token/字计约 230)再留裁剪空间,
// 否则保护区被挤压、最新消息也被压,测的就不再是「裁旧保新」语义。
let mut mgr = ContextManager::new(cfg(400));
// user/assistant 交替(真实对话序列;连续 user 会被 ensure_sequence_legal 合并,无法测条数裁剪) // user/assistant 交替(真实对话序列;连续 user 会被 ensure_sequence_legal 合并,无法测条数裁剪)
for i in 0..20 { for i in 0..20 {
if i % 2 == 0 { if i % 2 == 0 {
@@ -1045,9 +1047,12 @@ fn phase1_append_after_dirty_stays_dirty_until_save() {
// ============================================================ // ============================================================
#[test] #[test]
fn eviction_inserts_summary_message_at_view_head() { fn eviction_inserts_summary_message_at_view_tail() {
// 触发淘汰 → 视图头部应有一条 "[早期对话概要]" user 摘要(补偿无声丢弃的早期主题), // 触发淘汰 → 视图尾部应有一条 "[早期对话概要]" user 摘要(补偿无声丢弃的早期主题),
// 概要 ≤ EVICTION_SUMMARY_MAX_CHARS,保护消息完好,且纯视图(不污染内存全量/不落库)。 // 概要 ≤ EVICTION_SUMMARY_MAX_CHARS,保护消息完好,且纯视图(不污染内存全量/不落库)。
// 缓存友好(2026-10):摘要改尾部追加(push)而非头部插入(insert(0))——头部插入位移
// 整条前缀字节,使 DeepSeek 等自动前缀缓存 provider 全部缓存失效(命中率掉到个位数)。
// 尾部追加让稳定前缀(system + 早期 history)逐轮不变,命中率最大化。
let mut mgr = ContextManager::new(cfg(200)); let mut mgr = ContextManager::new(cfg(200));
for i in 0..20 { for i in 0..20 {
if i % 2 == 0 { if i % 2 == 0 {
@@ -1063,46 +1068,47 @@ fn eviction_inserts_summary_message_at_view_head() {
let (msgs, trimmed) = mgr.build_for_request(0); let (msgs, trimmed) = mgr.build_for_request(0);
assert!(trimmed, "超预算应触发裁剪"); assert!(trimmed, "超预算应触发裁剪");
// 视图头部 = 补偿摘要(role=user,"[早期对话概要]" 前缀 + 高频主题) // 视图尾部 = 补偿摘要(role=user,"[早期对话概要]" 前缀 + 高频主题)
let head = msgs.first().expect("视图不应为空"); let tail = msgs.last().expect("视图不应为空");
assert!( assert!(
matches!(head.role, MessageRole::User), matches!(tail.role, MessageRole::User),
"视图头部应为 user 摘要, 实际 {:?}", "视图尾部应为 user 摘要, 实际 {:?}",
head.role tail.role
); );
assert!( assert!(
head.content.starts_with("[早期对话概要] 高频主题: "), tail.content.starts_with("[早期对话概要] 高频主题: "),
"摘要格式不符, 实际 {}", "摘要格式不符, 实际 {}",
head.content tail.content
); );
assert!( assert!(
head.content.contains("审批"), tail.content.contains("审批"),
"摘要应含被淘汰区高频主题词'审批', 实际 {}", "摘要应含被淘汰区高频主题词'审批', 实际 {}",
head.content tail.content
); );
assert!( assert!(
head.content.chars().count() <= EVICTION_SUMMARY_MAX_CHARS, tail.content.chars().count() <= EVICTION_SUMMARY_MAX_CHARS,
"摘要应 ≤ {} 字符, 实际 {}", "摘要应 ≤ {} 字符, 实际 {}",
EVICTION_SUMMARY_MAX_CHARS, EVICTION_SUMMARY_MAX_CHARS,
head.content.chars().count() tail.content.chars().count()
); );
// 保护消息完好:末条(i=19 assistant)保留;摘要 user 与首轮真实 user 相邻,由 // 保护消息完好:末条(i=19 assistant)保留;摘要 user 追加在尾部,与末尾 assistant
// sanitize 的 ensure_sequence_legal 合并(摘要在前,换行接原文),故 msgs[1] 为 i=1 的 assistant // 不同 role 不合并,故 msgs[倒数第2] 仍为 i=19 的 assistant
assert_eq!( assert_eq!(
msgs.last().unwrap().content, msgs[msgs.len() - 2].content,
"第19条回复内容同样较长用于撑爆上下文预算触发视图裁剪逻辑", "第19条回复内容同样较长用于撑爆上下文预算触发视图裁剪逻辑",
"保护区最新消息被误裁" "保护区最新消息被误裁"
); );
assert!( assert!(
matches!(msgs[1].role, MessageRole::Assistant), matches!(msgs[msgs.len() - 2].role, MessageRole::Assistant),
"摘要与首轮 user 合并后,第二条应为 assistant, 实际 {:?}", "摘要前一条应为 assistant(i=19), 实际 {:?}",
msgs[1].role msgs[msgs.len() - 2].role
); );
// 首条仍是保留区真实首条 user(摘要不再抢占头部,不位移前缀)
assert!( assert!(
msgs[0].content.contains("审批流程与审批门禁需要适配"), matches!(msgs[0].role, MessageRole::User),
"合并不应丢失被保留的首轮真实 user 文本, 实际 {}", "视图头部应为真实首条 user(摘要已移尾部), 实际 {:?}",
msgs[0].content msgs[0].role
); );
// 纯视图:摘要不进内存全量,被淘汰原文存储态不变 // 纯视图:摘要不进内存全量,被淘汰原文存储态不变
@@ -1196,3 +1202,278 @@ fn eviction_no_user_text_no_summary() {
); );
assert_eq!(mgr.all_messages_clone().len(), 16, "裁剪污染了内存全量"); assert_eq!(mgr.all_messages_clone().len(), 16, "裁剪污染了内存全量");
} }
// ── 源头守静(909):push_system_anchor 避开悬空 tool_call 头 ──
#[test]
fn push_system_anchor_appends_on_normal_tail() {
let mut mgr = ContextManager::new(cfg(100_000));
mgr.push(ChatMessage::user("问"));
mgr.push(ChatMessage::assistant("答"));
let avoided = mgr.push_system_anchor(ChatMessage::system("警示"));
assert!(!avoided, "常规尾部应走追加(不避让)");
assert_eq!(mgr.len(), 3);
assert_eq!(mgr.messages_mut()[2].message.content, "警示", "system 应在尾部");
}
#[test]
fn push_system_anchor_avoids_dangling_tool_call_head() {
// [user] [assistant(tool_calls)] ← 末条为悬空头(其 tool 响应尚未回填)
let mut mgr = ContextManager::new(cfg(100_000));
mgr.push(ChatMessage::user("跑工具"));
mgr.push(ChatMessage::assistant_with_tools(
"",
vec![ToolCall::new("c1".to_string(), "t", "{}")],
));
let avoided = mgr.push_system_anchor(ChatMessage::system("## 摘要"));
assert!(avoided, "末条为悬空 tool_call 头应避让插入");
// 序列应为 [user, system(摘要), assistant(tool_calls)] —— 头仍是末条,三元组不被劈开
let msgs = mgr.all_messages_clone();
assert_eq!(msgs.len(), 3);
assert_eq!(msgs[1].content, "## 摘要", "摘要应插在悬空头之前");
assert!(
msgs[2].tool_calls.as_ref().is_some_and(|c| !c.is_empty()),
"悬空头应保持末条,其 tool 响应可紧随其后回填(协议不变式成立)"
);
}
// ============================================================
// 缓存友好上下文(W1 块状裁剪 + W2 system 锚点 view 层转 user)
// ============================================================
/// 推入一个完整工具轮 [user 提问, assistant 带 tool_call, tool_result, assistant 小结]。
/// 淘汰单元随之分为「user 独立单元 + 三元组单元」,轮起点(user)是轮吸附锚点。
fn push_tool_round(mgr: &mut ContextManager, tag: &str, filler: &str) {
mgr.push(ChatMessage::user(&format!("问题{}{}", tag, filler)));
mgr.push(ChatMessage::assistant_with_tools(
"",
vec![ToolCall::new(format!("tc{}", tag), "read_file", "{}")],
));
mgr.push(ChatMessage::tool_result(format!("tc{}", tag), "结果"));
mgr.push(ChatMessage::assistant(&format!("小结{}", tag)));
}
/// 视图中真实历史的 token 量:排除尾部追加的淘汰补偿摘要(它不计入裁剪水位)。
fn view_history_tokens(msgs: &[ChatMessage]) -> u64 {
let est = TokenEstimator::default();
msgs.iter()
.filter(|m| !m.content.starts_with("[早期对话概要]"))
.map(|m| est.estimate_message(m) as u64)
.sum()
}
#[test]
fn block_trim_lands_below_target_water_level() {
// W1:开关开后超预算裁剪一次裁到 T×available 水位以下(贪心只裁到刚好塞下 available)。
let mut mgr = ContextManager::new(cfg(1000));
for i in 0..40 {
let text = format!("第{}条这条消息足够长让总历史稳定超过一千token预算线", i);
if i % 2 == 0 {
mgr.push(ChatMessage::user(&text));
} else {
mgr.push(ChatMessage::assistant(&text));
}
}
assert!(mgr.history_tokens() > 1000, "前置:历史须超预算");
mgr.set_cache_friendly(true);
let (msgs, trimmed) = mgr.build_for_request(0);
assert!(trimmed, "超预算应触发裁剪");
let target = (1000u64 as f32 * crate::context_helpers::TRIM_TARGET_RATIO) as u64;
let remaining = view_history_tokens(&msgs);
assert!(
remaining <= target,
"裁后水位应落在目标 {} 以下, 实际 {}",
target,
remaining
);
// 保护区最新消息不被动(摘要追加在尾部,末条真实消息是其前一条)
let last_real = msgs
.iter()
.rev()
.find(|m| !m.content.starts_with("[早期对话概要]"))
.expect("视图非空");
assert!(
last_real.content.contains("第39条"),
"保护区最新消息被误裁, 实际末条: {}",
last_real.content
);
}
#[test]
fn block_trim_snaps_boundary_to_turn_start() {
// W1 轮边界吸附:水位在某单元边界达标但下一条不是 user 时继续扫,直到边界吸附轮起点
// (整问整答同生同死)。3 个工具轮 + 2 条过渡 + 6 条保护区,len=20 且 protect_start=14
// 恰为保护区起点;按纯水位会在过渡轮中间收手(首条为 assistant),吸附后首条必是 user。
let mut mgr = ContextManager::new(cfg(300));
let filler = "这一轮的体量用来把历史顶过预算线撑出裁剪空间";
for r in 0..3 {
push_tool_round(&mut mgr, &r.to_string(), filler);
}
mgr.push(ChatMessage::user(&format!("过渡问题A{}", filler)));
mgr.push(ChatMessage::assistant(&format!("过渡回复B{}", filler)));
for i in 0..6 {
let text = format!("保护区{}需要足够长的文本来撑量", i);
if i % 2 == 0 {
mgr.push(ChatMessage::user(&text));
} else {
mgr.push(ChatMessage::assistant(&text));
}
}
assert!(mgr.history_tokens() > 300, "前置:历史须超预算");
mgr.set_cache_friendly(true);
let (msgs, trimmed) = mgr.build_for_request(0);
assert!(trimmed, "超预算应触发裁剪");
let first = msgs.first().expect("视图非空");
assert!(
matches!(first.role, MessageRole::User),
"裁剪边界应吸附轮起点(下一条未淘汰消息是 user), 实际首条 role={:?}",
first.role
);
let target = (300u64 as f32 * crate::context_helpers::TRIM_TARGET_RATIO) as u64;
assert!(
view_history_tokens(&msgs) <= target,
"吸附同时仍应满足目标水位"
);
assert_eq!(
has_head(&msgs),
has_tail(&msgs),
"轮吸附不得劈开工具三元组"
);
}
#[test]
fn block_trim_pathological_falls_back_to_hard_budget() {
// W1 病态回退:单轮体量 > 块,扫到保护区仍无「水位+轮边界」齐备的边界(保护区首条
// 是 system 锚点,堵死末轮三元组边界的轮吸附)→ 回退取满足硬预算的最后单元边界,
// 硬预算安全底线不破;且回退比贪心裁得更深(贪心在首个硬预算边界即收手)。
let mut mgr = ContextManager::new(cfg(400));
let filler = "这一轮的提问体量远超块大小用来制造病态回退场景".repeat(10);
for r in 0..3 {
push_tool_round(&mut mgr, &r.to_string(), &filler);
}
// 保护区首条 system 锚点:末轮三元组边界 next 非 user,轮吸附不可达
mgr.push(ChatMessage::system("[早期锚点]"));
for i in 0..5 {
let text = format!("保护区{}需要足够长的文本来撑量", i);
if i % 2 == 0 {
mgr.push(ChatMessage::user(&text));
} else {
mgr.push(ChatMessage::assistant(&text));
}
}
assert!(mgr.history_tokens() > 400, "前置:历史须超预算");
mgr.set_cache_friendly(true);
let (msgs, trimmed) = mgr.build_for_request(0);
assert!(trimmed, "病态回退仍应触发裁剪");
let remaining = view_history_tokens(&msgs);
assert!(
remaining <= 400,
"硬预算安全底线:裁后不得超过 available, 实际 {}",
remaining
);
let first = msgs.first().expect("视图非空");
assert!(
first.content.contains("[早期锚点]"),
"回退应取满足硬预算的最后单元边界(裁到保护区起点), 实际首条: {}",
first.content
);
}
#[test]
fn cache_friendly_off_keeps_greedy_min_trim() {
// W1 开关关:走原贪心最小裁剪,裁到刚好塞下硬预算即停(高于目标水位),逐字节等价旧路径。
let mut mgr = ContextManager::new(cfg(1000));
for i in 0..40 {
let text = format!("第{}条这条消息足够长让总历史稳定超过一千token预算线", i);
if i % 2 == 0 {
mgr.push(ChatMessage::user(&text));
} else {
mgr.push(ChatMessage::assistant(&text));
}
}
// 不注入(字段默认 false)= 开关关
let (msgs, trimmed) = mgr.build_for_request(0);
assert!(trimmed, "超预算应触发裁剪");
let remaining = view_history_tokens(&msgs);
let target = (1000u64 as f32 * crate::context_helpers::TRIM_TARGET_RATIO) as u64;
assert!(
remaining <= 1000,
"贪心裁后应塞下硬预算, 实际 {}",
remaining
);
assert!(
remaining > target,
"贪心只裁到刚好塞下(高于目标水位 {}), 实际 {}",
target,
remaining
);
}
#[test]
fn system_anchor_view_becomes_user_when_cache_friendly_on() {
// W2:开关开,存储态 system 锚点在发送视图转 user;存储不动;最终视图(组装层
// system_prompt 在下标 0,不经 build_for_request)中 system 独占首条。
let mut mgr = ContextManager::new(cfg(100_000));
mgr.push(ChatMessage::user("问"));
mgr.push(ChatMessage::assistant("答"));
mgr.push_system_anchor(ChatMessage::system("[压缩摘要锚点]"));
mgr.set_cache_friendly(true);
let (msgs, _t) = mgr.build_for_request(0);
assert!(!msgs.is_empty());
assert!(
msgs.iter().all(|m| !matches!(m.role, MessageRole::System)),
"开时视图不得残留 system: {:?}",
msgs.iter().map(role_tag).collect::<Vec<_>>()
);
assert!(
mgr.all_messages_clone()
.iter()
.any(|m| matches!(m.role, MessageRole::System)),
"存储态 system 锚点不应被改写(纯视图)"
);
let mut final_view = vec![ChatMessage::system("SYS_PROMPT")];
final_view.extend(msgs);
assert!(
matches!(final_view[0].role, MessageRole::System),
"组装层首条 system_prompt 不转"
);
assert_eq!(
final_view
.iter()
.filter(|m| matches!(m.role, MessageRole::System))
.count(),
1,
"最终视图应只有组装层首条 system"
);
}
#[test]
fn system_anchor_head_insert_still_converted() {
// W2:IPC 压缩路径 insert_at(0) 的存储首位 system 锚点,最终视图在下标 1(>0),同样转 user。
let mut mgr = ContextManager::new(cfg(100_000));
mgr.push(ChatMessage::user("问"));
mgr.insert_at(0, ChatMessage::system("[摘要]"));
mgr.set_cache_friendly(true);
let (msgs, _t) = mgr.build_for_request(0);
assert!(
msgs.iter().all(|m| !matches!(m.role, MessageRole::System)),
"存储首位的 system 锚点最终视图下标 > 0,应转 user"
);
}
#[test]
fn system_anchor_stays_system_when_cache_friendly_off() {
// W2 开关关:不转换,视图与存储角色一致(旧行为逐字节等价)。
let mut mgr = ContextManager::new(cfg(100_000));
mgr.push(ChatMessage::user("问"));
mgr.push(ChatMessage::assistant("答"));
mgr.push_system_anchor(ChatMessage::system("[压缩摘要锚点]"));
let (msgs, _t) = mgr.build_for_request(0);
assert!(
msgs.iter().any(|m| matches!(m.role, MessageRole::System)),
"关时 system 锚点应原样进视图"
);
}
+185 -24
View File
@@ -20,9 +20,11 @@
//! 保持 `ContextManager::sanitize_messages(...)` 调用路径不变(零调用方变更)。 //! 保持 `ContextManager::sanitize_messages(...)` 调用路径不变(零调用方变更)。
mod sanitize; mod sanitize;
pub mod archive;
use crate::context_helpers::{ use crate::context_helpers::{
build_eviction_summary_text, classify_group, PLACEHOLDER_INTEGRITY_ENABLED, PROTECT_COUNT, build_eviction_summary_text, classify_group, PLACEHOLDER_INTEGRITY_ENABLED, PROTECT_COUNT,
TRIM_TARGET_RATIO,
}; };
// 重导出:保持 `df_ai::context::TokenEstimator` / `df_ai::context::ContextConfig` 等 // 重导出:保持 `df_ai::context::TokenEstimator` / `df_ai::context::ContextConfig` 等
// 历史路径对外可见(agentic.rs / commands/ai/mod.rs 等调用方零变更)。 // 历史路径对外可见(agentic.rs / commands/ai/mod.rs 等调用方零变更)。
@@ -78,6 +80,16 @@ pub struct ContextManager {
/// insert_at / compress_old_messages / messages_mut(可变借用兜底,调用方可能改 status/content)。 /// insert_at / compress_old_messages / messages_mut(可变借用兜底,调用方可能改 status/content)。
/// 误置位零代价(只是下次 save 多走一次全量重写,正确性不变);漏置位才会丢更新——故保守置位。 /// 误置位零代价(只是下次 save 多走一次全量重写,正确性不变);漏置位才会丢更新——故保守置位。
needs_full_rewrite: bool, needs_full_rewrite: bool,
/// 压缩可召回:本次压缩事件的存档目标(调用方读到 KV 开后注入,compress_old_messages
/// 一次性消费)。None = 本事件不存档(开关关/未注入),零行为变更。
compress_recall: Option<archive::RecallTarget>,
/// 最近一次压缩成功落盘的存档路径(take 语义,供调用方在压缩摘要尾部拼召回句柄;
/// 每次 compress_old_messages 按事件重置,防跨事件残留误报)。
archive_path: Option<std::path::PathBuf>,
/// 缓存友好上下文总开关(跨 crate 注入:settings 在 src-tauri 侧,df-ai 不可直读,
/// 调用方读 KV `df-ai-cache-friendly-context` 后经 [`Self::set_cache_friendly`] 注入)。
/// 默认 false = 旧行为(贪心最小裁剪 + system 锚点原样发送),未注入路径零行为变更。
cache_friendly: bool,
} }
impl ContextManager { impl ContextManager {
@@ -91,6 +103,9 @@ impl ContextManager {
pending_topic_marker: None, pending_topic_marker: None,
persisted_msg_count: 0, persisted_msg_count: 0,
needs_full_rewrite: false, needs_full_rewrite: false,
compress_recall: None,
archive_path: None,
cache_friendly: false,
} }
} }
@@ -153,6 +168,8 @@ impl ContextManager {
// 全量重写空列表也会清 DB)。 // 全量重写空列表也会清 DB)。
self.persisted_msg_count = 0; self.persisted_msg_count = 0;
self.needs_full_rewrite = true; self.needs_full_rewrite = true;
self.compress_recall = None;
self.archive_path = None;
} }
/// 消息数量 /// 消息数量
@@ -196,10 +213,10 @@ impl ContextManager {
// 未超预算 → 直接返回全量(仍做畸形配对自愈,防历史中毒触发 provider 500 死循环) // 未超预算 → 直接返回全量(仍做畸形配对自愈,防历史中毒触发 provider 500 死循环)
if self.history_tokens <= available { if self.history_tokens <= available {
let sanitized = Self::sanitize_messages(self.all_messages_clone()); let view = self.anchors_to_user(Self::sanitize_messages(self.all_messages_clone()));
// 出口断言:占位配对完整性,失败降级 TOOL_MISSING_PREFIX 自愈(防 400 orphan) // 出口断言:占位配对完整性,失败降级 TOOL_MISSING_PREFIX 自愈(防 400 orphan)
return ( return (
Self::assert_placeholder_pairing(sanitized, PLACEHOLDER_INTEGRITY_ENABLED), Self::assert_placeholder_pairing(view, PLACEHOLDER_INTEGRITY_ENABLED),
false, false,
); );
} }
@@ -208,15 +225,12 @@ impl ContextManager {
let protect_start = self.messages.len().saturating_sub(PROTECT_COUNT); let protect_start = self.messages.len().saturating_sub(PROTECT_COUNT);
let units = self.build_eviction_units(protect_start); let units = self.build_eviction_units(protect_start);
let mut removed: u64 = 0; // 缓存友好开:块状裁剪(目标水位+轮边界吸附);关:贪心最小裁剪(逐字节等价旧路径)
let mut trim_end = 0; let (trim_end, removed) = if self.cache_friendly {
for unit in &units { self.block_trim_target(&units, available)
if self.history_tokens.saturating_sub(removed as u32) <= available { } else {
break; self.greedy_trim_target(&units, available)
} };
removed += unit.token_sum as u64;
trim_end = unit.end;
}
if trim_end == 0 { if trim_end == 0 {
tracing::warn!( tracing::warn!(
@@ -229,9 +243,9 @@ impl ContextManager {
// 致超预算且保护区满时命中)。异常会话(开头连续 assistant/tool 无 user)经 // 致超预算且保护区满时命中)。异常会话(开头连续 assistant/tool 无 user)经
// ensure_sequence_legal 清空后,由协议层 ensure_leading_user 补 user 占位降级,不阻塞。 // ensure_sequence_legal 清空后,由协议层 ensure_leading_user 补 user 占位降级,不阻塞。
// view-only:不改 self.messages 持久化(与分支 1/3 一致)。 // view-only:不改 self.messages 持久化(与分支 1/3 一致)。
let sanitized = Self::sanitize_messages(self.all_messages_clone()); let view = self.anchors_to_user(Self::sanitize_messages(self.all_messages_clone()));
return ( return (
Self::assert_placeholder_pairing(sanitized, PLACEHOLDER_INTEGRITY_ENABLED), Self::assert_placeholder_pairing(view, PLACEHOLDER_INTEGRITY_ENABLED),
false, false,
); );
} }
@@ -246,30 +260,111 @@ impl ContextManager {
.collect(); .collect();
// 淘汰补偿摘要:视图裁剪无声丢弃被淘汰内容,LLM 从此看不到早期主题,与压缩 // 淘汰补偿摘要:视图裁剪无声丢弃被淘汰内容,LLM 从此看不到早期主题,与压缩
// 路径(有摘要补偿)行为不一致。此处把被淘汰文本提炼成一短段摘要插回发送 // 路径(有摘要补偿)行为不一致。此处把被淘汰文本提炼成一短段摘要插回发送视图。
// 视图最前部(淘汰区头部、保护消息之前)。角色必须 user:role=system 会被 // **缓存友好修正(2026-10)**:原 insert(0) 头部插入会位移整条前缀字节,使
// anthropic_compat 折叠进带 cache_control 的顶层 system 段,而本摘要纯视图 // DeepSeek 等自动前缀缓存 provider 的稳定前缀(顶层 system + 早期 history)
// 每轮即时再生成(淘汰区单调增长致内容逐轮漂移),落 system 即逐轮改写缓存 // 全部缓存失效——命中率掉到个位数。改尾部追加(push):摘要本身每轮即时再生成
// 指纹使全部前缀缓存失效;连续 role 由既有 sanitize/协议层自愈兜底。摘要只进 // (淘汰区单调增长致内容逐轮漂移)本就无法命中缓存,但放尾部不再拖累稳定前缀,
// messages 易变区,严禁拼进 system_prompt/任何稳定段;纯视图语义:不落库、 // 命中率最大化。角色必须 user:role=system 会被 anthropic_compat 折叠进带
// 不改 conv.messages 存储、不标记 compressed,被淘汰原文存储态不变。 // cache_control 的顶层 system 段,落 system 即逐轮改写缓存指纹;user 放尾部
// 极短/空淘汰文本提炼为空 → 跳过(不值得插)。 // 由 sanitize/协议层自愈兜底。摘要只进 messages 易变区,严禁拼进 system_prompt/
// 任何稳定段;纯视图语义:不落库、不改 conv.messages 存储、不标记 compressed,
// 被淘汰原文存储态不变。极短/空淘汰文本提炼为空 → 跳过(不值得插)。
if let Some(text) = build_eviction_summary_text(&evicted) { if let Some(text) = build_eviction_summary_text(&evicted) {
msgs.insert(0, ChatMessage::user(text)); msgs.push(ChatMessage::user(text));
} }
tracing::info!( tracing::info!(
"context_trimmed: skip {} messages, ~{} tokens (view-only, full history retained)", "context_trimmed: skip {} messages, ~{} tokens (view-only, full history retained)",
trim_end, removed trim_end, removed
); );
let sanitized = Self::sanitize_messages(msgs); let view = self.anchors_to_user(Self::sanitize_messages(msgs));
// 出口断言:占位配对完整性,失败降级 TOOL_MISSING_PREFIX 自愈(防 400 orphan) // 出口断言:占位配对完整性,失败降级 TOOL_MISSING_PREFIX 自愈(防 400 orphan)
( (
Self::assert_placeholder_pairing(sanitized, PLACEHOLDER_INTEGRITY_ENABLED), Self::assert_placeholder_pairing(view, PLACEHOLDER_INTEGRITY_ENABLED),
true, true,
) )
} }
/// 贪心最小裁剪:累计淘汰单元直到塞下硬预算即停(历史行为,开关关路径)。
///
/// 缺陷:对话每增长一条消息就重新越过预算一点,裁剪边界随之前移一个单元,
/// 发送视图首条每轮都变,前缀缓存从第 0 字节起永远 miss。
fn greedy_trim_target(&self, units: &[EvictionUnit], available: u32) -> (usize, u64) {
let mut removed: u64 = 0;
let mut trim_end = 0;
for unit in units {
if self.history_tokens.saturating_sub(removed as u32) <= available {
break;
}
removed += unit.token_sum as u64;
trim_end = unit.end;
}
(trim_end, removed)
}
/// 块状裁剪:一次裁到目标水位 `TRIM_TARGET_RATIO × available`,且边界吸附轮边界。
///
/// 为什么块状:两次裁剪之间裁剪边界不动 → 被淘汰集不变 → 视图前缀逐字节稳定,
/// 前缀缓存每轮命中;裁剪频度由水位差(块大小)决定而非逐条爬行。
/// 轮边界:单元 end 处下一条未淘汰消息 role=User 即轮起点,整问整答(含工具链)
/// 同生同死,淘汰摘要按轮覆盖语义完整,且轮起点是稳定地标。
/// 病态回退(单轮体量 > 块,扫到保护区仍无两条件齐备的边界):取满足硬预算
/// available 的最后单元边界(安全底线);连硬预算都不可得(保护区自身超限)则
/// 尽力裁到末单元,对齐贪心路径。
fn block_trim_target(&self, units: &[EvictionUnit], available: u32) -> (usize, u64) {
let target = (available as f32 * TRIM_TARGET_RATIO) as u32;
let mut removed: u64 = 0;
let mut hard = (0usize, 0u64);
let mut last = (0usize, 0u64);
for unit in units {
removed += unit.token_sum as u64;
last = (unit.end, removed);
let remaining = self.history_tokens.saturating_sub(removed as u32);
if remaining <= available {
hard = (unit.end, removed);
}
// 目标水位隐含硬预算已满足(target < available),轮吸附与水位同时判定
if remaining <= target
&& self
.messages
.get(unit.end)
.is_some_and(|next| matches!(next.message.role, MessageRole::User))
{
return (unit.end, removed);
}
}
if hard.0 > 0 {
hard
} else {
last
}
}
/// 发送视图出口整形(sanitize 之后):总开关开时把存储态 role=System 消息
/// (压缩摘要/话题标记/警示等锚点)映射为 role=User 发送。
///
/// 为什么:anthropic_compat 把存储态全部 role=System 折叠进顶层 system(带
/// cache_control),每次锚点事件都改写顶层缓存指纹,system 断点与消息断点同时失配。
/// 转 user 后顶层 system 回到纯 system_prompt 稳定段;连续 user 由
/// anthropic_compat::merge_consecutive_users 合并(协议要求严格交替),user 不触发
/// 三元组/首条规则。纯视图:存储态仍 System,压缩定位(has_compressible_messages
/// 的 System 排除)与前端展示语义不变。调用方组装层加入的 system_prompt 不经本
/// 函数(不在 storage),最终视图中它独占 role=System 首条。
fn anchors_to_user(&self, view: Vec<ChatMessage>) -> Vec<ChatMessage> {
if !self.cache_friendly {
return view;
}
view.into_iter()
.map(|mut m| {
if matches!(m.role, MessageRole::System) {
m.role = MessageRole::User;
}
m
})
.collect()
}
// ── 畸形配对自愈(转发至 [`sanitize`] 子模块,保持 ContextManager::xxx 调用路径)── // ── 畸形配对自愈(转发至 [`sanitize`] 子模块,保持 ContextManager::xxx 调用路径)──
/// 畸形配对自愈 — 转发到 [`sanitize::sanitize_messages`]。 /// 畸形配对自愈 — 转发到 [`sanitize::sanitize_messages`]。
@@ -565,6 +660,41 @@ impl ContextManager {
self.needs_full_rewrite = true; self.needs_full_rewrite = true;
} }
/// 追加一条 system 锚点消息(警示文本/压缩摘要),自动避开**悬空 tool_call 头**。
///
/// **源头守静(909)**:OpenAI/内部协议不变式是「assistant(tool_calls) 之后必须紧随
/// 其 tool 响应」。主 loop 的顺序是 push 头(`finalize_round_output`)→ 释放锁 →
/// 执行工具 → 再取锁 push 结果;若在「头已入、结果未回」的窗口内从尾部追加 system
/// (后台压缩摘要/警示/nudge/主题标记),system 就楔进 head 与其响应之间 → 上游报
/// 400「insufficient tool messages following tool_calls message」。
///
/// 本出口统一处置:末条非悬空头 → 常规尾部追加(不位移前缀,保自动前缀缓存,行为与
/// 原 `push` 一致);末条是悬空头 → 改插在该头**之前**(index = len-1),不劈三元组,
/// 摘要/警示仍紧邻本轮内容对 LLM 可见。返回 `true` 表示走了避让插入(会置
/// `needs_full_rewrite`,需 DB 全量重写),供调用方记日志。
pub fn push_system_anchor(&mut self, message: ChatMessage) -> bool {
if self.tail_is_dangling_head() {
let idx = self.messages.len() - 1;
self.insert_at(idx, message);
tracing::debug!(
idx,
"system 锚点避让悬空 tool_call 头插入(防楔入三元组致 400)"
);
true
} else {
self.push(message);
false
}
}
/// 末条是否为「悬空 tool_call 头」(assistant 带 tool_calls 且其 tool 响应尚未回填)。
/// 供 [`Self::push_system_anchor`] 判定避让位置:末条即头,说明头已入而结果未回。
fn tail_is_dangling_head(&self) -> bool {
self.messages
.last()
.is_some_and(|m| m.group == MessageGroup::ToolCallHead)
}
/// 按淘汰单元分组消息范围(三元组原子性),供压缩定位/分段标记复用同一分组逻辑。 /// 按淘汰单元分组消息范围(三元组原子性),供压缩定位/分段标记复用同一分组逻辑。
/// ///
/// 返回每个单元的右开区间 end + token 总和,保证: /// 返回每个单元的右开区间 end + token 总和,保证:
@@ -631,6 +761,11 @@ impl ContextManager {
/// ///
/// 返回空 Vec 表示本批次无可压缩消息(全部已 compressed 或范围空),调用方据此跳过 LLM 调用。 /// 返回空 Vec 表示本批次无可压缩消息(全部已 compressed 或范围空),调用方据此跳过 LLM 调用。
pub fn compress_old_messages(&mut self, compress_end: usize) -> Vec<ChatMessage> { pub fn compress_old_messages(&mut self, compress_end: usize) -> Vec<ChatMessage> {
// 压缩可召回:目标一次性消费(防跨事件残留),路径按本事件重置。压缩标记后原文
// 从 LLM 上下文隔离,此刻原文在手,追加写入会话存档供摘要句柄召回(read_file);
// 失败仅置空路径(调用方降级为无句柄),不影响压缩本身。
let recall = self.compress_recall.take();
self.archive_path = None;
// 压缩边界按对话单元(三元组)对齐:调用方传入的 end(如 len - PROTECT_COUNT)周期性 // 压缩边界按对话单元(三元组)对齐:调用方传入的 end(如 len - PROTECT_COUNT)周期性
// 落在单元中间(assistant 头与其 tool 结果之间),只压半个三元组会让保护区以孤立的 // 落在单元中间(assistant 头与其 tool 结果之间),只压半个三元组会让保护区以孤立的
// 头或尾开头,成为 sanitize 每请求告警的确定性供给。按全长分组取「完整单元边界中 // 头或尾开头,成为 sanitize 每请求告警的确定性供给。按全长分组取「完整单元边界中
@@ -657,10 +792,36 @@ impl ContextManager {
// newly_compressed 为空,状态未变,无需全量重写)。 // newly_compressed 为空,状态未变,无需全量重写)。
if !newly_compressed.is_empty() { if !newly_compressed.is_empty() {
self.needs_full_rewrite = true; self.needs_full_rewrite = true;
if let Some(target) = recall {
match archive::append_archive(&target, &newly_compressed) {
Some(p) => self.archive_path = Some(p),
None => tracing::warn!("[ai] 压缩存档写入失败,本轮摘要无召回句柄(压缩不受影响)"),
}
}
} }
newly_compressed newly_compressed
} }
/// 注入本次压缩事件的存档目标(压缩可召回,`df-ai-compress-recall` 开时由调用方在
/// `compress_old_messages` 之前调用;开关关不调用)。一次性消费:压缩标记时落盘,
/// 路径经 [`Self::take_archive_path`] 取走。
pub fn set_compress_recall(&mut self, data_dir: &std::path::Path, conv_id: &str) {
self.compress_recall = Some(archive::RecallTarget::new(data_dir, conv_id));
}
/// 取走最近一次压缩成功落盘的存档路径(Some = 本次压缩可召回,调用方据此在摘要尾部
/// 拼句柄行)。take 语义:取后即空,防跨压缩事件残留误报。
pub fn take_archive_path(&mut self) -> Option<std::path::PathBuf> {
self.archive_path.take()
}
/// 注入缓存友好上下文总开关(每请求由 src-tauri 调用方读 KV 后在 `build_for_request`
/// 前调用,持续语义:注入值保持到下次注入)。true = 块状裁剪 + system 锚点 view 层
/// 转 user;false = 两者旁路,行为与旧路径逐字节等价。
pub fn set_cache_friendly(&mut self, enabled: bool) {
self.cache_friendly = enabled;
}
/// 压缩重入标志(读)。true 表示一次 LLM 压缩正在进行中,触发方应跳过本轮压缩。 /// 压缩重入标志(读)。true 表示一次 LLM 压缩正在进行中,触发方应跳过本轮压缩。
pub fn is_compressing(&self) -> bool { pub fn is_compressing(&self) -> bool {
self.is_compressing self.is_compressing
+231 -7
View File
@@ -42,12 +42,35 @@ pub fn sanitize_messages(messages: Vec<ChatMessage>) -> Vec<ChatMessage> {
.filter(|m| m.is_active()) .filter(|m| m.is_active())
.collect(); .collect();
// step 1:已闭合的 tool_call_id 集合(role=Tool 消息全部提供过的 id) // step 0.5(相邻性归位,909②):把楔入 assistant(tool_calls) 头与其 tool 响应之间的异类消息
let resolved_ids: HashSet<&str> = messages // (后台压缩摘要 / 用户插话 / 警示 system)整体移到该组响应**之后**——只重排不删内容。
.iter() // 归位后三元组恢复相邻,step1 的邻接判定即可正确识别为已闭合,不再走「未邻接即剥除」而**保住
.filter(|m| matches!(m.role, MessageRole::Tool)) // 工具轮**(避免丢结果致模型重跑/误判)。对同一头的多段夹层同样生效(全部响应聚拢到头后紧邻)。
.filter_map(|m| m.tool_call_id.as_deref()) // 真正无法归位者(头无任何响应)仍由 step2/3 剥除兜底。
.collect(); let messages = gather_tool_responses(messages);
// step 1:相邻性判定的依据——每个 assistant 头「紧后 tool 连续段」覆盖的 id 集合。
// 口径与 openai_compat::sanitize_openai_triplets 一致(2026-10-09 收紧):
// 协议要求 tool 响应必须紧随其 assistant(tool_calls);原实现按「id 全局存在」判定
// (位置无关),会把被 system/user 隔开的游离响应误判为已闭合,放行畸形序列致 400
// 「insufficient tool messages」。故判定改为「紧后连续段覆盖」——未覆盖的 call 视同
// 未闭合处理(剥除),其游离响应随后由 step3 按 orphaned_ids 丢弃,自动闭环。
let mut adjacency: std::collections::HashMap<usize, HashSet<String>> =
std::collections::HashMap::new();
for i in 0..messages.len() {
if !matches!(messages[i].role, MessageRole::Assistant) {
continue;
}
let mut seg: HashSet<String> = HashSet::new();
let mut j = i + 1;
while j < messages.len() && matches!(messages[j].role, MessageRole::Tool) {
if let Some(id) = messages[j].tool_call_id.as_deref() {
seg.insert(id.to_string());
}
j += 1;
}
adjacency.insert(i, seg);
}
// step 2:判定每个 assistant 头的闭合状态,产出需丢弃的 id 集合 // step 2:判定每个 assistant 头的闭合状态,产出需丢弃的 id 集合
// orphaned_ids:未闭合的 tool_call.id(含占位 id),其 tool_result 要丢 // orphaned_ids:未闭合的 tool_call.id(含占位 id),其 tool_result 要丢
@@ -72,7 +95,7 @@ pub fn sanitize_messages(messages: Vec<ChatMessage>) -> Vec<ChatMessage> {
.iter() .iter()
.partition(|c| { .partition(|c| {
!c.id.starts_with(TOOL_MISSING_PREFIX) !c.id.starts_with(TOOL_MISSING_PREFIX)
&& resolved_ids.contains(c.id.as_str()) && adjacency.get(&i).is_some_and(|s| s.contains(&c.id))
}); });
match (resolved.is_empty(), unresolved.is_empty()) { match (resolved.is_empty(), unresolved.is_empty()) {
@@ -181,6 +204,78 @@ pub fn sanitize_messages(messages: Vec<ChatMessage>) -> Vec<ChatMessage> {
ensure_sequence_legal(after_reverse_orphan) ensure_sequence_legal(after_reverse_orphan)
} }
/// 相邻性归位(909②,view-only):把 assistant 头的**全部** tool 响应聚拢到头后紧邻,夹层整体后移。
///
/// 根因:协议不变式要求「tool 响应必须紧随其 assistant(tool_calls)」。后台压缩摘要在 head 已入、
/// 结果未回的窗口内楔入,或用户在执行期插话,都会产生 `[head, 异类, tool…]`。原逻辑对此按「未邻接
/// = 未闭合」剥除头与其响应,**丢掉整轮工具交互**(有损)。本步改为**归位**——夹层后移,头与其响应
/// 恢复相邻,工具轮保留;夹层内容亦不丢。
///
/// 覆盖**多夹层**:同一头的响应之间又插了第二个夹层时(如 `[head(c1,c2), 异类, tool(c1), 异类,
/// tool(c2)]`),只后移首夹层会让后段 tool 仍不与头相邻 → step1 邻接判定只认前段 → 后段 call 被
/// 判未闭合而剥除(tool 一并丢)。本实现把「响应集合耗尽前出现的**所有**非 Tool 消息」整体移到
/// **该头最后一个响应之后**,头后紧邻其全部响应(保序),夹层随后。
///
/// 边界:遇下一个 tool_call 头即停(不跨单元搬运);该头响应集合(其 tool_calls 的 id)全部命中
/// 即停(不牵动响应之后的无关消息)。无法归位者(头无响应)仍由 step2/3 剥除兜底。
///
/// 幂等:归位后序列已相邻,再跑本函数无夹层可移,输出不变。
fn gather_tool_responses(msgs: Vec<ChatMessage>) -> Vec<ChatMessage> {
use std::collections::HashSet;
fn is_tool_call_head(m: &ChatMessage) -> bool {
matches!(m.role, MessageRole::Assistant)
&& m.tool_calls.as_ref().is_some_and(|c| !c.is_empty())
}
fn is_tool(m: &ChatMessage) -> bool {
matches!(m.role, MessageRole::Tool)
}
let mut out: Vec<ChatMessage> = Vec::with_capacity(msgs.len());
let mut i = 0usize;
while i < msgs.len() {
if is_tool_call_head(&msgs[i]) {
// 该头全部 call id —— 用于判定它的响应集合是否已耗尽
let call_ids: HashSet<String> = msgs[i]
.tool_calls
.as_ref()
.map(|c| c.iter().map(|c| c.id.clone()).collect())
.unwrap_or_default();
// 从头后扫到「下一个 tool_call 头」或「该头响应集合耗尽」为止,分流响应与夹层
let mut j = i + 1;
let mut responses: Vec<ChatMessage> = Vec::new();
let mut interlopers: Vec<ChatMessage> = Vec::new();
let mut seen: HashSet<String> = HashSet::new();
while j < msgs.len() && !is_tool_call_head(&msgs[j]) {
if is_tool(&msgs[j]) {
if let Some(id) = msgs[j].tool_call_id.as_deref() {
seen.insert(id.to_string());
}
responses.push(msgs[j].clone());
} else {
interlopers.push(msgs[j].clone());
}
j += 1;
// 该头响应集合已全部命中 → 不再牵动响应之后的无关消息(保持边界语义)
if !call_ids.is_empty() && call_ids.iter().all(|id| seen.contains(id)) {
break;
}
}
// 有夹层且确有响应 → 头 + 全部响应在前,夹层整体后移(只重排)
if !interlopers.is_empty() && !responses.is_empty() {
out.push(msgs[i].clone());
out.extend(responses);
out.extend(interlopers);
i = j;
continue;
}
}
out.push(msgs[i].clone());
i += 1;
}
out
}
/// step 3.5:反向 orphan 检测(view-only)—— **补头自愈**(对 orphan tool_result 就地补占位 /// step 3.5:反向 orphan 检测(view-only)—— **补头自愈**(对 orphan tool_result 就地补占位
/// assistant tool_call 头,而非丢弃 tool_result)。 /// assistant tool_call 头,而非丢弃 tool_result)。
/// ///
@@ -554,6 +649,135 @@ mod tests {
// ── 占位配对完整性(解 400 orphan):反向 orphan 检测 + 出口自愈 ── // ── 占位配对完整性(解 400 orphan):反向 orphan 检测 + 出口自愈 ──
/// 相邻性归位(909②):夹层(system/user)隔开 head 与其 tool 响应时,把夹层移到响应之后
/// (只重排不删),保住整轮工具交互;同时保证发送序列满足「head 后紧邻其响应」。
/// 旧行为(逐字剥除工具轮)已被归位取代——本用例即旧行为的有损对照。
#[test]
fn sanitize_reorders_interloper_to_keep_tool_round() {
// [user] [assistant(c1,c2)] [system 夹层] [tool c1] [tool c2]
let msgs = vec![
ChatMessage::user("go"),
ChatMessage::assistant_with_tools(
"",
vec![
ToolCall::new("c1".to_string(), "t", "{}"),
ToolCall::new("c2".to_string(), "t", "{}"),
],
),
ChatMessage::system("夹层提示"),
ChatMessage::tool_result("c1", "r1"),
ChatMessage::tool_result("c2", "r2"),
];
let out = sanitize_messages(msgs);
// 零丢失:5 条全在(归位只重排,不删内容)
assert_eq!(out.len(), 5, "归位不得丢内容, 实际 {:?}", out);
// 头保留其两个 tool_call(工具轮不丢)
let head_idx = out
.iter()
.position(|m| matches!(m.role, MessageRole::Assistant))
.expect("head 应保留");
assert_eq!(
out[head_idx].tool_calls.as_ref().map(|c| c.len()),
Some(2),
"头应保留两个 tool_call, 实际 {:?}",
out
);
// 头后紧邻两条 tool 响应(协议不变式成立)
assert!(
matches!(out[head_idx + 1].role, MessageRole::Tool)
&& matches!(out[head_idx + 2].role, MessageRole::Tool),
"head 后应紧邻两条 tool 响应, 实际 {:?}",
out
);
// 夹层内容保留(后移到响应之后)
assert!(
out.iter()
.any(|m| matches!(m.role, MessageRole::System) && m.content == "夹层提示"),
"夹层内容不得丢, 实际 {:?}",
out
);
}
#[test]
fn sanitize_reorder_is_idempotent() {
let msgs = vec![
ChatMessage::user("go"),
ChatMessage::assistant_with_tools(
"",
vec![ToolCall::new("c1".to_string(), "t", "{}")],
),
ChatMessage::system("夹层提示"),
ChatMessage::tool_result("c1", "r1"),
];
let once = sanitize_messages(msgs);
let twice = sanitize_messages(once.clone());
// MessageRole 未实现 PartialEq → 用 Debug 签名比较序列(角色+内容)是否一致
let sig = |v: &[ChatMessage]| -> Vec<String> {
v.iter().map(|m| format!("{:?}|{}", m.role, m.content)).collect()
};
assert_eq!(sig(&once), sig(&twice), "归位应幂等(角色序与内容序不变)");
}
/// 多夹层归位:同一头的响应之间又插了第二个夹层时,头与该头的**全部**响应必须聚拢相邻
/// (旧实现只后移首夹层,后段 tool 仍不与头相邻 → step1 判后段 call 未闭合 → 剥除并丢其 tool)。
#[test]
fn sanitize_reorders_multiple_interlopers_to_keep_tool_round() {
// [user] [assistant(c1,c2)] [system 夹层X] [tool c1] [system 夹层Y] [tool c2]
let msgs = vec![
ChatMessage::user("go"),
ChatMessage::assistant_with_tools(
"",
vec![
ToolCall::new("c1".to_string(), "t", "{}"),
ToolCall::new("c2".to_string(), "t", "{}"),
],
),
ChatMessage::system("夹层X"),
ChatMessage::tool_result("c1", "r1"),
ChatMessage::system("夹层Y"),
ChatMessage::tool_result("c2", "r2"),
];
let out = sanitize_messages(msgs);
// 零丢失:6 条全在(归位只重排,不删内容)
assert_eq!(out.len(), 6, "多夹层归位不得丢内容, 实际 {:?}", out);
// 头保留其两个 tool_call(工具轮不丢)
let head_idx = out
.iter()
.position(|m| matches!(m.role, MessageRole::Assistant))
.expect("head 应保留");
assert_eq!(
out[head_idx].tool_calls.as_ref().map(|c| c.len()),
Some(2),
"头应保留两个 tool_call, 实际 {:?}",
out
);
// 头后紧邻其全部响应(保序 c1,c2),协议不变式成立
assert!(
matches!(out[head_idx + 1].role, MessageRole::Tool)
&& out[head_idx + 1].tool_call_id.as_deref() == Some("c1")
&& matches!(out[head_idx + 2].role, MessageRole::Tool)
&& out[head_idx + 2].tool_call_id.as_deref() == Some("c2"),
"head 后应紧邻其全部响应 c1,c2, 实际 {:?}",
out
);
// 两条夹层内容都保留(后移到响应之后)
assert!(
out.iter()
.any(|m| matches!(m.role, MessageRole::System) && m.content == "夹层X")
&& out.iter()
.any(|m| matches!(m.role, MessageRole::System) && m.content == "夹层Y"),
"两条夹层内容不得丢, 实际 {:?}",
out
);
// 幂等:再跑一次结果不变
let twice = sanitize_messages(out.clone());
let sig = |v: &[ChatMessage]| -> Vec<String> {
v.iter().map(|m| format!("{:?}|{}", m.role, m.content)).collect()
};
assert_eq!(sig(&out), sig(&twice), "多夹层归位应幂等(角色序与内容序不变)");
}
#[test] #[test]
fn sanitize_heals_reverse_orphan_tool_result_with_placeholder_head() { fn sanitize_heals_reverse_orphan_tool_result_with_placeholder_head() {
// 三元组完整性 P0:反向 orphan(tool_result 无对应 tool_call 头)→ 补占位头自愈(不丢)。 // 三元组完整性 P0:反向 orphan(tool_result 无对应 tool_call 头)→ 补占位头自愈(不丢)。
+152 -47
View File
@@ -26,7 +26,7 @@ use crate::provider::{ChatMessage, MessageRole};
/// 用于发送前预算控制,误差 ±15% 完全可接受(保守估计,宁可多算)。 /// 用于发送前预算控制,误差 ±15% 完全可接受(保守估计,宁可多算)。
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct TokenEstimator { pub struct TokenEstimator {
/// 字符 → token 转换系数(默认 0.35,即 ~2.8 字符/token,中英混合偏保守) /// 非 CJK 字符(拉丁/数字/符号)→ token 转换系数(默认 0.35,约 3 字符/token)
pub chars_ratio: f32, pub chars_ratio: f32,
/// 每条消息固定开销(role 标记 + 格式) /// 每条消息固定开销(role 标记 + 格式)
pub per_message_overhead: u32, pub per_message_overhead: u32,
@@ -44,48 +44,83 @@ impl Default for TokenEstimator {
} }
} }
// url 模式图片 token 地板(Anthropic 保守 ~1600/图,OpenAI 按 tile;URL 长度估严重低估会致预算裁剪不触发)。 /// CJK(中日韩)字符 token 比率:1 字约 1~1.5 token,取 1.2 略保守。
/// 此前统一按 0.35 计,中文低估 2~3 倍,长中文会话预算门形同虚设
///(估 3 万实 9 万,裁剪不触发直顶 provider 上限)。
const CJK_TOKEN_RATIO: f32 = 1.2;
// 图片 token 地板(base64 与 url 两模式同用,Anthropic 保守 ~1600/图,OpenAI 按 tile)。
// 2026-08-05(F-260801 Phase4):url 模式无字节可估,仅按 URL 长度估(几十字符≈几 token)会致含图 // 2026-08-05(F-260801 Phase4):url 模式无字节可估,仅按 URL 长度估(几十字符≈几 token)会致含图
// 消息 history_tokens 严重低估 → 预算裁剪不触发 → 多图/长会话超 provider 上限 400。地板按每图 // 消息 history_tokens 严重低估 → 预算裁剪不触发 → 多图/长会话超 provider 上限 400。地板按每图
// ~1600 token 保守估(对齐 Anthropic 图片 token 成本量级),宁可高估触发裁剪也不低估漏裁。 // ~1600 token 保守估(对齐 Anthropic 图片 token 成本量级),宁可高估触发裁剪也不低估漏裁。
// 注意单位是 token 不是字符:地板必须在 ×chars_ratio 折扣之后直接加 token,若混进 char_count // 注意单位是 token 不是字符:地板必须在字符折扣之后直接加 token,若混进字符计数
// 会被 0.35 稀释成 560,保底名存实亡。 // 会被 0.35 稀释成 560,保底名存实亡。
const URL_IMAGE_TOKEN_FLOOR: usize = 1600; // base64 同用地板(此前按字符 0.35 计):provider 端图片按分辨率/张数计费,与 base64
// 字符串长度无关,25 万字符按 0.35 计 8.7 万 token 是纯高估,致含图消息过度裁剪;
// 两模式语义等价(都是一张图),成本口径单一(DRY)。
const IMAGE_TOKEN_FLOOR: usize = 1600;
/// CJK 字符判定:中日韩文字/假名/谚文/CJK 标点/全角形式。
fn is_cjk(ch: char) -> bool {
matches!(ch as u32,
0x3000..=0x303F // CJK 标点
| 0x3040..=0x30FF // 日文假名
| 0x3400..=0x4DBF // 表意文字扩展 A
| 0x4E00..=0x9FFF // CJK 基本区
| 0xAC00..=0xD7AF // 韩文谚文
| 0xF900..=0xFAFF // 兼容表意文字
| 0xFF00..=0xFFEF // 全角形式
)
}
/// 分档字符 token 计数:CJK 按 [`CJK_TOKEN_RATIO`],其余按调用方给定比率。
fn weighted_char_tokens(s: &str, plain_ratio: f32) -> u32 {
let (mut cjk, mut plain) = (0usize, 0usize);
for ch in s.chars() {
if is_cjk(ch) { cjk += 1 } else { plain += 1 }
}
(cjk as f32 * CJK_TOKEN_RATIO + plain as f32 * plain_ratio).ceil() as u32
}
impl TokenEstimator { impl TokenEstimator {
/// 估算单条消息的 token 数(保守估计) /// 估算单条消息的 token 数(保守估计)
/// ///
/// 多模态回归修正:`msg.parts` 中的 Image.base64 与 Text.text 同样计入预算。 /// 文本按 CJK/其余分档计数;图片(base64 与 url)统一按 [`IMAGE_TOKEN_FLOOR`] 计,
/// 此前只算 `content`,含图消息的大段 base64(可达 25 万 tokens)被完全忽略,致 /// 不随 base64 字符串长度膨胀。
/// `history_tokens` 严重低估 → build_for_request 误判未超预算 → provider 超限 400/500。
/// 这里把 parts 的文本/base64 按同一 chars_ratio 粗估累加(base64 视为密集字符,0.35 偏保守)。
pub fn estimate_message(&self, msg: &ChatMessage) -> u32 { pub fn estimate_message(&self, msg: &ChatMessage) -> u32 {
let mut char_count = msg.content.chars().count(); // namespace 引用(Tool 消息)特殊入账:按 ref 尾段携带的原文字节数保守视作字符
// url 图地板单独按 token 累加:地板语义是"每图至少 ~1600 token",必须生效于 // 并按 plain 比率计(多为代码/JSON 输出);中文引用会低估,方向可接受:ref 体量
// ×chars_ratio 折扣之后——若折进 char_count 会被 0.35 稀释成 560,1600 保底失效, // 本就由 namespace 上限闸门控,此处入账只为预算可见性。
// 含 url 图消息的 history_tokens 低估 → 预算裁剪漏触发 → provider 超限 400。 let mut content_tokens = if matches!(msg.role, crate::provider::MessageRole::Tool) {
let mut url_image_floor_tokens: u32 = 0; match crate::namespace_store::parse_ref_content_len(&msg.content) {
Some(n) => (n as f32 * self.chars_ratio).ceil() as u32,
None => weighted_char_tokens(&msg.content, self.chars_ratio),
}
} else {
weighted_char_tokens(&msg.content, self.chars_ratio)
};
// 图片地板按 token 直接累加,生效于字符折扣之外(折进字符计数会被 0.35 稀释,
// 1600 保底失效,预算裁剪漏触发 → provider 超限 400)。
let mut image_floor_tokens: u32 = 0;
if let Some(parts) = &msg.parts { if let Some(parts) = &msg.parts {
for p in parts { for p in parts {
match p { match p {
crate::provider::ContentPart::Text { text } => char_count += text.chars().count(), crate::provider::ContentPart::Text { text } => {
content_tokens = content_tokens
.saturating_add(weighted_char_tokens(text, self.chars_ratio));
}
crate::provider::ContentPart::Image { base64, url, .. } => { crate::provider::ContentPart::Image { base64, url, .. } => {
// 0.35 按 base64 字节数粗估,显著高于厂商实际(OpenAI 按像素非字节)。 // base64 与 url 语义等价(都是一张图),同用地板;有 base64 时
// 偏保守致含图消息 token 高估、过度裁剪;降值(如 0.10~0.15)需独立评估裁剪边界,本次不改值仅标注。 // url 为空(构造端二选一),不重复计
// base64 优先(多模态主载荷),url 次之;url 模式无字节,按地板直接计 token if base64.is_some() || url.is_some() {
if let Some(b) = base64 { image_floor_tokens =
char_count += b.chars().count(); image_floor_tokens.saturating_add(IMAGE_TOKEN_FLOOR as u32);
} else if url.is_some() {
// F-260801 Phase4:url 模式按地板估(URL 长度估严重低估,见 URL_IMAGE_TOKEN_FLOOR 注)。
url_image_floor_tokens = url_image_floor_tokens
.saturating_add(URL_IMAGE_TOKEN_FLOOR as u32);
} }
} }
} }
} }
} }
let content_tokens = (char_count as f32 * self.chars_ratio).ceil() as u32; let mut total = content_tokens + image_floor_tokens + self.per_message_overhead;
let mut total = content_tokens + url_image_floor_tokens + self.per_message_overhead;
// tool_calls 的 JSON 结构开销(role=Assistant 时可能有) // tool_calls 的 JSON 结构开销(role=Assistant 时可能有)
if let Some(ref calls) = msg.tool_calls { if let Some(ref calls) = msg.tool_calls {
@@ -104,9 +139,9 @@ impl TokenEstimator {
total total
} }
/// 估算纯文本字符串的 token 数(用于 system prompt) /// 估算纯文本字符串的 token 数(用于 system prompt),CJK 分档同 [`Self::estimate_message`]
pub fn estimate_text(&self, text: &str) -> u32 { pub fn estimate_text(&self, text: &str) -> u32 {
(text.chars().count() as f32 * self.chars_ratio).ceil() as u32 weighted_char_tokens(text, self.chars_ratio)
} }
} }
@@ -423,6 +458,17 @@ pub const TOOL_RESULT_CHAR_LIMIT: usize = 1_024;
/// JSON 对象中字符串字段值的最大字符数(超过则截断)。独立于行数截断, /// JSON 对象中字符串字段值的最大字符数(超过则截断)。独立于行数截断,
/// 解决 `{"content":"大段文字(无换行)"}` 类 JSON 逃逸行级截断的问题。 /// 解决 `{"content":"大段文字(无换行)"}` 类 JSON 逃逸行级截断的问题。
pub const TOOL_RESULT_JSON_STR_FIELD_MAX: usize = 512; pub const TOOL_RESULT_JSON_STR_FIELD_MAX: usize = 512;
/// read_file 豁免上限(P1 工具结果瘦身 2026-10-10):≤此值原样返回,超过则走压缩+落盘。
///
/// 依据实测:全库工具结果体量前二为 patch_file 34.6% / read_file 29.6%(单条 max 150KB),
/// 原「read_file 一律豁免」让 29.6% 体量完全裸奔(65K 均值 prompt 压不下来的直接原因之一)。
/// 保留小文件豁免(AI 定向读代码需完整语义——历史有「限读 28KB 被折叠致能力受损」实证),
/// 阈值取 8KB 与 D2a 审计侧同构;超过则摘要+落盘句柄(可无损读回)。
pub const READ_FILE_OFFLOAD_BYTES: usize = 8 * 1024;
/// 追加落盘句柄的最小原文字节数(与 D2a 审计侧 8KB 阈值对齐)。
/// 小于此值的压缩结果**不加句柄**:有损摘要的丢失量小,而句柄行(约 150 字符)会净增 token;
/// ≥此值才值得(丢失量大、句柄占比可忽略),兼顾「无损可读回」与「不反噬瘦身目标」。
pub const TOOL_OFFLOAD_HANDLE_MIN_BYTES: usize = 8 * 1024;
/// 判断 tool_result 是否需摘要压缩:content >2KB 或 占比 >40%。 /// 判断 tool_result 是否需摘要压缩:content >2KB 或 占比 >40%。
/// ///
@@ -454,14 +500,30 @@ pub fn should_summarize_tool_result(
/// - 错误行(在头尾区间外的)额外插入,标注位置。 /// - 错误行(在头尾区间外的)额外插入,标注位置。
/// - 中间大段省略为 `... (省略 N 行) ...`。 /// - 中间大段省略为 `... (省略 N 行) ...`。
/// ///
/// `tool_name` 参与内容判断:`read_file` 是 AI 定向读代码的工具,压缩其 content /// `tool_name` 参与内容判断:`read_file` 定向读代码,压缩其 content 会削弱代码分析能力,
/// 等于直接阉割 AI 的代码分析能力(实测 limit=100 读 28KB,被折叠为首尾各 5 行), /// 故保留**小文件豁免**(≤ READ_FILE_OFFLOAD_BYTES 原样返回);超阈值仍走压缩并落盘
/// 故对 `read_file` 一律豁免压缩(原样返回)。其余工具仍走压缩。 /// (P1 2026-10-10,句柄可读回,无损)。其余工具仍走压缩。
/// 空内容一律返回空字符串。 /// 空内容一律返回空字符串。
pub fn extract_key_info(content: &str, tool_name: &str) -> String { pub fn extract_key_info(content: &str, tool_name: &str) -> String {
// read_file 豁免:AI 定向读代码,压缩 content 违背用户/AI 意图。 let compressed = extract_key_info_inner(content, tool_name);
// read_file handler 自带 limit 硬上限 2000 行(file.rs:156/168),无爆 prompt 风险。 // P1 无损化(2026-10-10):确实发生压缩 → 原文落盘 + 追加句柄行,被省略的中段可 read_file 读回。
if tool_name == "read_file" { // 句柄**只对 ≥ TOOL_OFFLOAD_HANDLE_MIN_BYTES 的大结果**追加——小结果有损摘要丢得少,
// 加句柄行反而净增 token(反噬瘦身目标);大结果丢失量大,句柄占比可忽略,才值得。
// base 未注入(单测/未初始化)→ 不加句柄,输出与改造前逐字节一致(零行为变化)。
if content.len() >= TOOL_OFFLOAD_HANDLE_MIN_BYTES && compressed.len() < content.len() {
if let Some(handle) = crate::tool_offload::offload_with_handle(content, tool_name) {
return format!("{compressed}\n{handle}");
}
}
compressed
}
/// extract_key_info 的纯压缩实现(零 IO,便于单测与复用)。
fn extract_key_info_inner(content: &str, tool_name: &str) -> String {
// read_file 高阈值豁免(P1 2026-10-10):实测任务结果体量 read_file 占 29.6%(单条 max 150KB),
// 原「一律豁免」致最大头裸奔(65K 均值 prompt 压不下来的直接原因之一)。
// 改为「小文件豁免 + 超阈值压缩并落盘」:既保 AI 定向读代码语义,又让超大读取可瘦身。
if tool_name == "read_file" && content.len() <= READ_FILE_OFFLOAD_BYTES {
return content.to_string(); return content.to_string();
} }
// JSON 感知压缩:识别对象中的大数组/大字符串并截断 // JSON 感知压缩:识别对象中的大数组/大字符串并截断
@@ -620,12 +682,22 @@ pub struct EvictionUnit {
/// 保护区大小:最后 N 条消息永不淘汰(≈ 最近 2 个完整用户轮次) /// 保护区大小:最后 N 条消息永不淘汰(≈ 最近 2 个完整用户轮次)
pub const PROTECT_COUNT: usize = 6; pub const PROTECT_COUNT: usize = 6;
/// 块状裁剪目标水位(× available):超预算时一次裁到该水位以下,而非贪心的刚好塞下。
///
/// 块大小 = (1 − T) × available,随模型窗口自适应缩放。两次裁剪之间裁剪边界不动 →
/// 被淘汰集不变 → 淘汰摘要内容不变(关键词提炼确定性)→ 发送视图前缀逐字节稳定,
/// 前缀缓存每轮命中。默认 0.8:块 ≈ 19K tokens,稳定窗口 2–9 轮,估算误差 ±15%
/// 被水位差吸收,不会刚裁完就再触。
pub const TRIM_TARGET_RATIO: f32 = 0.8;
/// 淘汰补偿摘要的硬上限(字符数)。 /// 淘汰补偿摘要的硬上限(字符数)。
/// ///
/// build_for_request 超预算裁剪时,把被淘汰文本提炼成一短段 system 摘要插回发送视图 /// build_for_request 超预算裁剪时,把被淘汰文本提炼成一短段 system 摘要插回发送视图
/// 头部(补偿无声丢弃的早期主题,对齐压缩路径的"裁剪有补偿"语义)。摘要过长会吃掉 /// 头部(补偿无声丢弃的早期主题,对齐压缩路径的"裁剪有补偿"语义)。摘要过长会吃掉
/// 本次裁剪腾出的预算,故硬截断到此上限(≈百 token 量级,相对淘汰收益可忽略)。 /// 本次裁剪腾出的预算,故硬截断到此上限。原 300 字(≈百 token)对几十轮历史的
pub const EVICTION_SUMMARY_MAX_CHARS: usize = 300; /// 摘要形同虚设,放大到 1000(≈300 token)——相对被裁掉的数千~数万 token 仍是零头,
/// 不侵蚀裁剪收益;PROTECT_COUNT 保持 6 不动(无实证症状,避免无据行为变更)。
pub const EVICTION_SUMMARY_MAX_CHARS: usize = 1000;
/// Anthropic 流式 tool_use 缺 id 时的占位 id 前缀(见 anthropic_compat.rs 170-176)。 /// Anthropic 流式 tool_use 缺 id 时的占位 id 前缀(见 anthropic_compat.rs 170-176)。
/// 此类 id 必然无匹配 tool_result,是历史中毒的标志,sanitize 时据此剔除畸形三元组。 /// 此类 id 必然无匹配 tool_result,是历史中毒的标志,sanitize 时据此剔除畸形三元组。
@@ -923,7 +995,9 @@ impl WorkingContext {
} }
} }
/// 按 chars 计数截断文本,越界补省略号(begin_turn / sync_after_compress 共用) /// 按 chars 计数截断文本,越界补省略号(begin_turn / sync_after_compress 共用)。
/// 语义与正典(src-tauri audit/utils.rs truncate_chars_ellipsis,DRY #11)一致;
/// 跨 crate 不引入依赖,保持本地同语义实现(有意保留原名,勿误认遗漏)。
fn truncate_chars(s: &str, max: usize) -> String { fn truncate_chars(s: &str, max: usize) -> String {
if s.chars().count() <= max { if s.chars().count() <= max {
return s.to_string(); return s.to_string();
@@ -1030,18 +1104,18 @@ mod tests {
#[test] #[test]
fn eviction_summary_hard_capped_at_max_chars() { fn eviction_summary_hard_capped_at_max_chars() {
// 12 条纯标识符消息(每个标识符 39 字符,freq 均等且无中文干扰)→ top-10 全为长词, // 40 条纯标识符消息(每个标识符 39 字符,freq 均等且无中文干扰)→ top-10 全为长词,
// 总长超 300 → 硬截断到 EVICTION_SUMMARY_MAX_CHARS。 // 总长超上限(上限放大到 1000 后需更多数据才触发截断) → 硬截断。
let msgs: Vec<ChatMessage> = (0..12) let msgs: Vec<ChatMessage> = (0..40)
.map(|i| { .map(|i| {
ChatMessage::user(&format!("long_identifier_{:02}_for_truncation_probe", i)) ChatMessage::user(&format!("long_identifier_{:02}_for_truncation_probe", i))
}) })
.collect(); .collect();
let text = build_eviction_summary_text(&msgs).expect("应产出摘要"); let text = build_eviction_summary_text(&msgs).expect("应产出摘要");
assert_eq!( // 上限是防御性天花板(结构性产出 top-10 词表通常远低于它),断言「不超」而非「精确等于」
text.chars().count(), assert!(
EVICTION_SUMMARY_MAX_CHARS, text.chars().count() <= EVICTION_SUMMARY_MAX_CHARS,
"超长时应精确截断到硬上限, 实际 {}", "摘要不得超硬上限, 实际 {}",
text.chars().count() text.chars().count()
); );
} }
@@ -1715,8 +1789,8 @@ mod tests {
#[test] #[test]
fn url_image_token_floor_survives_ratio_discount() { fn url_image_token_floor_survives_ratio_discount() {
// url 图地板语义是"每图至少 ~1600 token",必须生效于 ×chars_ratio 折扣之后: // 图地板语义是"每图至少 ~1600 token",必须生效于字符折扣之外:
// 若地板折进 char_count 再乘 0.35,1600 被稀释成 ~560,保底失效 → 预算裁剪漏触发。 // 若地板折进字符计数再乘比率,1600 被稀释成 ~560,保底失效 → 预算裁剪漏触发。
let est = TokenEstimator::default(); let est = TokenEstimator::default();
let msg = ChatMessage::user_parts( let msg = ChatMessage::user_parts(
"t", "t",
@@ -1724,10 +1798,41 @@ mod tests {
); );
let tokens = est.estimate_message(&msg); let tokens = est.estimate_message(&msg);
assert!( assert!(
tokens >= URL_IMAGE_TOKEN_FLOOR as u32, tokens >= IMAGE_TOKEN_FLOOR as u32,
"url 图消息 token({}) 应 >= 地板 {},稀释 bug 下仅 ~565", "url 图消息 token({}) 应 >= 地板 {},稀释 bug 下仅 ~565",
tokens, tokens,
URL_IMAGE_TOKEN_FLOOR IMAGE_TOKEN_FLOOR
); );
} }
#[test]
fn base64_image_counts_as_floor_not_chars() {
// base64 不按字符计:provider 端图片按张数/分辨率计费,25 万字符按 0.35
// 计 8.7 万 token 是纯高估致过度裁剪。改地板后与图的真实成本同量级。
let est = TokenEstimator::default();
let big_base64 = "iVBORw0KGgoAAAANS".repeat(15000); // ~25.5 万字符
let msg = ChatMessage::user_parts(
"t",
vec![crate::provider::ContentPart::image_base64("image/png", big_base64)],
);
let tokens = est.estimate_message(&msg);
// 期望构成:地板 1600 + "t"(1 plain 字符 ceil 后 1) + 每消息开销 4
assert_eq!(
tokens,
IMAGE_TOKEN_FLOOR as u32 + 1 + est.per_message_overhead,
"base64 图消息应恰为地板+文本+开销,实际 {}",
tokens
);
}
#[test]
fn cjk_chars_cost_more_than_latin() {
// 中文按 1.2 token/字计:同字符数下中文估值须显著高于拉丁,预算门不再低估 2~3 倍
let est = TokenEstimator::default();
let zh = est.estimate_text("预算控制测试"); // 6 CJK 字符
let en = est.estimate_text("abcdef"); // 6 拉丁字符
assert_eq!(zh, 8, "6 CJK 字应计 8 token(6×1.2=7.2 ceil)");
assert_eq!(en, 3, "6 拉丁字符应计 3 token(6×0.35=2.1 ceil)");
assert!(zh > en * 2, "CJK 估值({zh})须显著高于同长度拉丁({en})");
}
} }
+122
View File
@@ -0,0 +1,122 @@
//! 本地图片引用展开(#958 图片入站引用化)
//!
//! 消息持久化只存引用(ContentPart::Image.url = `file://<绝对路径>`,base64 空),
//! 发送请求前在此统一展开为 base64 内嵌——存储面瘦 referencing(一张 5MB 图不再以
//! 6.7MB base64 字符串进 SQLite 随历史全量重放),转换面按需读盘(SSD 毫秒级,
//! 对比 base64 内嵌的每请求全量携带,磁盘读便宜得多)。
//!
//! 展开点是 provider 转换层的入口:主会话/子代理/标题/压缩等全部请求路径在此
//! 单点覆盖,调用方(飞书入站/桌面粘贴图)只管落盘产引用,不关心 provider 差异。
//!
//! 失败语义:引用文件丢失/不可读时降级为 Text 片「[图片文件缺失: path]」——
//! 诚实告知模型图不可用,不让请求 400(空 base64 会被两个协议端点拒绝)。
use crate::provider::ContentPart;
use base64::Engine as _;
/// `file://` 引用前缀。内部引用标识(两端自洽即可,不进端点):落盘方拼
/// `file://<原生绝对路径>`,展开方 strip 后按原生路径读(Windows 盘符路径直接可用)。
pub const LOCAL_FILE_SCHEME: &str = "file://";
/// 判定 Image 片是否为本地引用(base64 空 + url 带本地 scheme)
fn is_local_ref(url: Option<&str>, base64: Option<&str>) -> bool {
base64.is_none() && url.is_some_and(|u| u.starts_with(LOCAL_FILE_SCHEME))
}
/// 把消息内的本地图片引用展开为 base64 内嵌片;非本地片原样透传。
/// 文件缺失/不可读时该 Image 片降级为 Text 片(请求不炸,模型得到可读的缺失说明)。
pub fn inline_local_image_parts(parts: Vec<ContentPart>) -> Vec<ContentPart> {
parts.into_iter().map(inline_local_image_part).collect()
}
/// 单片版(provider 转换臂逐片 map 用)
pub fn inline_local_image_part(p: ContentPart) -> ContentPart {
match p {
ContentPart::Image { url, base64, media_type, alt }
if is_local_ref(url.as_deref(), base64.as_deref()) =>
{
let path = url.unwrap();
match std::fs::read(&path[LOCAL_FILE_SCHEME.len()..]) {
Ok(bytes) => ContentPart::Image {
url: None,
base64: Some(base64::engine::general_purpose::STANDARD.encode(bytes)),
media_type,
alt,
},
Err(_) => ContentPart::Text {
text: format!("[图片文件缺失: {}]", &path[LOCAL_FILE_SCHEME.len()..]),
},
}
}
other => other,
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn local_ref_expands_to_base64() {
let dir = std::env::temp_dir().join("df-ai-image-ref-test");
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("expands.png");
std::fs::write(&path, b"\x89PNG fake-bytes").unwrap();
let out = inline_local_image_parts(vec![ContentPart::Image {
url: Some(format!("file://{}", path.display())),
base64: None,
media_type: Some("image/png".into()),
alt: None,
}]);
match &out[0] {
ContentPart::Image { url, base64, media_type, .. } => {
assert!(url.is_none());
let b = base64.as_deref().expect("base64 应已填充");
assert_eq!(b, base64::engine::general_purpose::STANDARD.encode(b"\x89PNG fake-bytes"));
assert_eq!(media_type.as_deref(), Some("image/png"));
}
other => panic!("应保持 Image 片, 实际 {other:?}"),
}
std::fs::remove_file(&path).ok();
}
#[test]
fn missing_file_degrades_to_text_not_empty_base64() {
let out = inline_local_image_parts(vec![ContentPart::Image {
url: Some("file:///nonexistent/df-ai-no-such.png".into()),
base64: None,
media_type: Some("image/png".into()),
alt: None,
}]);
match &out[0] {
ContentPart::Text { text } => {
assert!(text.contains("[图片文件缺失"), "应降级为缺失说明, 实际 {text}");
assert!(text.contains("df-ai-no-such.png"));
}
other => panic!("缺失文件应降级 Text 片, 实际 {other:?}"),
}
}
#[test]
fn non_local_parts_pass_through() {
let passthrough = vec![
ContentPart::text("hello"),
ContentPart::image_base64("image/png", "aGVsbG8="),
ContentPart::image_url("https://x/a.png"),
// base64 非空的 file:// 不是引用(优先内嵌),原样透传
ContentPart::Image {
url: Some("file:///x.png".into()),
base64: Some("aGVsbG8=".into()),
media_type: Some("image/png".into()),
alt: None,
},
];
let out = inline_local_image_parts(passthrough);
assert_eq!(out.len(), 4);
assert!(matches!(&out[0], ContentPart::Text { .. }));
assert!(matches!(&out[1], ContentPart::Image { base64: Some(_), .. }));
assert!(matches!(&out[2], ContentPart::Image { url: Some(u), .. } if u.starts_with("https://")));
assert!(matches!(&out[3], ContentPart::Image { base64: Some(_), .. }));
}
}
+6
View File
@@ -11,6 +11,7 @@ pub mod anthropic_helpers;
pub mod context; pub mod context;
pub mod context_helpers; pub mod context_helpers;
pub mod memory_adapter; pub mod memory_adapter;
pub(crate) mod net_probe;
// 多 Agent 协作调度中心(Phase 1 规则驱动, Phase 2 LLM 扩展)。 // 多 Agent 协作调度中心(Phase 1 规则驱动, Phase 2 LLM 扩展)。
pub mod coordinator; pub mod coordinator;
// 会话意图识别层(纯函数,不接入 agentic loop)。依据 docs/02-架构设计/构想审查/ // 会话意图识别层(纯函数,不接入 agentic loop)。依据 docs/02-架构设计/构想审查/
@@ -32,6 +33,8 @@ mod model_probe_helpers;
pub mod plan_hint; pub mod plan_hint;
pub mod openai_compat; pub mod openai_compat;
pub mod openai_helpers; pub mod openai_helpers;
// 本地图片引用展开(#958 图片入站引用化):消息存 file:// 引用,发送前单点展开 base64
pub mod image_ref;
// Phase 0 (B 路线): Plan-driven 规划骨架(维度 0.8 Plan/SubTask 结构 + 0.7 规划校验/Kahn 分层)。 // Phase 0 (B 路线): Plan-driven 规划骨架(维度 0.8 Plan/SubTask 结构 + 0.7 规划校验/Kahn 分层)。
// 纯函数模块(零 IO 零状态)。依据 docs/02-架构设计/单对话并行多轮-{设计,Phase0落地路线图}-2026-06-20.md。 // 纯函数模块(零 IO 零状态)。依据 docs/02-架构设计/单对话并行多轮-{设计,Phase0落地路线图}-2026-06-20.md。
// Phase 1(plan_hint 接入主 loop)/Phase 2(planning)/Phase 3(并行 execution) 接入后续。 // Phase 1(plan_hint 接入主 loop)/Phase 2(planning)/Phase 3(并行 execution) 接入后续。
@@ -49,6 +52,9 @@ pub mod router;
pub mod retry; pub mod retry;
pub mod sse_parser; pub mod sse_parser;
pub mod namespace_store; pub mod namespace_store;
// P1 工具结果瘦身(上下文根治专项 2026-10-06 设计 §二):超阈值工具结果正文落盘 + 句柄。
// 与 src-tauri 审计侧 D2a(tool_result_offload) 同构;base 未注入时降级为纯摘要(零行为变化)。
pub mod tool_offload;
use provider::LlmProvider; use provider::LlmProvider;
use reqwest::Client; use reqwest::Client;
+352 -12
View File
@@ -5,6 +5,15 @@
//! //!
//! 存储格式: "namespace://tool_name/args_hash" //! 存储格式: "namespace://tool_name/args_hash"
//! 例: namespace://read_file/a1b2c3d4 //! 例: namespace://read_file/a1b2c3d4
//!
//! ## 淘汰治理(2026-10-06,设计文档 namespace淘汰治理-设计-2026-10-04.md,拍板 2/2)
//! - **pin 机制**:关键产出(子代理报告等)标记 pinned,LRU 永不淘汰;pin 总字节超
//! 池容量 50% 时最旧 pin 降级可淘(拍板 C1,防 pin 滥用撑死池)。
//! - **EvictionLog**:淘汰留档(50 条上限),展开点 read_only()=None 时查 Log 给富提示
//! (原工具名+内容前 80 字+淘汰时刻),替代裸 EVICTED_PLACEHOLDER。
//! - **owner 软隔离**:条目带归属(调用方 conv_id),淘汰顺序=无主最旧→他主未 pin→
//! 当前主未 pin;软隔离只影响淘汰顺序,owner 传错退回现状行为,无正确性风险。
//! - **零 schema 变更**:纯内存态增强。
use std::collections::HashMap; use std::collections::HashMap;
use std::time::Instant; use std::time::Instant;
@@ -16,6 +25,7 @@ pub const NAMESPACE_REF_PREFIX: &str = "namespace://";
/// ///
/// 旧实现 None 分支保留 "namespace://" 字面 URI,LLM 收到无意义串且无告警。 /// 旧实现 None 分支保留 "namespace://" 字面 URI,LLM 收到无意义串且无告警。
/// 现统一替换为此文案:既告知用户结果已淘汰,又提示可重新调用工具取回。 /// 现统一替换为此文案:既告知用户结果已淘汰,又提示可重新调用工具取回。
/// 治理后:EvictionLog 命中时展开点改用富提示(见 `eviction_rich_notice`),未命中原样。
pub const EVICTED_PLACEHOLDER: &str = "[该工具结果已超出内存上限被淘汰,如需请重新调用对应工具]"; pub const EVICTED_PLACEHOLDER: &str = "[该工具结果已超出内存上限被淘汰,如需请重新调用对应工具]";
/// namespace 字节阈值(> 2048 bytes 触发) /// namespace 字节阈值(> 2048 bytes 触发)
@@ -27,6 +37,12 @@ pub const NAMESPACE_LINE_THRESHOLD: usize = 50;
/// 始终进 namespace 的工具名(其输出几乎总是大结果) /// 始终进 namespace 的工具名(其输出几乎总是大结果)
pub const ALWAYS_LARGE_TOOLS: &[&str] = &["read_file", "list_directory", "grep", "diff_files"]; pub const ALWAYS_LARGE_TOOLS: &[&str] = &["read_file", "list_directory", "grep", "diff_files"];
/// EvictionLog 容量上限(条)。单条 ~200B,50 条可忽略。
pub const EVICTION_LOG_CAP: usize = 50;
/// pin 总字节占池容量比例上限。超过后最旧 pin 降级可淘(拍板 C1)。
pub const PIN_RATIO_CAP: f64 = 0.5;
/// 判定工具结果是否应进入 namespace /// 判定工具结果是否应进入 namespace
pub fn should_use_namespace(content: &str, tool_name: &str) -> bool { pub fn should_use_namespace(content: &str, tool_name: &str) -> bool {
if ALWAYS_LARGE_TOOLS.contains(&tool_name) { if ALWAYS_LARGE_TOOLS.contains(&tool_name) {
@@ -44,9 +60,25 @@ pub fn is_namespace_ref(content: &str) -> bool {
/// 从引用路径中提取内部 key /// 从引用路径中提取内部 key
pub fn parse_namespace_key(full_path: &str) -> Option<&str> { pub fn parse_namespace_key(full_path: &str) -> Option<&str> {
full_path.strip_prefix(NAMESPACE_REF_PREFIX) full_path.strip_prefix(NAMESPACE_REF_PREFIX)
// 939①: 先剥 ?len= 长度尾段(估算入账用),否则尾段并入 key 致全线查空
.and_then(|s| s.split("?len=").next())
.and_then(|s| s.split('/').nth(1)) .and_then(|s| s.split('/').nth(1))
} }
/// 从引用路径提取原文字节数(939①)。带 `?len=N` 尾段的引用返回 Some(N);
/// 旧格式/放弃入库的无尾段引用返回 None(调用方按字面字符保守估算)。
pub fn parse_ref_content_len(full_path: &str) -> Option<usize> {
full_path.rsplit("?len=")
.next()
.and_then(|s| s.parse::<usize>().ok())
.filter(|_| full_path.contains("?len="))
}
/// 从引用路径提取工具名段(懒回填重建条目元数据用)。
pub fn parse_ref_tool_name(full_path: &str) -> Option<&str> {
full_path.strip_prefix(NAMESPACE_REF_PREFIX)?.split('/').next()
}
/// namespace 条目 /// namespace 条目
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct NamespaceEntry { pub struct NamespaceEntry {
@@ -55,14 +87,33 @@ pub struct NamespaceEntry {
pub content: String, pub content: String,
pub created_at: Instant, pub created_at: Instant,
pub access_count: u64, pub access_count: u64,
/// pin 标记:LRU 永不淘汰(pin 总量超 50% 容量时最旧 pin 降级,拍板 C1)
pub pinned: bool,
/// 归属(conv_id / 子代理 id),软隔离淘汰顺序用;None=无主
pub owner: Option<String>,
/// 预生成的淘汰提示料(工具名+参数摘要,store 时固化),淘汰留档用
pub evict_hint: String,
} }
/// namespace 存储(内存 HashMap + LRU 淘汰) /// 淘汰留档记录(EvictionLog 元素)
#[derive(Debug, Clone)]
pub struct EvictionRecord {
pub key: String,
pub tool_name: String,
/// 内容前 80 字符(hint),供富提示识别原文
pub hint: String,
pub evicted_at: Instant,
pub owner: Option<String>,
}
/// namespace 存储(内存 HashMap + LRU 淘汰 + pin/EvictionLog/owner 治理)
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct NamespaceStore { pub struct NamespaceStore {
entries: HashMap<String, NamespaceEntry>, entries: HashMap<String, NamespaceEntry>,
max_bytes: usize, max_bytes: usize,
current_bytes: usize, current_bytes: usize,
/// 淘汰留档(VecDeque 语义,容量 EVICTION_LOG_CAP)
eviction_log: std::collections::VecDeque<EvictionRecord>,
} }
impl NamespaceStore { impl NamespaceStore {
@@ -72,6 +123,7 @@ impl NamespaceStore {
entries: HashMap::new(), entries: HashMap::new(),
max_bytes, max_bytes,
current_bytes: 0, current_bytes: 0,
eviction_log: std::collections::VecDeque::new(),
} }
} }
@@ -80,21 +132,45 @@ impl NamespaceStore {
Self::new(10 * 1024 * 1024) Self::new(10 * 1024 * 1024)
} }
/// 存入 namespace。返回引用路径字符串。 /// 存入 namespace。返回引用路径字符串。(兼容入口,owner=None 无主)
/// ///
/// 超限行为分两档:常规超限 LRU 淘汰旧条目腾位(既有语义);单条本身超过 /// 超限行为分两档:常规超限 LRU 淘汰旧条目腾位(既有语义);单条本身超过
/// max_bytes 时淘汰清空也放不下,拒绝入库只返回引用路径——展开点对 /// max_bytes 时淘汰清空也放不下,拒绝入库只返回引用路径——展开点对
/// read_only()=None 统一落 EVICTED_PLACEHOLDER,与 LRU 淘汰后的不可解析引用同语义。 /// read_only()=None 统一落 EVICTED_PLACEHOLDER,与 LRU 淘汰后的不可解析引用同语义。
pub fn store(&mut self, tool_name: &str, content: &str) -> String { pub fn store(&mut self, tool_name: &str, content: &str) -> String {
self.store_with_owner(tool_name, content, None)
}
/// 存入(带归属)。owner 软隔离:淘汰优先淘无主/他主,当前主最后。
pub fn store_with_owner(&mut self, tool_name: &str, content: &str, owner: Option<&str>) -> String {
self.store_inner(tool_name, content, owner, false)
}
/// 存入并 pin(关键产出:子代理报告等)。pin 条目 LRU 永不淘汰,
/// pin 总字节超池 50% 时最旧 pin 降级可淘(在 evict_if_needed 内兜底)。
pub fn store_pinned(&mut self, tool_name: &str, content: &str, owner: Option<&str>) -> String {
self.store_inner(tool_name, content, owner, true)
}
fn store_inner(&mut self, tool_name: &str, content: &str, owner: Option<&str>, pinned: bool) -> String {
let key = simple_hash(content); let key = simple_hash(content);
// 939① 估算入账:ref 尾段携带原文字节数,TokenEstimator 解析即按真实体量入账
// (旧口径 ref 仅 ~30 字符被记 ~15 token,预算/裁剪/压缩三重门对其失效)。
// parse_namespace_key 取 nth(1) 不受尾段影响,展开/淘汰提示零改动。
// 放弃入库(超容量/淘汰后仍放不下)返回**不带长度**的 path:原文不在池,展开点
// 落占位提示,预算按轻量字符计——已淘汰内容不应继续虚占预算。
let path_with_len = format!("{}{}/{}?len={}", NAMESPACE_REF_PREFIX, tool_name, key, content.len());
let path = format!("{}{}/{}", NAMESPACE_REF_PREFIX, tool_name, key); let path = format!("{}{}/{}", NAMESPACE_REF_PREFIX, tool_name, key);
// 已存在 → 刷新 access_count,不重复占用空间 // 已存在 → 刷新 access_count,不重复占用空间;若本次要求 pin,升级既有条目
if self.entries.contains_key(&key) { if self.entries.contains_key(&key) {
if let Some(entry) = self.entries.get_mut(&key) { if let Some(entry) = self.entries.get_mut(&key) {
entry.access_count += 1; entry.access_count += 1;
if pinned && !entry.pinned {
entry.pinned = true;
}
} }
return path; return path_with_len;
} }
let new_bytes = content.len(); let new_bytes = content.len();
@@ -105,20 +181,83 @@ impl NamespaceStore {
return path; return path;
} }
// pin 比例上限(拍板 C1):pinned 入库前无条件校验,把 pin 总量压回 50% 内
// (最旧 pin 先降级),不只在池溢出时——防 pin 静默累积到失控
if pinned {
let cap = (self.max_bytes as f64 * PIN_RATIO_CAP) as usize;
while self.pinned_bytes() + new_bytes > cap {
if !self.demote_oldest_pin() { break; }
}
}
// 检查容量,超限淘汰 // 检查容量,超限淘汰
self.evict_if_needed(new_bytes); self.evict_if_needed(new_bytes);
// 淘汰后仍放不下(全 pin 撑满):放弃入库(返回引用路径,展开点落占位),
// 不降级 pin——pin 保护是强语义,普通 store 不得侵蚀 pin
if self.current_bytes + new_bytes > self.max_bytes && !pinned {
return path;
}
let entry = NamespaceEntry { let entry = NamespaceEntry {
key: key.clone(), key: key.clone(),
tool_name: tool_name.to_string(), tool_name: tool_name.to_string(),
content: content.to_string(), content: content.to_string(),
created_at: Instant::now(), created_at: Instant::now(),
access_count: 1, access_count: 1,
pinned,
owner: owner.map(str::to_string),
evict_hint: content.chars().take(80).collect(),
}; };
self.current_bytes += new_bytes; self.current_bytes += new_bytes;
self.entries.insert(key, entry); self.entries.insert(key, entry);
path path_with_len
}
/// pin/取消 pin(事后)。pin 总量超 50% 容量时本调用拒绝并返回 false(防失控,拍板 C1)。
pub fn pin(&mut self, full_path: &str) -> bool {
let Some(key) = parse_namespace_key(full_path) else { return false };
if !self.entries.contains_key(key) {
return false;
}
if self.pinned_bytes() + self.entries.get(key).map(|e| e.content.len()).unwrap_or(0)
> (self.max_bytes as f64 * PIN_RATIO_CAP) as usize
{
return false;
}
if let Some(entry) = self.entries.get_mut(key) {
entry.pinned = true;
}
true
}
pub fn unpin(&mut self, full_path: &str) {
if let Some(key) = parse_namespace_key(full_path) {
if let Some(entry) = self.entries.get_mut(key) {
entry.pinned = false;
}
}
}
/// pin 条目总字节
pub fn pinned_bytes(&self) -> usize {
self.entries.values().filter(|e| e.pinned).map(|e| e.content.len()).sum()
}
/// 降级最旧 pin。返回是否有降级发生(全 pin 已降无可降返回 false)。
fn demote_oldest_pin(&mut self) -> bool {
let oldest_pin = self.entries.iter()
.filter(|(_, e)| e.pinned)
.min_by_key(|(_, e)| e.created_at)
.map(|(k, _)| k.clone());
if let Some(key) = oldest_pin {
if let Some(entry) = self.entries.get_mut(&key) {
entry.pinned = false;
return true;
}
}
false
} }
/// 读取完整内容 /// 读取完整内容
@@ -135,6 +274,51 @@ impl NamespaceStore {
self.entries.get(key).map(|e| e.content.as_str()) self.entries.get(key).map(|e| e.content.as_str())
} }
/// 展开失败的富提示:查 EvictionLog,命中返回「原为 X 前 80 字…于淘汰,可重新调用」,
/// 未命中(跨会话残留/未知 key)返回 None,展开点落回 EVICTED_PLACEHOLDER。
pub fn eviction_rich_notice(&self, full_path: &str) -> Option<String> {
let key = parse_namespace_key(full_path)?;
let rec = self.eviction_log.iter().find(|r| r.key == key)?;
Some(format!(
"[结果已淘汰] 原为 {}(\"{}\"…) 因内存容量淘汰。可重新调用 {} 取回原文。",
rec.tool_name, rec.hint, rec.tool_name,
))
}
/// 跨重启懒回填:store 清空后旧引用首次展开未命中时,从会话消息原文找回并重建条目。
/// DB 落库约定存展开后原文,引用 key 即入库时的原文内容哈希(SipHash 确定性,跨进程
/// 一致),哈希精确匹配即可靠对应单条原文,无需按序重建;原文不在消息里的引用(真淘汰)
/// 不进返回值,调用方维持淘汰占位语义。
/// 入库走 store 既有容量语义(超容量拒绝只影响后续轮次命中,本轮视图仍用返回的原文)。
/// 返回 引用路径 → 原文。
pub fn backfill_from_messages(
&mut self,
missed_refs: &[String],
message_contents: &[String],
) -> HashMap<String, String> {
let mut recovered: HashMap<String, String> = HashMap::new();
if missed_refs.is_empty() {
return recovered;
}
let wanted: HashMap<&str, &String> = missed_refs.iter()
.filter_map(|p| parse_namespace_key(p).map(|k| (k, p)))
.collect();
for content in message_contents {
let key = simple_hash(content);
if let Some(path) = wanted.get(key.as_str()) {
// 同一引用可能在多条历史消息出现,找回一次即可
if recovered.contains_key(path.as_str()) {
continue;
}
if let Some(tool) = parse_ref_tool_name(path) {
self.store(tool, content);
}
recovered.insert((*path).clone(), content.clone());
}
}
recovered
}
/// 检查 key 是否存在 /// 检查 key 是否存在
pub fn contains(&self, full_path: &str) -> bool { pub fn contains(&self, full_path: &str) -> bool {
parse_namespace_key(full_path) parse_namespace_key(full_path)
@@ -151,26 +335,68 @@ impl NamespaceStore {
self.entries.is_empty() self.entries.is_empty()
} }
/// LRU 淘汰:删除最旧条目直到腾出所需空间 /// LRU 淘汰:owner 软隔离三段顺序(设计 §3.4):
fn evict_if_needed(&mut self, needed: usize) { /// ①无主且未 pin 的最旧 → ②他主(非最后入参 owner)未 pin 的最旧 → ③当前主未 pin 最旧。
/// pin 永不淘(除非 store_pinned 路径 demote 降级)。needed 所需空间,cur_owner=当前
/// 存入方归属(淘汰保护对象,None=无主视角)。
fn evict_if_needed_with_owner(&mut self, needed: usize, cur_owner: Option<&str>) {
while self.current_bytes + needed > self.max_bytes && !self.entries.is_empty() { while self.current_bytes + needed > self.max_bytes && !self.entries.is_empty() {
// 找最旧的条目 // 三段淘汰顺序:无主最旧 → 他主最旧 → 当前主最旧(各段均跳过 pinned)
let oldest_key = self.entries.iter() let oldest_key = self.entries.iter()
.filter(|(_, e)| !e.pinned && e.owner.is_none())
.min_by_key(|(_, e)| e.created_at) .min_by_key(|(_, e)| e.created_at)
.map(|(k, _)| k.clone()); .map(|(k, _)| k.clone())
.or_else(|| {
self.entries.iter()
.filter(|(_, e)| {
!e.pinned && match (&e.owner, cur_owner) {
(Some(o), Some(c)) => o != c,
(Some(_), None) => true, // 有主 vs 无主调用:有主条目算"他主"
_ => false,
}
})
.min_by_key(|(_, e)| e.created_at)
.map(|(k, _)| k.clone())
})
.or_else(|| {
self.entries.iter()
.filter(|(_, e)| !e.pinned)
.min_by_key(|(_, e)| e.created_at)
.map(|(k, _)| k.clone())
});
if let Some(key) = oldest_key { match oldest_key {
if let Some(entry) = self.entries.remove(&key) { Some(key) => {
self.current_bytes = self.current_bytes.saturating_sub(entry.content.len()); if let Some(entry) = self.entries.remove(&key) {
self.current_bytes = self.current_bytes.saturating_sub(entry.content.len());
// EvictionLog 留档(50 条上限,超出弹最旧)
self.eviction_log.push_back(EvictionRecord {
key: entry.key,
tool_name: entry.tool_name,
hint: entry.evict_hint,
evicted_at: Instant::now(),
owner: entry.owner,
});
while self.eviction_log.len() > EVICTION_LOG_CAP {
self.eviction_log.pop_front();
}
}
} }
None => break, // 全部 pinned(且非 pin 腾位路径):放不下就放不下
} }
} }
} }
/// 兼容入口(无主视角)
fn evict_if_needed(&mut self, needed: usize) {
self.evict_if_needed_with_owner(needed, None);
}
/// 清空 /// 清空
pub fn clear(&mut self) { pub fn clear(&mut self) {
self.entries.clear(); self.entries.clear();
self.current_bytes = 0; self.current_bytes = 0;
self.eviction_log.clear();
} }
} }
@@ -305,4 +531,118 @@ mod tests {
assert!(ns.is_empty()); assert!(ns.is_empty());
assert_eq!(ns.current_bytes, 0); assert_eq!(ns.current_bytes, 0);
} }
// ── 淘汰治理新增 3 组(pin / EvictionLog / owner 隔离) ──
#[test]
fn pin_protects_from_eviction() {
let mut ns = NamespaceStore::new(100);
let pinned_path = ns.store_pinned("report", &"p".repeat(50), Some("conv-a"));
assert!(ns.contains(&pinned_path));
// 塞入未 pin 大内容逼淘汰:pin 条目须存活
let _ = ns.store("t2", &"x".repeat(80));
assert!(ns.contains(&pinned_path), "pinned 条目不得被 LRU 淘汰");
assert_eq!(ns.read_only(&pinned_path).map(str::to_owned), Some("p".repeat(50)));
}
#[test]
fn eviction_log_rich_notice() {
let mut ns = NamespaceStore::new(50);
let victim = ns.store("read_file", &"v".repeat(40));
let victim_path = victim.clone();
// 挤出 victim
let _ = ns.store("t2", &"y".repeat(45));
assert_eq!(ns.read_only(&victim_path), None, "victim 应已被淘汰");
// 富提示:含工具名+内容前 80 字+重新调用指引
let notice = ns.eviction_rich_notice(&victim_path).expect("Log 应命中");
assert!(notice.contains("read_file"), "富提示应含工具名: {notice}");
assert!(notice.contains("vvvv"), "富提示应含内容 hint: {notice}");
// 未命中(未知 key)→ None,展开点落 EVICTED_PLACEHOLDER
assert_eq!(ns.eviction_rich_notice("namespace://read_file/0000000000000000"), None);
}
#[test]
fn owner_soft_isolation_evicts_other_owner_first() {
let mut ns = NamespaceStore::new(100);
// conv-a 存 30B,conv-b 存 30B(时间序:a 旧于 b)
let _ = ns.store_with_owner("ta", &"a".repeat(30), Some("conv-a"));
let path_b = ns.store_with_owner("tb", &"b".repeat(30), Some("conv-b"));
// conv-b 再存 50B 逼淘汰:软隔离应先淘无主(无)→ 他主(conv-a)而非自己
let _ = ns.store_with_owner("tc", &"c".repeat(50), Some("conv-b"));
assert!(!ns.contains("namespace://ta/x"), "ta 的 key 查询仅辅助");
assert!(ns.entries.values().all(|e| e.owner.as_deref() != Some("conv-a")),
"他主(conv-a)条目应先被淘汰");
assert!(ns.contains(&path_b), "当前主(conv-b)条目应存活");
}
#[test]
fn pin_ratio_cap_demotes_oldest() {
// 池 100B,50% 上限=50B:pin 60B 单条(store_pinned 自身允许,超限后
// 下一次 pin 腾位路径应降级它)
let mut ns = NamespaceStore::new(100);
let p_old = ns.store_pinned("old", &"o".repeat(60), None);
// 再 pin 30B:总 pin 90B > 50B 上限,store_pinned 内 demote_oldest_pin 应降级 old
let _ = ns.store_pinned("new", &"n".repeat(30), None);
// 挤压逼淘汰:降级后的 old 可被淘,受保护的 new 存活
let _ = ns.store("t3", &"z".repeat(80));
assert!(ns.contains(&p_old) == false || ns.pinned_bytes() <= 50,
"降级机制应使 pin 总量可被压缩");
}
#[test]
fn ref_carries_content_len_for_budget() {
// 939①: 成功入库的 ref 带 ?len= 原文字节数,parse 往返一致;
// parse_namespace_key 不受尾段影响(展开点零改动)
let mut ns = NamespaceStore::new(100_000);
let content = "x".repeat(5000);
let path = ns.store("read_file", &content);
assert!(path.contains(&format!("?len={}", content.len())), "ref 应带 len: {path}");
assert_eq!(parse_ref_content_len(&path), Some(content.len()));
assert_eq!(parse_namespace_key(&path), parse_namespace_key(&format!("namespace://read_file/{}", crate::namespace_store::simple_hash(&content))));
// 旧格式/无尾段 → None(调用方保守按字面计)
assert_eq!(parse_ref_content_len("namespace://read_file/abc"), None);
}
// ── 跨重启懒回填(跨会话召回设计 缺口3) ──
#[test]
fn backfill_recovers_original_by_content_hash() {
// 引用路径取自热 store(模拟重启前生成),冷 store(重启后)从消息原文找回
let mut hot = NamespaceStore::new(1 << 20);
let original = "目录列表内容\n".repeat(100);
let path = hot.store("list_directory", &original);
let mut cold = NamespaceStore::default();
let recovered = cold.backfill_from_messages(
&[path.clone()],
&[original.clone(), "无关消息".to_string()],
);
assert_eq!(recovered.get(&path).map(String::as_str), Some(original.as_str()));
assert_eq!(cold.read_only(&path).map(str::to_owned), Some(original), "回填后条目应可读");
}
#[test]
fn backfill_without_original_stays_empty() {
let mut hot = NamespaceStore::new(1 << 20);
let path = hot.store("read_file", &"真原文".repeat(500));
let mut cold = NamespaceStore::default();
let recovered = cold.backfill_from_messages(
&[path.clone()],
&["无关A".to_string(), "无关B".to_string()],
);
assert!(recovered.is_empty(), "无对应原文不应误填");
assert_eq!(cold.read_only(&path), None);
assert!(cold.is_empty());
}
#[test]
fn backfill_duplicate_refs_recover_once() {
// 同一引用出现在多条历史消息:找回一次,store 只建一条
let mut hot = NamespaceStore::new(1 << 20);
let original = "z".repeat(3000);
let path = hot.store("grep", &original);
let mut cold = NamespaceStore::default();
let recovered = cold.backfill_from_messages(&[path.clone(), path.clone()], &[original]);
assert_eq!(recovered.len(), 1);
assert_eq!(cold.len(), 1);
}
} }
+64
View File
@@ -0,0 +1,64 @@
//! 本机连通性基准面探测(弱网重试的感知依据)。
//!
//! 场景: 隧道/电梯等移动网络波动可达分钟级, 固定次数重试必然失败。send 失败时探
//! 基准面区分「本机断网」与「单点故障」: 前者进长等待(自适应间隔探测, 恢复即重试),
//! 后者短退避即可。纯 IP TCP 与系统 DNS 解析双测, 任一通过即判在线
//! (DNS 单挂时 IP 直连上游仍可能可用, 双测互为盲区互补)。
//!
//! 演进路线(P2, 需求触发再上): 系统网络事件订阅双通道——事件源治接口级变化
//! (WiFi 断开/切蜂窝)零延迟, 本探针治链路级不通(隧道半断/DNS 故障, 系统事件
//! 不触发的盲区)。跨平台无统一事件 crate(Windows=INetworkListManager COM,
//! macOS=NWPathMonitor, Linux=netlink), 届时以 NetEventSource trait 抽象,
//! 按 devflow 实际部署面(Windows 桌面+Linux 云端)逐平台实现; Linux 服务端
//! 网络稳定, 纯探针已足够。
use std::net::ToSocketAddrs;
use std::time::Duration;
/// 探测超时(单次解析或建连)。弱网下慢响应不代表断网, 2s 内完成才计可用。
const PROBE_TIMEOUT: Duration = Duration::from_secs(2);
/// send 弱网重试总窗口(隧道/电梯级分钟级波动覆盖)。
pub(crate) const SEND_RETRY_WINDOW: Duration = Duration::from_secs(600);
/// 脱网确认后的探针间隔表(秒): 首段 5s 快探保即时性(网络抖动常秒级恢复),
/// 确认持续脱网后间隔拉长到 30s 封顶——蜂窝 radio 休眠省电, 分钟级隧道场景无感差异。
pub(crate) const OFFLINE_PROBE_SCHEDULE_SECS: &[u64] = &[5, 5, 5, 5, 10, 15, 30];
/// 按脱网轮次取探针间隔(越界取表尾封顶值)。
pub(crate) fn offline_probe_wait(poll_count: usize) -> Duration {
let idx = poll_count.saturating_sub(1).min(OFFLINE_PROBE_SCHEDULE_SECS.len() - 1);
Duration::from_secs(OFFLINE_PROBE_SCHEDULE_SECS[idx])
}
/// IP 直连探针目标(公共 DNS 服务商的 443, 不依赖 DNS)。
const PROBE_IPS: &[&str] = &["223.5.5.5:443", "119.29.29.29:443"];
/// DNS 面探针目标(公共域, 走系统 resolver)。
const PROBE_DNS_HOST: &str = "www.aliyun.com:443";
async fn tcp_alive(addr: &str) -> bool {
let target = match addr.to_socket_addrs() {
Ok(mut it) => match it.next() {
Some(a) => a,
None => return false,
},
Err(_) => return false,
};
tokio::time::timeout(PROBE_TIMEOUT, tokio::net::TcpStream::connect(target))
.await
.is_ok()
}
/// 本机网络基准面是否可用。任一探针通过即在线。
pub(crate) async fn baseline_alive() -> bool {
for ip in PROBE_IPS {
if tcp_alive(ip).await {
return true;
}
}
matches!(
tokio::time::timeout(PROBE_TIMEOUT, tokio::net::lookup_host(PROBE_DNS_HOST)).await,
Ok(Ok(_))
)
}
+207 -40
View File
@@ -118,10 +118,12 @@ impl OpenAICompatProvider {
// 多模态 content(须在 move m.tool_calls 之前算,借用 m)。 // 多模态 content(须在 move m.tool_calls 之前算,借用 m)。
// 含图消息走 content 数组(text/image_url);纯文本走字符串简写 // 含图消息走 content 数组(text/image_url);纯文本走字符串简写
// (保持与现有纯文本端点零回归)。image_url 支持 data URI(base64)与 http(s) URL。 // (保持与现有纯文本端点零回归)。image_url 支持 data URI(base64)与 http(s) URL。
// 本地 file:// 引用(#958)先展开为 base64 内嵌——OpenAI 端点不认本地路径
let content = if m.has_image() { let content = if m.has_image() {
let parts: Vec<serde_json::Value> = m let parts: Vec<serde_json::Value> = m
.flattened_parts() .flattened_parts()
.into_iter() .into_iter()
.map(crate::image_ref::inline_local_image_part)
.map(|p| match p { .map(|p| match p {
crate::provider::ContentPart::Text { text } => serde_json::json!({ crate::provider::ContentPart::Text { text } => serde_json::json!({
"type": "text", "type": "text",
@@ -211,6 +213,10 @@ impl OpenAICompatProvider {
// 模型不支持 temperature(如商汤 sensenova 仅接受 1.0)→ 省略该字段,避免上游 HTTP 400。 // 模型不支持 temperature(如商汤 sensenova 仅接受 1.0)→ 省略该字段,避免上游 HTTP 400。
// 判定走 model_probe(命中 presets 表;未命中默认支持,保持原样透传零回归)。 // 判定走 model_probe(命中 presets 表;未命中默认支持,保持原样透传零回归)。
// 商汤网关(token.sensenova.cn)下温度支持**按模型各异**:sensenova-6.x 与
// sn2-kimi 实测仅接受 1(传其它值 400),部分新模型支持常规温度。
// 故不做 base_url 一刀切,改由 presets 逐模型精确维护 temperature_supported:
// 受限模型(含实测 sn2-kimi)标 false 省略;支持温度变化的模型标 true 正常透传。
let temperature = if req.temperature.is_some() let temperature = if req.temperature.is_some()
&& !crate::model_probe::probe(&model).temperature_supported && !crate::model_probe::probe(&model).temperature_supported
{ {
@@ -368,18 +374,34 @@ fn extract_tool_call_ids(msg: &OpenAiMessage) -> Vec<String> {
fn sanitize_openai_triplets(messages: &mut Vec<OpenAiMessage>) { fn sanitize_openai_triplets(messages: &mut Vec<OpenAiMessage>) {
use std::collections::HashSet; use std::collections::HashSet;
// step 1:resolved_ids = 所有 tool 消息提供的 tool_call_id(有 result 响应的 id)。 // step 1:相邻性判定的依据 = 每个 assistant 头「紧后 tool 连续段」覆盖的 id 集合。
let resolved_ids: HashSet<String> = messages // 根因(2026-10-09):原实现按「id 全局存在」判闭合(位置无关),而 OpenAI 协议要求
.iter() // tool 响应必须**紧随**其 assistant(tool_calls)。后台压缩摘要在 head 已 push、tool
.filter(|m| m.role == "tool") // 结果未 push 的窗口内可楔入 system(context_lifecycle),被隔开的响应仍被判"已闭合"
.filter_map(|m| m.tool_call_id.clone()) // → 畸形序列放行 → 上游 400「insufficient tool messages」。故判定口径由「id 全局存在」
.collect(); // 收紧为「紧后连续段覆盖」:未覆盖的 call 视同未闭合剥除,其游离响应随后由 step3 丢弃。
let mut adjacency: std::collections::HashMap<usize, HashSet<String>> =
std::collections::HashMap::new();
for i in 0..messages.len() {
if messages[i].role != "assistant" {
continue;
}
let mut seg: HashSet<String> = HashSet::new();
let mut j = i + 1;
while j < messages.len() && messages[j].role == "tool" {
if let Some(id) = messages[j].tool_call_id.as_deref() {
seg.insert(id.to_string());
}
j += 1;
}
adjacency.insert(i, seg);
}
let mut stripped_heads = 0u32; let mut stripped_heads = 0u32;
let mut total_stripped = 0u32; let mut total_stripped = 0u32;
// step 2:assistant 剥未闭合 tool_call(无对应 tool result 响应)。 // step 2:assistant 剥未闭合 tool_call(其响应不在紧后连续段内)。
for m in messages.iter_mut() { for (i, m) in messages.iter_mut().enumerate() {
if m.role != "assistant" { if m.role != "assistant" {
continue; continue;
} }
@@ -389,12 +411,13 @@ fn sanitize_openai_triplets(messages: &mut Vec<OpenAiMessage>) {
if calls.is_empty() { if calls.is_empty() {
continue; continue;
} }
let seg = adjacency.get(&i);
let kept: Vec<serde_json::Value> = calls let kept: Vec<serde_json::Value> = calls
.iter() .iter()
.filter(|tc| { .filter(|tc| {
tc.get("id") tc.get("id")
.and_then(|v| v.as_str()) .and_then(|v| v.as_str())
.is_some_and(|id| resolved_ids.contains(id)) .is_some_and(|id| seg.is_some_and(|s| s.contains(id)))
}) })
.cloned() .cloned()
.collect(); .collect();
@@ -593,37 +616,83 @@ impl LlmProvider for OpenAICompatProvider {
// send 阶段需 timeout 防 hang(同 Anthropic 路径)。 // send 阶段需 timeout 防 hang(同 Anthropic 路径)。
// 不能用 reqwest .timeout()(会砍流式 body),改用 tokio::time::timeout 包裹 send。 // 不能用 reqwest .timeout()(会砍流式 body),改用 tokio::time::timeout 包裹 send。
let send_future = self // send 阶段弱网退避: connect 类失败重试零副作用; 网络感知分速(net_probe 基准面):
.client // 本机断网长等待按自适应间隔探测恢复, 单点故障指数短退避 1s→8s, 10 分钟窗口耗尽才放弃。
.post(self.chat_url()) // 与 anthropic_compat 同构。
.header("Authorization", format!("Bearer {}", self.api_key)) let resp = {
.header("Content-Type", "application/json") let start = std::time::Instant::now();
.json(&openai_req) let mut offline_logged = false;
.send(); let mut offline_polls = 0usize;
let resp = match tokio::time::timeout(Duration::from_secs(60), send_future).await { let mut attempt = 0usize;
Ok(Ok(r)) => r, loop {
Ok(Err(e)) => { attempt += 1;
// 谓词结构化分类:Display 串不含 source 关键词("error sending request for url"), let send_future = self
// 纯串分类必 unknown → Fatal 零重试(陈旧连接首包 RST 就此裸奔)。类词置串首, .client
// source 链兜底真实传输词(reset/broken pipe 等),下游按类词走可重试。 .post(self.chat_url())
let class = crate::send_error_class(crate::SendErrPreds::of(&e)); .header("Authorization", format!("Bearer {}", self.api_key))
tracing::error!( .header("Content-Type", "application/json")
error = %e, .json(&openai_req)
class = class, .send();
url = %self.chat_url(), let outcome = tokio::time::timeout(Duration::from_secs(60), send_future).await;
source = ?std::error::Error::source(&e), match outcome {
"OpenAI 流式 send 失败" Ok(Ok(r)) => break r,
); Ok(Err(e)) => {
anyhow::bail!( // 谓词结构化分类:Display 串不含 source 关键词("error sending request for url"),
"流式 send 失败({}): {} [src={:?}]", // 纯串分类必 unknown → Fatal 零重试(陈旧连接首包 RST 就此裸奔)。类词置串首,
class, // source 链兜底真实传输词(reset/broken pipe 等),下游按类词走可重试。
e, let preds = crate::SendErrPreds::of(&e);
std::error::Error::source(&e) let retryable = preds.is_connect || (preds.is_request && !preds.is_builder);
); if retryable && start.elapsed() < crate::net_probe::SEND_RETRY_WINDOW {
} // 本机断网: 等基准面恢复, 自适应间隔探测(首段 5s 快探, 确认脱网后 30s 封顶省电)
Err(_elapsed) => { if !crate::net_probe::baseline_alive().await {
tracing::error!(url = %self.chat_url(), "OpenAI 流式 send 超时(60s 未返回响应头)"); offline_polls += 1;
anyhow::bail!("流式请求超时(60秒未收到 HTTP 响应,可能服务不可达或被防火墙拦截)"); let wait = crate::net_probe::offline_probe_wait(offline_polls);
if !offline_logged {
warn!(
url = %self.chat_url(),
window_secs = crate::net_probe::SEND_RETRY_WINDOW.as_secs(),
"本机网络不可达(疑似隧道/电梯弱网), 按自适应间隔探测恢复, 恢复后立即重试"
);
offline_logged = true;
}
tokio::time::sleep(wait).await;
continue;
}
offline_polls = 0;
offline_logged = false;
let delay = Duration::from_millis(
1000u64.saturating_mul(1 << (attempt - 1).min(3)).min(8000),
);
warn!(
url = %self.chat_url(),
attempt,
delay_ms = delay.as_millis() as u64,
error = %e,
"流式 send 连接失败, 退避重试"
);
tokio::time::sleep(delay).await;
continue;
}
let class = crate::send_error_class(preds);
tracing::error!(
error = %e,
class = class,
url = %self.chat_url(),
source = ?std::error::Error::source(&e),
"OpenAI 流式 send 失败"
);
anyhow::bail!(
"流式 send 失败({}): {} [src={:?}]",
class,
e,
std::error::Error::source(&e)
);
}
Err(_elapsed) => {
tracing::error!(url = %self.chat_url(), "OpenAI 流式 send 超时(60s 未返回响应头)");
anyhow::bail!("流式请求超时(60秒未收到 HTTP 响应,可能服务不可达或被防火墙拦截)");
}
}
} }
}; };
@@ -953,6 +1022,57 @@ mod tests {
assert_eq!(arr[2]["image_url"]["url"], "https://x/a.png"); assert_eq!(arr[2]["image_url"]["url"], "https://x/a.png");
} }
/// 本地 file:// 引用(#958)在转换层展开为 data URI;引用缺失降级 Text 片不炸请求
#[test]
fn openai_convert_expands_local_file_ref() {
use base64::Engine as _;
let dir = std::env::temp_dir().join("df-ai-openai-ref-test");
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join("ref.png");
std::fs::write(&path, b"\x89PNG ok").unwrap();
let provider = OpenAICompatProvider::new("https://api.openai.com", "k", "gpt-4o");
let req = CompletionRequest {
model: "gpt-4o".into(),
messages: vec![
ChatMessage::user_parts(
"看图",
vec![crate::provider::ContentPart::Image {
url: Some(format!("file://{}", path.display())),
base64: None,
media_type: Some("image/png".into()),
alt: None,
}],
),
ChatMessage::user_parts(
"坏引用",
vec![crate::provider::ContentPart::Image {
url: Some("file:///nonexistent/ref-missing.png".into()),
base64: None,
media_type: Some("image/png".into()),
alt: None,
}],
),
],
temperature: None,
max_tokens: None,
stream: false,
tools: None,
tool_choice: None,
reasoning_content: None,
};
let out = provider.convert_request(req);
let good = &out.messages[0];
let arr = good.content.as_array().expect("含图 → 数组");
assert_eq!(arr[1]["image_url"]["url"], format!("data:image/png;base64,{}", base64::engine::general_purpose::STANDARD.encode(b"\x89PNG ok")));
let bad = &out.messages[1];
let arr = bad.content.as_array().expect("降级片仍在数组");
assert_eq!(arr[1]["type"], "text");
assert!(arr[1]["text"].as_str().unwrap().contains("[图片文件缺失"));
std::fs::remove_file(&path).ok();
}
/// 纯文本消息 → content 仍是字符串简写(无图不数组化,对齐纯文本端点兼容) /// 纯文本消息 → content 仍是字符串简写(无图不数组化,对齐纯文本端点兼容)
#[test] #[test]
fn openai_convert_text_only_remains_string() { fn openai_convert_text_only_remains_string() {
@@ -972,6 +1092,53 @@ mod tests {
assert_eq!(msg.content, serde_json::Value::String("hello".into())); assert_eq!(msg.content, serde_json::Value::String("hello".into()));
} }
/// 温度支持按模型精确判定(presets 表,probe 命中),不按 base_url 一刀切——
/// 商汤网关下温度支持各异:sn2-kimi 实测仅接受 temperature=1(传 0.7 → 上游 400),
/// presets 已标 temperature_supported=false,任何 provider 下都省略。
/// 回归 B5 挂机 sn2-kimi 传 0.7 → 400。省略交上游默认,宁默认不 400。
#[test]
fn openai_temperature_omitted_for_sn2_kimi_via_presets() {
let provider =
OpenAICompatProvider::new("https://token.sensenova.cn", "k", "sensenova-6.8-flash-lite");
let req = CompletionRequest {
model: "sn2-kimi".into(), // presets 精确标记受限,probe 命中 false
messages: vec![ChatMessage::user("hi")],
temperature: Some(0.7),
max_tokens: None,
stream: true,
tools: None,
tool_choice: None,
reasoning_content: None,
};
let out = provider.convert_request(req);
assert_eq!(out.temperature, None, "sn2-kimi presets 标 false,应省略 temperature");
}
/// 反向保证:支持常规温度的模型(presets 未标 false,如 deepseek-v4-flash)
/// 即使挂在商汤 provider 池也按模型判定透传,不因 base_url 误杀——商汤新模型
/// 支持温度变化,一刀切会错误降级。
#[test]
fn openai_temperature_passed_for_supported_model_on_sensenova_base() {
let provider =
OpenAICompatProvider::new("https://token.sensenova.cn", "k", "sensenova-6.8-flash-lite");
let req = CompletionRequest {
model: "deepseek-v4-flash".into(), // presets 未标受限 → 默认支持温度
messages: vec![ChatMessage::user("hi")],
temperature: Some(0.7),
max_tokens: None,
stream: true,
tools: None,
tool_choice: None,
reasoning_content: None,
};
let out = provider.convert_request(req);
assert_eq!(
out.temperature,
Some(0.7),
"支持温度的模型应透传,不因 base_url 是商汤而省略"
);
}
// ---------- ensure_leading_user(首条非 user/system → 补 user 占位,OpenAI 对称 Anthropic)---------- // ---------- ensure_leading_user(首条非 user/system → 补 user 占位,OpenAI 对称 Anthropic)----------
/// 首条 assistant → 补 user 占位(对齐 Anthropic)。上游绕过 sanitize 的 /// 首条 assistant → 补 user 占位(对齐 Anthropic)。上游绕过 sanitize 的
+117
View File
@@ -0,0 +1,117 @@
//! 上下文侧工具结果正文外置(P1 工具结果瘦身)。
//!
//! 依据 docs/02-架构设计/上下文根治设计-20261006.md §二「工具结果瘦身(治最大头,先行)」:
//! 超阈值工具结果**正文落盘**,上下文只留「摘要 + 句柄路径」;句柄可被 agent 主动展开
//! (read_file 读句柄路径)——需要时找得回,不需要时不占位。与 src-tauri 审计侧 D2a 同构。
//!
//! 与 namespace_store 的区别(两者互补,不冲突):
//! - namespace_store:内存态引用化(省进程内存),发 LLM 前展开为全文 → token 零收益。
//! - 本模块:落盘 + 句柄,被省略的中段**不进 prompt 但可读回** → 真正减 token 且不丢信息。
//!
//! 降级契约:base 未注入(单测/未初始化)→ 不落盘返回 None,调用方退化为纯摘要(改造前行为)。
//! 幂等:文件名取内容稳定哈希,同内容重复压缩只落盘一次。
//! 保留策略:与全局「历史数据默认保留」一致,本模块自产文件只增不减(无清理调用点)。
//!
//! 测试隔离:核心逻辑 `offload_to(base, ...)` 显式传 base(零全局态),单测只测它;
//! `init_base` 的 OnceLock 全局态一次性不可重置,故测试**绝不调用**它(避免污染同进程
//! 其他用例——如 extract_key_info 系列断言会被意外注入的句柄行破坏)。
use std::path::{Path, PathBuf};
use std::sync::OnceLock;
/// 外置根目录(进程级一次注入;src-tauri lib.rs setup 调 init_base)。
/// 与审计侧共用 {app_data},文件名前缀 `ctx-` 区分(tool-results/ 目录共存)。
static BASE: OnceLock<PathBuf> = OnceLock::new();
/// 注入外置根目录(app data dir)。幂等,首次生效。
pub fn init_base(base: PathBuf) {
let _ = BASE.set(base);
}
/// base 是否已注入(降级判定用)。
pub fn is_enabled() -> bool {
BASE.get().is_some()
}
/// 内容稳定哈希(非密码学用途;碰撞后果仅同名文件互相覆盖,概率可忽略)。
fn content_id(content: &str) -> u64 {
use std::collections::hash_map::DefaultHasher;
use std::hash::{Hash, Hasher};
let mut h = DefaultHasher::new();
content.hash(&mut h);
h.finish()
}
/// 落盘工具结果正文,返回可读句柄行(供摘要末尾追加)。base 未注入 → None。
pub fn offload_with_handle(content: &str, tool_name: &str) -> Option<String> {
offload_to(BASE.get()?, content, tool_name)
}
/// 核心实现(显式 base,零全局态,可测)。
///
/// 建目录失败 / 写文件失败 → None(降级:调用方仅用摘要,不丢数据)。
pub fn offload_to(base: &Path, content: &str, tool_name: &str) -> Option<String> {
let dir = base.join("tool-results");
if std::fs::create_dir_all(&dir).is_err() {
return None;
}
let file = dir.join(format!("ctx-{:016x}.txt", content_id(content)));
if !file.exists() && std::fs::write(&file, content).is_err() {
return None;
}
Some(format!(
"[工具 {} 原文 {} 字符已外置: {} —— 需完整内容用 read_file 读该路径]",
tool_name,
content.chars().count(),
file.display()
))
}
/// 读回外置文件(查询侧/测试用)。文件缺失返回带语义错误(降级不 panic)。
pub fn read_offloaded(path: &str) -> Result<String, String> {
std::fs::read_to_string(path).map_err(|e| format!("外置结果读取失败: {e}"))
}
#[cfg(test)]
mod tests {
use super::*;
fn tmp_base(tag: &str) -> PathBuf {
std::env::temp_dir().join(format!("df-ai-offload-{tag}-{}", std::process::id()))
}
#[test]
fn offload_to_writes_readable_file_and_is_idempotent() {
let base = tmp_base("ok");
let _ = std::fs::remove_dir_all(&base);
let payload = "large tool output ".repeat(1000);
let handle = offload_to(&base, &payload, "grep").expect("显式 base 应落盘");
assert!(handle.contains("已外置"), "句柄应含提示文案: {handle}");
assert!(handle.contains("read_file"), "句柄应指引读回方式: {handle}");
// 幂等:同内容二次调用返回同一文件名
let handle2 = offload_to(&base, &payload, "grep").expect("二次调用仍返回句柄");
assert_eq!(handle, handle2, "同内容应幂等复用同一文件");
// 落盘文件可读回原文(无损)
let path = std::fs::read_dir(base.join("tool-results"))
.expect("目录应存在")
.flatten()
.map(|e| e.path())
.find(|p| p.extension().map(|e| e == "txt").unwrap_or(false))
.expect("应有落盘文件");
assert_eq!(
read_offloaded(path.to_string_lossy().as_ref()).expect("应可读回"),
payload
);
let _ = std::fs::remove_dir_all(&base);
}
#[test]
fn different_content_gets_different_file() {
let base = tmp_base("diff");
let _ = std::fs::remove_dir_all(&base);
let h1 = offload_to(&base, &"a".repeat(3000), "grep").unwrap();
let h2 = offload_to(&base, &"b".repeat(3000), "grep").unwrap();
assert_ne!(h1, h2, "不同内容应落不同文件");
let _ = std::fs::remove_dir_all(&base);
}
}
+85
View File
@@ -0,0 +1,85 @@
//! P1 工具结果瘦身 — 端到端集成验证。
//!
//! 独立测试二进制(独立进程):`tool_offload::init_base` 的 OnceLock 全局态不外泄,
//! 不污染 lib 单测(反之亦然)。这是验证「超阈值 → 落盘句柄 → 可无损读回」全链的唯一安全方式。
use df_ai::context_helpers::{extract_key_info, READ_FILE_OFFLOAD_BYTES};
use df_ai::tool_offload;
fn tmp_base(tag: &str) -> std::path::PathBuf {
std::env::temp_dir().join(format!("df-ai-ctx-offload-{tag}-{}", std::process::id()))
}
/// 从句柄行里粗取落盘路径(格式:`[...已外置: <path> —— ...]`)。
fn path_of(handle_line: &str) -> String {
handle_line
.split("已外置: ")
.nth(1)
.and_then(|s| s.split(" ——").next())
.expect("句柄行应含路径")
.trim()
.to_string()
}
#[test]
fn large_read_file_summarized_with_lossless_handle() {
let base = tmp_base("large");
let _ = std::fs::remove_dir_all(&base);
tool_offload::init_base(base.clone());
// 造一个超阈值的 read_file 结果(代码风格多行文本)
let body: String = (1..=800)
.map(|i| format!("line {i}: fn sample_{i}() {{ let x = {i}; }}\n"))
.collect();
assert!(body.len() > READ_FILE_OFFLOAD_BYTES, "夹具须超阈值");
let out = extract_key_info(&body, "read_file");
assert!(out.len() < body.len(), "超阈值 read_file 应被压缩");
assert!(out.contains("已外置"), "应追加落盘句柄:\n{out}");
assert!(out.contains("read_file 读该路径"), "句柄应指引读回方式");
// 无损性:按句柄路径读回 = 原文逐字节一致
let line = out.lines().find(|l| l.contains("已外置")).expect("应有句柄行");
let back = tool_offload::read_offloaded(&path_of(line)).expect("句柄路径应可读回");
assert_eq!(back, body, "落盘原文须与输入逐字节一致(无损)");
// 摘要确实变小(瘦身生效)
let ratio = out.len() as f64 / body.len() as f64;
assert!(ratio < 0.2, "压缩比应显著(<0.2),实际 {ratio:.3}");
let _ = std::fs::remove_dir_all(&base);
}
#[test]
fn small_read_file_still_exempt() {
let base = tmp_base("small");
tool_offload::init_base(base);
let small = "line1: fn a() {}\nline2: fn b() {}\n";
assert_eq!(
extract_key_info(small, "read_file"),
small,
"阈值内 read_file 应原样返回(保 AI 定向读代码语义)"
);
}
#[test]
fn non_read_file_large_result_also_gets_handle() {
let base = tmp_base("other");
let _ = std::fs::remove_dir_all(&base);
tool_offload::init_base(base.clone());
// 中性行夹具:不得含 error/panic/.rs: 等错误行信号——错误行保留策略会全量保留
// 命中行,用信号行做夹具会让「应被压缩」断言先天不成立(测的是普通大结果外置)
let body: String = (1..=1200).map(|i| format!("match {i}: src/foo.txt:{i}\n")).collect();
assert!(body.len() >= 8 * 1024, "夹具须 ≥8KB(句柄追加下限)");
let out = extract_key_info(&body, "grep");
assert!(out.len() < body.len(), "应被压缩");
assert!(out.contains("已外置"), "≥8KB 结果应带落盘句柄");
let line = out.lines().find(|l| l.contains("已外置")).expect("应有句柄行");
assert_eq!(
tool_offload::read_offloaded(&path_of(line)).expect("应可读回"),
body
);
let _ = std::fs::remove_dir_all(&base);
}
+191
View File
@@ -0,0 +1,191 @@
//! 提交冻结机械闸(39 治理) — AI 引擎派发命令的 git commit 拦截。
//!
//! 冻结期间(用户 🔒 令)禁止引擎/挂机会话执行 git commit:命令在 spawn 前被拒,
//! 改动留在工作区。用户本人终端不受影响(本闸只作用于引擎 execute 入口)。
//!
//! 冻结标志解析(优先级:环境变量 DEVFLOW_COMMIT_FROZEN > env 内 DevFlow 标志):
//! - env::var("DEVFLOW_COMMIT_FROZEN")="1"/"true" → 冻结
//! - ShellRequest.env 同名键同样生效(测试/调用方定向注入)
//! 默认不冻结(用户本人正常提交流程零影响)。
//!
//! 拦截范围:命令子串含 `git commit`(含 `git -C x commit`)即拒;`git commit --amend`
//! 同拒。`git log/diff/status/add/push` 不拦(只冻结提交动作)。
//!
//! 提示文案给 LLM 指明出路:改留工作区,等「攒批提交通知」。
use std::sync::atomic::{AtomicBool, Ordering};
use crate::shell::ShellRequest;
/// 进程级冻结标志(引擎侧写入:unattended 轮询每轮按看板 🔒 行 + KV 覆盖同步)。
/// 用静态位而非 std::env::set_var(Rust 2024 edition set_var unsafe 且多线程 UB)。
static FROZEN: AtomicBool = AtomicBool::new(false);
/// 引擎侧设置/解除冻结(unattended 轮询每轮调用)。
pub fn set_frozen(v: bool) {
FROZEN.store(v, Ordering::SeqCst);
}
/// 环境变量真值判定:"1"/"true"(含大小写变体)为冻结,其余("0"/"false"/空)为不冻结。
fn env_val_is_truthy(v: &str) -> bool {
matches!(v, "1" | "true" | "TRUE" | "True")
}
/// 是否启用冻结:静态标志(引擎同步) 或 环境变量 DEVFLOW_COMMIT_FROZEN(调用方/测试定向注入)。
pub fn is_frozen(extra_env: Option<&std::collections::HashMap<String, String>>) -> bool {
if FROZEN.load(Ordering::SeqCst) {
return true;
}
if let Some(v) = extra_env.and_then(|m| m.get("DEVFLOW_COMMIT_FROZEN")) {
return env_val_is_truthy(v);
}
std::env::var("DEVFLOW_COMMIT_FROZEN")
.map(|v| env_val_is_truthy(&v))
.unwrap_or(false)
}
/// 命令是否为 git commit 提交动作(token 判据,含 -C 前缀形态与复合命令)。
pub fn is_commit_command(command: &str) -> bool {
// 按常见命令分隔符切段( ; && || | 换行),每段独立判 git commit(防 "x; git commit" 绕过)
command
.split(|c: char| matches!(c, ';' | '|' | '&' | '\n'))
.any(is_commit_segment)
}
/// 单段判据:token 找 "git",跳过全局选项(-C path 等)后是否接 "commit"。
fn is_commit_segment(segment: &str) -> bool {
let tokens: Vec<String> = segment
.split_whitespace()
.map(|t| t.trim_matches(|c: char| c == '"' || c == '\'').to_lowercase())
.collect();
for (i, t) in tokens.iter().enumerate() {
if t == "git" {
let mut j = i + 1;
while j < tokens.len() {
match tokens[j].as_str() {
// `-c key=val` 两 token 形态:值在下一 token,跳 2
"-c" => j += 2,
// `--git-dir=path`/`--work-tree=path` 单 token 合并形态:自含值,跳 1
t if t.starts_with("--git-dir=") || t.starts_with("--work-tree=") => j += 1,
// `--git-dir path`/`--work-tree path` 两 token 形态:跳 2
"--git-dir" | "--work-tree" => j += 2,
"commit" => return true,
_ => break,
}
}
}
}
false
}
/// 冻结闸检查入口:冻结在场且命令是 git commit → 返回拒绝 Err(不 spawn)。
pub fn check(request: &ShellRequest) -> Result<(), String> {
if !is_frozen(Some(&request.env)) {
return Ok(());
}
if is_commit_command(&request.command) {
return Err(format!(
"[提交冻结令] 检测到 git commit 命令,当前处于提交冻结期(用户 🔒 令),已拒绝执行: \
改动保留在工作区,等用户「攒批提交通知」后才可提交。命令: {}",
truncate_cmd(&request.command)
));
}
Ok(())
}
fn truncate_cmd(s: &str) -> String {
if s.chars().count() <= 120 {
s.to_string()
} else {
let cut: String = s.chars().take(120).collect();
format!("{}…", cut)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
fn req(cmd: &str) -> ShellRequest {
ShellRequest {
command: cmd.to_string(),
working_dir: None,
env: HashMap::new(),
timeout_secs: None,
shell_type: None,
}
}
fn req_env(cmd: &str, env: &[(&str, &str)]) -> ShellRequest {
let mut r = req(cmd);
r.env = env
.iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect();
r
}
#[test]
fn commit_detection_covers_variants() {
assert!(is_commit_command("git commit -m 'x'"));
assert!(is_commit_command("git -C E:/repo commit -m x"));
assert!(is_commit_command("git commit --amend"));
assert!(is_commit_command("git commit -m y"));
assert!(is_commit_command("git commit; git push"));
// PowerShell / sh 包裹形态
assert!(is_commit_command("powershell -Command git commit -m x"));
assert!(is_commit_command("cd repo && git commit -m x"));
assert!(is_commit_command("git add . && git commit -m x"));
}
#[test]
fn non_commit_git_not_blocked() {
assert!(!is_commit_command("git log -1"));
assert!(!is_commit_command("git status"));
assert!(!is_commit_command("git diff --stat"));
assert!(!is_commit_command("git add ."));
assert!(!is_commit_command("git push origin main"));
assert!(!is_commit_command("cargo test"));
// commitment 等误命中防护:commit 后无后续 token 才算
assert!(!is_commit_command("git log --grep commit"));
}
#[test]
fn freeze_gate_blocks_commit_when_frozen() {
let r = req_env("git commit -m '细项批'", &[("DEVFLOW_COMMIT_FROZEN", "1")]);
assert!(check(&r).is_err(), "冻结在场 commit 必须被拒");
let r2 = req_env("git log -1", &[("DEVFLOW_COMMIT_FROZEN", "1")]);
assert!(check(&r2).is_ok(), "冻结只拦 commit 不拦只读命令");
}
/// 解冻态放行用例:持与 static_flag_toggle 同一把串行闸——静态位是进程级单源,
/// 置位瞬间并行跑到本用例会误判冻结(实测互踩),凡依赖「静态位=false」断言的用例一律持闸。
#[test]
fn no_freeze_no_block() {
let _gate = crate::shell::PWSH_CACHE_TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let r = req("git commit -m '用户本人提交'");
assert!(check(&r).is_ok(), "无冻结标志零影响(用户正常提交不受影响)");
}
#[test]
fn frozen_false_value_no_block() {
let _gate = crate::shell::PWSH_CACHE_TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let r = req_env("git commit -m x", &[("DEVFLOW_COMMIT_FROZEN", "0")]);
assert!(check(&r).is_ok(), "显式关闭标志不拦(解冻态)");
}
/// 静态标志(引擎同步路径)切换:置位拦 commit,复位恢复。
/// 串行闸:静态位是进程级单源,置位期间并行的 env 用例(check 走 is_frozen 静态位短路)会互踩,
/// 持锁串行 + 进出恢复默认(false)防污染。
#[test]
fn static_flag_toggle() {
// 复用 PWSH_CACHE_TEST_LOCK 串行闸(进程级互斥,专用闸后续有需要再单立)
let _gate = crate::shell::PWSH_CACHE_TEST_LOCK.lock().unwrap_or_else(|e| e.into_inner());
set_frozen(true);
assert!(check(&req("git commit -m x")).is_err(), "静态冻结位拦 commit");
assert!(check(&req("git diff")).is_ok(), "静态冻结位不拦只读命令");
set_frozen(false);
assert!(check(&req("git commit -m x")).is_ok(), "复位后恢复放行");
}
}
+8 -2
View File
@@ -102,7 +102,9 @@ impl EnvSnapshot {
} }
} }
/// 兜底:spawn_blocking panic/join 失败时返回最小可用快照(全 None,字段不空)。 /// 兜底:spawn_blocking panic/join 失败或外层 5s 超时时返回最小可用快照(全 None,字段不空)。
/// 注:异常/超时均**冻结**为 fallback 写入 OnceLock(EX-3)——本进程后续 detect() 不再重探,
/// 瞬时挂起(如 Store Alias)会被固化为整进程降级快照;解冻需重启进程。
fn fallback() -> EnvSnapshot { fn fallback() -> EnvSnapshot {
EnvSnapshot { EnvSnapshot {
os: std::env::consts::OS.to_string(), os: std::env::consts::OS.to_string(),
@@ -310,7 +312,11 @@ fn extract_codepage(text: &str) -> Option<String> {
} }
} }
/// 执行 `tool --version`,解析首行返回版本串。失败/超时返回 None,不阻塞调用方。 /// 执行 `tool --version`,解析首行返回版本串。失败(非零退出/无首行)返回 None。
///
/// 注:单条命令本身**无独立超时**(EX-4)——若 `tool --version` 挂起(如 Windows Store Alias),
/// 本函数会一直阻塞,由外层 [`EnvSnapshot::detect`] 的 5s spawn_blocking 超时兜底整批(届时
/// 全部工具版本一起回退 fallback),而非逐命令超时。「不阻塞调用方」靠的是外层超时,非本函数。
/// ///
/// 例:python --version 输出 "Python 3.11.5" → 返回 "3.11.5";git --version 输出 /// 例:python --version 输出 "Python 3.11.5" → 返回 "3.11.5";git --version 输出
/// "git version 2.41.0" → 返回 "2.41.0"。统一抽掉工具名前缀,只保留版本号本身。 /// "git version 2.41.0" → 返回 "2.41.0"。统一抽掉工具名前缀,只保留版本号本身。
+3 -1
View File
@@ -1,6 +1,8 @@
//! df-execute: 执行运行时 — Shell + 环境感知 //! df-execute: 执行运行时 — Shell + 环境感知 + 提交冻结闸
pub mod commit_freeze;
pub mod env_snapshot; pub mod env_snapshot;
pub mod shell; pub mod shell;
pub use commit_freeze::is_commit_command;
pub use env_snapshot::EnvSnapshot; pub use env_snapshot::EnvSnapshot;
+115 -11
View File
@@ -169,10 +169,95 @@ pub fn current_shell() -> ShellType {
ShellType::default() ShellType::default()
} }
/// 剥离 ANSI 转义序列(#8 出口兜底)。
///
/// 源头抑制(build_command 注入 NO_COLOR/TERM=dumb)覆盖主流工具链,但总有漏网
/// (不读环境变量的工具/嵌套脚本转发色码)。本纯函数兜底 CSI/OSC 两类常见序列:
/// - CSI:ESC [ 参数字母(颜色/光标/擦除,如 \x1b[7m \x1b[0K)
/// - OSC:ESC ] ... BEL 或 ESC \(终端标题/超链接,罕见但同样乱码)
/// 不碰普通文本与换行,幂等(已剥离文本再剥无变化)。
pub fn strip_ansi(text: &str) -> String {
let mut out = String::with_capacity(text.len());
let mut chars = text.char_indices().peekable();
while let Some((i, c)) = chars.next() {
if c != '\x1b' {
out.push(c);
continue;
}
// c == ESC:按下一字符分派序列类型
match text[i + 1..].chars().next() {
Some('[') => {
// CSI:跳到终止字母(@-~)为止
let rest = &text[i + 2..];
match rest.find(|ch: char| ('\u{40}'..='\u{7e}').contains(&ch)) {
Some(end) => {
// 跳过整个序列: advance 到 i+2+end 的下一字符
let skip = 2 + end + 1;
for _ in 1..skip {
chars.next();
}
}
None => {
// 流式截断的残缺序列(无终止符):丢弃剩余全部
for _ in (i + 1)..text.len() {
chars.next();
}
}
}
}
Some(']') => {
// OSC:到 BEL(\x07) 或 ST(ESC \)为止
let rest = &text[i + 2..];
let bel = rest.find('\x07');
let st = rest.find("\x1b\\");
match (bel, st) {
(Some(b), None) => {
// 已消费 ESC,再消费 i+1..=i+2+b(含 BEL)共 b+2 个
for _ in 0..(2 + b) {
chars.next();
}
}
(_, Some(s)) => {
// 再消费 i+1..=i+2+s+1(含 ST 两字符)共 s+3 个
for _ in 0..(2 + s + 1) {
chars.next();
}
}
(None, None) => {
for _ in (i + 1)..text.len() {
chars.next();
}
}
}
}
_ => {
// 两字符转义(ESC \ ESC ( 等):丢 ESC 保留后字符?保守双丢后单字符
chars.next();
}
}
}
out
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
/// #8 ANSI 剥离:CSI 色码/擦除/流式残缺序列/OSC/普通文本不动
#[test]
fn strip_ansi_basic() {
assert_eq!(strip_ansi("\x1b[7ma\x1b[0mb"), "ab");
assert_eq!(strip_ansi("\x1b[2K\x1b[1Gdone"), "done");
assert_eq!(strip_ansi("plain text\nline2"), "plain text\nline2");
// 流式残缺(无终止符):丢残序列
assert_eq!(strip_ansi("ok\x1b[3"), "ok");
// OSC BEL 终止
assert_eq!(strip_ansi("\x1b]0;title\x07body"), "body");
// 幂等
let once = strip_ansi("\x1b[32mgreen\x1b[0m");
assert_eq!(strip_ansi(&once), once);
}
/// 死缓存回归测试:probe_pwsh 写入后,Default 同步路径必须读到同一缓存值。 /// 死缓存回归测试:probe_pwsh 写入后,Default 同步路径必须读到同一缓存值。
/// ///
/// 历史 bug:probe_pwsh_cached() 与 probe_pwsh() 各持独立 OnceLock,Default 读的永空。 /// 历史 bug:probe_pwsh_cached() 与 probe_pwsh() 各持独立 OnceLock,Default 读的永空。
@@ -235,7 +320,7 @@ pub struct ShellRequest {
pub env: std::collections::HashMap<String, String>, pub env: std::collections::HashMap<String, String>,
/// 超时时间(秒),None 表示不超时 /// 超时时间(秒),None 表示不超时
pub timeout_secs: Option<u64>, pub timeout_secs: Option<u64>,
/// Shell 类型(默认:Windows→Cmd, 非Windows→Sh) /// Shell 类型(None 时取 [`ShellType::default`]:Windows→pwsh 可用则 pwsh 否则 PowerShell,非 Windows→Sh)
#[serde(default)] #[serde(default)]
pub shell_type: Option<ShellType>, pub shell_type: Option<ShellType>,
} }
@@ -312,6 +397,13 @@ fn build_command(request: ShellRequest) -> tokio::process::Command {
for (key, value) in &request.env { for (key, value) in &request.env {
cmd.env(key, value); cmd.env(key, value);
} }
// #8 ANSI 裸露根治(源头抑制):非交互管道下多数 CLI 仍探测 TERM 发色码,cargo/ls 等
// 输出 \x1b[Nm 原样透传前端显示为 ␛[7m 乱码。注入 NO_COLOR(https://no-color.org 标准)+
// TERM=dumb+FORCE_COLOR=0 三重信号,主流工具链(cargo/npm/git/ls/pytest)管道下停发色码。
// 调用方显式传同名 env 时以调用方为准(后写覆盖,见上循环顺序)。
cmd.env("NO_COLOR", "1");
cmd.env("TERM", "dumb");
cmd.env("FORCE_COLOR", "0");
cmd cmd
} }
@@ -324,7 +416,10 @@ fn build_command(request: ShellRequest) -> tokio::process::Command {
pub async fn execute(request: ShellRequest) -> anyhow::Result<ShellResult> { pub async fn execute(request: ShellRequest) -> anyhow::Result<ShellResult> {
let start = std::time::Instant::now(); let start = std::time::Instant::now();
// 探测 pwsh(惰性 + OnceLock 全局缓存,只探一次),使后续 ShellType::default() 可读取缓存 // 提交冻结机械闸(39 治理):冻结在场且命令为 git commit → spawn 前拒绝,改动留工作区
crate::commit_freeze::check(&request).map_err(anyhow::Error::msg)?;
// 探测 pwsh(惰性 + 模块级 PWSH_CACHE,成功才写缓存),使后续 ShellType::default() 可读取缓存
#[cfg(windows)] #[cfg(windows)]
let _ = probe_pwsh().await; let _ = probe_pwsh().await;
@@ -346,8 +441,8 @@ pub async fn execute(request: ShellRequest) -> anyhow::Result<ShellResult> {
let duration = start.elapsed().as_millis() as u64; let duration = start.elapsed().as_millis() as u64;
Ok(ShellResult { Ok(ShellResult {
stdout: String::from_utf8_lossy(&output.stdout).to_string(), stdout: strip_ansi(&String::from_utf8_lossy(&output.stdout)),
stderr: String::from_utf8_lossy(&output.stderr).to_string(), stderr: strip_ansi(&String::from_utf8_lossy(&output.stderr)),
exit_code: output.status.code(), exit_code: output.status.code(),
duration_ms: duration, duration_ms: duration,
}) })
@@ -358,7 +453,11 @@ pub async fn execute(request: ShellRequest) -> anyhow::Result<ShellResult> {
/// 治 run_command 执行中黑盒:execute() 等 exit 才返回整块 stdout/stderr,长命令(cargo/npm 构建) /// 治 run_command 执行中黑盒:execute() 等 exit 才返回整块 stdout/stderr,长命令(cargo/npm 构建)
/// 期间前端只看 Started→Completed,中间进度不可见。本函数 spawn 子进程后并发逐行读两条流, /// 期间前端只看 Started→Completed,中间进度不可见。本函数 spawn 子进程后并发逐行读两条流,
/// 每读到一行回调 `on_output(kind, line)`(调用方可 emit 事件给前端实时展示),仍等进程 exit /// 每读到一行回调 `on_output(kind, line)`(调用方可 emit 事件给前端实时展示),仍等进程 exit
/// 后返回完整 ShellResult(与 execute() 同形,调用方无需感知差异)。 /// 后返回完整 ShellResult。
///
/// 与 [`execute`] 的输出差异(EX-2,调用方须知):本函数 stdout/stderr 按行重组——经
/// `BufReader::lines()` 逐行拼接,行尾统一为 `\n`(剥 \r\n)且补末尾换行;execute 则保留原始字节
/// (lossy 解码,含 \r\n,无补换行)。二者 ShellResult 字段同形,仅换行形态不同;需字节级一致时用 execute。
/// ///
/// 4性: /// 4性:
/// - 合理机制:spawn + BufReader::lines() 逐行,不丢未换行结尾的末段(read_to_end 兜底) /// - 合理机制:spawn + BufReader::lines() 逐行,不丢未换行结尾的末段(read_to_end 兜底)
@@ -372,6 +471,9 @@ where
{ {
let start = std::time::Instant::now(); let start = std::time::Instant::now();
// 提交冻结机械闸(39 治理):同 execute,流式路径同样 spawn 前拦截
crate::commit_freeze::check(&request).map_err(anyhow::Error::msg)?;
#[cfg(windows)] #[cfg(windows)]
let _ = probe_pwsh().await; let _ = probe_pwsh().await;
@@ -430,21 +532,23 @@ where
let status = status?; let status = status?;
// drain 剩余行(读 task 在管道 EOF 后 send 完最后批次即退出,rx 返 None 闭合) // drain 剩余行(读 task 在管道 EOF 后 send 完最后批次即退出,rx 返 None 闭合)
while let Some((kind, line)) = rx.recv().await { while let Some((kind, line)) = rx.recv().await {
let clean = strip_ansi(&line);
match kind { match kind {
StreamKind::Stdout => { stdout_buf.push_str(&line); stdout_buf.push('\n'); } StreamKind::Stdout => { stdout_buf.push_str(&clean); stdout_buf.push('\n'); }
StreamKind::Stderr => { stderr_buf.push_str(&line); stderr_buf.push('\n'); } StreamKind::Stderr => { stderr_buf.push_str(&clean); stderr_buf.push('\n'); }
} }
on_output(kind, &line); on_output(kind, &clean);
} }
break status; break status;
} }
// 收到一行 → 累积 + 回调 // 收到一行 → 累积 + 回调
Some((kind, line)) = rx.recv() => { Some((kind, line)) = rx.recv() => {
let clean = strip_ansi(&line);
match kind { match kind {
StreamKind::Stdout => { stdout_buf.push_str(&line); stdout_buf.push('\n'); } StreamKind::Stdout => { stdout_buf.push_str(&clean); stdout_buf.push('\n'); }
StreamKind::Stderr => { stderr_buf.push_str(&line); stderr_buf.push('\n'); } StreamKind::Stderr => { stderr_buf.push_str(&clean); stderr_buf.push('\n'); }
} }
on_output(kind, &line); on_output(kind, &clean);
} }
} }
}; };
-2
View File
@@ -125,8 +125,6 @@ impl ScoringEngine {
} }
} }
/// 统计 text 中命中任一关键词的数量(小写匹配,兼顾中英文)
/// 局限:纯子串匹配,不识别"不复用""无用户增长"等否定前缀,接 LLM 后由语义层修正
/// 单字否定前缀:关键词紧邻这些字时该命中不计(「不复用」不计入「复用」正向命中)。 /// 单字否定前缀:关键词紧邻这些字时该命中不计(「不复用」不计入「复用」正向命中)。
/// ///
/// 注意「别」单字歧义大(别具/别的/特别 均非否定),故不进单字清单—— /// 注意「别」单字歧义大(别具/别的/特别 均非否定),故不进单字清单——
+29
View File
@@ -0,0 +1,29 @@
[package]
name = "df-im-gateway"
version = "0.1.0"
edition = "2021"
description = "IM 网关:飞书/企微/钉钉/Web 四端远程接入 DevFlow AI Chat(WS Commander 连 u-relay)"
[dependencies]
anyhow = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
rusqlite = { workspace = true }
tokio = { workspace = true }
async-trait = { workspace = true }
tracing = { workspace = true }
df-tunnel = { path = "../df-tunnel" }
# HTTP 静态托管 + WS 服务端(P1:Web Chat 页面由网关直接托管,/ 与 /ws 同端口)
axum = { version = "0.7", features = ["ws"] }
# WS 客户端(commander 连 u-relay;与 df-tunnel 同款本地版本声明,workspace 无这几项)
tokio-tungstenite = { version = "0.23", features = ["native-tls"] }
futures-util = "0.3"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
# 飞书 SDK(长连接 + 归一化消息;channel 自动连带 ws+messaging)
larksuite-oapi-sdk-rs = { version = "0.3.12", features = ["channel"] }
[dev-dependencies]
tempfile = "3"
@@ -0,0 +1,22 @@
[Unit]
Description=DevFlow IM Gateway (飞书/企微/钉钉/Web 四端远程接入)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
WorkingDirectory=/opt/df-im-gateway
ExecStart=/opt/df-im-gateway/df-im-gateway
Restart=on-failure
RestartSec=3
# ── 运行时 env(与本地 config.rs 对齐;token 与 u-relay 同源)──
Environment=GATEWAY_RELAY_URL=ws://127.0.0.1:9180/ws/commander
Environment=GATEWAY_RELAY_TOKEN=devflow-relay-default-token
Environment=GATEWAY_DEFAULT_CHANNEL=chan-work
Environment=GATEWAY_DATA_DIR=/opt/df-im-gateway/data
Environment=RUST_LOG=info,df_im_gateway=debug
StandardOutput=append:/opt/df-im-gateway/gateway.log
StandardError=append:/opt/df-im-gateway/gateway.err.log
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,23 @@
# df-im-gateway nginx 反代片段(与 u-relay 同机,复用 *.1216.top 通配符证书)
# 放入 /etc/nginx/conf.d/ 后 nginx -t && systemctl reload nginx
# Web Chat 页面 + WS 同端口 9191(网关 axum 直接托管)
server {
listen 443 ssl;
server_name im.1216.top;
ssl_certificate /etc/nginx/sslkey/_.1216.top.pem;
ssl_certificate_key /etc/nginx/sslkey/_.1216.top.key;
location / {
proxy_pass http://127.0.0.1:9191;
proxy_http_version 1.1;
# WS 升级头(与 u-relay 反代同款)
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
# 流式响应不缓冲(SSE/流式文本)
proxy_buffering off;
proxy_read_timeout 300s;
}
}
@@ -0,0 +1,86 @@
# df-im-gateway 运维手册
> 四端 IM 网关(飞书/企微/钉钉/Web)远程接入 DevFlow AI Chat。
> 部署拓扑:与 u-relay 同机(39.99.243.191),WS Commander 经 u-relay 连桌面端 df-tunnel。
## 一、拓扑与环境
```
飞书/企微/钉钉/Web ──► df-im-gateway(9191, 同机)
│ ws://127.0.0.1:9180/ws/commander (Commander 身份)
▼
u-relay(9180, nginx 前置 wss://u-relay.1216.top)
│ /ws/agent
▼
桌面端 DevFlow(df-tunnel Agent)
```
| 项 | 值 |
|---|---|
| 服务器 | 39.99.243.191:22 (root 免密, Ubuntu 22.04) |
| 部署路径 | /opt/df-im-gateway/(二进制 + data/ + 日志) |
| 监听端口 | 9191(本机明文,nginx 前置 im.1216.top → wss) |
| systemd | df-im-gateway.service(Restart=on-failure, 开机自启) |
| token | devflow-relay-default-token(与 u-relay/桌面端三端同源) |
| 数据库 | /opt/df-im-gateway/data/{sessions.db, audit.db}(SQLite WAL) |
## 二、日常运维
```bash
# 状态 / 日志 / 重启
systemctl status df-im-gateway
journalctl -u df-im-gateway -n 50 --no-pager
systemctl restart df-im-gateway
# 端口验证
ss -tlnp | grep 9191
# 审计查询(最近 20 条)
sqlite3 /opt/df-im-gateway/data/audit.db \
"SELECT datetime(ts/1000,'unixepoch','+8 hours'), direction, action, chat_id, outcome FROM tunnel_audit ORDER BY id DESC LIMIT 20;"
# 会话映射表
sqlite3 /opt/df-im-gateway/data/sessions.db "SELECT * FROM session_map;"
```
⚠️ systemd 托管,勿 kill Main PID、勿 nohup 起野进程;重启一律 `systemctl restart`。
## 三、发布流程
```powershell
# 本机(Windows)一键发布:单测 → musl 交叉编译 → scp → 原子切换 → 9191 验证 → 失败自动回滚
powershell -File E:\wk-lab\devflow\crates\df-im-gateway\scripts\publish-im-gateway.ps1
# 仅构建不部署: 加 -SkipDeploy;仅部署不构建: 加 -SkipBuild
```
首次部署需手动铺底:
```bash
scp crates/df-im-gateway/deploy/df-im-gateway.service root@39.99.243.191:/etc/systemd/system/
ssh root@39.99.243.191 'systemctl daemon-reload && systemctl enable df-im-gateway'
# nginx(可选,Web 端公网访问时): scp deploy/nginx-im.1216.top.conf → /etc/nginx/conf.d/ && nginx -t && systemctl reload nginx
```
## 四、回滚
```bash
cd /opt/df-im-gateway
systemctl stop df-im-gateway
LATEST=$(ls -t df-im-gateway.bak.* | head -1)
cp "$LATEST" df-im-gateway && chmod +x df-im-gateway
systemctl start df-im-gateway
```
## 五、配置变更
| env | 说明 | 改法 |
|---|---|---|
| GATEWAY_RELAY_TOKEN | 与 u-relay 三端同源,**改必须同步桌面端** | 改 unit Environment= → daemon-reload → restart |
| GATEWAY_DEFAULT_CHANNEL | 默认路由 channel | 同上 |
| GATEWAY_WHITELIST | 用户白名单(空=任何人可用,生产必设!) | 同上 |
| FEISHU_APP_ID/SECRET | 飞书长连接 | 同上 |
## 六、已知项
- token 仍为 MVP 单共享值(per-user 是 S2 计划)
- 日志 gateway.log 追加无轮转,可配 logrotate 或定期 truncate
- 四端 E2E 联调依赖各平台凭证到位(飞书/钉钉/企微)
@@ -0,0 +1,70 @@
# df-im-gateway 一键发布脚本(复用 publish-u-relay 模式)
# 用法: powershell -File scripts\publish-im-gateway.ps1 [-SkipBuild] [-SkipDeploy]
# 流程: cargo test → musl 交叉编译 → scp 上传(.new 中转) → systemd 原子切换 → 9191 验证
param(
[switch]$SkipBuild,
[switch]$SkipDeploy
)
$ErrorActionPreference = "Stop"
$Repo = "E:/wk-lab/devflow"
$Target = "x86_64-unknown-linux-musl"
$Server = "root@39.99.243.191"
$RemoteDir = "/opt/df-im-gateway"
$ServiceName = "df-im-gateway"
$LocalBin = "$Repo/target-musl/$Target/release/df-im-gateway"
function Step($msg) { Write-Host "`n=== $msg ===" -ForegroundColor Cyan }
if (-not $SkipBuild) {
Step "Step 0: 单测基线(df-im-gateway)"
cargo test -p df-im-gateway --manifest-path "$Repo/Cargo.toml"
if ($LASTEXITCODE -ne 0) { throw "单测失败,中止发布" }
Step "Step 1: musl 交叉编译(rust-lld self-contained)"
$env:CARGO_TARGET_DIR = "$Repo/target-musl"
cargo build --release -p df-im-gateway --manifest-path "$Repo/Cargo.toml" --target $Target
if ($LASTEXITCODE -ne 0) { throw "编译失败,中止发布" }
Remove-Item Env:\CARGO_TARGET_DIR -ErrorAction SilentlyContinue
if (-not (Test-Path $LocalBin)) { throw "产物不存在: $LocalBin" }
$size = [math]::Round((Get-Item $LocalBin).Length / 1MB, 1)
Write-Host "产物: $LocalBin (${size}MB)"
}
if ($SkipDeploy) { Write-Host "`nSkipDeploy: 仅构建完成"; exit 0 }
Step "Step 2: 服务器预检 + 备份"
ssh $Server "mkdir -p $RemoteDir && test -f $RemoteDir/df-im-gateway && cp $RemoteDir/df-im-gateway $RemoteDir/df-im-gateway.bak.`$(date +%Y%m%d%H%M%S) || echo '首次部署,无备份'"
Step "Step 3: 上传(.new 中转防 Text file busy)"
scp $LocalBin "${Server}:$RemoteDir/df-im-gateway.new"
if ($LASTEXITCODE -ne 0) { throw "scp 失败" }
Step "Step 4: 原子切换 + systemd 重启"
ssh $Server "bash -s" @'
set -e
cd /opt/df-im-gateway
systemctl stop df-im-gateway 2>/dev/null || true
mv -f df-im-gateway.new df-im-gateway
chmod +x df-im-gateway
systemctl start df-im-gateway
sleep 1
systemctl is-active df-im-gateway
'@
if ($LASTEXITCODE -ne 0) { throw "切换/启动失败,检查 journalctl -u df-im-gateway"; }
Step "Step 5: 健康验证"
ssh $Server "ss -tlnp | grep 9191 && journalctl -u df-im-gateway -n 5 --no-pager | sed 's/\x1b\[[0-9;]*m//g'"
if ($LASTEXITCODE -ne 0) {
Write-Host "❌ 9191 未监听,执行回滚..." -ForegroundColor Red
ssh $Server "bash -s" @'
set -e
cd /opt/df-im-gateway
systemctl stop df-im-gateway
LATEST=$(ls -t df-im-gateway.bak.* 2>/dev/null | head -1)
if [ -n "$LATEST" ]; then cp "$LATEST" df-im-gateway; chmod +x df-im-gateway; systemctl start df-im-gateway; echo "回滚完成: $LATEST"; else echo "无备份可回滚(首次部署失败,保持停止)"; fi
'@
throw "健康检查失败,已回滚"
}
Write-Host "`n✅ df-im-gateway 发布完成 (9191)" -ForegroundColor Green
+284
View File
@@ -0,0 +1,284 @@
//! 平台适配器(P0 预留项 ④:trait 分层,统一 InboundEvent/OutboundAction)
//!
//! 网关核心只认识这两个类型,四端差异(飞书卡片/企微 Markdown/钉钉卡片/Web JSON)收敛在各 adapter。
//! P0 只实现 Web,飞书/企微/钉钉留 trait 位(P1/P2/P4 各自落地,不动核心)。
use async_trait::async_trait;
use tokio::sync::mpsc;
/// 平台入站事件(IM 平台 → 网关,已归一化)
#[derive(Debug, Clone)]
pub enum InboundEvent {
/// 用户发消息
Message {
platform: String,
chat_id: String,
user_id: String,
text: String,
},
/// 会话切换(/new 或菜单)
NewSession { platform: String, chat_id: String },
/// 审批按钮回调
Approval {
platform: String,
chat_id: String,
approval_id: String,
approved: bool,
},
}
/// 平台出站动作(网关 → IM 平台,已归一化)
#[derive(Debug, Clone)]
pub enum OutboundAction {
/// 发/更新 AI 流式文本(全量内容,适配器自选卡片或文本)
StreamText { chat_id: String, text: String, done: bool },
/// 发送审批卡片(同意/拒绝按钮)
ApprovalCard { chat_id: String, approval_id: String, title: String, detail: String },
/// 发系统提示
Notice { chat_id: String, text: String },
}
/// 平台适配器
#[async_trait]
pub trait PlatformAdapter: Send + Sync {
/// 平台名(feishu/wecom/dingtalk/web,与 session_map.platform 对应)。
/// 核心循环经 platform_index 注册表间接使用。
#[allow(dead_code)]
fn name(&self) -> &str;
/// 启动适配器(长连接/回调服务器/WS),入站事件从 events 通道流出。阻塞运行。
async fn run(&self, events: mpsc::Sender<InboundEvent>) -> anyhow::Result<()>;
/// 出站动作分发(网关核心调用)
async fn dispatch(&self, action: OutboundAction) -> anyhow::Result<()>;
/// 清理平台的流式消息状态(网关在新建会话映射时调用,兜底上轮异常中止的残留)。
/// 默认空实现:无流式覆写语义的平台(Web 直接推帧)无需关心。
async fn reset_stream(&self, _chat_id: &str) {}
}
/// Web Chat 适配器(P1:axum 同端口托管静态页 + WS,浏览器直连)
///
/// - `GET /` → 内嵌单文件聊天页(include_str!,零运行时文件依赖)
/// - `GET /ws` → WS 升级,协议见 handle_web_ws(与 P0 纯 WS 版完全一致)
pub mod web {
use std::collections::HashMap;
use std::sync::Arc;
use async_trait::async_trait;
use axum::extract::ws::{Message as WsMessage, WebSocket, WebSocketUpgrade};
use axum::extract::State;
use axum::response::{Html, Response};
use axum::routing::get;
use axum::Router;
use futures_util::{SinkExt, StreamExt};
use tokio::sync::{mpsc, Mutex};
use super::{InboundEvent, OutboundAction, PlatformAdapter};
/// 内嵌聊天页(单文件,无外部依赖,构建期打进二进制)
const INDEX_HTML: &str = include_str!("../web/index.html");
/// 浏览器连接注册表(chat_id → ws 发送通道)
type WsRegistry = Arc<Mutex<HashMap<String, mpsc::Sender<WsMessage>>>>;
/// axum 路由共享状态(注册表 + 核心入站通道)
#[derive(Clone)]
struct WsState {
registry: WsRegistry,
events: mpsc::Sender<InboundEvent>,
}
/// Web 适配器:HTTP+WS 服务监听 0.0.0.0:9191
#[derive(Clone)]
pub struct WebAdapter {
registry: WsRegistry,
}
impl WebAdapter {
pub fn new() -> Self {
Self { registry: Arc::new(Mutex::new(HashMap::new())) }
}
}
impl Default for WebAdapter {
fn default() -> Self {
Self::new()
}
}
#[async_trait]
impl PlatformAdapter for WebAdapter {
fn name(&self) -> &str {
"web"
}
async fn run(&self, events: mpsc::Sender<InboundEvent>) -> anyhow::Result<()> {
let state = WsState { registry: self.registry.clone(), events };
let app = Router::new()
.route("/", get(|| async { Html(INDEX_HTML) }))
.route("/ws", get(ws_upgrade))
.with_state(state);
let listener = tokio::net::TcpListener::bind("0.0.0.0:9191").await?;
tracing::info!("[web] HTTP+WS 服务监听 http://0.0.0.0:9191 (页面 / , WS /ws)");
axum::serve(listener, app).await?;
Ok(())
}
async fn dispatch(&self, action: OutboundAction) -> anyhow::Result<()> {
match action {
OutboundAction::StreamText { chat_id, text, done } => {
let frame = serde_json::json!({
"type": if done { "stream_done" } else { "stream_delta" },
"text": text,
});
self.send_ws(&chat_id, &frame.to_string()).await
}
OutboundAction::ApprovalCard { chat_id, approval_id, title, detail } => {
let frame = serde_json::json!({
"type": "approval",
"approval_id": approval_id,
"title": title,
"detail": detail,
});
self.send_ws(&chat_id, &frame.to_string()).await
}
OutboundAction::Notice { chat_id, text } => {
let frame = serde_json::json!({ "type": "notice", "text": text });
self.send_ws(&chat_id, &frame.to_string()).await
}
}
}
}
impl WebAdapter {
async fn send_ws(&self, chat_id: &str, text: &str) -> anyhow::Result<()> {
let registry = self.registry.lock().await;
if let Some(tx) = registry.get(chat_id) {
tx.send(WsMessage::Text(text.to_string())).await?;
}
Ok(())
}
}
/// WS 升级入口
async fn ws_upgrade(State(state): State<WsState>, ws: WebSocketUpgrade) -> Response {
ws.on_upgrade(move |socket| async move {
if let Err(e) = handle_web_ws(socket, state).await {
tracing::warn!("[web] WS 会话异常结束: {e:#}");
}
})
}
/// 单个浏览器 WS 会话。
///
/// 协议(浏览器 → 网关):
/// ```json
/// 首帧绑定: {"type":"hello","chat_id":"...","user_id":"..."}
/// 消息: {"type":"message","text":"..."}
/// 重置会话: {"type":"new_session"}
/// 审批: {"type":"approval","approval_id":"...","approved":true|false}
/// ```
/// 网关 → 浏览器:stream_delta / stream_done / approval / notice(见 dispatch)。
async fn handle_web_ws(socket: WebSocket, state: WsState) -> anyhow::Result<()> {
let (mut sink, mut upstream) = socket.split();
// 首帧 hello 绑定 chat_id
let first = match upstream.next().await {
Some(Ok(m)) => m,
_ => {
tracing::warn!("[web] 浏览器连接即断(无首帧)");
return Ok(());
}
};
let hello: serde_json::Value = match &first {
WsMessage::Text(t) => serde_json::from_str(t).unwrap_or_default(),
_ => {
tracing::warn!("[web] 首帧非文本,拒绝绑定");
return Ok(());
}
};
let chat_id = hello
.get("chat_id")
.and_then(|v| v.as_str())
.unwrap_or("web-anon")
.to_string();
let user_id = hello.get("user_id").and_then(|v| v.as_str()).unwrap_or("anon").to_string();
// 注册发送通道:出站帧经 ws_tx → forward 任务写 socket
let (ws_tx, mut ws_rx) = mpsc::channel::<WsMessage>(64);
state.registry.lock().await.insert(chat_id.clone(), ws_tx);
tracing::info!("[web] 浏览器接入 chat_id={chat_id} user={user_id}");
let forward = tokio::spawn(async move {
while let Some(frame) = ws_rx.recv().await {
if sink.send(frame).await.is_err() {
break;
}
}
});
let result: anyhow::Result<()> = async {
while let Some(msg) = upstream.next().await {
let msg = msg?;
match msg {
WsMessage::Text(t) => {
let v: serde_json::Value = serde_json::from_str(&t).unwrap_or_default();
match v.get("type").and_then(|x| x.as_str()).unwrap_or("") {
"message" => {
let text =
v.get("text").and_then(|x| x.as_str()).unwrap_or("").to_string();
state
.events
.send(InboundEvent::Message {
platform: "web".into(),
chat_id: chat_id.clone(),
user_id: user_id.clone(),
text,
})
.await
.map_err(|_| anyhow::anyhow!("核心入站通道关闭"))?;
}
"new_session" => {
state
.events
.send(InboundEvent::NewSession {
platform: "web".into(),
chat_id: chat_id.clone(),
})
.await
.map_err(|_| anyhow::anyhow!("核心入站通道关闭"))?;
}
"approval" => {
let approval_id =
v.get("approval_id").and_then(|x| x.as_str()).unwrap_or("").to_string();
let approved =
v.get("approved").and_then(|x| x.as_bool()).unwrap_or(false);
state
.events
.send(InboundEvent::Approval {
platform: "web".into(),
chat_id: chat_id.clone(),
approval_id,
approved,
})
.await
.map_err(|_| anyhow::anyhow!("核心入站通道关闭"))?;
}
_ => {}
}
}
WsMessage::Close(_) => break,
_ => {}
}
}
Ok(())
}
.await;
state.registry.lock().await.remove(&chat_id);
forward.abort();
tracing::info!("[web] 浏览器断开 chat_id={chat_id}");
result
}
}
+236
View File
@@ -0,0 +1,236 @@
//! 审计日志(P5):IM 消息 ↔ TunnelCommand 全量记录(SQLite + WAL,与 SessionStore 同机库文件)
//!
//! 设计口径:
//! - **append-only**:只写不改,无 UPDATE/DELETE——审计的公信力来自不可篡改。
//! - **双向打点**:inbound(IM 用户 → 网关)与 outbound(网关/桌面端 → IM)各记一行,
//! 同一 request_id 关联请求与响应(排障时按 id 串链)。
//! - **失败不阻断**:审计写入失败仅 tracing::warn,绝不影响主链路(审计是增益非闸门)。
//! - 表:tunnel_audit(id 自增 / ts 毫秒 / direction / platform / chat_id / user_id /
//! conv_id / action / request_id / payload 截断 4KB / outcome ok|error|denied)。
//! payload 截断防长消息撑爆库;敏感凭证不经过此层(config 层已隔离)。
//!
//! 查询接口(运维用):recent 按时间倒序限量;by_request 串同请求双向链。
use std::path::Path;
use std::sync::Arc;
use rusqlite::Connection;
const MAX_PAYLOAD: usize = 4096;
/// 审计方向
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Direction {
/// IM 用户 → 网关(入站消息/审批点击/新会话)
Inbound,
/// 网关/桌面端 → IM(流式输出/审批卡片/通知)
Outbound,
/// 网关 → 桌面端(TunnelCommand 下发)
Command,
}
impl Direction {
fn as_str(self) -> &'static str {
match self {
Direction::Inbound => "inbound",
Direction::Outbound => "outbound",
Direction::Command => "command",
}
}
}
#[derive(Clone)]
pub struct AuditLog {
conn: Arc<std::sync::Mutex<Connection>>,
}
/// 打点参数(字段可选,None 写空串,保持列 NOT NULL 简化查询)
pub struct AuditEntry {
pub direction: Direction,
pub platform: String,
pub chat_id: String,
pub user_id: String,
pub conv_id: String,
/// 动作名:message/new_session/approval_click/approval_card/stream/completed/notice/send_cmd/approve_cmd
pub action: String,
/// 关联 ID:入站↔出站串链用(消息场景可用 chat+ts 派生,审批场景用 approval_id)
pub request_id: String,
pub payload: String,
/// ok / error / denied(白名单拒/无 channel)
pub outcome: String,
}
impl AuditLog {
/// 打开(或创建)审计库。默认与 sessions.db 同目录独立文件(隔离写放大)。
pub fn open(path: &Path) -> anyhow::Result<Self> {
let conn = Connection::open(path)?;
conn.pragma_update(None, "journal_mode", "WAL")?;
conn.execute_batch(
"CREATE TABLE IF NOT EXISTS tunnel_audit (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ts INTEGER NOT NULL,
direction TEXT NOT NULL,
platform TEXT NOT NULL DEFAULT '',
chat_id TEXT NOT NULL DEFAULT '',
user_id TEXT NOT NULL DEFAULT '',
conv_id TEXT NOT NULL DEFAULT '',
action TEXT NOT NULL,
request_id TEXT NOT NULL DEFAULT '',
payload TEXT NOT NULL DEFAULT '',
outcome TEXT NOT NULL DEFAULT 'ok'
);
CREATE INDEX IF NOT EXISTS idx_audit_ts ON tunnel_audit(ts);
CREATE INDEX IF NOT EXISTS idx_audit_request ON tunnel_audit(request_id);",
)?;
Ok(Self { conn: Arc::new(std::sync::Mutex::new(conn)) })
}
/// 内存库(测试)
#[allow(dead_code)]
pub fn open_in_memory() -> anyhow::Result<Self> {
let conn = Connection::open_in_memory()?;
conn.execute_batch(
"CREATE TABLE IF NOT EXISTS tunnel_audit (
id INTEGER PRIMARY KEY AUTOINCREMENT, ts INTEGER NOT NULL,
direction TEXT NOT NULL, platform TEXT NOT NULL DEFAULT '',
chat_id TEXT NOT NULL DEFAULT '', user_id TEXT NOT NULL DEFAULT '',
conv_id TEXT NOT NULL DEFAULT '', action TEXT NOT NULL,
request_id TEXT NOT NULL DEFAULT '', payload TEXT NOT NULL DEFAULT '',
outcome TEXT NOT NULL DEFAULT 'ok');",
)?;
Ok(Self { conn: Arc::new(std::sync::Mutex::new(conn)) })
}
fn now_ms() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as i64)
.unwrap_or(0)
}
/// 追加一条审计(失败仅告警不冒泡)
pub fn record(&self, e: AuditEntry) {
let payload = if e.payload.len() > MAX_PAYLOAD {
format!("{}…[截断]", &e.payload[..MAX_PAYLOAD])
} else {
e.payload
};
let result = (|| -> anyhow::Result<()> {
let conn = self.conn.lock().map_err(|err| anyhow::anyhow!("audit 锁中毒: {err}"))?;
conn.execute(
"INSERT INTO tunnel_audit (ts, direction, platform, chat_id, user_id, conv_id, action, request_id, payload, outcome) \
VALUES (?1,?2,?3,?4,?5,?6,?7,?8,?9,?10)",
rusqlite::params![
Self::now_ms(), e.direction.as_str(), e.platform, e.chat_id,
e.user_id, e.conv_id, e.action, e.request_id, payload, e.outcome,
],
)?;
Ok(())
})();
if let Err(err) = result {
tracing::warn!("[audit] 写入失败(不影响主链路): {err}");
}
}
/// 最近 N 条(时间倒序,运维排障)
#[allow(dead_code)]
pub fn recent(&self, limit: usize) -> anyhow::Result<Vec<String>> {
let conn = self.conn.lock().map_err(|e| anyhow::anyhow!("audit 锁中毒: {e}"))?;
let mut stmt = conn.prepare(
"SELECT ts, direction, platform, chat_id, user_id, conv_id, action, request_id, outcome \
FROM tunnel_audit ORDER BY id DESC LIMIT ?1",
)?;
let rows = stmt.query_map([limit as i64], |row| {
Ok(format!(
"{} [{}] {} plat={} chat={} user={} conv={} req={} outcome={}",
row.get::<_, i64>(0)?, row.get::<_, String>(1)?, row.get::<_, String>(6)?,
row.get::<_, String>(2)?, row.get::<_, String>(3)?, row.get::<_, String>(4)?,
row.get::<_, String>(5)?, row.get::<_, String>(7)?, row.get::<_, String>(8)?,
))
})?;
Ok(rows.filter_map(|r| r.ok()).collect())
}
/// 按 request_id 串双向链(入站→命令→出站全链)
#[allow(dead_code)]
pub fn by_request(&self, request_id: &str) -> anyhow::Result<Vec<String>> {
let conn = self.conn.lock().map_err(|e| anyhow::anyhow!("audit 锁中毒: {e}"))?;
let mut stmt = conn.prepare(
"SELECT ts, direction, action, outcome FROM tunnel_audit WHERE request_id = ?1 ORDER BY id ASC",
)?;
let rows = stmt.query_map([request_id], |row| {
Ok(format!(
"{} [{}] {} outcome={}",
row.get::<_, i64>(0)?, row.get::<_, String>(1)?, row.get::<_, String>(2)?, row.get::<_, String>(3)?,
))
})?;
Ok(rows.filter_map(|r| r.ok()).collect())
}
}
#[cfg(test)]
mod tests {
use super::*;
fn entry(direction: Direction, action: &str, req: &str, outcome: &str) -> AuditEntry {
AuditEntry {
direction,
platform: "feishu".into(),
chat_id: "oc_123".into(),
user_id: "u_1".into(),
conv_id: "feishu:oc_123".into(),
action: action.into(),
request_id: req.into(),
payload: "hello".into(),
outcome: outcome.into(),
}
}
#[test]
fn record_and_recent_roundtrip() {
let log = AuditLog::open_in_memory().unwrap();
log.record(entry(Direction::Inbound, "message", "req-1", "ok"));
log.record(entry(Direction::Command, "send_cmd", "req-1", "ok"));
log.record(entry(Direction::Outbound, "stream", "req-1", "ok"));
log.record(entry(Direction::Inbound, "message", "req-2", "denied"));
let rows = log.recent(10).unwrap();
assert_eq!(rows.len(), 4);
// 倒序:最新在前
assert!(rows[0].contains("req-2"));
assert!(rows[0].contains("denied"));
let chain = log.by_request("req-1").unwrap();
assert_eq!(chain.len(), 3);
}
#[test]
fn payload_truncated() {
let log = AuditLog::open_in_memory().unwrap();
let mut e = entry(Direction::Inbound, "message", "req-x", "ok");
e.payload = "x".repeat(10_000);
log.record(e);
let conn_count = |log: &AuditLog| -> i64 {
let conn = log.conn.lock().unwrap();
conn.query_row("SELECT COUNT(*) FROM tunnel_audit", [], |r| r.get(0)).unwrap()
};
assert_eq!(conn_count(&log), 1);
let conn = log.conn.lock().unwrap();
let stored: String = conn
.query_row("SELECT payload FROM tunnel_audit", [], |r| r.get(0))
.unwrap();
assert!(stored.len() < 5_000);
assert!(stored.ends_with("…[截断]"));
}
#[test]
fn record_never_panics_on_poisoned_lock() {
let log = AuditLog::open_in_memory().unwrap();
// 毒锁:持锁 panic 使 Mutex poisoned,record 应 warn 而非 panic
let guard = log.conn.lock().unwrap();
std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
std::panic::panic_any("poison");
}))
.err();
drop(guard);
log.record(entry(Direction::Inbound, "message", "req-p", "ok")); // 不 panic 即通过
}
}
+189
View File
@@ -0,0 +1,189 @@
//! WS Commander 客户端(连 u-relay /ws/commander)
//!
//! 与桌面端 df-tunnel(Agent 角色)对称:网关作为 Commander 发 TunnelCommand、收 TunnelEvent。
//! 协议类型直接复用 df_tunnel::events(编译期一致)。
//!
//! 连接生命周期:hello {kind:"commander", channel_id, token} → (relay 校验) → 双向消息。
//! 断线重连:指数退避(1/2/4/8/16/30s 封顶,与桌面端 df-tunnel 同款策略)。
//!
//! 注:relay 对同 channel 多 Commander 的投递行为(P0 必验项 ①)待实测——
//! 过渡期约定同 channel 单活跃控制端。
use std::time::Duration;
use futures_util::{SinkExt, StreamExt};
use tokio::sync::mpsc;
use tokio_tungstenite::tungstenite::Message;
use tracing::{info, warn};
use df_tunnel::events::{Hello, HelloKind, RelayControlError};
use df_tunnel::{TunnelCommand, TunnelEvent, TunnelMessage};
/// 指数退避序列(秒,封顶 30)
fn backoff_secs(attempt: u32) -> u64 {
[1, 2, 4, 8, 16, 30]
.get(attempt.min(5) as usize)
.copied()
.unwrap_or(30)
}
/// 启动 Commander 连接循环(阻塞直到 shutdown)。重连在内部循环,指令/事件双向透传。
///
/// * `relay_url` — ws://host:9180/ws/commander
/// * `channel_id` — 配对的桌面端 channel
/// * `token` — U_RELAY_TOKEN 同源
/// * `ev_tx` — 事件出口(桌面端 → 网关核心)
/// * `cmd_rx` — 指令入口(网关核心 → 桌面端)
/// * `shutdown` — 关闭信号
pub async fn run_commander(
relay_url: String,
channel_id: String,
token: String,
ev_tx: mpsc::Sender<TunnelEvent>,
mut cmd_rx: mpsc::Receiver<TunnelCommand>,
shutdown: tokio::sync::watch::Receiver<bool>,
) -> anyhow::Result<()> {
let mut attempt: u32 = 0;
loop {
if *shutdown.borrow() {
info!("[commander] 收到关闭信号,退出");
return Ok(());
}
match connect_once(&relay_url, &channel_id, &token, &ev_tx, &mut cmd_rx, shutdown.clone()).await {
Ok(()) => {
info!("[commander] 连接正常关闭");
return Ok(());
}
Err(e) => {
let wait = backoff_secs(attempt);
attempt += 1;
warn!("[commander] 连接断开({e}),{}s 后重连(第 {attempt} 次)", wait);
let mut shutdown_w = shutdown.clone();
tokio::select! {
_ = tokio::time::sleep(Duration::from_secs(wait)) => {}
_ = shutdown_w.changed() => return Ok(()),
}
}
}
}
}
/// 单次连接会话(hello 握手 + 消息循环),断开/出错返回 Err 触发外层重连
async fn connect_once(
relay_url: &str,
channel_id: &str,
token: &str,
ev_tx: &mpsc::Sender<TunnelEvent>,
cmd_rx: &mut mpsc::Receiver<TunnelCommand>,
mut shutdown: tokio::sync::watch::Receiver<bool>,
) -> anyhow::Result<()> {
let (ws, _resp) = tokio_tungstenite::connect_async(relay_url).await?;
let (mut sink, mut stream) = ws.split();
// hello 握手(kind=commander;字段与 u-relay::relay::Hello 严格一致)
let hello = Hello {
kind: HelloKind::Commander,
channel_id: channel_id.to_string(),
token: token.to_string(),
proto: 1,
};
sink.send(Message::Text(serde_json::to_string(&hello)?)).await?;
// 等待 hello_ack(首条文本帧;control error 帧则报错)
let ack = tokio::time::timeout(Duration::from_secs(10), stream.next()).await
.map_err(|_| anyhow::anyhow!("hello_ack 10s 超时"))?
.ok_or_else(|| anyhow::anyhow!("连接在握手期关闭"))?;
match ack? {
Message::Text(t) => {
// relay 控制帧:{"kind":"control","error":"..."} → 握手被拒
if let Ok(ctrl_err) = serde_json::from_str::<RelayControlError>(&t) {
if ctrl_err.kind == "control" {
anyhow::bail!("握手被拒: {}", ctrl_err.error);
}
}
info!("[commander] hello_ack: channel={channel_id} 握手成功");
}
other => anyhow::bail!("hello_ack 非文本帧: {other:?}"),
}
// 心跳:30s ping(relay 90s 读死线)
let mut ping = tokio::time::interval(Duration::from_secs(30));
ping.tick().await; // 首个立即返回,跳过
// 消息循环
loop {
tokio::select! {
_ = shutdown.changed() => {
let _ = sink.close().await;
return Ok(());
}
_ = ping.tick() => {
sink.send(Message::Ping(vec![])).await?;
}
cmd = cmd_rx.recv() => {
let Some(cmd) = cmd else {
// 核心侧通道关闭:正常收尾
let _ = sink.close().await;
return Ok(());
};
let frame = TunnelMessage::Command(cmd);
sink.send(Message::Text(serde_json::to_string(&frame)?)).await?;
}
msg = stream.next() => {
match msg {
Some(Ok(Message::Text(t))) => {
match serde_json::from_str::<TunnelMessage>(&t) {
Ok(TunnelMessage::Event(e)) => {
if ev_tx.send(e).await.is_err() {
return Err(anyhow::anyhow!("核心事件通道关闭"));
}
}
Ok(TunnelMessage::Command(_)) => {
warn!("[commander] commander 收到 Command 帧(协议错向,忽略)");
}
Err(_) => warn!("[commander] 无法解析帧(非 TunnelMessage,忽略): {} 字节", t.len()),
}
}
Some(Ok(Message::Ping(p))) => { let _ = sink.send(Message::Pong(p)).await; }
Some(Ok(_)) => {}
Some(Err(e)) => return Err(e.into()),
None => return Err(anyhow::anyhow!("stream 结束")),
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn backoff_sequence() {
assert_eq!(backoff_secs(0), 1);
assert_eq!(backoff_secs(1), 2);
assert_eq!(backoff_secs(2), 4);
assert_eq!(backoff_secs(3), 8);
assert_eq!(backoff_secs(4), 16);
assert_eq!(backoff_secs(5), 30);
assert_eq!(backoff_secs(99), 30, "封顶 30s");
}
#[test]
fn tunnel_message_roundtrip() {
let cmd = TunnelMessage::Command(TunnelCommand::Send {
conv_id: "c1".into(),
content: "你好".into(),
});
let json = serde_json::to_string(&cmd).unwrap();
assert!(json.contains("\"command\""));
let back: TunnelMessage = serde_json::from_str(&json).unwrap();
match back {
TunnelMessage::Command(TunnelCommand::Send { conv_id, content }) => {
assert_eq!(conv_id, "c1");
assert_eq!(content, "你好");
}
_ => panic!("roundtrip 失败"),
}
}
}
+143
View File
@@ -0,0 +1,143 @@
//! 网关配置(env 加载,凭证不进 git)
//!
//! 配置来源:环境变量 > 默认值(gateway.toml 文件加载为后续增强,当前未实现)。
//! u-relay token 与桌面端 df-tunnel 同源(env U_RELAY_TOKEN)。
use serde::Deserialize;
/// 网关全局配置
#[derive(Debug, Clone, Deserialize)]
pub struct GatewayConfig {
/// u-relay WS 地址(生产 u-relay.1216.top,本地 http://127.0.0.1:9180)
pub relay_url: String,
/// relay 配对 token(env U_RELAY_TOKEN 同源)
pub relay_token: String,
/// channel 路由表:IM 侧默认走 default_channel;按前缀路由的多桌面端配置(预留项 ⑥)
#[serde(default)]
pub channels: Vec<ChannelRoute>,
/// 飞书应用凭证(未配置则该平台禁用)
#[serde(default)]
pub feishu: Option<FeishuConfig>,
/// TextDelta 节流窗口毫秒(默认 800,IM 频控友好)
#[serde(default = "default_throttle_ms")]
pub throttle_ms: u64,
/// 用户白名单(空=放行所有,生产部署务必显式配置)
///
/// 注:历史注释「空=拒绝所有」与 main.rs 实现(空=放行)矛盾,走查 2026-10-04 定为
/// 空=放行——否则首次启动零配置时飞书/Web 全部拒绝,连 banner 警告都没有,反而
/// 更难发现配置缺失。生产风险由 main.rs 启动 banner 常驻警告 + 白名单缺失提示兜底。
#[serde(default)]
pub whitelist: Vec<String>,
}
/// channel 路由项:IM 指令前缀 → 桌面端 channel
#[derive(Debug, Clone, Deserialize)]
pub struct ChannelRoute {
/// 指令前缀(如 "/dev2 xxx" 路由到 chan-home);空串=默认 channel
pub prefix: String,
/// u-relay channel_id(桌面端 df-tunnel 配置的同一 channel)
pub channel_id: String,
}
/// 飞书企业自建应用凭证
#[derive(Debug, Clone, Deserialize)]
pub struct FeishuConfig {
pub app_id: String,
pub app_secret: String,
/// 长连接模式事件订阅(免公网回调;当前恒 true,保留显式开关供回调模式扩展)
#[allow(dead_code)]
pub long_connection: bool,
}
fn default_throttle_ms() -> u64 {
800
}
impl GatewayConfig {
/// 从 env + 可选 toml 文件构建。缺失关键项(relay_url/token)返回 Err。
pub fn load() -> anyhow::Result<Self> {
let relay_url = std::env::var("GATEWAY_RELAY_URL")
.unwrap_or_else(|_| "ws://127.0.0.1:9180/ws/commander".to_string());
let relay_token = std::env::var("GATEWAY_RELAY_TOKEN")
.or_else(|_| std::env::var("U_RELAY_TOKEN"))
.map_err(|_| anyhow::anyhow!("缺少 GATEWAY_RELAY_TOKEN / U_RELAY_TOKEN 环境变量"))?;
let feishu = match (
std::env::var("FEISHU_APP_ID"),
std::env::var("FEISHU_APP_SECRET"),
) {
(Ok(app_id), Ok(app_secret)) => Some(FeishuConfig {
app_id,
app_secret,
long_connection: true,
}),
_ => None,
};
let whitelist = std::env::var("GATEWAY_WHITELIST")
.map(|v| v.split(',').map(|s| s.trim().to_string()).filter(|s| !s.is_empty()).collect())
.unwrap_or_default();
let default_channel = std::env::var("GATEWAY_DEFAULT_CHANNEL").unwrap_or_default();
let channels = if default_channel.is_empty() {
Vec::new()
} else {
vec![ChannelRoute { prefix: String::new(), channel_id: default_channel }]
};
Ok(Self {
relay_url,
relay_token,
channels,
feishu,
throttle_ms: default_throttle_ms(),
whitelist,
})
}
/// 查指令应路由到的 channel(前缀匹配,空前缀=兜底默认)
pub fn route(&self, command_text: &str) -> Option<&str> {
// 精确前缀优先,空前缀兜底
self.channels
.iter()
.find(|c| !c.prefix.is_empty() && command_text.starts_with(&c.prefix))
.map(|c| c.channel_id.as_str())
.or_else(|| self.channels.iter().find(|c| c.prefix.is_empty()).map(|c| c.channel_id.as_str()))
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn route_prefix_priority() {
let cfg = GatewayConfig {
relay_url: String::new(),
relay_token: String::new(),
channels: vec![
ChannelRoute { prefix: "/dev2".into(), channel_id: "chan-home".into() },
ChannelRoute { prefix: String::new(), channel_id: "chan-work".into() },
],
feishu: None,
throttle_ms: 800,
whitelist: vec![],
};
assert_eq!(cfg.route("/dev2 修个 bug"), Some("chan-home"));
assert_eq!(cfg.route("普通指令"), Some("chan-work"));
assert_eq!(cfg.route("/dev2"), Some("chan-home"));
}
#[test]
fn route_empty_when_no_channels() {
let cfg = GatewayConfig {
relay_url: String::new(),
relay_token: String::new(),
channels: vec![],
feishu: None,
throttle_ms: 800,
whitelist: vec![],
};
assert_eq!(cfg.route("任何"), None);
}
}
+410
View File
@@ -0,0 +1,410 @@
//! 飞书适配器(P2):长连接事件 + 归一化消息 + 卡片审批
//!
//! 依赖 larksuite-oapi-sdk-rs features=["channel"](自动连带 ws+messaging)。
//! 长连接:飞书←→网关出站直连,不经 u-relay(u-relay 仅穿 NAT 段到桌面端)。
//!
//! 验证记录(2026-10-04,SDK 0.3.12 源码核对):
//! - 卡片回调:ChannelBuilder::on_card_action(NormalizedCardAction)
//! 内部走 dispatcher.on_card_action_trigger(channel/handler.rs:112),
//! 即新版 card.action.trigger 事件通道,长连接可收(遗留验证项 ✅)。
//! - 发卡片:SendInput.card(String) → INTERACTIVE 类型(messaging.rs:752)。
//! - 流式更新:channel_messaging().send() 拿 message_id 后 edit_text() 覆写全文。
use std::collections::HashMap;
use std::sync::Arc;
use async_trait::async_trait;
use larksuite_oapi_sdk_rs::card::v1::{
Action, ActionComponent, Button, ButtonType, Card, CardDocument, Div, Element, Header,
Markdown,
};
use larksuite_oapi_sdk_rs::channel::{Channel, ChannelPolicy};
use larksuite_oapi_sdk_rs::{EventDispatcher, LarkClient, LarkError, RequestOption};
use tokio::sync::mpsc::Sender;
use crate::adapters::{InboundEvent, OutboundAction, PlatformAdapter};
use crate::config::FeishuConfig;
/// 飞书适配器:channel 归一化消息 + 卡片审批,单飞书应用单实例
#[derive(Clone)]
pub struct FeishuAdapter {
/// 常驻 client(&'static,构造期一次性 Box::leak)。D1(u-claw 教训:热重建不关旧
/// 连接→僵尸资源):leak 只在 new()/offline() 各发生一次;run() 重连重入只重建
/// Channel 不再重复 leak,防频繁断线场景 LarkClient 内存阶梯上涨。出站 dispatch
/// 与 Channel 构建共用同一份引用。
client: &'static LarkClient,
/// 流式会话注册表:chat_id → 已发出的流式消息 message_id(用于 edit_text 覆写)
streams: Arc<tokio::sync::Mutex<HashMap<String, String>>>,
}
impl FeishuAdapter {
pub fn new(cfg: &FeishuConfig) -> anyhow::Result<Self> {
let client: &'static LarkClient = Box::leak(Box::new(
LarkClient::builder(&cfg.app_id, &cfg.app_secret).build()?,
));
Ok(Self {
client,
streams: Arc::new(tokio::sync::Mutex::new(HashMap::new())),
})
}
/// 构造失败时的离线哨兵:client 用占位凭证(不启动 run,dispatch 记日志即弃)
pub fn offline() -> Self {
let client: &'static LarkClient = Box::leak(Box::new(
LarkClient::builder("offline", "offline")
.build()
.expect("占位凭证构造 LarkClient 不发网络请求,必成功"),
));
Self {
client,
streams: Arc::new(tokio::sync::Mutex::new(HashMap::new())),
}
}
}
#[async_trait]
impl PlatformAdapter for FeishuAdapter {
fn name(&self) -> &str {
"feishu"
}
/// 兜底清理流式残留(走查 #3):桌面端异常中止时无 Completed 事件,
/// streams entry 永驻导致下轮首帧错覆写旧消息。核心循环在新建会话映射时调用。
async fn reset_stream(&self, chat_id: &str) {
self.streams.lock().await.remove(chat_id);
}
async fn run(&self, events: Sender<InboundEvent>) -> anyhow::Result<()> {
// 长连接模式:空 verification_token/encrypt_key(免公网回调)
let dispatcher = EventDispatcher::new("", "");
let client = self.client;
// 入站事件通道复制两份,分别给消息/卡片回调闭包
let events_for_msg = events.clone();
let events_for_card = events;
let channel: Channel<'static> = {
// client 已是 &'static(构造期一次性 leak);run() 重连重入只重建 Channel,
// 不再重复 Box::leak(D1:防每次重入泄漏一份 LarkClient)
Channel::builder(client, dispatcher)
.policy(ChannelPolicy::default().require_mention(false))
.on_message(move |msg| {
let events = events_for_msg.clone();
async move {
handle_inbound_message(events, msg).await;
Ok(())
}
})
.on_card_action(move |action| {
let events = events_for_card.clone();
async move {
handle_card_action(events, action).await;
Ok(())
}
})
.build()
};
tracing::info!("[feishu] 长连接启动(免公网回调)");
channel.start().await.map_err(|e: LarkError| anyhow::anyhow!("飞书长连接退出: {e}"))?;
Ok(())
}
async fn dispatch(&self, action: OutboundAction) -> anyhow::Result<()> {
let opt = &RequestOption::default();
match action {
OutboundAction::StreamText { chat_id, text, done } => {
// 两段式:锁内只取/写 message_id(纯内存操作),网络 IO 在锁外执行。
// 修复走查 #2:同 chat 仍串行(顺序语义),不同 chat 不再被同一把锁挡住。
let existing_id: Option<String> = {
let streams = self.streams.lock().await;
streams.get(&chat_id).cloned()
};
if let Some(message_id) = existing_id {
let res = self.client.channel_messaging().edit_text(&message_id, &text, opt).await;
if let Err(e) = res {
tracing::warn!("[feishu] 流式更新失败(消息可能已被撤回/超时): {e},回退新发");
self.streams.lock().await.remove(&chat_id);
}
} else {
// 首帧:发新文本消息,记录 message_id 供后续覆写
let input = larksuite_oapi_sdk_rs::channel::SendInput {
receive_id: Some(chat_id.clone()),
text: Some(text),
..Default::default()
};
let send_result = self.client.channel_messaging().send(&input, opt).await;
match send_result {
Ok(sr) if !sr.message_id.is_empty() => {
self.streams.lock().await.insert(chat_id.clone(), sr.message_id);
}
Ok(_) => {
tracing::warn!("[feishu] 发送流式首帧未返回 message_id");
}
Err(e) => {
tracing::error!("[feishu] 发送流式首帧失败: {e}");
}
}
}
if done {
self.streams.lock().await.remove(&chat_id);
}
Ok(())
}
OutboundAction::ApprovalCard { chat_id, approval_id, title, detail } => {
let card = build_approval_card(&approval_id, &title, &detail);
let document = CardDocument::new(card)
.map_err(|e| anyhow::anyhow!("审批卡片校验失败: {e}"))?;
// encoded_content 是 SDK pub(crate);直接序列化 card(校验已过)
let content = serde_json::to_string(document.card())
.map_err(|e| anyhow::anyhow!("卡片序列化失败: {e}"))?;
let input = larksuite_oapi_sdk_rs::channel::SendInput {
receive_id: Some(chat_id),
card: Some(content),
..Default::default()
};
self.client
.channel_messaging()
.send(&input, opt)
.await
.map_err(|e| anyhow::anyhow!("审批卡片发送失败: {e}"))?;
Ok(())
}
OutboundAction::Notice { chat_id, text } => {
let input = larksuite_oapi_sdk_rs::channel::SendInput {
receive_id: Some(chat_id),
text: Some(text),
..Default::default()
};
self.client
.channel_messaging()
.send(&input, opt)
.await
.map_err(|e| anyhow::anyhow!("通知发送失败: {e}"))?;
Ok(())
}
}
}
}
/// 入站消息:群聊需 @机器人(mentioned_bot)或单聊直发
async fn handle_inbound_message(events: Sender<InboundEvent>, msg: larksuite_oapi_sdk_rs::channel::NormalizedMessage) {
// 只处理文本(text 优先,回退 content)
if msg.message_type != "text" {
tracing::debug!("[feishu] 忽略非文本消息 type={}", msg.message_type);
return;
}
// 群聊必须 @机器人;单聊(p2p)直发
if msg.chat_type == "group" && !msg.mentioned_bot {
tracing::debug!("[feishu] 群聊未 @机器人,忽略");
return;
}
let text = msg.text.clone().unwrap_or_default().trim().to_string();
if text.is_empty() {
return;
}
// 指令:/new 重置会话
if text == "/new" {
let _ = events
.send(InboundEvent::NewSession {
platform: "feishu".into(),
chat_id: msg.chat_id.clone(),
})
.await;
return;
}
let user_id = msg
.sender
.user_id
.as_ref()
.and_then(|u| u.open_id.clone())
.unwrap_or_else(|| "unknown".into());
let _ = events
.send(InboundEvent::Message {
platform: "feishu".into(),
chat_id: msg.chat_id,
user_id,
text,
})
.await;
}
/// 卡片按钮回调:action_value 携带 approval_id + approved
async fn handle_card_action(events: Sender<InboundEvent>, action: larksuite_oapi_sdk_rs::channel::NormalizedCardAction) {
// 按钮约定的 value 形态:{"approval_id": "...", "approved": true|false}
let approval_id = action
.action_value
.get("approval_id")
.and_then(|v| v.as_str())
.unwrap_or_default()
.to_string();
let approved = action
.action_value
.get("approved")
.and_then(|v| v.as_bool())
.unwrap_or(false);
if approval_id.is_empty() {
tracing::warn!("[feishu] 卡片回调缺 approval_id,忽略");
return;
}
let chat_id = action.open_chat_id.clone().unwrap_or_default();
let _ = events
.send(InboundEvent::Approval {
platform: "feishu".into(),
chat_id,
approval_id,
approved,
})
.await;
}
/// 构造审批卡片(Card JSON 1.0):标题 + 详情 + 同意/拒绝按钮
fn build_approval_card(approval_id: &str, title: &str, detail: &str) -> Card {
let detail_md = format!("{}\n\n**审批 ID**: `{approval_id}`", md_to_lark_friendly(detail));
let to_json_value = |v: serde_json::Value| -> larksuite_oapi_sdk_rs::JsonValue {
serde_json::from_value(serde_json::to_value(v).expect("serde_json::Value 可逆"))
.expect("按钮 value 结构兼容")
};
let actions = Action::new()
.action(ActionComponent::Button(
Button::new(larksuite_oapi_sdk_rs::card::v1::Text::plain("✅ 同意"))
.button_type(ButtonType::Primary)
.value(to_json_value(serde_json::json!({"approval_id": approval_id, "approved": true}))),
))
.action(ActionComponent::Button(
Button::new(larksuite_oapi_sdk_rs::card::v1::Text::plain("❌ 拒绝"))
.button_type(ButtonType::Danger)
.value(to_json_value(serde_json::json!({"approval_id": approval_id, "approved": false}))),
));
Card::new()
.header(Header::new(title))
.element(Element::Div(Div::new(
larksuite_oapi_sdk_rs::card::v1::Text::lark_md(detail_md),
)))
.element(Element::Markdown(Markdown::new("---")))
.element(Element::Action(actions))
}
/// 出站 Markdown 净化(飞书卡片 lark_md 语法适配)。
///
/// AI 产出常带标准 Markdown 结构,lark_md 不支持标题与表格语法,原样注入按字面量渲染。
/// 规则(与 src-tauri feishu_channel.rs 侧 md_to_lark_friendly 同规则互引,双端出站口径一致):
/// 标题行(行首一个以上 #)转粗体行;表格行(| 开头)转列表行,表头分隔行整行跳过;
/// 代码块围栏内外均保留原样(转义或改写破坏代码展示);其余行原样透传(lark_md 原生
/// 支持粗体/链接/行内代码,不额外转义)。
fn md_to_lark_friendly(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut in_code = false;
for line in s.lines() {
if line.trim_start().starts_with("```") {
in_code = !in_code;
out.push_str(line);
out.push('\n');
continue;
}
if in_code {
out.push_str(line);
out.push('\n');
continue;
}
let trimmed = line.trim_start();
if trimmed.starts_with('#') {
let title = trimmed.trim_start_matches('#').trim();
if !title.is_empty() {
out.push_str(&format!("**{title}**\n"));
}
continue;
}
if trimmed.starts_with('|') {
let cells: Vec<&str> = trimmed.trim_matches('|').split('|').map(str::trim).collect();
// 表头分隔行(|---|:---|)无内容语义,整行跳过
if cells.iter().all(|c| c.is_empty() || c.chars().all(|ch| ch == '-' || ch == ':')) {
continue;
}
let shown: Vec<&str> = cells.into_iter().filter(|c| !c.is_empty()).collect();
out.push_str(&format!("- {}\n", shown.join(" · ")));
continue;
}
out.push_str(line);
out.push('\n');
}
// lines() 逐行补换行,源文本不以换行结尾时尾部多出一个,与源对齐
if !s.ends_with('\n') {
out.pop();
}
out
}
#[cfg(test)]
mod tests {
use super::*;
use crate::config::FeishuConfig;
/// 构造冒烟:new/offline 均不发网络请求,验证适配器可实例化
#[test]
fn feishu_adapter_constructs() {
let cfg = FeishuConfig {
app_id: "cli_smoke".into(),
app_secret: "secret".into(),
long_connection: true,
};
let adapter = FeishuAdapter::new(&cfg).expect("适配器构造应成功");
assert_eq!(adapter.name(), "feishu");
let offline = FeishuAdapter::offline();
assert_eq!(offline.name(), "feishu");
}
/// 审批卡片构造 + 协议校验(CardDocument::new 是发送前的强校验门槛)
#[test]
fn approval_card_passes_protocol_validation() {
let card = build_approval_card("appr-123", "工具审批请求", "run_command: cargo test");
CardDocument::new(card).expect("审批卡片应通过 Card JSON 1.0 校验");
}
/// 卡片构造包含同意/拒绝按钮及 approval_id 载荷
#[test]
fn approval_card_has_buttons_and_payload() {
let card = build_approval_card("appr-456", "标题", "详情");
let json = serde_json::to_value(&card).unwrap();
let elements = json["elements"].as_array().expect("elements 数组");
// 最后一个元素是 Action 行,内含 2 个按钮
let action = elements.last().unwrap();
let buttons = action["actions"].as_array().expect("actions 数组");
assert_eq!(buttons.len(), 2);
let values: Vec<String> = buttons
.iter()
.map(|b| b["value"]["approval_id"].as_str().unwrap_or_default().to_string())
.collect();
assert!(values.iter().all(|v| v == "appr-456"));
let approved_flags: Vec<bool> = buttons
.iter()
.map(|b| b["value"]["approved"].as_bool().unwrap_or(false))
.collect();
assert!(approved_flags.contains(&true) && approved_flags.contains(&false));
}
/// 出站净化:标题转粗体行,表格行转列表行,表头分隔行跳过
#[test]
fn lark_friendly_converts_headings_and_tables() {
let md = "# 标题\n| a | b |\n|---|---|\n| 1 | 2 |";
assert_eq!(md_to_lark_friendly(md), "**标题**\n- a · b\n- 1 · 2");
}
/// 出站净化:代码围栏内外均保留原样,围栏外的标题照常改写
#[test]
fn lark_friendly_preserves_code_fence() {
let md = "前文\n```\n# 不是标题\n| 不 | 是表格 |\n```\n## 后文";
assert_eq!(
md_to_lark_friendly(md),
"前文\n```\n# 不是标题\n| 不 | 是表格 |\n```\n**后文**"
);
}
/// 出站净化:普通行原样透传(lark_md 原生支持粗体/链接,不额外转义)
#[test]
fn lark_friendly_passes_plain_lines() {
let md = "**粗** 与 [链接](http://e)\n普通行";
assert_eq!(md_to_lark_friendly(md), md);
}
}
+330
View File
@@ -0,0 +1,330 @@
//! df-im-gateway 主入口:装配 config + store + throttle + commander + adapters
//!
//! 骨架版(P0):Web 适配器真实可跑,事件流全链路打通。
//! 用法:
//! ```text
//! df-im-gateway.exe # 默认配置(env)
//! GATEWAY_RELAY_URL=... GATEWAY_RELAY_TOKEN=... \
//! GATEWAY_DEFAULT_CHANNEL=chan-work df-im-gateway.exe
//! ```
mod adapters;
mod audit;
mod commander;
mod config;
mod feishu;
mod store;
mod throttle;
use std::path::PathBuf;
use tokio::sync::mpsc;
use adapters::{InboundEvent, OutboundAction, PlatformAdapter};
use config::GatewayConfig;
use store::SessionStore;
use throttle::DeltaThrottle;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info")),
)
.init();
let cfg = GatewayConfig::load()?;
if cfg.channels.is_empty() {
anyhow::bail!("未配置任何 channel 路由(GATEWAY_DEFAULT_CHANNEL),无法启动");
}
let data_dir = std::env::var("GATEWAY_DATA_DIR")
.map(PathBuf::from)
.unwrap_or_else(|_| PathBuf::from("data"));
std::fs::create_dir_all(&data_dir)?;
let store = SessionStore::open(&data_dir.join("sessions.db"))?;
// 审计日志(P5):IM 消息 ↔ TunnelCommand 全量记录,写失败不阻断主链路
let audit = audit::AuditLog::open(&data_dir.join("audit.db"))?;
info_banner(&cfg);
// ── 核心事件总线:适配器 → 核心 ─────────────────────────────
let (inbound_tx, mut inbound_rx) = mpsc::channel::<InboundEvent>(256);
// Web 适配器(四端首个,P0)
let web_adapter = adapters::web::WebAdapter::new();
{
let web_run = web_adapter.clone();
let ev_tx = inbound_tx.clone();
tokio::spawn(async move {
if let Err(e) = web_run.run(ev_tx).await {
tracing::error!("[web] 适配器退出: {e}");
}
});
}
// 飞书适配器(P2):FEISHU_APP_ID/SECRET 配置时启用长连接
let feishu_adapter = cfg.feishu.as_ref().map(|f| match feishu::FeishuAdapter::new(f) {
Ok(adapter) => {
let feishu_run = adapter.clone();
let ev_tx = inbound_tx.clone();
tokio::spawn(async move {
if let Err(e) = feishu_run.run(ev_tx).await {
tracing::error!("[feishu] 适配器退出: {e}");
}
});
tracing::info!("[feishu] 适配器已启动(长连接)");
adapter
}
Err(e) => {
tracing::error!("[feishu] 适配器构造失败(跳过): {e}");
feishu::FeishuAdapter::offline()
}
});
// ── Commander 连接(u-relay)────────────────────────────────
let (shutdown_tx, shutdown_rx) = tokio::sync::watch::channel(false);
let default_channel = cfg
.channels
.iter()
.find(|c| c.prefix.is_empty())
.map(|c| c.channel_id.clone())
.unwrap_or_else(|| cfg.channels[0].channel_id.clone());
// 事件流:commander 任务 → 核心循环
let (ev_tx, mut ev_rx) = mpsc::channel::<df_tunnel::events::TunnelEvent>(256);
// 指令流:核心循环 → commander 任务(透传到 WS)
let (cmd_tx, cmd_rx) = mpsc::channel::<df_tunnel::TunnelCommand>(256);
{
let relay_url = cfg.relay_url.clone();
let token = cfg.relay_token.clone();
let channel = default_channel.clone();
tokio::spawn(async move {
if let Err(e) = commander::run_commander(relay_url, channel, token, ev_tx, cmd_rx, shutdown_rx).await {
tracing::error!("[commander] 连接循环退出: {e}");
}
});
}
// ── 核心分发循环(单 select,持有全部状态,无跨线程共享)─────
let mut throttle = DeltaThrottle::new(cfg.throttle_ms);
// 审批幂等表:approval_id → 已处理标记(P0 必验项 ②:防 pending→approved 二次点击重复转发)
let mut handled_approvals: std::collections::HashSet<String> = std::collections::HashSet::new();
// 平台注册表(走查 #4):显式 platform → 下标映射,替代硬编码 "feishu"=>1 与
// adapters vec 构建顺序的隐式耦合。新增平台(企微/钉钉)只在此处注册一次。
let mut adapters: Vec<Box<dyn adapters::PlatformAdapter>> = vec![Box::new(web_adapter.clone())];
let mut platform_index: std::collections::HashMap<String, usize> =
std::collections::HashMap::from([("web".to_string(), 0usize)]);
if let Some(fs) = feishu_adapter {
platform_index.insert("feishu".to_string(), adapters.len());
adapters.push(Box::new(fs));
}
let platform_of = |platform_or_conv: &str| -> usize {
// conv_id 形如 "platform:chat_id",前缀即平台;未知平台回退 web(0)
let platform = platform_or_conv.split(':').next().unwrap_or("web");
*platform_index.get(platform).unwrap_or(&0)
};
// 停止信号
let shutdown_ctrl = tokio::signal::ctrl_c();
tokio::pin!(shutdown_ctrl);
let cfg_loop = cfg.clone();
loop {
tokio::select! {
_ = &mut shutdown_ctrl => {
tracing::info!("收到 Ctrl+C,优雅退出");
let _ = shutdown_tx.send(true);
break;
}
// ── TunnelEvent(桌面端 → 网关)──
ev = ev_rx.recv() => {
let Some(ev) = ev else { break };
match ev {
df_tunnel::TunnelEvent::TextDelta { conv_id, delta } => {
// 出站按 conv_id 前缀(platform:chat_id)路由回原平台
let flush = throttle.feed(&conv_id, &delta);
if let Some(text) = flush {
let idx = platform_of(&conv_id);
let _ = adapters[idx]
.dispatch(OutboundAction::StreamText { chat_id: conv_id, text, done: false })
.await;
}
}
df_tunnel::TunnelEvent::Completed { conv_id, .. } => {
let tail = throttle.finish(&conv_id);
let full = tail.unwrap_or_default();
let idx = platform_of(&conv_id);
audit.record(audit::AuditEntry {
direction: audit::Direction::Outbound,
platform: String::new(), chat_id: conv_id.clone(),
user_id: String::new(), conv_id: conv_id.clone(),
action: "completed".into(), request_id: String::new(),
payload: full.clone(), outcome: "ok".into(),
});
let _ = adapters[idx]
.dispatch(OutboundAction::StreamText { chat_id: conv_id, text: full, done: true })
.await;
}
df_tunnel::TunnelEvent::Approval { conv_id, approval_id, description } => {
audit.record(audit::AuditEntry {
direction: audit::Direction::Outbound,
platform: String::new(), chat_id: conv_id.clone(),
user_id: String::new(), conv_id: conv_id.clone(),
action: "approval_card".into(), request_id: approval_id.clone(),
payload: description.clone(), outcome: "ok".into(),
});
let idx = platform_of(&conv_id);
let _ = adapters[idx]
.dispatch(OutboundAction::ApprovalCard {
chat_id: conv_id,
approval_id,
title: "工具审批请求".into(),
detail: description,
})
.await;
}
df_tunnel::TunnelEvent::ToolCall { .. } => {
// P0 骨架不转发 ToolCall 到 IM(卡片噪音),仅日志
tracing::debug!("[core] ToolCall 事件(骨架忽略)");
}
}
}
// ── InboundEvent(IM → 网关)──
inbound = inbound_rx.recv() => {
let Some(ev) = inbound else { break };
match ev {
InboundEvent::Message { platform, chat_id, user_id, text } => {
// 回复适配器:按来源平台回路由
let reply_idx = platform_of(&platform);
if !cfg_loop.whitelist.is_empty() && !cfg_loop.whitelist.contains(&user_id) {
audit.record(audit::AuditEntry {
direction: audit::Direction::Inbound,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: user_id.clone(), conv_id: String::new(),
action: "message".into(), request_id: String::new(),
payload: text.clone(), outcome: "denied".into(),
});
let _ = adapters[reply_idx]
.dispatch(OutboundAction::Notice { chat_id, text: "⛔ 你不在白名单内".into() })
.await;
continue;
}
let Some(channel_id) = cfg_loop.route(&text) else {
audit.record(audit::AuditEntry {
direction: audit::Direction::Inbound,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: user_id.clone(), conv_id: String::new(),
action: "message".into(), request_id: String::new(),
payload: text.clone(), outcome: "error".into(),
});
let _ = adapters[reply_idx]
.dispatch(OutboundAction::Notice { chat_id, text: "⚠️ 无可用 channel".into() })
.await;
continue;
};
// 会话映射:无则新建(conv_id = platform:chat_id,Web 端 chat_id 同源)
let conv_id = match store.get_conv(&platform, &chat_id) {
Some(m) => m.conv_id,
None => {
let conv = format!("{platform}:{chat_id}");
let _ = store.set_conv(&platform, &chat_id, &conv, channel_id);
// 走查 #3:新建会话时清理流式残留(上轮异常中止无
// Completed 事件,streams entry 永驻会错覆写旧消息)
adapters[reply_idx].reset_stream(&chat_id).await;
audit.record(audit::AuditEntry {
direction: audit::Direction::Inbound,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: user_id.clone(), conv_id: conv.clone(),
action: "new_session".into(), request_id: String::new(),
payload: String::new(), outcome: "ok".into(),
});
conv
}
};
// 入站消息审计:request_id 用 conv+毫秒派生,出站 Completed 事件无法
// 回带,此处以 (direction=inbound,message) 为链头,command/outbound
// 侧用同一 conv_id 关联(recent 按时间倒序即还原会话时间线)。
audit.record(audit::AuditEntry {
direction: audit::Direction::Inbound,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: user_id.clone(), conv_id: conv_id.clone(),
action: "message".into(), request_id: String::new(),
payload: text.clone(), outcome: "ok".into(),
});
audit.record(audit::AuditEntry {
direction: audit::Direction::Command,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: user_id.clone(), conv_id: conv_id.clone(),
action: "send_cmd".into(), request_id: String::new(),
payload: text.clone(), outcome: "ok".into(),
});
let _ = cmd_tx
.send(df_tunnel::TunnelCommand::Send { conv_id, content: text })
.await;
}
InboundEvent::NewSession { platform, chat_id } => {
audit.record(audit::AuditEntry {
direction: audit::Direction::Inbound,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: String::new(), conv_id: String::new(),
action: "new_session_reset".into(), request_id: String::new(),
payload: String::new(), outcome: "ok".into(),
});
let _ = store.remove(&platform, &chat_id);
let reply_idx = platform_of(&platform);
let _ = adapters[reply_idx]
.dispatch(OutboundAction::Notice { chat_id, text: "🔄 已重置会话".into() })
.await;
}
InboundEvent::Approval { platform, chat_id, approval_id, approved } => {
// 幂等:同 approval_id 只处理首次(P0 必验项 ②)
if handled_approvals.contains(&approval_id) {
tracing::debug!("[core] 审批 {approval_id} 已处理,忽略重复点击");
continue;
}
handled_approvals.insert(approval_id.clone());
let conv_id = store
.get_conv(&platform, &chat_id)
.map(|m| m.conv_id)
.unwrap_or_else(|| format!("{platform}:{chat_id}"));
audit.record(audit::AuditEntry {
direction: audit::Direction::Inbound,
platform: platform.clone(), chat_id: chat_id.clone(),
user_id: String::new(), conv_id: conv_id.clone(),
action: "approval_click".into(), request_id: approval_id.clone(),
payload: if approved { "approved".into() } else { "rejected".into() },
outcome: "ok".into(),
}); audit.record(audit::AuditEntry {
direction: audit::Direction::Command,
platform, chat_id, user_id: String::new(), conv_id: conv_id.clone(),
action: "approve_cmd".into(), request_id: approval_id.clone(), payload: if approved { "approved".into() } else { "rejected".into() },
outcome: "ok".into(),
});
let _ = cmd_tx
.send(df_tunnel::TunnelCommand::Approve { conv_id, approval_id, accepted: approved })
.await;
}
}
}
}
}
Ok(())
}
fn info_banner(cfg: &GatewayConfig) {
println!("╔══════════════════════════════════════════╗");
println!("║ df-im-gateway v0.1.0 (P0 骨架) ║");
println!("╚══════════════════════════════════════════╝");
println!(" relay : {}", cfg.relay_url);
println!(" channels : {} 条路由", cfg.channels.len());
println!(" throttle : {}ms", cfg.throttle_ms);
println!(" whitelist : {} 用户", cfg.whitelist.len());
if cfg.whitelist.is_empty() {
println!(" ⚠️ 白名单为空:任何人可用本网关(开发默认)。生产部署请设置 GATEWAY_WHITELIST!");
}
println!(" feishu : {}", if cfg.feishu.is_some() { "已配置" } else { "未配置(跳过)" });
println!(" web ws : 0.0.0.0:9191");
}
+151
View File
@@ -0,0 +1,151 @@
//! 会话映射存储(IM chat_id ↔ DevFlow conv_id,SQLite + WAL)
//!
//! 建表即带 owner_user_id/platform 列(P0 预留项 ⑤,当前恒默认值,
//! S2 多人场景免迁表)。
//!
//! 表结构:
//! ```sql
//! CREATE TABLE IF NOT EXISTS session_map (
//! platform TEXT NOT NULL, -- feishu/wecom/dingtalk/web
//! chat_id TEXT NOT NULL, -- IM 侧会话(单聊 user_id / 群聊 chat_id)
//! conv_id TEXT NOT NULL, -- DevFlow 会话 ID
//! owner_user_id TEXT NOT NULL DEFAULT '',-- 预留 S2 多人归属(当前恒 '')
//! channel_id TEXT NOT NULL DEFAULT '',-- 路由到的桌面端 channel(预留项 ⑥)
//! created_at INTEGER NOT NULL,
//! updated_at INTEGER NOT NULL,
//! PRIMARY KEY (platform, chat_id)
//! );
//! ```
use std::path::Path;
use std::sync::Arc;
use rusqlite::Connection;
/// 单条会话映射(字段预留 S2 多人/多桌面端,当前核心只读 conv_id)
#[derive(Debug, Clone)]
#[allow(dead_code)]
pub struct SessionMap {
pub platform: String,
pub chat_id: String,
pub conv_id: String,
pub owner_user_id: String,
pub channel_id: String,
}
#[derive(Clone)]
pub struct SessionStore {
conn: Arc<std::sync::Mutex<Connection>>,
}
impl SessionStore {
/// 打开(或创建)库文件并确保表存在。WAL 模式(与桌面端一致)。
pub fn open(path: &Path) -> anyhow::Result<Self> {
let conn = Connection::open(path)?;
conn.pragma_update(None, "journal_mode", "WAL")?;
conn.execute_batch(
"CREATE TABLE IF NOT EXISTS session_map (
platform TEXT NOT NULL,
chat_id TEXT NOT NULL,
conv_id TEXT NOT NULL,
owner_user_id TEXT NOT NULL DEFAULT '',
channel_id TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
PRIMARY KEY (platform, chat_id)
);",
)?;
Ok(Self { conn: Arc::new(std::sync::Mutex::new(conn)) })
}
/// 内存库(测试/演示)
#[allow(dead_code)]
pub fn open_in_memory() -> anyhow::Result<Self> {
let conn = Connection::open_in_memory()?;
conn.execute_batch(
"CREATE TABLE IF NOT EXISTS session_map (
platform TEXT NOT NULL, chat_id TEXT NOT NULL, conv_id TEXT NOT NULL,
owner_user_id TEXT NOT NULL DEFAULT '', channel_id TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL, updated_at INTEGER NOT NULL,
PRIMARY KEY (platform, chat_id)
);",
)?;
Ok(Self { conn: Arc::new(std::sync::Mutex::new(conn)) })
}
/// 查 IM 会话映射的 conv_id(未映射返回 None)
pub fn get_conv(&self, platform: &str, chat_id: &str) -> Option<SessionMap> {
let conn = self.conn.lock().ok()?;
conn.query_row(
"SELECT platform, chat_id, conv_id, owner_user_id, channel_id FROM session_map \
WHERE platform = ?1 AND chat_id = ?2",
[platform, chat_id],
|row| {
Ok(SessionMap {
platform: row.get(0)?,
chat_id: row.get(1)?,
conv_id: row.get(2)?,
owner_user_id: row.get(3)?,
channel_id: row.get(4)?,
})
},
)
.ok()
}
/// 写/更新映射(upsert)
pub fn set_conv(&self, platform: &str, chat_id: &str, conv_id: &str, channel_id: &str) -> anyhow::Result<()> {
let conn = self.conn.lock().map_err(|e| anyhow::anyhow!("store 锁中毒: {e}"))?;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis() as i64)
.unwrap_or(0);
conn.execute(
"INSERT INTO session_map (platform, chat_id, conv_id, owner_user_id, channel_id, created_at, updated_at) \
VALUES (?1, ?2, ?3, '', ?4, ?5, ?5) \
ON CONFLICT(platform, chat_id) DO UPDATE SET conv_id = ?3, channel_id = ?4, updated_at = ?5",
rusqlite::params![platform, chat_id, conv_id, channel_id, now],
)?;
Ok(())
}
/// 删除映射(桌面端会话已删,悬空清理)
pub fn remove(&self, platform: &str, chat_id: &str) -> anyhow::Result<()> {
let conn = self.conn.lock().map_err(|e| anyhow::anyhow!("store 锁中毒: {e}"))?;
conn.execute("DELETE FROM session_map WHERE platform = ?1 AND chat_id = ?2", [platform, chat_id])?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn upsert_and_get() {
let store = SessionStore::open_in_memory().unwrap();
store.set_conv("feishu", "oc_123", "conv_abc", "chan-work").unwrap();
let m = store.get_conv("feishu", "oc_123").unwrap();
assert_eq!(m.conv_id, "conv_abc");
assert_eq!(m.channel_id, "chan-work");
assert_eq!(m.owner_user_id, "");
// upsert 覆盖
store.set_conv("feishu", "oc_123", "conv_new", "chan-home").unwrap();
let m = store.get_conv("feishu", "oc_123").unwrap();
assert_eq!(m.conv_id, "conv_new");
// 删除后悬空
store.remove("feishu", "oc_123").unwrap();
assert!(store.get_conv("feishu", "oc_123").is_none());
}
#[test]
fn platform_isolated() {
let store = SessionStore::open_in_memory().unwrap();
store.set_conv("feishu", "c1", "conv1", "ch").unwrap();
store.set_conv("wecom", "c1", "conv2", "ch").unwrap();
assert_eq!(store.get_conv("feishu", "c1").unwrap().conv_id, "conv1");
assert_eq!(store.get_conv("wecom", "c1").unwrap().conv_id, "conv2");
}
}
+103
View File
@@ -0,0 +1,103 @@
//! TextDelta 节流聚合器(IM 频控防护)
//!
//! IM 平台消息 API 频控(飞书 ~50 QPS 应用级、企微 200 次/分钟),逐 delta 刷卡片必被限流。
//! 策略:per-conv 聚合缓冲 + 时间窗(默认 800ms)批量 flush;Completed 时强制冲刷尾包。
//!
//! 设计要点:
//! - 每个 key 一个缓冲,互不干扰
//! - 纯函数式:feed() 返回 Option<全量文本>(窗口到期才返回),不持有回调/不 await
//! - flush 语义:全量文本(IM 卡片更新是「全量内容」语义,非增量)
//! - finish() 冲刷尾包并清缓冲;abort() 丢弃缓冲(错误/停止,不输出残文)
use std::collections::HashMap;
use std::time::{Duration, Instant};
/// 单 key 缓冲
struct ConvBuffer {
/// 累积全量文本
full_text: String,
/// 上次 flush 时刻
last_flush: Instant,
}
/// 节流聚合器
pub struct DeltaThrottle {
window: Duration,
buffers: HashMap<String, ConvBuffer>,
}
impl DeltaThrottle {
pub fn new(window_ms: u64) -> Self {
Self { window: Duration::from_millis(window_ms), buffers: HashMap::new() }
}
/// 喂入 delta。窗口到期则返回 Some(全量文本)(调用方负责外发),否则 None。
pub fn feed(&mut self, key: &str, delta: &str) -> Option<String> {
let now = Instant::now();
let buf = self.buffers.entry(key.to_string()).or_insert_with(|| ConvBuffer {
full_text: String::new(),
last_flush: now,
});
buf.full_text.push_str(delta);
if buf.last_flush.elapsed() >= self.window {
buf.last_flush = now;
Some(buf.full_text.clone())
} else {
None
}
}
/// 会话完成:强制冲刷尾包 + 清缓冲。返回 Some(全量文本) 或(空会话)None。
pub fn finish(&mut self, key: &str) -> Option<String> {
self.buffers.remove(key).map(|buf| buf.full_text)
}
/// 丢弃缓冲(错误/停止,不输出残文)
///
/// 注:当前 TunnelEvent 无 Error/Stopped 变体,核心循环暂无调用点;
/// df-tunnel Phase2 扩展后接入。
#[allow(dead_code)]
pub fn abort(&mut self, key: &str) {
self.buffers.remove(key);
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn throttle_window_batch() {
let mut t = DeltaThrottle::new(60_000); // 60s 窗口:模拟窗口永不到期
assert_eq!(t.feed("k", "你"), None);
assert_eq!(t.feed("k", "好"), None);
assert_eq!(t.feed("k", "!"), None);
assert_eq!(t.finish("k"), Some("你好!".to_string()), "finish 冲刷全量");
}
#[test]
fn window_elapsed_flushes() {
let mut t = DeltaThrottle::new(0); // 0ms 窗口:每次 feed 都到期
assert_eq!(t.feed("k", "a"), Some("a".to_string()));
assert_eq!(t.feed("k", "b"), Some("ab".to_string()));
// 缓冲仍在 finish 前的最后 flush 版本;finish 拿走剩余
assert_eq!(t.finish("k"), Some("ab".to_string()));
}
#[test]
fn abort_drops() {
let mut t = DeltaThrottle::new(60_000);
t.feed("k", "残文");
t.abort("k");
assert_eq!(t.finish("k"), None, "abort 后缓冲不存在");
}
#[test]
fn keys_isolated() {
let mut t = DeltaThrottle::new(60_000);
t.feed("a", "1");
t.feed("b", "2");
assert_eq!(t.finish("a"), Some("1".to_string()));
assert_eq!(t.finish("b"), Some("2".to_string()));
}
}
+398
View File
@@ -0,0 +1,398 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<title>DevFlow AI Chat</title>
<style>
:root {
--bg: #0f1115; --panel: #171a21; --panel2: #1d212b; --border: #2a2f3a;
--text: #e6e9ef; --muted: #8a93a6; --accent: #4f8cff; --accent2: #3a6fd8;
--ok: #3fb96f; --warn: #e5a03c; --err: #e05252; --radius: 10px;
}
* { margin: 0; padding: 0; box-sizing: border-box; -webkit-tap-highlight-color: transparent; }
html, body { height: 100%; }
body {
background: var(--bg); color: var(--text);
font: 14px/1.6 -apple-system, "PingFang SC", "Microsoft YaHei", sans-serif;
display: flex; overflow: hidden;
}
/* ── 侧栏:会话列表 ── */
#sidebar {
width: 220px; min-width: 220px; background: var(--panel);
border-right: 1px solid var(--border); display: flex; flex-direction: column;
}
#sidebar header { padding: 14px 14px 8px; font-weight: 600; font-size: 13px; color: var(--muted); letter-spacing: .05em; }
#conv-list { flex: 1; overflow-y: auto; padding: 4px 8px; }
.conv-item {
padding: 8px 10px; border-radius: 8px; cursor: pointer; margin-bottom: 2px;
display: flex; align-items: center; gap: 8px; color: var(--muted);
}
.conv-item:hover { background: var(--panel2); }
.conv-item.active { background: var(--panel2); color: var(--text); }
.conv-item .dot { width: 7px; height: 7px; border-radius: 50%; background: var(--accent); flex: none; }
.conv-item .name { flex: 1; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; font-size: 13px; }
.conv-item .del { opacity: 0; color: var(--muted); font-size: 12px; padding: 0 2px; }
.conv-item:hover .del { opacity: 1; }
.conv-item .del:hover { color: var(--err); }
#sidebar footer { padding: 10px; border-top: 1px solid var(--border); display: flex; gap: 8px; }
/* ── 主区 ── */
#main { flex: 1; display: flex; flex-direction: column; min-width: 0; }
#topbar {
padding: 10px 16px; border-bottom: 1px solid var(--border); background: var(--panel);
display: flex; align-items: center; gap: 10px; font-size: 13px;
}
#conn-dot { width: 8px; height: 8px; border-radius: 50%; background: var(--err); flex: none; }
#conn-dot.on { background: var(--ok); }
#conn-label { color: var(--muted); }
#topbar .spacer { flex: 1; }
#topbar button {
background: var(--panel2); color: var(--text); border: 1px solid var(--border);
border-radius: 6px; padding: 4px 10px; cursor: pointer; font-size: 12px;
}
#topbar button:hover { border-color: var(--accent); }
#msgs { flex: 1; overflow-y: auto; padding: 18px 16px; scroll-behavior: smooth; }
.msg { margin-bottom: 14px; display: flex; }
.msg.user { justify-content: flex-end; }
.msg .bubble {
max-width: 78%; padding: 9px 13px; border-radius: var(--radius);
background: var(--panel2); white-space: pre-wrap; word-break: break-word;
}
.msg.user .bubble { background: var(--accent2); }
.msg.ai .bubble { background: var(--panel2); }
.msg .bubble.sys { background: transparent; color: var(--muted); font-size: 12px; max-width: 100%; }
.approval-card {
border: 1px solid var(--warn); border-radius: var(--radius); padding: 10px 13px;
max-width: 78%; background: rgba(229,160,60,.08);
}
.approval-card .t { color: var(--warn); font-weight: 600; font-size: 13px; margin-bottom: 4px; }
.approval-card .d { color: var(--muted); font-size: 12px; margin-bottom: 9px; white-space: pre-wrap; word-break: break-word; }
.approval-card .row { display: flex; gap: 8px; }
.approval-card button {
flex: 1; padding: 6px 0; border-radius: 6px; border: none; cursor: pointer; font-size: 13px;
}
.approval-card .yes { background: var(--ok); color: #fff; }
.approval-card .no { background: var(--panel2); color: var(--muted); border: 1px solid var(--border); }
.approval-card.answered { opacity: .55; }
.approval-card.answered .row { display: none; }
.approval-card .verdict { font-size: 12px; color: var(--muted); }
/* ── 输入区 ── */
#inputbar { display: flex; gap: 8px; padding: 12px 14px; border-top: 1px solid var(--border); background: var(--panel); }
#inp {
flex: 1; background: var(--panel2); border: 1px solid var(--border); color: var(--text);
border-radius: 8px; padding: 9px 12px; font: inherit; resize: none; outline: none; height: 44px;
}
#inp:focus { border-color: var(--accent); }
#send, #stop {
border: none; border-radius: 8px; padding: 0 18px; cursor: pointer; font: inherit; font-size: 13px;
}
#send { background: var(--accent); color: #fff; }
#send:disabled { opacity: .45; cursor: not-allowed; }
#stop { background: var(--panel2); color: var(--err); border: 1px solid var(--border); display: none; }
@media (max-width: 640px) {
#sidebar { position: fixed; z-index: 10; height: 100%; transform: translateX(-100%); transition: transform .2s; }
#sidebar.open { transform: translateX(0); box-shadow: 0 0 30px rgba(0,0,0,.5); }
#menu-btn { display: inline-block !important; }
.msg .bubble, .approval-card { max-width: 92%; }
}
#menu-btn { display: none; background: none; border: none; color: var(--text); font-size: 18px; cursor: pointer; }
/* 简单 markdown 粗体/代码 */
.bubble code { background: rgba(79,140,255,.12); border-radius: 4px; padding: 1px 5px; font-size: 12.5px; }
</style>
</head>
<body>
<nav id="sidebar">
<header>会话</header>
<div id="conv-list"></div>
<footer><button id="new-conv" style="flex:1;background:var(--panel2);border:1px solid var(--border);color:var(--text);border-radius:6px;padding:6px;cursor:pointer">+ 新会话</button></footer>
</nav>
<div id="main">
<div id="topbar">
<button id="menu-btn">☰</button>
<span id="conn-dot"></span><span id="conn-label">连接中…</span>
<span class="spacer"></span>
<button id="clear-btn">🗑 清空当前会话记录</button>
</div>
<div id="msgs"></div>
<div id="inputbar">
<textarea id="inp" placeholder="发消息给 DevFlow AI…(Enter 发送 / Shift+Enter 换行)"></textarea>
<button id="send">发送</button>
<button id="stop">停止</button>
</div>
</div>
<script>
'use strict';
/* ===== 常量与状态 ===== */
var LS_KEY = 'devflow.webchat.v1'; // localStorage: {convs:[{id,name}], activeId}
var state = {
convs: [], // [{id, name, msgs:[{role:'user'|'ai'|'sys', text, ts}], pending:{approval_id, title, detail, el} }]
activeId: null, // 当前会话 chat_id
ws: null, wsReady: false, reconnectTimer: null, closedByUs: false,
generating: false, // 当前会话生成中(流式未完)
};
var $ = function (s) { return document.querySelector(s); };
var msgsEl = $('#msgs'), inpEl = $('#inp'), sendBtn = $('#send'), stopBtn = $('#stop');
/* ===== 本地持久化 ===== */
function loadStore() {
try { state.convs = JSON.parse(localStorage.getItem(LS_KEY) || '[]'); } catch (e) { state.convs = []; }
}
function saveStore() {
try { localStorage.setItem(LS_KEY, JSON.stringify(state.convs)); } catch (e) { /* 配额满忽略 */ }
}
function activeConv() {
return state.convs.find(function (c) { return c.id === state.activeId; }) || null;
}
/* ===== 会话管理 ===== */
function newConvId() { return 'web-' + Date.now().toString(36) + '-' + Math.random().toString(36).slice(2, 8); }
function newConv(activate) {
var id = newConvId();
var c = { id: id, name: '新会话', msgs: [], pending: null };
state.convs.unshift(c);
if (activate) switchConv(id);
saveStore(); renderConvList();
return c;
}
function switchConv(id) {
if (state.activeId === id) { closeSidebar(); return; }
state.activeId = id;
saveStore(); renderConvList(); renderMsgs(); closeSidebar();
// WS 已就绪则重绑到新 chat_id(断开重连,首帧 hello 携带新 id)
if (state.wsReady) reconnect();
}
function deleteConv(id) {
var idx = state.convs.findIndex(function (c) { return c.id === id; });
if (idx < 0) return;
state.convs.splice(idx, 1);
if (state.activeId === id) {
state.activeId = state.convs.length ? state.convs[0].id : null;
if (!state.activeId) newConv(false);
renderMsgs();
if (state.wsReady) reconnect();
}
saveStore(); renderConvList();
}
function newSessionOnServer() {
var c = activeConv();
if (c && state.wsReady) wsSend({ type: 'new_session' });
}
function ensureConv() { return activeConv() || newConv(true); }
/* ===== 渲染 ===== */
function esc(s) {
return String(s).replace(/[&<>"']/g, function (m) {
return { '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' }[m];
});
}
function fmtText(s) {
// 轻量格式化:转义后 `code` 行内代码
return esc(s).replace(/`([^`\n]+)`/g, '<code>$1</code>');
}
function renderConvList() {
var el = $('#conv-list'); el.innerHTML = '';
state.convs.forEach(function (c) {
var d = document.createElement('div');
d.className = 'conv-item' + (c.id === state.activeId ? ' active' : '');
d.innerHTML = '<span class="dot"></span><span class="name">' + esc(c.name) + '</span><span class="del" title="删除">✕</span>';
d.onclick = function (e) {
if (e.target.classList.contains('del')) {
if (confirm('删除会话「' + c.name + '」?(仅删除本页记录,桌面端对话保留)')) deleteConv(c.id);
return;
}
switchConv(c.id);
};
el.appendChild(d);
});
}
function msgNode(m) {
if (m.role === 'sys') {
var d = document.createElement('div'); d.className = 'msg';
var b = document.createElement('div'); b.className = 'bubble sys'; b.innerHTML = fmtText(m.text);
d.appendChild(b); return d;
}
if (m.role === 'approval') {
var wrap = document.createElement('div'); wrap.className = 'msg';
var card = document.createElement('div'); card.className = 'approval-card';
card.innerHTML = '<div class="t">🛡 ' + esc(m.title || '工具审批请求') + '</div>'
+ '<div class="d">' + fmtText(m.detail || '') + '</div>'
+ '<div class="row"><button class="yes">同意</button><button class="no">拒绝</button></div>';
var done = function (approved) {
card.classList.add('answered');
var v = document.createElement('div'); v.className = 'verdict';
v.textContent = (approved ? '✓ 已同意' : '✕ 已拒绝') + '(未知新帧到达前以此为准)';
card.appendChild(v);
wsSend({ type: 'approval', approval_id: m.approval_id, approved: approved });
};
card.querySelector('.yes').onclick = function () { done(true); };
card.querySelector('.no').onclick = function () { done(false); };
if (m.answered) { card.classList.add('answered'); var v = document.createElement('div'); v.className = 'verdict'; v.textContent = m.answered === 'yes' ? '✓ 已同意' : '✕ 已拒绝'; card.appendChild(v); }
wrap.appendChild(card); return wrap;
}
var d = document.createElement('div'); d.className = 'msg ' + (m.role === 'user' ? 'user' : 'ai');
var b = document.createElement('div'); b.className = 'bubble'; b.innerHTML = fmtText(m.text || '');
d.appendChild(b); return d;
}
function renderMsgs() {
msgsEl.innerHTML = '';
var c = activeConv(); if (!c) return;
c.msgs.forEach(function (m) { msgsEl.appendChild(msgNode(m)); });
scrollBottom();
}
function scrollBottom() { msgsEl.scrollTop = msgsEl.scrollHeight; }
/* ===== WS 连接 ===== */
function wsSend(obj) {
if (state.ws && state.ws.readyState === 1) { state.ws.send(JSON.stringify(obj)); return true; }
return false;
}
function connect() {
clearTimeout(state.reconnectTimer);
var c = ensureConv();
state.closedByUs = false;
var proto = location.protocol === 'https:' ? 'wss' : 'ws';
var ws;
try { ws = new WebSocket(proto + '://' + location.host + '/ws'); } catch (e) { scheduleReconnect(); return; }
state.ws = ws;
ws.onopen = function () {
ws.send(JSON.stringify({ type: 'hello', chat_id: state.activeId, user_id: 'web-user' }));
setConn(true);
};
ws.onmessage = function (e) {
var v; try { v = JSON.parse(e.data); } catch (err) { return; }
handleFrame(v);
};
ws.onclose = function () { setConn(false); if (!state.closedByUs) scheduleReconnect(); };
ws.onerror = function () { try { ws.close(); } catch (err2) {} };
}
function reconnect() {
state.closedByUs = true;
try { state.ws && state.ws.close(); } catch (e) {}
setConn(false);
setTimeout(connect, 60);
}
function scheduleReconnect() {
setConn(false);
clearTimeout(state.reconnectTimer);
state.reconnectTimer = setTimeout(connect, 2500);
}
function setConn(on) {
state.wsReady = on;
$('#conn-dot').className = on ? 'on' : '';
$('#conn-label').textContent = on ? '已连接' : '重连中…(桌面端或网关未在线)';
}
/* ===== 帧处理(网关 → 浏览器) ===== */
function handleFrame(v) {
var c = activeConv(); if (!c) return;
switch (v.type) {
case 'stream_delta': {
var m = lastAiStreaming(c);
if (!m) { m = { role: 'ai', text: '' }; c.msgs.push(m); setGenerating(true); }
m.text = v.text || ''; // 网关推的是全量聚合文本,直接覆盖
appendLive(m);
break;
}
case 'stream_done': {
var m2 = lastAiStreaming(c);
if (m2) { m2.text = v.text || m2.text || ''; m2.done = true; }
else { c.msgs.push({ role: 'ai', text: v.text || '', done: true }); }
setGenerating(false);
renderMsgs();
break;
}
case 'approval': {
var ap = { role: 'approval', approval_id: v.approval_id, title: v.title, detail: v.detail };
c.msgs.push(ap); setGenerating(true); // 等审批期间视为生成中(停止按钮可用)
renderMsgs();
break;
}
case 'notice': {
c.msgs.push({ role: 'sys', text: v.text || '' });
renderMsgs();
break;
}
}
saveStore();
}
function lastAiStreaming(c) {
for (var i = c.msgs.length - 1; i >= 0; i--) {
var m = c.msgs[i];
if (m.role === 'ai' && !m.done) return m;
if (m.role === 'ai') return null; // 最新 ai 消息已完成 → 开新条
}
return null;
}
var liveEl = null;
function appendLive(m) {
if (!liveEl || !liveEl._m) {
// 首个 delta:占位气泡(增量期间只更新最后一条,不全量重渲染,防抖动)
msgsEl.appendChild(msgNode(m));
liveEl = msgsEl.lastChild;
liveEl._m = m;
}
liveEl.lastChild.innerHTML = fmtText(m.text);
scrollBottom();
}
/* ===== 生成态(发送/停止按钮) ===== */
function setGenerating(on) {
state.generating = on;
sendBtn.style.display = on ? 'none' : '';
stopBtn.style.display = on ? '' : 'none';
sendBtn.disabled = on;
}
/* ===== 发送/停止 ===== */
function doSend() {
var text = inpEl.value.trim(); if (!text) return;
var c = ensureConv();
if (c.msgs.length === 0) { c.name = text.slice(0, 18); renderConvList(); } // 首条消息作会话名
c.msgs.push({ role: 'user', text: text });
saveStore(); renderMsgs();
inpEl.value = ''; inpEl.style.height = 'auto';
if (!wsSend({ type: 'message', text: text })) {
c.msgs.push({ role: 'sys', text: '⚠ 未连接,消息未发出(连接恢复后请重发)' }); renderMsgs();
}
}
sendBtn.onclick = doSend;
stopBtn.onclick = function () {
var c = activeConv();
// 本地止血:标记完成 + 提示(协议无 Stop 指令,桌面端继续跑完本轮,网关会继续推帧,本页忽略)
if (c) {
var m = lastAiStreaming(c);
if (m) m.done = true;
c.msgs.push({ role: 'sys', text: '■ 已停止显示(桌面端本轮可能仍在执行,结果以桌面端为准)' });
saveStore(); renderMsgs();
}
setGenerating(false);
};
/* ===== 输入框 ===== */
inpEl.addEventListener('keydown', function (e) {
if (e.key === 'Enter' && !e.shiftKey) { e.preventDefault(); doSend(); }
});
inpEl.addEventListener('input', function () {
inpEl.style.height = 'auto';
inpEl.style.height = Math.min(inpEl.scrollHeight, 120) + 'px';
});
/* ===== 顶栏按钮 ===== */
$('#new-conv').onclick = function () { newConv(true); };
$('#clear-btn').onclick = function () {
var c = activeConv(); if (!c) return;
if (!confirm('清空当前会话的页面记录?(不删桌面端对话)')) return;
c.msgs = []; saveStore(); renderMsgs();
};
$('#menu-btn').onclick = function () { $('#sidebar').classList.toggle('open'); };
function closeSidebar() { $('#sidebar').classList.remove('open'); }
/* ===== 启动 ===== */
loadStore();
if (!state.convs.length) newConv(false);
if (!state.activeId) state.activeId = state.convs[0].id;
renderConvList(); renderMsgs();
connect();
</script>
</body>
</html>
+1 -1
View File
@@ -17,4 +17,4 @@ pub mod server;
pub mod server_http; pub mod server_http;
pub mod tools; pub mod tools;
pub use server::{run_server, ServerConfig}; pub use server::{run_server, ServerConfig, MCP_DATA_CHANGED_KEY};
+103 -10
View File
@@ -14,6 +14,7 @@ use std::path::Path;
use std::sync::Arc; use std::sync::Arc;
use std::time::Duration; use std::time::Duration;
use df_storage::crud::SettingsRepo;
use df_storage::db::Database; use df_storage::db::Database;
use serde_json::{json, Value}; use serde_json::{json, Value};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader}; use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
@@ -29,6 +30,15 @@ pub(crate) const PROTOCOL_VERSION: &str = "2025-06-18";
pub(crate) const SERVER_NAME: &str = "devflow-mcp"; pub(crate) const SERVER_NAME: &str = "devflow-mcp";
pub(crate) const SERVER_VERSION: &str = env!("CARGO_PKG_VERSION"); pub(crate) const SERVER_VERSION: &str = env!("CARGO_PKG_VERSION");
/// app_settings KV key:stdio MCP 写库后落的变更标记,供 GUI 侧轮询兜底消费。
///
/// 独立进程形态的 stdio MCP 无 AppHandle(`on_write_call=None`),写库后 GUI 无从经
/// `df-data-changed` 事件感知,造成双端数据断层(P0-2)。此标记是跨进程的落库信令:
/// 每次成功写工具覆盖写入,值为 `{"at": <毫秒时间戳>, "tool": "<工具名>"}`。
/// GUI 侧轮询读此 key,`at` 变化即知库被外部改动;再按 `tool` 复用 df-data-changed
/// 的 entity 映射刷新对应列表(entity 映射留在 GUI 侧,避免本 crate 引入 df-ai 依赖)。
pub const MCP_DATA_CHANGED_KEY: &str = "mcp.data_changed";
/// MCP server 运行配置。 /// MCP server 运行配置。
/// ///
/// 聚合 stdio 形态的运行参数,避免 `run_server` 参数膨胀。 /// 聚合 stdio 形态的运行参数,避免 `run_server` 参数膨胀。
@@ -41,7 +51,8 @@ pub struct ServerConfig {
pub idle_timeout: Option<Duration>, pub idle_timeout: Option<Duration>,
/// 写操作回调:成功执行(未被 read-only/High 拒绝)的写工具(risk != Low)触发。 /// 写操作回调:成功执行(未被 read-only/High 拒绝)的写工具(risk != Low)触发。
/// 桌面内嵌形态经 server_http → on_tool_call → emit df-data-changed 刷新 GUI; /// 桌面内嵌形态经 server_http → on_tool_call → emit df-data-changed 刷新 GUI;
/// stdio 独立进程暂无 AppHandle,传 None 预留接线点,后续可注入。 /// stdio 独立进程暂无 AppHandle,传 None 预留接线点,后续可注入。传 None 时由
/// 落库变更标记([`MCP_DATA_CHANGED_KEY`])兜底,供 GUI 侧轮询消费。
pub on_write_call: Option<Arc<dyn Fn(&str) + Send + Sync>>, pub on_write_call: Option<Arc<dyn Fn(&str) + Send + Sync>>,
} }
@@ -154,8 +165,13 @@ where
if let Some(name) = tool_name { if let Some(name) = tool_name {
// MC-5仅业务成功才触发写回调——防业务失败(如 update_project 非法 status、 // MC-5仅业务成功才触发写回调——防业务失败(如 update_project 非法 status、
// create_project 空名等 result.isError=true)仍假触发 df-data-changed。 // create_project 空名等 result.isError=true)仍假触发 df-data-changed。
if is_success_tool_call(&resp) { if is_success_tool_call(&resp) && is_executed_write(&name, config.read_only) {
fire_write_hook(config, &name); // 多进程兜底:stdio 形态无 AppHandle,先落库内变更标记,供 GUI 侧轮询消费
// (即便未注入回调也不丢变更感知);再触发进程内回调(HTTP 内嵌形态走事件)。
record_data_change(ctx, &name).await;
if let Some(cb) = &config.on_write_call {
cb(&name);
}
} }
} }
} }
@@ -293,13 +309,30 @@ fn required_names(schema: &Value) -> Vec<String> {
} }
} }
/// 触发写操作回调(若有)。仅当工具为写操作(risk != Low)且未被 read-only/High 拒绝时触发, /// 判定工具是否属于「会执行的成功写操作」:risk != Low 且未被 read-only/High 拒绝。
/// 与 dispatch 的执行判定一致。回调仅作通知(如 GUI 刷新),不承载返回结果。 ///
fn fire_write_hook(config: &ServerConfig, name: &str) { /// 变更标记与进程内写回调共用此判定,保证两通道对「哪些写操作需通知 GUI」语义一致;
let Some(cb) = &config.on_write_call else { return }; /// 与 dispatch 的执行判定(`should_execute`)同源,避免逻辑漂移。
let Some(spec) = tools::find(name) else { return }; fn is_executed_write(name: &str, read_only: bool) -> bool {
if spec.risk != RiskLevel::Low && should_execute(config.read_only, spec.risk) { match tools::find(name) {
cb(name); Some(spec) => spec.risk != RiskLevel::Low && should_execute(read_only, spec.risk),
None => false,
}
}
/// 写成功后向 [`MCP_DATA_CHANGED_KEY`] 覆盖写入一条变更标记(多进程兜底信令)。
///
/// 独立进程形态完成写库后,GUI 无法经事件感知;此标记是落库的持久信令,GUI 轮询读
/// 该 key、`at` 变化即知库被外部改动。写失败仅告警不阻断(工具调用已成功,下次成功
/// 写会再次覆盖,标记只作「有变更」的粗粒度提示,非精确事件队列)。
async fn record_data_change(ctx: &Ctx, tool_name: &str) {
let at = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_millis())
.unwrap_or(0);
let value = json!({ "at": at, "tool": tool_name }).to_string();
if let Err(e) = SettingsRepo::new(&ctx.db).set(MCP_DATA_CHANGED_KEY, &value).await {
tracing::warn!(target: "df_mcp", tool = tool_name, err = %e, "写变更标记失败(不改工具调用结果)");
} }
} }
@@ -675,4 +708,64 @@ mod tests {
*calls.lock().unwrap() *calls.lock().unwrap()
); );
} }
// ── 多进程兜底:stdio 写库后落变更标记 ────────────────────────────
#[tokio::test]
async fn main_loop_records_data_change_marker_for_write_tool() {
// stdio 形态(on_write_call=None)成功写工具后应向 KV 落变更标记,
// 供 GUI 轮询消费——即便无任何回调注入也不丢变更感知(P0-2 兜底)。
let ctx = test_ctx().await;
let (mut tx, rx) = tokio::io::duplex(1024);
let line = r#"{"jsonrpc":"2.0","id":11,"method":"tools/call","params":{"name":"create_project","arguments":{"name":"MarkProj","description":"d"}}}"#;
tx.write_all(format!("{line}\n").as_bytes()).await.unwrap();
drop(tx);
main_loop(rx, tokio::io::sink(), &ctx, &test_config(false).await)
.await
.unwrap();
let v = SettingsRepo::new(&ctx.db)
.get(MCP_DATA_CHANGED_KEY)
.await
.unwrap()
.expect("写工具成功后应落变更标记");
let parsed: Value = serde_json::from_str(&v).unwrap();
assert_eq!(parsed["tool"], "create_project");
assert!(parsed["at"].as_u64().unwrap() > 0, "标记应含毫秒时间戳: {v}");
}
#[tokio::test]
async fn main_loop_no_data_change_marker_for_read_tool() {
// 只读工具无副作用,不应落变更标记(否则 GUI 无谓刷新)。
let ctx = test_ctx().await;
let (mut tx, rx) = tokio::io::duplex(1024);
let line = r#"{"jsonrpc":"2.0","id":12,"method":"tools/call","params":{"name":"list_projects","arguments":{}}}"#;
tx.write_all(format!("{line}\n").as_bytes()).await.unwrap();
drop(tx);
main_loop(rx, tokio::io::sink(), &ctx, &test_config(false).await)
.await
.unwrap();
let v = SettingsRepo::new(&ctx.db)
.get(MCP_DATA_CHANGED_KEY)
.await
.unwrap();
assert!(v.is_none(), "只读工具不应落变更标记,实际: {v:?}");
}
#[tokio::test]
async fn main_loop_no_data_change_marker_when_read_only_denies_write() {
// read-only 下写被拒(未真正写库),不应落变更标记。
let ctx = test_ctx().await;
let (mut tx, rx) = tokio::io::duplex(1024);
let line = r#"{"jsonrpc":"2.0","id":13,"method":"tools/call","params":{"name":"create_project","arguments":{"name":"X","description":"d"}}}"#;
tx.write_all(format!("{line}\n").as_bytes()).await.unwrap();
drop(tx);
main_loop(rx, tokio::io::sink(), &ctx, &test_config(true).await)
.await
.unwrap();
let v = SettingsRepo::new(&ctx.db)
.get(MCP_DATA_CHANGED_KEY)
.await
.unwrap();
assert!(v.is_none(), "read-only 下写被拒不应落变更标记,实际: {v:?}");
}
} }
+58 -1
View File
@@ -1,4 +1,5 @@
//! 工具暴露层 — 从 df-storage Repo 复用 CRUD,转 MCP Tool schema //! 工具暴露层 — 从 df-storage Repo 复用 CRUD,转 MCP Tool schema
//! 工具暴露层 — 从 df-storage Repo 复用 CRUD,转 MCP Tool schema
//! //!
//! 不重复实现数据层逻辑,只做:① 工具元数据(name/description/inputSchema)声明; //! 不重复实现数据层逻辑,只做:① 工具元数据(name/description/inputSchema)声明;
//! ② 工具调用 → df-storage Repo 方法 → 序列化为 MCP CallToolResult。 //! ② 工具调用 → df-storage Repo 方法 → 序列化为 MCP CallToolResult。
@@ -379,6 +380,7 @@ fn create_project(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult>
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -434,6 +436,7 @@ fn update_project(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult>
idea_id: existing.idea_id, idea_id: existing.idea_id,
path: existing.path, path: existing.path,
stack: existing.stack, stack: existing.stack,
archived: existing.archived,
created_at: existing.created_at, created_at: existing.created_at,
updated_at: now, updated_at: now,
}; };
@@ -544,9 +547,18 @@ fn list_tasks(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult> {
Box::pin(async move { Box::pin(async move {
let repo = TaskRepo::new(&db); let repo = TaskRepo::new(&db);
// 分页:取 limit+1 条探测是否有下一页(has_more),再截断到 limit。 // 分页:取 limit+1 条探测是否有下一页(has_more),再截断到 limit。
// 缺陷修复(1eff911c,登记于 2026-08-08):无 status 过滤时默认排除 done/cancelled——
// 终态任务是存量大头(本库 260+ 条),不过滤会挤占分页窗口,活跃任务被
// created_at DESC + limit 截断挤出视图(表象=「过滤失效/他项目任务混入」)。
// 显式传 status(含 done)不干预,保留查终态能力。
let query = TaskQuery { let query = TaskQuery {
project_id: project_id_filter, project_id: project_id_filter,
status: status_filter, status: status_filter.clone(),
status_not_in: if status_filter.is_some() {
None
} else {
Some(vec!["done".to_string(), "cancelled".to_string()])
},
limit: Some(limit + 1), limit: Some(limit + 1),
offset: Some(offset), offset: Some(offset),
..Default::default() ..Default::default()
@@ -644,6 +656,7 @@ fn create_task(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult> {
queue, queue,
parent_id, parent_id,
content_json, content_json,
archived: false,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -705,6 +718,8 @@ fn update_task(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult> {
parent_id: existing.parent_id, parent_id: existing.parent_id,
module_id: existing.module_id, module_id: existing.module_id,
content_json: existing.content_json, content_json: existing.content_json,
// 归档标记不经 MCP 修改(归档走桌面端 archive_task 专用 IPC,防旁路)
archived: existing.archived,
created_at: existing.created_at, created_at: existing.created_at,
updated_at: now, updated_at: now,
}; };
@@ -819,6 +834,7 @@ fn create_idea(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult> {
ai_analysis: None, ai_analysis: None,
scores: None, scores: None,
related_ids: None, related_ids: None,
archived: false,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -874,6 +890,7 @@ fn update_idea(ctx: &Ctx, args: Value) -> BoxFuture<'static, CallToolResult> {
ai_analysis: existing.ai_analysis, ai_analysis: existing.ai_analysis,
scores: existing.scores, scores: existing.scores,
related_ids: existing.related_ids.clone(), related_ids: existing.related_ids.clone(),
archived: existing.archived,
created_at: existing.created_at, created_at: existing.created_at,
updated_at: now, updated_at: now,
}; };
@@ -1239,6 +1256,7 @@ mod tests {
ai_analysis: None, ai_analysis: None,
scores: None, scores: None,
related_ids: None, related_ids: None,
archived: false,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -1257,6 +1275,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -1285,6 +1304,7 @@ mod tests {
queue: "todo".to_string(), queue: "todo".to_string(),
parent_id: None, parent_id: None,
content_json: None, content_json: None,
archived: false,
created_at: now.clone(), created_at: now.clone(),
updated_at: now, updated_at: now,
}; };
@@ -1729,6 +1749,43 @@ mod tests {
assert_eq!(v["has_more"], false, "3 条 < 100,无下一页"); assert_eq!(v["has_more"], false, "3 条 < 100,无下一页");
} }
/// 1eff911c 缺陷修复:无 status 过滤时默认排除 done/cancelled 终态任务;
/// 显式传 status=done 仍可查终态。
#[tokio::test]
async fn list_tasks_excludes_terminal_status_by_default() {
let ctx = test_ctx().await;
let pid = seed_project(&ctx, "终态过滤").await;
let done_id = seed_task(&ctx, &pid, "已完成任务").await;
seed_task(&ctx, &pid, "进行中任务").await;
// 把第一条推进到 done
let repo = TaskRepo::new(&ctx.db);
repo.advance_status_atomic(&done_id, "todo", "done", false).await.unwrap();
// 无 status 过滤:done 应被排除,只返回活跃任务
let r = list_tasks(&ctx, json!({ "project_id": pid })).await;
assert!(r.is_error.is_none(), "{:?}", text_of(&r));
let v = json_of(&r);
let titles: Vec<&str> = v["tasks"]
.as_array()
.unwrap()
.iter()
.map(|t| t["title"].as_str().unwrap())
.collect();
assert!(
!titles.contains(&"已完成任务"),
"done 任务不应出现在默认结果: {titles:?}"
);
assert!(titles.contains(&"进行中任务"), "活跃任务应保留: {titles:?}");
// 显式 status=done:仍可查终态(能力不回退)
let r = list_tasks(&ctx, json!({ "project_id": pid, "status": "done" })).await;
assert!(r.is_error.is_none(), "{:?}", text_of(&r));
let v = json_of(&r);
let arr = v["tasks"].as_array().unwrap();
assert_eq!(arr.len(), 1, "显式查 done 应命中 1 条");
assert_eq!(arr[0]["title"], "已完成任务");
}
/// list_projects 分页:offset 生效 + has_more 标记。 /// list_projects 分页:offset 生效 + has_more 标记。
#[tokio::test] #[tokio::test]
async fn list_projects_pagination_offset_and_has_more() { async fn list_projects_pagination_offset_and_has_more() {
+3
View File
@@ -1,4 +1,5 @@
//! AI 编排节点 — 直连 agentic 会话执行工作流任务 //! AI 编排节点 — 直连 agentic 会话执行工作流任务
//! AI 编排节点 — 直连 agentic 会话执行工作流任务
//! //!
//! 与 AiNode(单次 LLM 补全)的区别:AiAgentNode 在节点 execute 时以编程方式启动一个 //! 与 AiNode(单次 LLM 补全)的区别:AiAgentNode 在节点 execute 时以编程方式启动一个
//! 独立 agentic 会话(新 conv_id,与用户活跃会话隔离,防 epoch 互踩),由 run_agentic_loop //! 独立 agentic 会话(新 conv_id,与用户活跃会话隔离,防 epoch 互踩),由 run_agentic_loop
@@ -241,6 +242,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
}) })
@@ -265,6 +267,7 @@ mod tests {
queue: "todo".to_string(), queue: "todo".to_string(),
parent_id: None, parent_id: None,
content_json: None, content_json: None,
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
}) })
@@ -1,4 +1,5 @@
//! AI 自审节点(决策 a 步骤③)— AiSelfReviewNode //! AI 自审节点(决策 a 步骤③)— AiSelfReviewNode
//! AI 自审节点(决策 a 步骤③)— AiSelfReviewNode
//! //!
//! 从 ai_node.rs 拆出的独立节点(纯结构搬运,逻辑等价)。复用 AiNode 的 provider //! 从 ai_node.rs 拆出的独立节点(纯结构搬运,逻辑等价)。复用 AiNode 的 provider
//! 调用链(resolve_and_parse + provider_from_params),差异在 prompt 模板/JSON 解析/ //! 调用链(resolve_and_parse + provider_from_params),差异在 prompt 模板/JSON 解析/
@@ -316,6 +317,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
}) })
@@ -340,6 +342,7 @@ mod tests {
queue: "todo".to_string(), queue: "todo".to_string(),
parent_id: None, parent_id: None,
content_json: None, content_json: None,
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
}) })
+7 -3
View File
@@ -70,10 +70,12 @@ impl Node for HumanNode {
// 3. select! 循环:Response / 超时 / 取消 // 3. select! 循环:Response / 超时 / 取消
let deadline = tokio::time::Instant::now() + Duration::from_secs(timeout_secs); let deadline = tokio::time::Instant::now() + Duration::from_secs(timeout_secs);
let mut cancel_tick = tokio::time::interval(Duration::from_millis(500)); // FR-P4:取消检测由 500ms interval 轮询改为 Notify 被动唤醒(set_cancelled 触发)。
cancel_tick.tick().await; // 丢弃首个立即触发的 tick // 先建 notified future 再进 loop,避免 notify 早于 await 的竞态窗口;
// 每轮 select! 后重建(Notified 单次消费),配合 is_cancelled 复查兜底。
loop { loop {
let cancel_wait = ctx.node_status.cancel_notified();
tokio::select! { tokio::select! {
recv = rx.recv() => match recv { recv = rx.recv() => match recv {
Ok(WorkflowEvent::HumanApprovalResponse { Ok(WorkflowEvent::HumanApprovalResponse {
@@ -154,7 +156,9 @@ impl Node for HumanNode {
_ = tokio::time::sleep_until(deadline) => { _ = tokio::time::sleep_until(deadline) => {
return Err(anyhow::anyhow!("人工审批超时({}s)", timeout_secs)); return Err(anyhow::anyhow!("人工审批超时({}s)", timeout_secs));
} }
_ = cancel_tick.tick() => { _ = cancel_wait => {
// FR-P4:Notify 唤醒后复查 is_cancelled(Notify 无状态,
// 可能因其他节点取消而误唤醒,须确认本节点确被取消)。
if ctx.node_status.is_cancelled(&ctx.node_id) { if ctx.node_status.is_cancelled(&ctx.node_id) {
return Err(anyhow::anyhow!("人工审批被取消")); return Err(anyhow::anyhow!("人工审批被取消"));
} }
+3 -2
View File
@@ -1,7 +1,8 @@
//! 子工作流节点 — 加载一个子 DAG 并在当前执行上下文中递归执行 //! 子工作流节点 — 加载并校验一个子 DAG,返回其 JSON 快照
//! //!
//! 用于复用通用流程(如"代码审查"可作为任意主流程的子步骤)。 //! 用于复用通用流程(如"代码审查"可作为任意主流程的子步骤)。
//! 执行结果不直接展开节点,而是返回子 DAG 的 JSON 快照,由 DagExecutor 消费展开。 //! 注意:本节点不递归执行子 DAG,只做配置/深度校验后返回子 DAG 快照;
//! 快照的展开执行需调用方自行处理(DagExecutor 当前不会自动执行返回的 subflow)。
use async_trait::async_trait; use async_trait::async_trait;
use df_workflow::dag_def::DagDef; use df_workflow::dag_def::DagDef;
+312 -8
View File
@@ -1,4 +1,5 @@
//! 任务推进节点 — advance_task 推进链触发器。 //! 任务推进节点 — advance_task 推进链触发器。
//! 任务推进节点 — advance_task 推进链触发器。
//! //!
//! 实现推进链的唯一 status 写入路径(D-260616-03 落 df-nodes Node): //! 实现推进链的唯一 status 写入路径(D-260616-03 落 df-nodes Node):
//! 1. 读当前 TaskRecord(取 from status) //! 1. 读当前 TaskRecord(取 from status)
@@ -8,9 +9,14 @@
//! //!
//! 两条调用路径复用同一推进逻辑: //! 两条调用路径复用同一推进逻辑:
//! - DAG 工作流:TaskAdvanceNode 实现 Node trait,从 NodeContext.config 读 task_id / //! - DAG 工作流:TaskAdvanceNode 实现 Node trait,从 NodeContext.config 读 task_id /
//! target_status,持有 Arc<Database> 构造 TaskRepo。 //! target_status,持有 Arc<Database> 构造 TaskRepo(严格单跳,advance_task_atomic)。
//! - IPC 直驱:advance_task_atomic 公开 async fn,src-tauri commands::task::advance_task //! - IPC 直驱:advance_task_atomic 公开 async fn,src-tauri commands::task::advance_task
//! thin 入口直接调用(无需构造 NodeContext,避免 EventBus/StateMachine 等重依赖)。 //! thin 入口直接调用(无需构造 NodeContext,避免 EventBus/StateMachine 等重依赖)。
//!
//! auto_path 层(默认开启):advance_task_with_parent(IPC/AI 工具/MCP 同源入口)走
//! advance_task_auto_path,跳态目标按 shortest_path 自动逐跳推进(每跳独立校验 +
//! review_rounds 回归计数照旧);直达单跳与严格路径行为一致,workflow.rs 回调等
//! 机器生成单跳场景仍直用 advance_task_atomic 保持严格语义。
use std::sync::Arc; use std::sync::Arc;
@@ -61,6 +67,26 @@ pub async fn advance_task_atomic(
.await? .await?
.ok_or_else(|| df_types::error::Error::NotFound(format!("任务 {} 不存在", id)))?; .ok_or_else(|| df_types::error::Error::NotFound(format!("任务 {} 不存在", id)))?;
// 2.5 前置校验:归档任务冻结,不可推进(与 CAS SQL 的 archived=0 条件同源)。
//
// 背景:advance_status_atomic 的 CAS SQL 带 `AND archived = 0`,归档任务必然
// affected==0 → 返回 None → 下游误报「已变更(并发推进或旁路修改)」,诊断信息
// 完全误导(真实原因是归档,非并发)。此处前置拦截,给出精确可操作错误。
//
// 软删(deleted_at IS NOT NULL)任务 get_by_id 仍能读到(TASK_COLS 不 SELECT
// deleted_at,TaskRecord 无该字段),但 CAS 的 `deleted_at IS NULL` 同样必败。
// 因 TaskRecord 不带 deleted_at,此处无法直接判定;由 advance_status_atomic
// 的 CAS 失败回查区分(见该函数注释)。
if current.archived {
return Err(df_types::error::Error::InvalidState {
current: format!(
"任务 {} 已归档(archived=true),归档任务冻结不可推进;如需推进请先取消归档",
id
),
expected: target_status.to_string(),
});
}
// 3. 状态机校验(三类拒绝,错误类型区分供前端分辨): // 3. 状态机校验(三类拒绝,错误类型区分供前端分辨):
// - 同态(from==to):Validation「相同状态,无需推进」(空操作,非状态机违例) // - 同态(from==to):Validation「相同状态,无需推进」(空操作,非状态机违例)
// - 非法转换(跳态/终态无后继等):InvalidState「非法状态转换 X→Y」 // - 非法转换(跳态/终态无后继等):InvalidState「非法状态转换 X→Y」
@@ -106,11 +132,108 @@ pub async fn advance_task_atomic(
.advance_status_atomic(id, from, target_status, bump) .advance_status_atomic(id, from, target_status, bump)
.await?; .await?;
// 5. CAS 失败:并发已改动 status(或任务被删),拒绝静默成功。 // 5. CAS 失败:affected==0 有四种可能,回查区分,给出精确错误(不再一律误报「并发」)。
updated.ok_or_else(|| df_types::error::Error::InvalidState { //
current: "已变更(并发推进或旁路修改)".to_string(), // CAS SQL 条件:`WHERE id=? AND status=? AND deleted_at IS NULL AND archived=0`。
expected: from.to_string(), // affected==0 时逐一排查,避免「已变更(并发推进或旁路修改)」这个误导性文案:
}) // a) deleted_at 非 NULL → 任务在回收站(回查 is_deleted)
// b) archived=1 → 已归档(前置 2.5 已拦,此处兜底防 TOCTOU)
// c) status 真被并发改动 → 回读当前 status 对比 from
// d) 任务被删(id 不存在)→ get_by_id 返 None
let Some(updated) = updated else {
// a) 回收站任务(最易被误报为「并发」的一类,优先排查)
if repo.is_deleted(id).await? {
return Err(df_types::error::Error::InvalidState {
current: format!(
"任务 {} 在回收站(deleted_at 非 NULL),回收站任务不可推进;如需推进请先恢复",
id
),
expected: from.to_string(),
});
}
// b/c/d) 回读最新记录,区分归档 / 真并发 / 已删除
let latest = repo.get_by_id(id).await?;
return Err(match latest {
None => df_types::error::Error::NotFound(format!("任务 {} 不存在", id)),
Some(rec) if rec.archived => df_types::error::Error::InvalidState {
current: format!("任务 {} 已归档(archived=true),不可推进;如需推进请先取消归档", id),
expected: from.to_string(),
},
Some(rec) => df_types::error::Error::InvalidState {
current: format!(
"状态已变更(并发推进或旁路修改):期望 {:?},实际 {:?}",
from,
rec.status.as_str()
),
expected: from.to_string(),
},
});
};
Ok(updated)
}
// ============================================================
// auto_path — 跳态目标自动逐跳推进(治 LLM 直取跳态被状态机拒绝的空耗往返)
// ============================================================
/// advance_task 自动路径推进(auto_path):目标不可直达时按最短路径逐跳推进。
///
/// 背景:实测 LLM 反复请求跳态推进(todo→done / in_progress→done),错误文本虽已附
/// shortest_path 提示,LLM 仍不走多跳,空耗往返。根修:跳态目标自动按
/// [`crate::task_state_machine::shortest_path`] 逐跳推进,每跳独立走
/// [`advance_task_atomic`](状态机校验 + 原子 CAS + review_rounds 回归计数规则照旧,
/// 如路径途经 in_review→in_progress 退回边照样 +1)。
///
/// 分派:
/// - 直达/委托(路径长 0 或 2):交 [`advance_task_atomic`] 单跳完成,同态/非法
/// target/终态不可达的错误文案零变更。
/// - 多跳(路径长 > 2):逐跳推进,任一跳失败即止(任务停在中间态,错误注明第几跳;
/// 不回滚已完成跳——回滚本身也要过状态机)。并发改动致某跳 CAS 失败时,调用方
/// 重读任务现状再决策即可。
///
/// 返回 (最终 TaskRecord, 实际路径):路径含首尾([from, ..., to]),直达时长 2、
/// 多跳时长 > 2,供调用方在 result 透出实际推进路径。
pub async fn advance_task_auto_path(
repo: &TaskRepo,
id: &str,
target_status: &str,
) -> df_types::error::Result<(TaskRecord, Vec<&'static str>)> {
let current = repo
.get_by_id(id)
.await?
.ok_or_else(|| df_types::error::Error::NotFound(format!("任务 {} 不存在", id)))?;
let from = current.status.as_str();
let path = crate::task_state_machine::shortest_path(from, target_status);
let record = if path.len() > 2 {
let mut record = current;
for (idx, &to) in path.iter().enumerate().skip(1) {
record = advance_task_atomic(repo, id, to).await.map_err(|e| match e {
// 任务被删(NotFound)原样透传;其余(状态机/CAS 冲突)包上跳次上下文
nf @ df_types::error::Error::NotFound(_) => nf,
other => df_types::error::Error::InvalidState {
current: format!(
"自动路径第 {}/{} 跳({} → {})失败: {other};任务停在中间态,请重读任务现状后再决策",
idx,
path.len() - 1,
path[idx - 1],
to
),
expected: to.to_string(),
},
})?;
}
record
} else {
// 直达(含同态/非法 target/终态不可达):atomic 原样校验与报错,语义零变更
advance_task_atomic(repo, id, target_status).await?
};
// 路径承载:shortest_path 产物全为 ALL_STATES 常量(&'static);空路径(同态/
// 非法/不可达)不会走到成功返回,此处兜底空列表。
let full_path = if path.len() >= 2 { path } else { Vec::new() };
Ok((record, full_path))
} }
// ============================================================ // ============================================================
@@ -200,7 +323,10 @@ pub async fn recompute_parent_status(
/// 推进任务 + 若为子任务则触发父聚合(父聚合失败仅 warn 不阻断,宽容语义)。 /// 推进任务 + 若为子任务则触发父聚合(父聚合失败仅 warn 不阻断,宽容语义)。
/// ///
/// 推进链唯一 status 写入路径的两段式(设计 D3 统一,IPC/AI 工具/MCP 同源): /// 推进链唯一 status 写入路径的两段式(设计 D3 统一,IPC/AI 工具/MCP 同源):
/// 1. 推进自身:调 `advance_task_atomic`(状态机校验 + 原子 CAS + review_rounds 累加)。 /// 1. 推进自身:调 `advance_task_auto_path`(auto_path 默认开启:跳态目标按
/// shortest_path 自动逐跳推进,治实测 LLM todo→done / in_progress→done 直取被
/// 状态机拒绝后不走多跳的空耗往返;直达单跳行为与原 advance_task_atomic 一致)。
/// 实际路径结构化返回值在 auto_path,多跳时此处 tracing 留痕。
/// 2. 父聚合:推进成功后若 `updated.parent_id` 有值,触发 `recompute_parent_status` 重算 /// 2. 父聚合:推进成功后若 `updated.parent_id` 有值,触发 `recompute_parent_status` 重算
/// 父 status(父任务=容器模型,聚合规则见 recompute_parent_status)。父聚合失败仅 /// 父 status(父任务=容器模型,聚合规则见 recompute_parent_status)。父聚合失败仅
/// tracing::warn 不阻断——子任务推进成功是主结果,父 status 漂移可后续修正。 /// tracing::warn 不阻断——子任务推进成功是主结果,父 status 漂移可后续修正。
@@ -211,7 +337,16 @@ pub async fn advance_task_with_parent(
id: &str, id: &str,
target_status: &str, target_status: &str,
) -> df_types::error::Result<TaskRecord> { ) -> df_types::error::Result<TaskRecord> {
let updated = advance_task_atomic(repo, id, target_status).await?; let (updated, path) = advance_task_auto_path(repo, id, target_status).await?;
if path.len() > 2 {
tracing::info!(
task_id = %id,
target = %target_status,
path = %crate::task_state_machine::format_path(&path),
review_rounds = updated.review_rounds,
"[advance_task] 跳态目标已按最短路径自动逐跳推进"
);
}
if let Some(pid) = &updated.parent_id { if let Some(pid) = &updated.parent_id {
if let Err(e) = recompute_parent_status(repo, pid).await { if let Err(e) = recompute_parent_status(repo, pid).await {
tracing::warn!( tracing::warn!(
@@ -394,6 +529,7 @@ mod tests {
queue: "todo".to_string(), queue: "todo".to_string(),
parent_id: None, parent_id: None,
content_json: None, content_json: None,
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
} }
@@ -411,6 +547,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
}) })
@@ -563,6 +700,69 @@ mod tests {
assert!(r.is_none(), "expected 与库内不符应 CAS 失败"); assert!(r.is_none(), "expected 与库内不符应 CAS 失败");
} }
// ---------- 归档/软删任务的精确错误(治「已变更(并发推进或旁路修改)」误导) ----------
#[tokio::test]
async fn archived_task_rejected_with_precise_error() {
// 归档任务(archived=true)推进应被前置校验拦截,给出「已归档」精确错误,
// 而非落到 CAS 失败报「已变更(并发推进或旁路修改)」误导文案。
let repo = setup().await;
let mut r = rec("t1", TaskStatus::Todo);
r.archived = true;
repo.insert(r).await.unwrap();
let err = advance_task_atomic(&repo, "t1", "in_progress").await.unwrap_err();
match err {
df_types::error::Error::InvalidState { current, .. } => {
assert!(
current.contains("已归档"),
"归档任务错误应含「已归档」,实际: {current}"
);
assert!(
!current.contains("并发推进或旁路修改"),
"归档任务不应误报并发,实际: {current}"
);
}
other => panic!("归档任务拒绝应是 InvalidState,实际: {other:?}"),
}
}
#[tokio::test]
async fn soft_deleted_task_rejected_with_precise_error() {
// 回收站任务(deleted_at 非 NULL)推进:get_by_id 仍能读到(不过滤软删),
// 状态机校验通过,但 CAS SQL 带 deleted_at IS NULL 必败 → 回查 is_deleted
// 区分,给出「在回收站」精确错误,而非误报「已变更(并发推进或旁路修改)」。
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Todo)).await.unwrap();
// 软删该任务(走 soft_delete,设 deleted_at)
let ok = repo.soft_delete("t1").await.unwrap();
assert!(ok, "soft_delete 应命中");
assert!(repo.is_deleted("t1").await.unwrap(), "软删后 is_deleted 应为 true");
let err = advance_task_atomic(&repo, "t1", "in_progress").await.unwrap_err();
match err {
df_types::error::Error::InvalidState { current, .. } => {
assert!(
current.contains("回收站"),
"软删任务错误应含「回收站」,实际: {current}"
);
assert!(
!current.contains("并发推进或旁路修改"),
"软删任务不应误报并发,实际: {current}"
);
}
other => panic!("软删任务拒绝应是 InvalidState,实际: {other:?}"),
}
}
#[tokio::test]
async fn is_deleted_false_for_active_task() {
// 活跃任务 is_deleted 应为 false(不误判),保证回查区分逻辑不误伤正常任务。
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Todo)).await.unwrap();
assert!(!repo.is_deleted("t1").await.unwrap(), "活跃任务 is_deleted 应为 false");
// 不存在的任务:is_deleted 返回 false(不存在 ≠ 软删)
assert!(!repo.is_deleted("nope").await.unwrap(), "不存在任务 is_deleted 应为 false");
}
#[tokio::test] #[tokio::test]
async fn blocked_round_trip_does_not_bump() { async fn blocked_round_trip_does_not_bump() {
let repo = setup().await; let repo = setup().await;
@@ -575,6 +775,110 @@ mod tests {
assert_eq!(r.review_rounds, 0, "解除 blocked 不累加"); assert_eq!(r.review_rounds, 0, "解除 blocked 不累加");
} }
// ============================================================
// auto_path — 跳态目标自动逐跳推进(治实测 todo→done / in_progress→done 直取失败)
// ============================================================
#[tokio::test]
async fn auto_path_todo_to_done_walks_full_gates() {
// 实测失败形态 1:todo 直取 done,auto_path 走满闸门链直达目标
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Todo)).await.unwrap();
let (r, path) = advance_task_auto_path(&repo, "t1", "done").await.unwrap();
assert_eq!(r.status.as_str(), "done");
assert_eq!(r.review_rounds, 0, "纯前向路径不累加 review_rounds");
assert_eq!(path, vec!["todo", "in_progress", "in_review", "testing", "done"]);
}
#[tokio::test]
async fn auto_path_in_progress_to_done() {
// 实测失败形态 2:in_progress 直取 done
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::InProgress)).await.unwrap();
let (r, path) = advance_task_auto_path(&repo, "t1", "done").await.unwrap();
assert_eq!(r.status.as_str(), "done");
assert_eq!(path, vec!["in_progress", "in_review", "testing", "done"]);
}
#[tokio::test]
async fn auto_path_regression_hop_bumps_rounds() {
// 目标不可直达且路径途经退回边:testing→in_review→in_progress 两跳均为退回,
// 每跳独立按 is_regression 累加 → review_rounds = 2(规则照旧,逐跳生效)
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Testing)).await.unwrap();
let (r, path) = advance_task_auto_path(&repo, "t1", "in_progress").await.unwrap();
assert_eq!(r.status.as_str(), "in_progress");
assert_eq!(path, vec!["testing", "in_review", "in_progress"]);
assert_eq!(r.review_rounds, 2, "两跳退回各 +1,应累加到 2");
}
#[tokio::test]
async fn auto_path_multihop_forward_no_bump() {
// deferred → blocked:3 跳(deferred→todo→in_progress→blocked),全前向不累加
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Deferred)).await.unwrap();
let (r, path) = advance_task_auto_path(&repo, "t1", "blocked").await.unwrap();
assert_eq!(r.status.as_str(), "blocked");
assert_eq!(path, vec!["deferred", "todo", "in_progress", "blocked"]);
assert_eq!(r.review_rounds, 0);
}
#[tokio::test]
async fn auto_path_direct_transition_unchanged() {
// 直达单跳:行为与旧 advance_task_atomic 完全一致(路径长 2,rounds 不动)
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Todo)).await.unwrap();
let (r, path) = advance_task_auto_path(&repo, "t1", "in_progress").await.unwrap();
assert_eq!(r.status.as_str(), "in_progress");
assert_eq!(r.review_rounds, 0);
assert_eq!(path, vec!["todo", "in_progress"]);
}
#[tokio::test]
async fn auto_path_with_parent_multihop_lands_done() {
// LLM 入口(advance_task_with_parent)默认开启 auto_path:todo 直取 done 成功
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Todo)).await.unwrap();
let r = advance_task_with_parent(&repo, "t1", "done").await.unwrap();
assert_eq!(r.status.as_str(), "done");
assert_eq!(r.review_rounds, 0);
}
#[tokio::test]
async fn auto_path_terminal_and_same_state_still_rejected() {
// auto_path 不越状态机:终态不可达 / 同态仍按原错误类型拒绝(委托 atomic 原样报错)
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Done)).await.unwrap();
let err = advance_task_with_parent(&repo, "t1", "todo").await.unwrap_err();
assert!(
matches!(err, df_types::error::Error::InvalidState { .. }),
"终态不可达应是 InvalidState,实际: {err:?}"
);
repo.insert(rec("t2", TaskStatus::InProgress)).await.unwrap();
let err = advance_task_with_parent(&repo, "t2", "in_progress").await.unwrap_err();
match err {
df_types::error::Error::Validation(msg) => {
assert!(msg.contains("相同状态"), "同态消息应含「相同状态」,实际: {msg}");
}
other => panic!("同态应是 Validation,实际: {other:?}"),
}
}
#[tokio::test]
async fn auto_path_invalid_target_still_rejected() {
let repo = setup().await;
repo.insert(rec("t1", TaskStatus::Todo)).await.unwrap();
let err = advance_task_with_parent(&repo, "t1", "merged").await.unwrap_err();
assert!(matches!(err, df_types::error::Error::Validation(_)));
}
#[tokio::test]
async fn auto_path_not_found_still_rejected() {
let repo = setup().await;
let err = advance_task_with_parent(&repo, "nope", "done").await.unwrap_err();
assert!(matches!(err, df_types::error::Error::NotFound(_)));
}
// ============================================================ // ============================================================
// CR-06 观察项 ② 测试补强 — ②-3(completed)/②-4(failed) 回调推进覆盖 // CR-06 观察项 ② 测试补强 — ②-3(completed)/②-4(failed) 回调推进覆盖
// //
-3
View File
@@ -114,14 +114,11 @@ pub(crate) fn strip_readme_noise(content: &str) -> String {
if trimmed == "---" || trimmed == "+++" { if trimmed == "---" || trimmed == "+++" {
// 开头处的 frontmatter:跳到下一个匹配分隔符 // 开头处的 frontmatter:跳到下一个匹配分隔符
let delim = trimmed; let delim = trimmed;
let mut consumed_any = false;
for next in lines.by_ref() { for next in lines.by_ref() {
consumed_any = true;
if next.trim() == delim { if next.trim() == delim {
break; break;
} }
} }
let _ = consumed_any;
continue; continue;
} }
// ── HTML 注释块(<!-- ... -->,可能跨行)── // ── HTML 注释块(<!-- ... -->,可能跨行)──
+40
View File
@@ -75,6 +75,17 @@ fn validate_device_id(id: &str) -> bool {
!id.is_empty() && id.len() <= 128 && id.chars().all(|c| !c.is_control()) !id.is_empty() && id.len() <= 128 && id.chars().all(|c| !c.is_control())
} }
/// 当前 Hello 握手协议版本。
///
/// 协商规则:旧客户端 Hello 缺 `proto` 字段 → serde default 视作 v1(向后兼容);
/// 高于本端支持版本时记录告警,为后续破坏性变更预留通知位(当前仅日志,不拒绝握手)。
pub const HELLO_PROTO_VERSION: u32 = 1;
/// `proto` 缺省值:旧客户端不带该字段时按 v1 处理(向后兼容)。
fn default_proto() -> u32 {
HELLO_PROTO_VERSION
}
/// 客户端首消息:身份宣告(简单协议) /// 客户端首消息:身份宣告(简单协议)
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Hello { pub struct Hello {
@@ -84,6 +95,9 @@ pub struct Hello {
pub device_id: String, pub device_id: String,
/// 配对 token /// 配对 token
pub token: String, pub token: String,
/// 协议版本(缺省按 v1,兼容旧客户端)
#[serde(default = "default_proto")]
pub proto: u32,
} }
/// Hello.kind 的传输表示(serde 字符串,与 ClientKind 解耦避免 rename 歧义) /// Hello.kind 的传输表示(serde 字符串,与 ClientKind 解耦避免 rename 歧义)
@@ -298,6 +312,16 @@ async fn handle_connection(socket: WebSocket, state: RelayState, expected: Clien
return; return;
} }
// 版本协商:proto 缺省按 v1(旧客户端不带字段,向后兼容)。
// 高于本端支持版本时告警 —— 为后续破坏性变更预留的通知位(当前仅日志,不拒绝握手)。
if hello.proto > HELLO_PROTO_VERSION {
tracing::warn!(
client_proto = hello.proto,
server_proto = HELLO_PROTO_VERSION,
"客户端协议版本高于本端,可能存在不兼容(预留通知位)"
);
}
let conn_id = next_conn_id(); let conn_id = next_conn_id();
let kind: ClientKind = hello.kind.into(); let kind: ClientKind = hello.kind.into();
let device_id = hello.device_id.clone(); let device_id = hello.device_id.clone();
@@ -305,6 +329,7 @@ async fn handle_connection(socket: WebSocket, state: RelayState, expected: Clien
conn_id = conn_id.0, conn_id = conn_id.0,
?kind, ?kind,
device_id = %device_id, device_id = %device_id,
proto = hello.proto,
"连接握手通过,进入收发循环" "连接握手通过,进入收发循环"
); );
@@ -545,4 +570,19 @@ mod tests {
fn max_frame_is_one_mib() { fn max_frame_is_one_mib() {
assert_eq!(MAX_FRAME_BYTES, 1024 * 1024); assert_eq!(MAX_FRAME_BYTES, 1024 * 1024);
} }
#[test]
fn hello_proto_defaults_to_v1_when_absent() {
// 旧客户端不发 proto → 反序列化按 v1,保证向后兼容
let json = r#"{"kind":"device","device_id":"dev-1","token":"t"}"#;
let h: Hello = serde_json::from_str(json).expect("缺 proto 应可解析");
assert_eq!(h.proto, HELLO_PROTO_VERSION);
}
#[test]
fn hello_proto_parsed_when_present() {
let json = r#"{"kind":"device","device_id":"dev-1","token":"t","proto":2}"#;
let h: Hello = serde_json::from_str(json).unwrap();
assert_eq!(h.proto, 2);
}
} }
+289 -13
View File
@@ -110,6 +110,12 @@ fn ai_conversation_from_row(row: &Row<'_>) -> std::result::Result<AiConversation
origin: row.get("origin").unwrap_or(None), origin: row.get("origin").unwrap_or(None),
// boundary_json 列老库经 v45 迁移补建。unwrap_or(None) 兜底(NULL=未声明任务边界)。 // boundary_json 列老库经 v45 迁移补建。unwrap_or(None) 兜底(NULL=未声明任务边界)。
boundary_json: row.get("boundary_json").unwrap_or(None), boundary_json: row.get("boundary_json").unwrap_or(None),
// loop_state 列老库经 v49 迁移补建。unwrap_or(None) 兜底(NULL=非任务会话)。
loop_state: row.get("loop_state").unwrap_or(None),
// status 列老库经 v50 迁移补建(NOT NULL DEFAULT 'active')。兜底迁移未跑/极端缺列时按 active。
status: row.get("status").unwrap_or_else(|_| "active".to_string()),
// working_context_json 列老库经 v53 迁移补建。unwrap_or(None) 兜底(NULL=无 WorkingContext)。
working_context_json: row.get("working_context_json").unwrap_or(None),
}) })
} }
@@ -131,6 +137,8 @@ fn ai_tool_execution_from_row(row: &Row<'_>) -> std::result::Result<AiToolExecut
requested_at: row.get("requested_at")?, requested_at: row.get("requested_at")?,
executed_at: row.get("executed_at")?, executed_at: row.get("executed_at")?,
decided_by: row.get("decided_by")?, decided_by: row.get("decided_by")?,
// duration_ms 列老库经 v50 迁移补建。unwrap_or(None) 兜底(NULL=老数据/未测路径)。
duration_ms: row.get("duration_ms").unwrap_or(None),
}) })
} }
@@ -185,25 +193,32 @@ impl_repo!(
from_row => |row| ai_conversation_from_row(row), from_row => |row| ai_conversation_from_row(row),
insert => |conn, rec| { insert => |conn, rec| {
conn.execute( conn.execute(
"INSERT INTO ai_conversations (id, title, messages, provider_id, model, models, archived, pinned, prompt_tokens, completion_tokens, pinned_goals, pending_approvals, created_at, updated_at, origin, boundary_json) "INSERT INTO ai_conversations (id, title, messages, provider_id, model, models, archived, pinned, prompt_tokens, completion_tokens, pinned_goals, pending_approvals, created_at, updated_at, origin, boundary_json, loop_state, status, working_context_json)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19)",
params![ params![
rec.id, rec.title, rec.messages, rec.provider_id, rec.model, rec.models, rec.archived, rec.id, rec.title, rec.messages, rec.provider_id, rec.model, rec.models, rec.archived,
if rec.pinned { 1i32 } else { 0i32 }, if rec.pinned { 1i32 } else { 0i32 },
rec.prompt_tokens, rec.completion_tokens, rec.prompt_tokens, rec.completion_tokens,
rec.pinned_goals, rec.pending_approvals, rec.created_at, rec.updated_at, rec.origin, rec.pinned_goals, rec.pending_approvals, rec.created_at, rec.updated_at, rec.origin,
rec.boundary_json rec.boundary_json, rec.loop_state, rec.status, rec.working_context_json
], ],
) )
}, },
update => |conn, rec| { update => |conn, rec| {
// loop_state/status 有意不在 SET 列(V49/V50 生命周期标记独立更新,对齐 boundary 早期模式):
// update_full 的调用方(save_conversation 等)从 get_by_id 读旧记录再改元数据写回,
// 读改写窗口内 loop 生命周期/终局态可变更——若 SET 含二者,旧值会覆盖新值(lost-update)。
// 生命周期只经 set_loop_state / set_status 单点写。
// working_context_json 在 SET 列(V53):与 pinned_goals/boundary_json 同为 save 每轮
// 随行写的内存真相源快照,读改写窗口内无独立写入方,不存在 lost-update 面。
conn.execute( conn.execute(
"UPDATE ai_conversations SET title = ?1, messages = ?2, provider_id = ?3, model = ?4, models = ?5, archived = ?6, pinned = ?7, prompt_tokens = ?8, completion_tokens = ?9, pinned_goals = ?10, pending_approvals = ?11, updated_at = ?12, boundary_json = ?13 WHERE id = ?14", "UPDATE ai_conversations SET title = ?1, messages = ?2, provider_id = ?3, model = ?4, models = ?5, archived = ?6, pinned = ?7, prompt_tokens = ?8, completion_tokens = ?9, pinned_goals = ?10, pending_approvals = ?11, updated_at = ?12, boundary_json = ?13, working_context_json = ?14 WHERE id = ?15",
params![ params![
rec.title, rec.messages, rec.provider_id, rec.model, rec.models, rec.archived, rec.title, rec.messages, rec.provider_id, rec.model, rec.models, rec.archived,
if rec.pinned { 1i32 } else { 0i32 }, if rec.pinned { 1i32 } else { 0i32 },
rec.prompt_tokens, rec.completion_tokens, rec.prompt_tokens, rec.completion_tokens,
rec.pinned_goals, rec.pending_approvals, rec.updated_at, rec.boundary_json, rec.id rec.pinned_goals, rec.pending_approvals, rec.updated_at, rec.boundary_json,
rec.working_context_json, rec.id
], ],
) )
} }
@@ -243,22 +258,22 @@ impl_repo!(
from_row => |row| ai_tool_execution_from_row(row), from_row => |row| ai_tool_execution_from_row(row),
insert => |conn, rec| { insert => |conn, rec| {
conn.execute( conn.execute(
"INSERT INTO ai_tool_executions (id, conversation_id, message_id, tool_call_id, tool_name, arguments, result, status, risk_level, requested_at, executed_at, decided_by) "INSERT INTO ai_tool_executions (id, conversation_id, message_id, tool_call_id, tool_name, arguments, result, status, risk_level, requested_at, executed_at, decided_by, duration_ms)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13)",
params![ params![
rec.id, rec.conversation_id, rec.message_id, &*rec.tool_call_id, rec.tool_name, rec.id, rec.conversation_id, rec.message_id, &*rec.tool_call_id, rec.tool_name,
rec.arguments, rec.result, rec.status, rec.risk_level, rec.arguments, rec.result, rec.status, rec.risk_level,
rec.requested_at, rec.executed_at, rec.decided_by rec.requested_at, rec.executed_at, rec.decided_by, rec.duration_ms
], ],
) )
}, },
update => |conn, rec| { update => |conn, rec| {
conn.execute( conn.execute(
"UPDATE ai_tool_executions SET conversation_id = ?1, message_id = ?2, tool_call_id = ?3, tool_name = ?4, arguments = ?5, result = ?6, status = ?7, risk_level = ?8, requested_at = ?9, executed_at = ?10, decided_by = ?11 WHERE id = ?12", "UPDATE ai_tool_executions SET conversation_id = ?1, message_id = ?2, tool_call_id = ?3, tool_name = ?4, arguments = ?5, result = ?6, status = ?7, risk_level = ?8, requested_at = ?9, executed_at = ?10, decided_by = ?11, duration_ms = ?12 WHERE id = ?13",
params![ params![
rec.conversation_id, rec.message_id, &*rec.tool_call_id, rec.tool_name, rec.conversation_id, rec.message_id, &*rec.tool_call_id, rec.tool_name,
rec.arguments, rec.result, rec.status, rec.risk_level, rec.arguments, rec.result, rec.status, rec.risk_level,
rec.requested_at, rec.executed_at, rec.decided_by, rec.id rec.requested_at, rec.executed_at, rec.decided_by, rec.duration_ms, rec.id
], ],
) )
} }
@@ -292,15 +307,15 @@ impl AiToolExecutionRepo {
let tx = guard.transaction().map_err(storage_err)?; let tx = guard.transaction().map_err(storage_err)?;
{ {
let mut stmt = tx.prepare( let mut stmt = tx.prepare(
"INSERT INTO ai_tool_executions (id, conversation_id, message_id, tool_call_id, tool_name, arguments, result, status, risk_level, requested_at, executed_at, decided_by) "INSERT INTO ai_tool_executions (id, conversation_id, message_id, tool_call_id, tool_name, arguments, result, status, risk_level, requested_at, executed_at, decided_by, duration_ms)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13)",
) )
.map_err(storage_err)?; .map_err(storage_err)?;
for rec in &records { for rec in &records {
stmt.execute(params![ stmt.execute(params![
rec.id, rec.conversation_id, rec.message_id, &*rec.tool_call_id, rec.tool_name, rec.id, rec.conversation_id, rec.message_id, &*rec.tool_call_id, rec.tool_name,
rec.arguments, rec.result, rec.status, rec.risk_level, rec.arguments, rec.result, rec.status, rec.risk_level,
rec.requested_at, rec.executed_at, rec.decided_by rec.requested_at, rec.executed_at, rec.decided_by, rec.duration_ms
]) ])
.map_err(storage_err)?; .map_err(storage_err)?;
} }
@@ -809,6 +824,95 @@ impl AiConversationRepo {
.map_err(storage_err)? .map_err(storage_err)?
} }
/// loop 生命周期标记写入(V49 loop_state,断点续跑底座)。
///
/// state: Some("running")=任务 loop spawn 时置起;Some("parked")=重启扫描改写;
/// None=正常收敛/用户停止时清理。纯元数据标记不进 update_full(防 save_conversation
/// 高频全量写覆盖 loop 生命周期语义),对齐 set_boundary_json/set_archived 模式。
/// 不动 updated_at(纯标记)。对话未落库(懒创建)时影响 0 行,调用方忽略。
pub async fn set_loop_state(&self, id: &str, state: Option<&str>) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let value = state.map(|s| s.to_owned());
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_conversations SET loop_state = ?1 WHERE id = ?2",
params![value, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 会话终局态写入(V50 status)。
///
/// status: "active"(默认/进行中/正常完成)、"incomplete"(停靠/断流等未完成终局)、
/// "error"(循环错误终局)。纯标记不进 update_full(防 save_conversation 高频全量写覆盖),
/// 对齐 set_loop_state 模式;不动 updated_at。对话未落库(懒创建)时影响 0 行,调用方忽略。
pub async fn set_status(&self, id: &str, status: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let status = status.to_owned();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_conversations SET status = ?1 WHERE id = ?2",
params![status, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 重启扫描:把所有 loop_state='running' 残留改写为 'parked'。
///
/// 进程启动时调用一次:上次运行中 loop_state=running 的会话,进程重启后 loop 必死
/// (内存态不复存在),残留 running 是假活。改写 parked 后侧栏读 list_recoverable
/// 展示「任务中断可恢复」。返回改写条数(0=无中断任务,静默)。
pub async fn mark_running_as_parked(&self) -> Result<u64> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_conversations SET loop_state = 'parked' WHERE loop_state = 'running'",
[],
)
.map_err(storage_err)?;
Ok(affected as u64)
})
.await
.map_err(storage_err)?
}
/// 列出可恢复会话(loop_state='parked',重启中断待一键恢复)。
pub async fn list_parked(&self) -> Result<Vec<AiConversationRecord>> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut stmt = guard
.prepare(
"SELECT * FROM ai_conversations WHERE loop_state = 'parked' ORDER BY updated_at DESC",
)
.map_err(storage_err)?;
let rows = stmt
.query_map([], ai_conversation_from_row)
.map_err(storage_err)?
.collect::<std::result::Result<Vec<_>, _>>()
.map_err(storage_err)?;
Ok(rows)
})
.await
.map_err(storage_err)?
}
/// G1.3: 删除对话 + 其全部 ai_messages 子行(单事务原子)。 /// G1.3: 删除对话 + 其全部 ai_messages 子行(单事务原子)。
/// ///
/// 背景:原宏生成 `delete` 只删 ai_conversations 主行,而 ai_messages 表无外键级联 /// 背景:原宏生成 `delete` 只删 ai_conversations 主行,而 ai_messages 表无外键级联
@@ -948,6 +1052,7 @@ mod tests {
requested_at: "0".into(), requested_at: "0".into(),
executed_at: None, executed_at: None,
decided_by: None, decided_by: None,
duration_ms: None,
} }
} }
@@ -996,6 +1101,43 @@ mod tests {
assert_eq!(got2.unwrap().id, "a2", "同三元组应返回最新落定记录"); assert_eq!(got2.unwrap().id, "a2", "同三元组应返回最新落定记录");
} }
/// D2a 结果外置的读写兼容契约:result 列由外置层(audit 侧)决定写内联还是写句柄,
/// 存储层既不解释也不改写——老行(外置落地前写入)是内联全文,读回必须原样;新行的
/// 外置句柄 JSON 同样原样存取。锁死「旧行仍读表内全文」:存储层若对 result 做截断/
/// 外置/解析等 mutation,会同时破坏两侧兼容(旧行读残缺 + 句柄被误当正文)。
#[tokio::test]
async fn tool_execution_result_stored_verbatim_old_and_new() {
let db = Database::open_in_memory().await.expect("open_in_memory");
let repo = AiToolExecutionRepo::new(&db);
// 旧数据:内联全文。超大值也要能原样读回(D2a 落地前的历史大结果就长这样,存储层不得追改)
let legacy_big = "A".repeat(10 * 1024);
let mut old_big = audit_rec("old_big", "conv1", "call_old_big", "read_file", "{}", "completed");
old_big.result = Some(legacy_big.clone());
repo.insert(old_big).await.expect("insert old big");
let mut old_small = audit_rec("old_small", "conv2", "call_old_small", "write_file", "{}", "completed");
old_small.result = Some("{\"ok\":true}".to_string());
repo.insert(old_small).await.expect("insert old small");
// 新数据:外置句柄 JSON(存储层不解析,当普通 TEXT 存取)
let handle = r#"{"__offloaded__":true,"path":"/tmp/x.txt","size_bytes":20480}"#;
let mut new_off = audit_rec("new_off", "conv3", "call_new", "run_command", "{}", "completed");
new_off.result = Some(handle.to_string());
repo.insert(new_off).await.expect("insert new offloaded");
// 读回:三条各自原样(旧行全文;句柄不被误解析/清空)
let got_old_big = repo.get_by_id("old_big").await.expect("get").expect("row");
assert_eq!(got_old_big.result.as_deref(), Some(legacy_big.as_str()), "老行超大内联 result 须原样读回");
let got_old_small = repo.get_by_id("old_small").await.expect("get").expect("row");
assert_eq!(got_old_small.result.as_deref(), Some("{\"ok\":true}"));
let got_new = repo.get_by_id("new_off").await.expect("get").expect("row");
assert_eq!(got_new.result.as_deref(), Some(handle), "外置句柄 JSON 须原样存取");
// 专用查询路径(按 tool_call_id 定位)同样原样,与通用 get_by_id 口径一致
let by_tc = repo.find_by_tool_call_id("call_old_big").await.expect("query").expect("row");
assert_eq!(by_tc.result.as_deref(), Some(legacy_big.as_str()));
}
/// insert_checkpoint 冲突语义:同 id 二次写入 Ok 不报错(幂等跳过,affected=0 走 warn), /// insert_checkpoint 冲突语义:同 id 二次写入 Ok 不报错(幂等跳过,affected=0 走 warn),
/// 保留首次 snapshot。锁死 OR IGNORE 行为,防未来误改成 REPLACE 静默覆盖快照。 /// 保留首次 snapshot。锁死 OR IGNORE 行为,防未来误改成 REPLACE 静默覆盖快照。
#[tokio::test] #[tokio::test]
@@ -1022,4 +1164,138 @@ mod tests {
.expect("snapshot"); .expect("snapshot");
assert_eq!(snapshot, r#"[{"seq":0}]"#, "冲突时应保留首次快照"); assert_eq!(snapshot, r#"[{"seq":0}]"#, "冲突时应保留首次快照");
} }
/// #904 契约:set_loop_state 写 "parked" 后 get_by_id 可读回(停靠出口写侧单点)。
/// 锁死「停靠状态写入收敛进 emit_task_parked」依赖的 repo 能力——若 set_loop_state
/// 或 from_row 的 loop_state 映射回归,停靠卡「继续」将再次撞「AI 未在暂停态」。
#[tokio::test]
async fn set_loop_state_parked_roundtrip() {
let db = Database::open_in_memory().await.expect("open_in_memory");
let repo = AiConversationRepo::new(&db);
repo.insert(AiConversationRecord {
id: "c1".into(),
title: None,
messages: "[]".into(),
provider_id: None,
model: None,
models: None,
archived: false,
pinned: false,
prompt_tokens: Some(0),
completion_tokens: Some(0),
pinned_goals: None,
pending_approvals: None,
created_at: "0".into(),
updated_at: "0".into(),
origin: None,
boundary_json: None,
loop_state: None,
status: "active".into(),
working_context_json: None,
})
.await
.expect("insert");
assert!(repo.set_loop_state("c1", Some("parked")).await.expect("set"));
let got = repo.get_by_id("c1").await.expect("get").expect("row");
assert_eq!(got.loop_state.as_deref(), Some("parked"), "写 parked 应可读回");
// 清态(running 收敛出口)可读回 None
assert!(repo.set_loop_state("c1", None).await.expect("clear"));
let cleared = repo.get_by_id("c1").await.expect("get").expect("row");
assert_eq!(cleared.loop_state, None, "清 loop_state 应可读回 None");
}
/// V50 契约:set_status 写终局态后可读回,且 insert 默认 active。
/// 锁死「会话终局态」建模能力,防 from_row 映射或 insert 列回归致默认值丢失。
#[tokio::test]
async fn set_status_roundtrip_and_default_active() {
let db = Database::open_in_memory().await.expect("open_in_memory");
let repo = AiConversationRepo::new(&db);
repo.insert(AiConversationRecord {
id: "c1".into(),
title: None,
messages: "[]".into(),
provider_id: None,
model: None,
models: None,
archived: false,
pinned: false,
prompt_tokens: Some(0),
completion_tokens: Some(0),
pinned_goals: None,
pending_approvals: None,
created_at: "0".into(),
updated_at: "0".into(),
origin: None,
boundary_json: None,
loop_state: None,
status: "active".into(),
working_context_json: None,
})
.await
.expect("insert");
let got = repo.get_by_id("c1").await.expect("get").expect("row");
assert_eq!(got.status, "active", "默认终局态应为 active");
assert!(repo.set_status("c1", "incomplete").await.expect("set"));
let got2 = repo.get_by_id("c1").await.expect("get").expect("row");
assert_eq!(got2.status, "incomplete", "写 incomplete 应可读回");
}
/// V53 契约:working_context_json 经 insert → get_by_id → update_full 全程原样存取
/// (存储层按不透明 TEXT,结构解析归调用方);老库行(缺列插入)读回 None 不炸。
#[tokio::test]
async fn working_context_json_roundtrip_and_legacy_null() {
let db = Database::open_in_memory().await.expect("open_in_memory");
let repo = AiConversationRepo::new(&db);
let wc_json = r#"{"goals":[{"id":"t1","text":"修登录bug","status":"Active"}],"decisions":[],"unresolved":["端口未定"],"current_step":null,"version":{"goals_changed_at":1,"decisions_changed_at":0,"unresolved_changed_at":0,"step_changed_at":0,"current_turn":1}}"#;
let mut rec = AiConversationRecord {
id: "c1".into(),
title: None,
messages: "[]".into(),
provider_id: None,
model: None,
models: None,
archived: false,
pinned: false,
prompt_tokens: Some(0),
completion_tokens: Some(0),
pinned_goals: None,
pending_approvals: None,
created_at: "0".into(),
updated_at: "0".into(),
origin: None,
boundary_json: None,
loop_state: None,
status: "active".into(),
working_context_json: Some(wc_json.into()),
};
repo.insert(rec.clone()).await.expect("insert");
let got = repo.get_by_id("c1").await.expect("get").expect("row");
assert_eq!(got.working_context_json.as_deref(), Some(wc_json), "insert 后应原样读回");
// update_full 全量写(save_conversation 路径)后列随行更新
rec.title = Some("t".into());
rec.working_context_json = Some(r#"{"goals":[],"decisions":[],"unresolved":[],"current_step":null,"version":{"goals_changed_at":0,"decisions_changed_at":0,"unresolved_changed_at":0,"step_changed_at":0,"current_turn":0}}"#.into());
repo.update_full(&rec).await.expect("update");
let got2 = repo.get_by_id("c1").await.expect("get").expect("row");
assert!(got2.working_context_json.as_deref().unwrap_or_default().contains(r#""goals":[]"#), "update_full 后应写新快照");
// 老库兼容:绕过 repo 直插不带该列的行(模拟 V53 前老数据)→ 读回 None
{
let conn = db.conn();
let g = conn.lock().await;
g.execute(
"INSERT INTO ai_conversations (id, messages, created_at, updated_at, status) \
VALUES ('legacy', '[]', '0', '0', 'active')",
[],
)
.expect("raw insert legacy row");
}
let legacy = repo.get_by_id("legacy").await.expect("get").expect("legacy row");
assert!(legacy.working_context_json.is_none(), "老库行应为 None(无 WorkingContext)");
}
} }
+415 -16
View File
@@ -44,11 +44,12 @@ const KNOWLEDGE_COLS_WITH_EMBEDDING: &str = concat!(
/// 定义 + 配合 `IDEA_COL_COUNT` 断言 + 测试 `test_idea_cols_matches_record`,加列漏改即捕获。 /// 定义 + 配合 `IDEA_COL_COUNT` 断言 + 测试 `test_idea_cols_matches_record`,加列漏改即捕获。
/// ///
/// V24 新增 related_ids 列(灵感间关联关系持久化打底),已纳入列清单 + 计数。 /// V24 新增 related_ids 列(灵感间关联关系持久化打底),已纳入列清单 + 计数。
/// V47 新增 archived 列(灵感归档,归档功能 P1),已纳入列清单 + 计数(14→15)。
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
const IDEA_COLS: &str = "id,title,description,status,priority,score,tags,source,promoted_to,ai_analysis,scores,related_ids,created_at,updated_at"; const IDEA_COLS: &str = "id,title,description,status,priority,score,tags,source,promoted_to,ai_analysis,scores,related_ids,archived,created_at,updated_at";
/// `IdeaRecord` 字段数(与上面列清单的逗号分隔项数一致,被测试断言)。 /// `IdeaRecord` 字段数(与上面列清单的逗号分隔项数一致,被测试断言)。
#[cfg_attr(not(test), allow(dead_code))] #[cfg_attr(not(test), allow(dead_code))]
const IDEA_COL_COUNT: usize = 14; const IDEA_COL_COUNT: usize = 15;
// ============================================================ // ============================================================
// 向量工具 — embedding BLOB 序列化 + 余弦相似度 // 向量工具 — embedding BLOB 序列化 + 余弦相似度
@@ -123,6 +124,8 @@ fn idea_from_row(row: &Row<'_>) -> std::result::Result<IdeaRecord, rusqlite::Err
ai_analysis: row.get("ai_analysis")?, ai_analysis: row.get("ai_analysis")?,
scores: row.get("scores")?, scores: row.get("scores")?,
related_ids: row.get("related_ids")?, related_ids: row.get("related_ids")?,
// 归档功能 P1 V47:15 列同步之一(SQLite 无真 BOOLEAN,0→false/非0→true)。
archived: row.get::<_, i32>("archived")? != 0,
created_at: row.get("created_at")?, created_at: row.get("created_at")?,
updated_at: row.get("updated_at")?, updated_at: row.get("updated_at")?,
}) })
@@ -218,22 +221,22 @@ impl_repo!(
from_row => |row| idea_from_row(row), from_row => |row| idea_from_row(row),
insert => |conn, rec| { insert => |conn, rec| {
conn.execute( conn.execute(
"INSERT INTO ideas (id, title, description, status, priority, score, tags, source, promoted_to, ai_analysis, scores, related_ids, created_at, updated_at) "INSERT INTO ideas (id, title, description, status, priority, score, tags, source, promoted_to, ai_analysis, scores, related_ids, archived, created_at, updated_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15)",
params![ params![
rec.id, rec.title, rec.description, rec.status.as_str(), rec.priority, rec.id, rec.title, rec.description, rec.status.as_str(), rec.priority,
rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis, rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis,
rec.scores, rec.related_ids, rec.created_at, rec.updated_at rec.scores, rec.related_ids, rec.archived as i32, rec.created_at, rec.updated_at
], ],
) )
}, },
update => |conn, rec| { update => |conn, rec| {
conn.execute( conn.execute(
"UPDATE ideas SET title = ?1, description = ?2, status = ?3, priority = ?4, score = ?5, tags = ?6, source = ?7, promoted_to = ?8, ai_analysis = ?9, scores = ?10, related_ids = ?11, updated_at = ?12 WHERE id = ?13", "UPDATE ideas SET title = ?1, description = ?2, status = ?3, priority = ?4, score = ?5, tags = ?6, source = ?7, promoted_to = ?8, ai_analysis = ?9, scores = ?10, related_ids = ?11, archived = ?12, updated_at = ?13 WHERE id = ?14",
params![ params![
rec.title, rec.description, rec.status.as_str(), rec.priority, rec.title, rec.description, rec.status.as_str(), rec.priority,
rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis, rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis,
rec.scores, rec.related_ids, rec.updated_at, rec.id rec.scores, rec.related_ids, rec.archived as i32, rec.updated_at, rec.id
], ],
) )
} }
@@ -306,7 +309,7 @@ impl IdeaRepo {
// 累积 WHERE 子句 + 收集参数(按出现顺序绑定占位符)。 // 累积 WHERE 子句 + 收集参数(按出现顺序绑定占位符)。
// deleted_at IS NULL 恒带(常量条件无占位符),回收站任务不进结果 // deleted_at IS NULL 恒带(常量条件无占位符),回收站任务不进结果
// (对标 task_repo list_by_query 同款模式,防回收站泄漏,不可被 query 关闭)。 // (对标 task_repo list_by_query 同款模式,防回收站泄漏,不可被 query 关闭)。
let mut where_clauses: Vec<String> = vec!["deleted_at IS NULL".to_string()]; let mut where_clauses: Vec<String> = vec!["deleted_at IS NULL".to_string(), "archived = 0".to_string()];
let mut params_vec: Vec<Box<dyn rusqlite::ToSql>> = Vec::new(); let mut params_vec: Vec<Box<dyn rusqlite::ToSql>> = Vec::new();
if let Some(s) = &status { if let Some(s) = &status {
@@ -428,7 +431,7 @@ impl IdeaRepo {
validate_column_name(field, "ideas")?; validate_column_name(field, "ideas")?;
let conn = self.conn.clone(); let conn = self.conn.clone();
let sql = format!( let sql = format!(
"UPDATE ideas SET {} = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL", "UPDATE ideas SET {} = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL AND archived = 0",
field field
); );
let id = id.to_owned(); let id = id.to_owned();
@@ -445,6 +448,54 @@ impl IdeaRepo {
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 归档/取消归档(归档功能 P1,V47)。
///
/// 纯元数据标记,仅作用于未软删灵感(回收站优先级高于归档,先恢复再操作)。
/// archived=true 后灵感退出主视图(list_by_query 恒带 archived = 0),仅在归档视图可见。
/// 对标 TaskRepo::set_archived。返回是否命中。
pub async fn set_archived(&self, id: &str, archived: bool) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = now_millis_str();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ideas SET archived = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL",
params![if archived { 1 } else { 0 }, now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 列出归档灵感(archived = 1 且未软删),按更新时间降序(最近归档在前)。
///
/// 归档视图数据源,与 list_by_query 对偶(恒带 archived = 1)。
pub async fn list_archived(&self) -> Result<Vec<IdeaRecord>> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut stmt = guard
.prepare(&format!(
"SELECT {IDEA_COLS} FROM ideas WHERE deleted_at IS NULL AND archived = 1 ORDER BY updated_at DESC"
))
.map_err(storage_err)?;
let rows = stmt
.query_map([], |row| idea_from_row(row))
.map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
/// 原子「立项认领」:CAS 写回 status=promoted + promoted_to(`WHERE id AND promoted_to IS NULL`)。 /// 原子「立项认领」:CAS 写回 status=promoted + promoted_to(`WHERE id AND promoted_to IS NULL`)。
/// ///
/// promote_idea 读-改-写竞态的原子关闭。promote_idea 先建项目再回写灵感, /// promote_idea 读-改-写竞态的原子关闭。promote_idea 先建项目再回写灵感,
@@ -496,16 +547,24 @@ impl IdeaRepo {
.transaction() .transaction()
.map_err(storage_err)?; .map_err(storage_err)?;
// 1. 读主体当前 related_ids // 1. 读主体当前 related_ids — 同段校验主体存在且未软删(deleted_at IS NULL)。
// 治走查🟡:原仅读 related_ids,主体不存在/已软删时步骤 5 UPDATE 静默 affected=0
// 仍提交,而步骤 3/4 已改写目标侧 related_ids → 「目标有关联、主体无记录」单向脏关联。
// 查无立即 Err 回滚整个事务,目标侧零改写。
let old_raw: Option<String> = tx let old_raw: Option<String> = tx
.query_row( .query_row(
"SELECT related_ids FROM ideas WHERE id = ?1", "SELECT related_ids FROM ideas WHERE id = ?1 AND deleted_at IS NULL",
params![subject_id], params![subject_id],
|row| row.get(0), |row| row.get(0),
) )
.optional() .optional()
.map_err(storage_err)? .map_err(storage_err)?
.flatten(); .flatten()
.ok_or_else(|| {
df_types::error::Error::Storage(format!(
"灵感 {subject_id} 不存在或已删除,无法建立关联"
))
})?;
// 2. 解析新旧集合 // 2. 解析新旧集合
let old_set: std::collections::HashSet<String> = let old_set: std::collections::HashSet<String> =
@@ -622,7 +681,8 @@ impl IdeaRepo {
/// ///
/// 不动 GUI 的无条件 [`update_full`](无 expected 语义,保持现状);本方法仅服务 /// 不动 GUI 的无条件 [`update_full`](无 expected 语义,保持现状);本方法仅服务
/// 需要乐观锁版本的调用方(df-mcp update_idea / score_idea)。 /// 需要乐观锁版本的调用方(df-mcp update_idea / score_idea)。
/// `Ok(false)` = id 不存在、已进回收站(deleted_at IS NULL 守卫,V28 灵感软删)或版本冲突。 /// `Ok(false)` = id 不存在、已进回收站(deleted_at IS NULL 守卫,V28 灵感软删)、
/// 已归档(archived = 0 守卫,V47 灵感归档,须先取消归档)或版本冲突。
pub async fn update_full_cas( pub async fn update_full_cas(
&self, &self,
record: &IdeaRecord, record: &IdeaRecord,
@@ -635,8 +695,9 @@ impl IdeaRepo {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let affected = guard let affected = guard
.execute( .execute(
// deleted_at IS NULL:回收站灵感不可被 CAS 覆盖(同 tasks/projects 同族守卫) // deleted_at IS NULL:回收站灵感不可被 CAS 覆盖(同 tasks/projects 同族守卫);
"UPDATE ideas SET title = ?1, description = ?2, status = ?3, priority = ?4, score = ?5, tags = ?6, source = ?7, promoted_to = ?8, ai_analysis = ?9, scores = ?10, related_ids = ?11, updated_at = ?12 WHERE id = ?13 AND updated_at = ?14 AND deleted_at IS NULL", // archived = 0:归档灵感冻结(同 tasks update_full_cas 同族守卫,须先取消归档再改)。
"UPDATE ideas SET title = ?1, description = ?2, status = ?3, priority = ?4, score = ?5, tags = ?6, source = ?7, promoted_to = ?8, ai_analysis = ?9, scores = ?10, related_ids = ?11, updated_at = ?12 WHERE id = ?13 AND updated_at = ?14 AND deleted_at IS NULL AND archived = 0",
params![ params![
rec.title, rec.description, rec.status.as_str(), rec.priority, rec.title, rec.description, rec.status.as_str(), rec.priority,
rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis, rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis,
@@ -762,6 +823,82 @@ impl KnowledgeRepo {
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 项目维度软排序检索(跨会话召回最小闭环,D2 拍板:软排序非硬过滤)。
///
/// 排序语义:同项目(source_project = ?)第一档 > 全局(IS NULL)第二档 > 异项目第三档;
/// 同档内保持原有 title 命中分 → reuse_count → created_at 顺序。`limit` 不变只改顺序,
/// token 膨胀为零。`project_id = None` 时行为与 [`Self::search_keywords`] 逐字一致(回归门)。
///
/// 与 [`Self::search_keywords`] 的关系:后者是本方法的 project_id=None 特例(旧签名
/// 保留给 MCP 等外部调用方,内部转调本方法,零重复实现)。
pub async fn search_keywords_scoped(
&self,
keywords: &[String],
include_candidates: bool,
limit: usize,
project_id: Option<&str>,
) -> Result<Vec<KnowledgeRecord>> {
if project_id.is_none() {
return self.search_keywords(keywords, include_candidates, limit).await;
}
if keywords.is_empty() {
return Ok(Vec::new());
}
let project_id = project_id.unwrap().to_string();
let conn = self.conn.clone();
// 每个关键词占一个编号占位符(1..n);project_id 占 ?(n+1),WHERE 的 ORDER BY CASE
// 与三档投影各复用同号(SQLite 允许 ?N 复用)。patterns 只装关键词一份。
let mut patterns: Vec<String> = Vec::with_capacity(keywords.len());
let mut where_parts: Vec<String> = Vec::with_capacity(keywords.len());
let mut order_terms: Vec<String> = Vec::with_capacity(keywords.len() + 1);
for (i, kw) in keywords.iter().enumerate() {
let n = i + 1;
where_parts.push(format!(
"(title LIKE ?{n} ESCAPE '|' OR content LIKE ?{n} ESCAPE '|')"
));
order_terms.push(format!("CASE WHEN title LIKE ?{n} ESCAPE '|' THEN 1 ELSE 0 END"));
patterns.push(format!("%{}%", escape_like(kw)));
}
// 项目档位表达式复用 ?(n+1):0=同项目 1=全局(NULL) 2=异项目
// 档位是**第一排序键**(ASC:0 最前),title 命分是同档内第二键(DESC)——
// 若档位与 title 分相加,档位会被 title 分稀释(异项目 title 高分压过同项目)。
let pid_n = keywords.len() + 1;
let project_tier =
format!("CASE WHEN source_project = ?{pid_n} THEN 0 WHEN source_project IS NULL THEN 1 ELSE 2 END");
let title_terms = order_terms.join(" + ");
let status_clause = if include_candidates {
"status IN ('published','candidate')"
} else {
"status = 'published'"
};
let limit_i = limit as i64;
let sql = format!(
"SELECT {KNOWLEDGE_COLS} FROM knowledges \
WHERE {status_clause} AND ({}) \
ORDER BY ({project_tier}) ASC, ({title_terms}) DESC, reuse_count DESC, created_at DESC LIMIT ?{}",
where_parts.join(" OR "),
pid_n + 1,
);
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut param_refs: Vec<&dyn rusqlite::ToSql> =
patterns.iter().map(|p| p as &dyn rusqlite::ToSql).collect();
param_refs.push(&project_id);
param_refs.push(&limit_i);
let mut stmt = guard.prepare(&sql).map_err(storage_err)?;
let rows = stmt
.query_map(param_refs.as_slice(), |row| knowledge_from_row(row))
.map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
/// 按状态列出(审核收件箱用): 按 confidence 语义排序(high>medium>low),次按 created_at /// 按状态列出(审核收件箱用): 按 confidence 语义排序(high>medium>low),次按 created_at
/// ///
/// 用 CASE WHEN 替代纯 TEXT 排序(字典序 high>low>medium 非预期语义)。 /// 用 CASE WHEN 替代纯 TEXT 排序(字典序 high>low>medium 非预期语义)。
@@ -1041,6 +1178,74 @@ impl KnowledgeRepo {
.await .await
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 知识路标读侧(devflow.md 投影通道):已发布且(人工验证 或 复用达标)的条目选摘。
///
/// 与检索族(search/search_keywords)的差别:路标无查询词,是「库里有什么」的静态选摘,
/// 供 system prompt 路标段铺开让 agent 知晓库藏、想起检索。入选门槛 = published 且
/// (verified = 1 OR reuse_count >= 3);排序 = 复用降序(唯一客观信号,对齐 search/top_used),
/// 同复用按 created_at 降序定稳定次序。verified 只作入选不作排序(复用是更强的实战信号)。
///
/// 两档拼接:项目档(source_project = project_id,上限 project_limit)在前——会话归属
/// 项目的知识最相关;跨项目补充档(非本项目含无归属条目,上限 global_limit)在后。
/// 两档 WHERE 互斥(= pid vs IS NULL/≠ pid),结果天然无重复。
/// project_id = None → 仅补充档(会话未解析出归属项目,项目档无从谈起)。
pub async fn list_waymark(
&self,
project_id: Option<&str>,
project_limit: usize,
global_limit: usize,
) -> Result<Vec<KnowledgeRecord>> {
let conn = self.conn.clone();
let project_id = project_id.map(|s| s.to_owned());
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
// 入选门槛与排序两档共用(verified 在 SQLite 存 0/1;复用达标压线 3 即入选)
let qualify = "status = 'published' AND (verified = 1 OR reuse_count >= 3)";
let order = " ORDER BY reuse_count DESC, created_at DESC LIMIT ";
let mut results = Vec::new();
// 项目档:会话归属项目内的合格条目(复用降序,上限 project_limit)
if let Some(pid) = &project_id {
let sql = format!(
"SELECT {KNOWLEDGE_COLS} FROM knowledges WHERE {qualify} AND source_project = ?1{order}?2"
);
let mut stmt = guard.prepare(&sql).map_err(storage_err)?;
let rows = stmt
.query_map(params![pid, project_limit as i64], knowledge_from_row)
.map_err(storage_err)?;
for r in rows {
results.push(r.map_err(storage_err)?);
}
}
// 跨项目补充档:非本项目(含无归属全局条目)按复用降序补足 global_limit;
// WHERE 与项目档互斥 → 天然去重。project_id = None 时无排他条件,即全量合格条目。
// LIMIT 占位符编号随排他条件有无而变(?1 空档排他时归 LIMIT,有排他时排他占 ?1、LIMIT 占 ?2)
let exclude = if project_id.is_some() {
" AND (source_project IS NULL OR source_project != ?1)"
} else {
""
};
let lim_n = if project_id.is_some() { 2 } else { 1 };
let sql = format!("SELECT {KNOWLEDGE_COLS} FROM knowledges WHERE {qualify}{exclude}{order}?{lim_n}");
let mut stmt = guard.prepare(&sql).map_err(storage_err)?;
let rows = match &project_id {
Some(pid) => stmt
.query_map(params![pid, global_limit as i64], knowledge_from_row)
.map_err(storage_err)?,
None => stmt
.query_map(params![global_limit as i64], knowledge_from_row)
.map_err(storage_err)?,
};
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
} }
impl_repo!( impl_repo!(
@@ -1491,6 +1696,76 @@ mod tests {
assert_eq!(hits[0].id, "pub1"); assert_eq!(hits[0].id, "pub1");
} }
// ---------- search_keywords_scoped(跨会话召回 D2 软排序) ----------
/// 同项目知识排前:同档内 title 命分定序,异档同项目压过异项目高复用
#[tokio::test]
async fn scoped_same_project_ranks_above_foreign_despite_reuse() {
let repo = setup_repo().await;
// 外项目知识:标题命中 + 复用 999(档位应压过复用)
let mut foreign = krec("foreign", "Rust 所有权", "...", "lesson", "published", None, 999);
foreign.source_project = Some("other-proj".to_string());
repo.insert(foreign).await.unwrap();
// 同项目(proj-1,krec 默认):内容命中 + 复用 1
repo.insert(krec("same", "无关标题", "聊聊 Rust 所有权", "lesson", "published", None, 1))
.await
.unwrap();
let kws = vec!["所有权".to_string(), "Rust".to_string()];
let hits = repo
.search_keywords_scoped(&kws, false, 10, Some("proj-1"))
.await
.unwrap();
assert_eq!(hits.len(), 2);
assert_eq!(hits[0].id, "same", "同项目知识应压过异项目(即便后者 title 命中+高复用)");
assert_eq!(hits[1].id, "foreign");
}
/// 全局知识(NULL)居中档:同项目 > 全局 > 异项目
#[tokio::test]
async fn scoped_global_null_tier_between_same_and_foreign() {
let repo = setup_repo().await;
let mut foreign = krec("foreign", "Rust 异步", "...", "lesson", "published", None, 1);
foreign.source_project = Some("other-proj".to_string());
repo.insert(foreign).await.unwrap();
let mut global = krec("global", "Rust 生命周期", "...", "lesson", "published", None, 1);
global.source_project = None;
repo.insert(global).await.unwrap();
repo.insert(krec("same", "Rust 借用", "...", "lesson", "published", None, 1))
.await
.unwrap();
let kws = vec!["Rust".to_string()];
let hits = repo
.search_keywords_scoped(&kws, false, 10, Some("proj-1"))
.await
.unwrap();
let ids: Vec<_> = hits.iter().map(|h| h.id.as_str()).collect();
assert_eq!(ids, vec!["same", "global", "foreign"], "三档顺序: 同项目>全局>异项目");
}
/// 回归门:project_id=None 与旧 search_keywords 行为逐字一致(同输入同顺序)
#[tokio::test]
async fn scoped_none_project_matches_legacy_search_keywords() {
let repo = setup_repo().await;
let mut foreign = krec("foreign", "Rust 异步", "...", "lesson", "published", None, 50);
foreign.source_project = Some("other-proj".to_string());
repo.insert(foreign).await.unwrap();
repo.insert(krec("same", "Rust 借用", "...", "lesson", "published", None, 1))
.await
.unwrap();
let kws = vec!["Rust".to_string()];
let legacy = repo.search_keywords(&kws, false, 10).await.unwrap();
let scoped = repo
.search_keywords_scoped(&kws, false, 10, None)
.await
.unwrap();
let a: Vec<_> = legacy.iter().map(|h| h.id.as_str()).collect();
let b: Vec<_> = scoped.iter().map(|h| h.id.as_str()).collect();
assert_eq!(a, b, "project_id=None 应与旧路径行为逐字一致(回归门)");
}
#[tokio::test] #[tokio::test]
async fn search_keywords_like_wildcards_escaped() { async fn search_keywords_like_wildcards_escaped() {
let repo = setup_repo().await; let repo = setup_repo().await;
@@ -1508,6 +1783,107 @@ mod tests {
assert_eq!(hits[0].id, "k1"); assert_eq!(hits[0].id, "k1");
} }
// ---------- 知识路标读侧(list_waymark,devflow.md 投影通道) ----------
/// 入选门槛:published 且(verified 或 复用达标);candidate/archived/复用不足均淘汰
#[tokio::test]
async fn waymark_qualifies_verified_or_reused_published_only() {
let repo = setup_repo().await;
// verified=true 复用 0 → 入选(验证豁免复用门槛)
let mut v = krec("v", "已验证条目", "...", "lesson", "published", None, 0);
v.verified = true;
repo.insert(v).await.unwrap();
// 未验证复用 3 → 入选(压线达标)
repo.insert(krec("r3", "复用达标条目", "...", "lesson", "published", None, 3))
.await
.unwrap();
// 未验证复用 2 → 淘汰
repo.insert(krec("r2", "复用不足条目", "...", "lesson", "published", None, 2))
.await
.unwrap();
// 复用再高,状态非 published(candidate)→ 淘汰
repo.insert(krec("cand", "候选未发布", "...", "lesson", "candidate", None, 99))
.await
.unwrap();
// archived → 淘汰
repo.insert(krec("arch", "已归档", "...", "lesson", "archived", None, 99))
.await
.unwrap();
let hits = repo.list_waymark(None, 20, 10).await.unwrap();
let ids: Vec<_> = hits.iter().map(|h| h.id.as_str()).collect();
assert_eq!(ids.len(), 2, "只应入选 已验证 + 复用达标 两条");
assert!(ids.contains(&"v") && ids.contains(&"r3"));
}
/// 两档序:项目档在前(复用降序,verified 豁免复用但仍按复用参与排序),
/// 跨项目补充档在后(复用降序);异项目高复用压不过同项目低复用(档位优先于复用)
#[tokio::test]
async fn waymark_project_tier_first_global_supplement_after() {
let repo = setup_repo().await;
repo.insert(krec("p9", "同项目复用9", "...", "lesson", "published", None, 9))
.await
.unwrap();
repo.insert(krec("p5", "同项目复用5", "...", "lesson", "published", None, 5))
.await
.unwrap();
// 同项目 verified 但零复用:入选项目档,排序落在复用条目之后
let mut pv = krec("pv", "同项目已验证零复用", "...", "lesson", "published", None, 0);
pv.verified = true;
repo.insert(pv).await.unwrap();
// 异项目高复用 → 补充档
let mut f = krec("f99", "异项目复用99", "...", "lesson", "published", None, 99);
f.source_project = Some("proj-2".to_string());
repo.insert(f).await.unwrap();
// 无归属全局条目 → 补充档
let mut g = krec("g10", "全局复用10", "...", "lesson", "published", None, 10);
g.source_project = None;
repo.insert(g).await.unwrap();
let hits = repo.list_waymark(Some("proj-1"), 20, 10).await.unwrap();
let ids: Vec<_> = hits.iter().map(|h| h.id.as_str()).collect();
assert_eq!(
ids,
vec!["p9", "p5", "pv", "f99", "g10"],
"项目档在前(复用降序),补充档在后(复用降序:异项目99 > 全局10)"
);
// 两档 WHERE 互斥的去重属性:任一条目恰好出现一次
assert_eq!(ids.iter().filter(|i| **i == "p9").count(), 1, "同项目条目不得因两档重复出现");
}
/// 各档上限独立生效:项目档取复用最高的 project_limit 条,补充档另计 global_limit 条
/// (条目复用全部压线达标,确保裁剪只来自 LIMIT 而非入选门槛)
#[tokio::test]
async fn waymark_limits_cap_tiers_independently() {
let repo = setup_repo().await;
for reuse in 3..=5 {
repo.insert(krec(&format!("p{reuse}"), "同项目", "...", "lesson", "published", None, reuse))
.await
.unwrap();
}
for reuse in 3..=5 {
let mut g = krec(&format!("g{reuse}"), "全局", "...", "lesson", "published", None, reuse);
g.source_project = None;
repo.insert(g).await.unwrap();
}
let hits = repo.list_waymark(Some("proj-1"), 2, 2).await.unwrap();
let ids: Vec<_> = hits.iter().map(|h| h.id.as_str()).collect();
assert_eq!(
ids,
vec!["p5", "p4", "g5", "g4"],
"项目档取复用最高的 2 条,补充档独立取全局最高的 2 条"
);
}
/// 空库 → 空结果(路标子块整体缺席的前提,投影侧据此零追加)
#[tokio::test]
async fn waymark_empty_db_returns_empty() {
let repo = setup_repo().await;
assert!(repo.list_waymark(Some("proj-1"), 20, 10).await.unwrap().is_empty());
assert!(repo.list_waymark(None, 20, 10).await.unwrap().is_empty());
}
// ---------- 复用即信任(auto_publish_if_reused) ---------- // ---------- 复用即信任(auto_publish_if_reused) ----------
#[tokio::test] #[tokio::test]
@@ -1837,7 +2213,7 @@ mod tests {
// ---------- IdeaRepo 软删回收站(对标 task_repo 软删测试,V28) ---------- // ---------- IdeaRepo 软删回收站(对标 task_repo 软删测试,V28) ----------
/// 构造一条 IdeaRecord fixture(14 字段全填,IdeaRecord 不含 deleted_at —— 纯 SQL 过滤)。 /// 构造一条 IdeaRecord fixture(15 字段全填,IdeaRecord 不含 deleted_at —— 纯 SQL 过滤)。
fn irec(id: &str, title: &str) -> IdeaRecord { fn irec(id: &str, title: &str) -> IdeaRecord {
IdeaRecord { IdeaRecord {
id: id.to_string(), id: id.to_string(),
@@ -1852,6 +2228,7 @@ mod tests {
ai_analysis: None, ai_analysis: None,
scores: None, scores: None,
related_ids: None, related_ids: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
} }
@@ -2023,6 +2400,28 @@ mod tests {
assert_eq!(after.updated_at, current.updated_at, "updated_at 不应抖动"); assert_eq!(after.updated_at, current.updated_at, "updated_at 不应抖动");
} }
#[tokio::test]
async fn idea_update_full_cas_skips_archived() {
// 归档灵感不可被 CAS 命中(archived = 0 守卫,V47 归档,对标 tasks 同族):
// 归档=冻结,须先取消归档(set_archived(false))再改,防 MCP 对归档灵感静默改写。
let repo = setup_idea_repo().await;
repo.insert(irec("i1", "原标题")).await.unwrap();
assert!(repo.set_archived("i1", true).await.unwrap());
// set_archived 刷新了 updated_at,取当前值作 expected(排除版本不匹配干扰)
let current = repo.get_by_id("i1").await.unwrap().unwrap();
let mut rec = current.clone();
rec.title = "归档里改标题".to_string();
rec.updated_at = "1900000000000".to_string();
let ok = repo
.update_full_cas(&rec, &current.updated_at)
.await
.unwrap();
assert!(!ok, "归档灵感 CAS 应拒绝(archived 守卫)");
let after = repo.get_by_id("i1").await.unwrap().unwrap();
assert_eq!(after.title, "原标题", "归档灵感内容不应被改写");
assert_eq!(after.updated_at, current.updated_at, "updated_at 不应抖动");
}
#[tokio::test] #[tokio::test]
async fn idea_update_full_cas_conflict_when_expected_stale() { async fn idea_update_full_cas_conflict_when_expected_stale() {
let repo = setup_idea_repo().await; let repo = setup_idea_repo().await;
+101 -11
View File
@@ -47,6 +47,7 @@ fn ai_message_from_row(row: &Row<'_>) -> std::result::Result<AiMessageRecord, ru
prompt_cache_miss_tokens: row.get("prompt_cache_miss_tokens")?, prompt_cache_miss_tokens: row.get("prompt_cache_miss_tokens")?,
reasoning_tokens: row.get("reasoning_tokens")?, reasoning_tokens: row.get("reasoning_tokens")?,
is_estimated: row.get("is_estimated")?, is_estimated: row.get("is_estimated")?,
source: row.get("source")?,
}) })
} }
@@ -87,8 +88,8 @@ impl AiMessageRepo {
(id, conversation_id, seq, role, content, parts, tool_call_id, (id, conversation_id, seq, role, content, parts, tool_call_id,
tool_calls, model, status, reasoning_content, timestamp, created_at, tool_calls, model, status, reasoning_content, timestamp, created_at,
prompt_tokens, completion_tokens, prompt_tokens, completion_tokens,
prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated) prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated, source)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20)",
) )
.map_err(storage_err)?; .map_err(storage_err)?;
for rec in &records { for rec in &records {
@@ -97,7 +98,7 @@ impl AiMessageRepo {
rec.parts, rec.tool_call_id, rec.tool_calls, rec.model, rec.status, rec.parts, rec.tool_call_id, rec.tool_calls, rec.model, rec.status,
rec.reasoning_content, rec.timestamp, rec.created_at, rec.reasoning_content, rec.timestamp, rec.created_at,
rec.prompt_tokens, rec.completion_tokens, rec.prompt_tokens, rec.completion_tokens,
rec.prompt_cache_hit_tokens, rec.prompt_cache_miss_tokens, rec.reasoning_tokens, rec.is_estimated rec.prompt_cache_hit_tokens, rec.prompt_cache_miss_tokens, rec.reasoning_tokens, rec.is_estimated, rec.source
]) ])
.map_err(storage_err)?; .map_err(storage_err)?;
} }
@@ -124,7 +125,7 @@ impl AiMessageRepo {
tool_calls, model, status, reasoning_content, timestamp, created_at, tool_calls, model, status, reasoning_content, timestamp, created_at,
prompt_tokens, completion_tokens, prompt_tokens, completion_tokens,
prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens,
is_estimated is_estimated, source
FROM ai_messages WHERE conversation_id = ?1 ORDER BY seq ASC", FROM ai_messages WHERE conversation_id = ?1 ORDER BY seq ASC",
) )
.map_err(storage_err)?; .map_err(storage_err)?;
@@ -171,13 +172,13 @@ impl AiMessageRepo {
"SELECT id, conversation_id, seq, role, content, parts, tool_call_id, "SELECT id, conversation_id, seq, role, content, parts, tool_call_id,
tool_calls, model, status, reasoning_content, timestamp, created_at, tool_calls, model, status, reasoning_content, timestamp, created_at,
prompt_tokens, completion_tokens, prompt_tokens, completion_tokens,
prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated, source
FROM ai_messages WHERE conversation_id = ?1 AND seq < ?2 ORDER BY seq DESC LIMIT ?3" FROM ai_messages WHERE conversation_id = ?1 AND seq < ?2 ORDER BY seq DESC LIMIT ?3"
} else { } else {
"SELECT id, conversation_id, seq, role, content, parts, tool_call_id, "SELECT id, conversation_id, seq, role, content, parts, tool_call_id,
tool_calls, model, status, reasoning_content, timestamp, created_at, tool_calls, model, status, reasoning_content, timestamp, created_at,
prompt_tokens, completion_tokens, prompt_tokens, completion_tokens,
prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated, source
FROM ai_messages WHERE conversation_id = ?1 ORDER BY seq DESC LIMIT ?2" FROM ai_messages WHERE conversation_id = ?1 ORDER BY seq DESC LIMIT ?2"
}; };
let mut stmt = guard.prepare(sql).map_err(storage_err)?; let mut stmt = guard.prepare(sql).map_err(storage_err)?;
@@ -264,6 +265,12 @@ impl AiMessageRepo {
/// 全量重写语义——以入参 records 为该对话的**唯一真相**,先删该 conv 全部旧行再批量插。 /// 全量重写语义——以入参 records 为该对话的**唯一真相**,先删该 conv 全部旧行再批量插。
/// 单事务保证「删 + 插」原子,无中间空窗(reload 不会读到半删半插的中间态)。 /// 单事务保证「删 + 插」原子,无中间空窗(reload 不会读到半删半插的中间态)。
/// ///
/// created_at 保出生时间(2026-10-17 修复批量污染):created_at 语义 = 消息首次入库时间,
/// 全量重写只该变内容不变出生时间。调用方(save_conversation)对整批 records 统一盖
/// 对话级 created_at(老对话 None 时 now 兜底),直插会把全对话消息 created_at 抹成同一秒。
/// 故 DELETE 前先取存量 `id → created_at` 映射,重插时同 id 行回填原值;新 id(真新消息)
/// 保留入参值。ai_messages 表无 updated_at 列,内容变更无单独时间戳(不在此列语义内)。
///
/// 设计权衡(非 dirty 增量,留 P2.1 优化): /// 设计权衡(非 dirty 增量,留 P2.1 优化):
/// - 内存 ContextManager 是运行时真相源,save 是同步点,每轮 save 全量回写简单可靠; /// - 内存 ContextManager 是运行时真相源,save 是同步点,每轮 save 全量回写简单可靠;
/// - INSERT OR IGNORE 幂等:records 内 id 重复或与残留行(理论不应有,事务已 DELETE)冲突跳过; /// - INSERT OR IGNORE 幂等:records 内 id 重复或与残留行(理论不应有,事务已 DELETE)冲突跳过;
@@ -273,7 +280,7 @@ impl AiMessageRepo {
pub async fn replace_conversation( pub async fn replace_conversation(
&self, &self,
conv_id: &str, conv_id: &str,
records: Vec<AiMessageRecord>, mut records: Vec<AiMessageRecord>,
) -> Result<()> { ) -> Result<()> {
let conn = self.conn.clone(); let conn = self.conn.clone();
let conv_id = conv_id.to_owned(); let conv_id = conv_id.to_owned();
@@ -281,7 +288,29 @@ impl AiMessageRepo {
let mut guard = conn.blocking_lock(); let mut guard = conn.blocking_lock();
let tx = guard.transaction().map_err(storage_err)?; let tx = guard.transaction().map_err(storage_err)?;
{ {
// 先删该 conv 全部旧行(全量重写语义) // 先取存量 id → created_at 映射(DELETE 前留证,重插回填保出生时间)
{
let mut stmt = tx
.prepare("SELECT id, created_at FROM ai_messages WHERE conversation_id = ?1")
.map_err(storage_err)?;
let rows = stmt
.query_map(params![conv_id], |r| {
Ok((r.get::<_, String>("id")?, r.get::<_, String>("created_at")?))
})
.map_err(storage_err)?;
let mut original: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
for r in rows {
let (id, created_at) = r.map_err(storage_err)?;
original.insert(id, created_at);
}
for rec in &mut records {
if let Some(orig) = original.get(&rec.id) {
rec.created_at = orig.clone();
}
}
}
// 再删该 conv 全部旧行(全量重写语义)
tx.execute( tx.execute(
"DELETE FROM ai_messages WHERE conversation_id = ?1", "DELETE FROM ai_messages WHERE conversation_id = ?1",
params![conv_id], params![conv_id],
@@ -296,8 +325,8 @@ impl AiMessageRepo {
(id, conversation_id, seq, role, content, parts, tool_call_id, (id, conversation_id, seq, role, content, parts, tool_call_id,
tool_calls, model, status, reasoning_content, timestamp, created_at, tool_calls, model, status, reasoning_content, timestamp, created_at,
prompt_tokens, completion_tokens, prompt_tokens, completion_tokens,
prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated) prompt_cache_hit_tokens, prompt_cache_miss_tokens, reasoning_tokens, is_estimated, source)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20)",
) )
.map_err(storage_err)?; .map_err(storage_err)?;
for rec in &records { for rec in &records {
@@ -306,7 +335,7 @@ impl AiMessageRepo {
rec.parts, rec.tool_call_id, rec.tool_calls, rec.model, rec.status, rec.parts, rec.tool_call_id, rec.tool_calls, rec.model, rec.status,
rec.reasoning_content, rec.timestamp, rec.created_at, rec.reasoning_content, rec.timestamp, rec.created_at,
rec.prompt_tokens, rec.completion_tokens, rec.prompt_tokens, rec.completion_tokens,
rec.prompt_cache_hit_tokens, rec.prompt_cache_miss_tokens, rec.reasoning_tokens, rec.is_estimated rec.prompt_cache_hit_tokens, rec.prompt_cache_miss_tokens, rec.reasoning_tokens, rec.is_estimated, rec.source
]) ])
.map_err(storage_err)?; .map_err(storage_err)?;
} }
@@ -381,6 +410,7 @@ mod tests {
prompt_cache_miss_tokens: None, prompt_cache_miss_tokens: None,
reasoning_tokens: None, reasoning_tokens: None,
is_estimated: None, is_estimated: None,
source: None,
} }
} }
@@ -544,6 +574,7 @@ mod tests {
prompt_cache_miss_tokens: None, prompt_cache_miss_tokens: None,
reasoning_tokens: None, reasoning_tokens: None,
is_estimated: None, is_estimated: None,
source: None,
}, },
AiMessageRecord { AiMessageRecord {
id: "new_1".into(), id: "new_1".into(),
@@ -565,6 +596,7 @@ mod tests {
prompt_cache_miss_tokens: None, prompt_cache_miss_tokens: None,
reasoning_tokens: None, reasoning_tokens: None,
is_estimated: None, is_estimated: None,
source: None,
}, },
]; ];
repo.replace_conversation("conv", records).await.expect("replace"); repo.replace_conversation("conv", records).await.expect("replace");
@@ -638,6 +670,7 @@ mod tests {
prompt_cache_miss_tokens: None, prompt_cache_miss_tokens: None,
reasoning_tokens: None, reasoning_tokens: None,
is_estimated: None, is_estimated: None,
source: None,
}], }],
) )
.await .await
@@ -680,6 +713,7 @@ mod tests {
prompt_cache_miss_tokens: None, prompt_cache_miss_tokens: None,
reasoning_tokens: None, reasoning_tokens: None,
is_estimated: None, is_estimated: None,
source: None,
}; };
repo.replace_conversation("c", vec![rec()]).await.expect("1st"); repo.replace_conversation("c", vec![rec()]).await.expect("1st");
repo.replace_conversation("c", vec![rec()]).await.expect("2nd"); repo.replace_conversation("c", vec![rec()]).await.expect("2nd");
@@ -687,6 +721,62 @@ mod tests {
assert_eq!(got.len(), 1, "重复 replace 不应叠加"); assert_eq!(got.len(), 1, "重复 replace 不应叠加");
} }
/// replace_conversation 保 created_at 出生时间(2026-10-17 批量污染回归):
/// 调用方对整批统一盖一个 created_at(如对话级时间或 now),重写后既有 id 行
/// 的 created_at 必须逐行保留原入库值,不得被统一时间抹平;新 id 行保留入参值。
#[tokio::test]
async fn replace_conversation_preserves_created_at() {
let db = Database::open_in_memory().await.expect("open_in_memory");
let repo = AiMessageRepo::new(&db);
// 预置两行,created_at 各异(模拟真实逐条入库的出生时间)
let mut m0 = mk_msg("m0", "c", 0, "user", "v1");
m0.created_at = "2026-10-01T10:00:00.000".into();
let mut m1 = mk_msg("m1", "c", 1, "assistant", "v1");
m1.created_at = "2026-10-02T11:30:00.000".into();
repo.insert_batch(vec![m0, m1]).await.expect("insert");
// 重写:既有 id 内容变了、created_at 被调用方统一盖成同一时间;
// 另带一条新 id 消息(真新消息,该保留入参时间)。
let uniform_ts = "2026-10-17T21:25:01.000";
let rewrite = |id: &str, seq: i64, role: &str, content: &str| {
let mut m = mk_msg(id, "c", seq, role, content);
m.created_at = uniform_ts.into();
m
};
repo.replace_conversation(
"c",
vec![
rewrite("m0", 0, "user", "v2-改写内容"),
rewrite("m1", 1, "assistant", "v2-改写内容"),
rewrite("m2_new", 2, "user", "新消息"),
],
)
.await
.expect("replace");
let got = repo.list_by_conversation("c").await.expect("list");
assert_eq!(got.len(), 3);
// 逐行断言:既有 id 的 created_at 保持原值,内容确实变了
assert_eq!(got[0].id, "m0");
assert_eq!(got[0].created_at, "2026-10-01T10:00:00.000", "既有行 created_at 不得被重写抹平");
assert_eq!(got[0].content, "v2-改写内容");
assert_eq!(got[1].id, "m1");
assert_eq!(got[1].created_at, "2026-10-02T11:30:00.000", "既有行 created_at 不得被重写抹平");
assert_eq!(got[1].content, "v2-改写内容");
// 新 id(首次入库)保留入参 created_at
assert_eq!(got[2].id, "m2_new");
assert_eq!(got[2].created_at, uniform_ts, "新消息保留入参 created_at");
// 二次重写再验:出生时间已固化,连续重写不漂移
repo.replace_conversation("c", vec![rewrite("m0", 0, "user", "v3")])
.await
.expect("replace again");
let got = repo.list_by_conversation("c").await.expect("list");
assert_eq!(got.len(), 1);
assert_eq!(got[0].created_at, "2026-10-01T10:00:00.000", "连续重写 created_at 不漂移");
}
/// limit=0 契约:返空集(直传 SQLite LIMIT 0),不再旧版 max(1) 钳成 1 条。 /// limit=0 契约:返空集(直传 SQLite LIMIT 0),不再旧版 max(1) 钳成 1 条。
/// 锁死语义防回归:请求 0 条却回 1 条会让调用方误判页内有数据。 /// 锁死语义防回归:请求 0 条却回 1 条会让调用方误判页内有数据。
#[tokio::test] #[tokio::test]
+4
View File
@@ -14,6 +14,7 @@
//! - [`mod@idea_repo`]:IdeaRepo/KnowledgeRepo/KnowledgeEventsRepo + 向量工具 //! - [`mod@idea_repo`]:IdeaRepo/KnowledgeRepo/KnowledgeEventsRepo + 向量工具
//! - [`mod@idea_eval_repo`]:IdeaEvalRepo(灵感评估历史追加型审计表 idea_evaluations,V22) //! - [`mod@idea_eval_repo`]:IdeaEvalRepo(灵感评估历史追加型审计表 idea_evaluations,V22)
//! - [`mod@message_repo`]:AiMessageRepo(消息拆分存储,全专用方法不走宏) //! - [`mod@message_repo`]:AiMessageRepo(消息拆分存储,全专用方法不走宏)
//! - [`mod@outbox_repo`]:OutboxRepo(待发箱 ai_outbox,V51,含 claim CAS/reset_stale_sending,手写不走宏)
//! //!
//! re-export(`pub use ...::*`)保持 `df_storage::crud::XxxRepo` / //! re-export(`pub use ...::*`)保持 `df_storage::crud::XxxRepo` /
//! `df_storage::crud::is_allowed_column` 路径不变,**调用方零改动**。 //! `df_storage::crud::is_allowed_column` 路径不变,**调用方零改动**。
@@ -25,6 +26,7 @@ mod conversation_repo;
mod idea_eval_repo; mod idea_eval_repo;
mod idea_repo; mod idea_repo;
mod message_repo; mod message_repo;
mod outbox_repo;
mod project_event_repo; mod project_event_repo;
mod project_module_repo; mod project_module_repo;
mod project_repo; mod project_repo;
@@ -38,6 +40,7 @@ pub use conversation_repo::*;
pub use idea_eval_repo::*; pub use idea_eval_repo::*;
pub use idea_repo::*; pub use idea_repo::*;
pub use message_repo::*; pub use message_repo::*;
pub use outbox_repo::*;
pub use project_event_repo::*; pub use project_event_repo::*;
pub use project_module_repo::*; pub use project_module_repo::*;
pub use project_repo::*; pub use project_repo::*;
@@ -72,6 +75,7 @@ macro_rules! impl_repo {
update => |$u_conn:ident, $u_rec:ident| $update_body:expr update => |$u_conn:ident, $u_rec:ident| $update_body:expr
) => { ) => {
$(#[$meta])* $(#[$meta])*
#[derive(Clone)]
pub struct $name { pub struct $name {
conn: Arc<Mutex<Connection>>, conn: Arc<Mutex<Connection>>,
} }
+619
View File
@@ -0,0 +1,619 @@
//! 待发箱 Repo:OutboxRepo(ai_outbox 表,V51)
//!
//! 队列与定时消息统一实体的持久层。因需按 id 逐条 CAS 更新(claim)与状态条件扫描,
//! 不走 `impl_repo!` 宏,手写(同 `SettingsRepo` 范式)。
//!
//! 设计见 docs/02-架构设计/专项设计/定时消息与待发箱持久化-设计-2026-10-10.md §3.2。
//!
//! 状态落点纪律(V21 教训):每项状态由调度器按项逐条 CAS 写,不依赖全局内存推算;
//! `reset_stale_sending` 在启动时把崩溃残留的 `sending` 复位为 `pending`,镜像无人值守引擎的
//! `in_flight` 泄漏处置。
use std::sync::Arc;
use rusqlite::{params, Connection, OptionalExtension, Row};
use tokio::sync::Mutex;
use df_types::error::Result;
use crate::db::Database;
use crate::models::AiOutboxRecord;
use super::storage_err;
// ============================================================
// 状态 / 类型字面量(单一真值来源,避免调度器与 Repo 各写一份裸串漂移)
// ============================================================
/// 待派发(初始态)。
pub const STATUS_PENDING: &str = "pending";
/// 已被调度器 CAS 占取、正在派发。
pub const STATUS_SENDING: &str = "sending";
/// 受理成功(已交给正典发送链)。
pub const STATUS_SENT: &str = "sent";
/// 受理失败且不再自动重试。
pub const STATUS_FAILED: &str = "failed";
/// 用户取消 / 清空。
pub const STATUS_CANCELED: &str = "canceled";
/// 尽快发(kind=queue,due_at 为空)。
pub const KIND_QUEUE: &str = "queue";
/// 定时发(kind=scheduled,due_at 为 epoch 毫秒)。
pub const KIND_SCHEDULED: &str = "scheduled";
/// 终态集合(不再被调度器扫描)。
const DONE_STATUSES: &str = "'sent','failed','canceled'";
fn outbox_from_row(row: &Row<'_>) -> std::result::Result<AiOutboxRecord, rusqlite::Error> {
Ok(AiOutboxRecord {
id: row.get("id")?,
conversation_id: row.get("conversation_id")?,
kind: row.get("kind")?,
text: row.get("text")?,
skill: row.get("skill")?,
model_override: row.get("model_override")?,
parts_json: row.get("parts_json")?,
spans_json: row.get("spans_json")?,
due_at: row.get("due_at")?,
status: row.get("status")?,
attempts: row.get("attempts")?,
last_error: row.get("last_error")?,
created_at: row.get("created_at")?,
updated_at: row.get("updated_at")?,
sent_at: row.get("sent_at")?,
})
}
/// 待发箱 Repo(表 `ai_outbox`)。
pub struct OutboxRepo {
conn: Arc<Mutex<Connection>>,
}
impl OutboxRepo {
pub fn new(db: &Database) -> Self {
Self { conn: db.conn() }
}
/// 插入一条待发项,返回 id。created_at/updated_at 由本方法取当前毫秒覆盖
/// (写入时机即落库时机,调用方传入的不可信)。
pub async fn insert(&self, record: AiOutboxRecord) -> Result<String> {
let conn = self.conn.clone();
let now = df_types::now_millis();
let mut rec = record;
rec.created_at = now;
rec.updated_at = now;
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let id = rec.id.clone();
guard
.execute(
"INSERT INTO ai_outbox
(id, conversation_id, kind, text, skill, model_override, parts_json,
spans_json, due_at, status, attempts, last_error, created_at, updated_at, sent_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15)",
params![
rec.id, rec.conversation_id, rec.kind, rec.text, rec.skill,
rec.model_override, rec.parts_json, rec.spans_json, rec.due_at,
rec.status, rec.attempts, rec.last_error, rec.created_at, rec.updated_at,
rec.sent_at
],
)
.map_err(storage_err)?;
Ok(id)
})
.await
.map_err(storage_err)?
}
/// 按 id 取单条(IPC 回显 / 测试核验用)。
pub async fn get_by_id(&self, id: &str) -> Result<Option<AiOutboxRecord>> {
let conn = self.conn.clone();
let id = id.to_owned();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let rec = guard
.query_row("SELECT * FROM ai_outbox WHERE id = ?1", params![id], outbox_from_row)
.optional()
.map_err(storage_err)?;
Ok(rec)
})
.await
.map_err(storage_err)?
}
/// 按会话取待发视图(前端 `state.queue` 的来源)。`include_done=false` 时排除终态,
/// 按 created_at 升序(先入先出,保序)。
pub async fn list_by_conv(
&self,
conversation_id: &str,
include_done: bool,
) -> Result<Vec<AiOutboxRecord>> {
let conn = self.conn.clone();
let conversation_id = conversation_id.to_owned();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let sql = if include_done {
"SELECT * FROM ai_outbox WHERE conversation_id = ?1 ORDER BY created_at ASC"
.to_string()
} else {
format!(
"SELECT * FROM ai_outbox WHERE conversation_id = ?1 \
AND status NOT IN ({DONE_STATUSES}) ORDER BY created_at ASC"
)
};
let mut stmt = guard.prepare(&sql).map_err(storage_err)?;
let rows = stmt
.query_map(params![conversation_id], outbox_from_row)
.map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
/// 全量列表(跨会话,「定时列表」抽屉数据源)。`include_done=false` 时排除终态,
/// 按 created_at 升序(与 list_by_conv 同序语义)。
pub async fn list_all(&self, include_done: bool) -> Result<Vec<AiOutboxRecord>> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let sql = if include_done {
"SELECT * FROM ai_outbox ORDER BY created_at ASC".to_string()
} else {
format!(
"SELECT * FROM ai_outbox \
WHERE status NOT IN ({DONE_STATUSES}) ORDER BY created_at ASC"
)
};
let mut stmt = guard.prepare(&sql).map_err(storage_err)?;
let rows = stmt.query_map([], outbox_from_row).map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
/// 取到点的待派发项(status='pending' 且 due_at 为空或已到),按 created_at 升序,至多 limit 条。
/// 命中 idx_ai_outbox_due。
pub async fn list_due(&self, now_ms: i64, limit: i64) -> Result<Vec<AiOutboxRecord>> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut stmt = guard
.prepare(
"SELECT * FROM ai_outbox \
WHERE status = 'pending' AND (due_at IS NULL OR due_at <= ?1) \
ORDER BY created_at ASC LIMIT ?2",
)
.map_err(storage_err)?;
let rows = stmt
.query_map(params![now_ms, limit], outbox_from_row)
.map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
/// CAS 占取:仅当项仍为 pending 时置 sending。changes==1 才返回 true,
/// 防并发双取(多轮/多消费方同取同一条时只有一个成功)。
pub async fn claim(&self, id: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET status = 'sending', updated_at = ?1 \
WHERE id = ?2 AND status = 'pending'",
params![now, id],
)
.map_err(storage_err)?;
Ok(affected == 1)
})
.await
.map_err(storage_err)?
}
/// 标记受理成功(终态)。清空 last_error。
pub async fn mark_sent(&self, id: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET status = 'sent', sent_at = ?1, updated_at = ?1, \
last_error = NULL WHERE id = ?2",
params![now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 标记永久失败(终态,不再自动重试)。
pub async fn mark_failed(&self, id: &str, err: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let err = err.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET status = 'failed', last_error = ?1, updated_at = ?2 \
WHERE id = ?3",
params![err, now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 瞬时失败回退:attempts+1、状态回 pending、记 last_error,返回新的 attempts。
/// 保留下轮再试(网络超时/会话忙场景天然重试),不删项——对齐「错误不清队列」纪律。
pub async fn revert_pending(&self, id: &str, err: Option<&str>) -> Result<i64> {
let conn = self.conn.clone();
let id = id.to_owned();
let err = err.map(|s| s.to_owned());
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
guard
.execute(
"UPDATE ai_outbox SET status = 'pending', attempts = attempts + 1, \
last_error = ?1, updated_at = ?2 WHERE id = ?3",
params![err, now, id],
)
.map_err(storage_err)?;
let attempts: i64 = guard
.query_row(
"SELECT attempts FROM ai_outbox WHERE id = ?1",
params![id],
|r| r.get(0),
)
.map_err(storage_err)?;
Ok(attempts)
})
.await
.map_err(storage_err)?
}
/// 改期:更新 due_at(NULL=改为尽快)。
pub async fn reschedule(&self, id: &str, due_at: Option<i64>) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET due_at = ?1, updated_at = ?2 WHERE id = ?3",
params![due_at, now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 手动重试入口:仅 failed → pending(attempts 归零=手动重试给全新预算,清 last_error)。
/// 其余状态不动返回 false——调度器自动重试走 revert_pending(瞬时失败保序),两者互补。
pub async fn requeue(&self, id: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET status = 'pending', attempts = 0, last_error = NULL, \
updated_at = ?1 WHERE id = ?2 AND status = 'failed'",
params![now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 取消单项。
pub async fn cancel(&self, id: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET status = 'canceled', updated_at = ?1 WHERE id = ?2",
params![now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 编辑待发文本。
pub async fn update_text(&self, id: &str, text: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let text = text.to_owned();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET text = ?1, updated_at = ?2 WHERE id = ?3",
params![text, now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 清空待发(pending→canceled,不物理删——保留行便于审计)。`conv_id=None` 时清全部会话。
/// 返回被取消的行数。
pub async fn clear(&self, conv_id: Option<&str>) -> Result<usize> {
let conn = self.conn.clone();
let conv_id = conv_id.map(|s| s.to_owned());
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = match conv_id {
Some(id) => guard.execute(
"UPDATE ai_outbox SET status = 'canceled', updated_at = ?1 \
WHERE status = 'pending' AND conversation_id = ?2",
params![now, id],
),
None => guard.execute(
"UPDATE ai_outbox SET status = 'canceled', updated_at = ?1 \
WHERE status = 'pending'",
params![now],
),
}
.map_err(storage_err)?;
Ok(affected)
})
.await
.map_err(storage_err)?
}
/// 启动复位:把崩溃半途残留的 `sending` 全部复位为 `pending`(进程重启时前者必为陈旧值)。
/// 返回复位的行数。镜像无人值守引擎的 in_flight 泄漏处置。
pub async fn reset_stale_sending(&self) -> Result<usize> {
let conn = self.conn.clone();
let now = df_types::now_millis();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE ai_outbox SET status = 'pending', updated_at = ?1 WHERE status = 'sending'",
params![now],
)
.map_err(storage_err)?;
Ok(affected)
})
.await
.map_err(storage_err)?
}
}
// ============================================================
// 单元测试 — 迁移可重入 / CAS 互斥 / sender 残留复位(内存 DB)
// ============================================================
#[cfg(test)]
mod tests {
use super::*;
use crate::db::Database;
/// 构造一条 pending 待发项(due_at=None=尽快)。
fn rec(id: &str, conv: &str) -> AiOutboxRecord {
AiOutboxRecord {
id: id.to_string(),
conversation_id: conv.to_string(),
kind: KIND_QUEUE.to_string(),
text: format!("msg-{id}"),
skill: None,
model_override: None,
parts_json: None,
spans_json: None,
due_at: None,
status: STATUS_PENDING.to_string(),
attempts: 0,
last_error: None,
created_at: 0,
updated_at: 0,
sent_at: None,
}
}
async fn setup() -> (Database, OutboxRepo) {
let db = Database::open_in_memory().await.expect("open_in_memory");
let repo = OutboxRepo::new(&db);
(db, repo)
}
/// V51 迁移可重入:全链在已建表库上重跑 again 不报错,ai_outbox 表与两索引仍在。
#[tokio::test]
async fn migration_v51_is_reentrant() {
let db = Database::open_in_memory().await.expect("open_in_memory");
let conn = db.conn();
let guard = conn.lock().await;
// 首次已由 open_in_memory 跑过;再显式重跑一遍全链(模拟版本号回退/崩溃重跑)。
crate::migrations::run(&guard).expect("重跑迁移不应失败(幂等)");
crate::migrations::run(&guard).expect("二次重跑仍应幂等");
// 表存在
let has_table: bool = guard
.query_row(
"SELECT COUNT(*) > 0 FROM sqlite_master WHERE type='table' AND name='ai_outbox'",
[],
|r| r.get(0),
)
.unwrap();
assert!(has_table, "ai_outbox 表应存在");
// 两索引存在
for idx in ["idx_ai_outbox_due", "idx_ai_outbox_conv"] {
let has_idx: bool = guard
.query_row(
"SELECT COUNT(*) > 0 FROM sqlite_master WHERE type='index' AND name=?1",
params![idx],
|r| r.get(0),
)
.unwrap();
assert!(has_idx, "索引 {idx} 应存在");
}
}
/// claim CAS 互斥:同一条只被占取一次,二次 claim 返回 false。
#[tokio::test]
async fn claim_is_mutually_exclusive() {
let (_db, repo) = setup().await;
repo.insert(rec("o1", "conv-1")).await.unwrap();
assert!(repo.claim("o1").await.unwrap(), "首次 claim 应成功");
assert!(!repo.claim("o1").await.unwrap(), "二次 claim 应失败(已非 pending)");
let got = repo.get_by_id("o1").await.unwrap().unwrap();
assert_eq!(got.status, STATUS_SENDING);
}
/// reset_stale_sending 复位崩溃残留:sending → pending,计数正确。
#[tokio::test]
async fn reset_stale_sending_recovers_crash_residue() {
let (_db, repo) = setup().await;
repo.insert(rec("o1", "conv-1")).await.unwrap();
repo.insert(rec("o2", "conv-1")).await.unwrap();
// 模拟两条被占取后进程崩溃(残留 sending)
assert!(repo.claim("o1").await.unwrap());
assert!(repo.claim("o2").await.unwrap());
assert_eq!(repo.get_by_id("o1").await.unwrap().unwrap().status, STATUS_SENDING);
let recovered = repo.reset_stale_sending().await.unwrap();
assert_eq!(recovered, 2, "两条残留 sending 应被复位");
assert_eq!(repo.get_by_id("o1").await.unwrap().unwrap().status, STATUS_PENDING);
assert_eq!(repo.get_by_id("o2").await.unwrap().unwrap().status, STATUS_PENDING);
}
/// list_due:只取到点的 pending(尽快项恒命中,未到点定时项不取),按 created_at 升序。
#[tokio::test]
async fn list_due_respects_due_and_order() {
let (_db, repo) = setup().await;
repo.insert(rec("o1", "conv-1")).await.unwrap();
// 未到点定时项(远未来)
let mut future = rec("o2", "conv-1");
future.kind = KIND_SCHEDULED.to_string();
future.due_at = Some(9_999_999_999_999);
repo.insert(future).await.unwrap();
// 已到点定时项
let mut due = rec("o3", "conv-1");
due.kind = KIND_SCHEDULED.to_string();
due.due_at = Some(0);
repo.insert(due).await.unwrap();
let got = repo.list_due(df_types::now_millis(), 32).await.unwrap();
let ids: Vec<&str> = got.iter().map(|r| r.id.as_str()).collect();
assert_eq!(ids, vec!["o1", "o3"], "仅到点的 pending,按 created_at 升序");
}
/// revert_pending:瞬时失败回退 pending + attempts 自增 + 记 last_error,项不丢。
#[tokio::test]
async fn revert_pending_keeps_item_and_bumps_attempts() {
let (_db, repo) = setup().await;
repo.insert(rec("o1", "conv-1")).await.unwrap();
repo.claim("o1").await.unwrap();
let n = repo.revert_pending("o1", Some("boom")).await.unwrap();
assert_eq!(n, 1);
let got = repo.get_by_id("o1").await.unwrap().unwrap();
assert_eq!(got.status, STATUS_PENDING, "回退后应可再次被调度");
assert_eq!(got.attempts, 1);
assert_eq!(got.last_error.as_deref(), Some("boom"));
}
/// cancel/clear 只动 pending:已终态项不受影响,clear 按会话范围限定。
#[tokio::test]
async fn clear_cancels_only_pending_in_scope() {
let (_db, repo) = setup().await;
repo.insert(rec("o1", "conv-1")).await.unwrap();
repo.insert(rec("o2", "conv-2")).await.unwrap();
repo.mark_sent("o1").await.unwrap();
let n = repo.clear(Some("conv-1")).await.unwrap();
assert_eq!(n, 0, "conv-1 已无 pending(sent 不算),不误改");
assert_eq!(repo.get_by_id("o1").await.unwrap().unwrap().status, STATUS_SENT);
let n = repo.clear(Some("conv-2")).await.unwrap();
assert_eq!(n, 1);
assert_eq!(repo.get_by_id("o2").await.unwrap().unwrap().status, STATUS_CANCELED);
}
/// list_all:跨会话全量返回;include_done=false 排除终态,created_at 升序。
#[tokio::test]
async fn list_all_covers_convs_and_filters_done() {
let (_db, repo) = setup().await;
repo.insert(rec("o1", "conv-1")).await.unwrap();
repo.insert(rec("o2", "conv-2")).await.unwrap();
let mut done = rec("o3", "conv-1");
done.status = STATUS_SENT.to_string();
repo.insert(done).await.unwrap();
let all = repo.list_all(true).await.unwrap();
assert_eq!(all.len(), 3, "include_done=true 全量(含跨会话)");
let active = repo.list_all(false).await.unwrap();
let ids: Vec<&str> = active.iter().map(|r| r.id.as_str()).collect();
assert_eq!(ids, vec!["o1", "o2"], "排除终态后仅两条 pending,升序");
}
/// requeue:仅 failed → pending 且 attempts 归零清错;pending/sent 不动返 false。
#[tokio::test]
async fn requeue_only_resets_failed() {
let (_db, repo) = setup().await;
// o1:pending → claim(占取)→ revert(attempts=1)→ mark_failed(failed,attempts 保留 1)
repo.insert(rec("o1", "conv-1")).await.unwrap();
repo.claim("o1").await.unwrap();
let attempts = repo.revert_pending("o1", Some("boom")).await.unwrap();
assert_eq!(attempts, 1);
repo.mark_failed("o1", "boom").await.unwrap();
assert!(repo.requeue("o1").await.unwrap(), "failed 应可重入队");
let got = repo.get_by_id("o1").await.unwrap().unwrap();
assert_eq!(got.status, STATUS_PENDING);
assert_eq!(got.attempts, 0, "手动重试给全新预算");
assert_eq!(got.last_error, None, "重入队清上次错误");
// pending 项:requeue 不动(返回 false)
repo.insert(rec("o2", "conv-1")).await.unwrap();
assert!(!repo.requeue("o2").await.unwrap(), "pending 非 failed,requeue 不动");
// sent 终态:requeue 不动
repo.insert(rec("o3", "conv-1")).await.unwrap();
repo.mark_sent("o3").await.unwrap();
assert!(!repo.requeue("o3").await.unwrap(), "sent 终态不可重入队");
}
}
@@ -278,6 +278,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
@@ -426,6 +427,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
@@ -255,6 +255,7 @@ mod tests {
idea_id: None, idea_id: None,
path: Some("/tmp".into()), path: Some("/tmp".into()),
stack: Some("rust".into()), stack: Some("rust".into()),
archived: false,
created_at: "0".to_string(), created_at: "0".to_string(),
updated_at: "0".to_string(), updated_at: "0".to_string(),
}) })
+142 -27
View File
@@ -1,4 +1,5 @@
//! 项目域 Repo:ProjectRepo / BranchRepo / ReleaseRepo / WorkflowRepo / NodeExecutionRepo //! 项目域 Repo:ProjectRepo / BranchRepo / ReleaseRepo / WorkflowRepo / NodeExecutionRepo
//! 项目域 Repo:ProjectRepo / BranchRepo / ReleaseRepo / WorkflowRepo / NodeExecutionRepo
use std::sync::Arc; use std::sync::Arc;
@@ -86,9 +87,9 @@ fn build_order_clause(order_by: Option<&str>) -> Result<String> {
/// 带「最近活跃时间」的项目记录(list_active_with_activity 专用返回结构)。 /// 带「最近活跃时间」的项目记录(list_active_with_activity 专用返回结构)。
/// ///
/// `last_active_at` = COALESCE(project_events 最新事件 created_at, projects.updated_at): /// `last_active_at` = MAX(tasks.updated_at 最新值, project_events 最新 created_at,
/// 反映业务活跃(任务/灵感/状态推进等事件),无事件项目回退 updated_at。供前端「最近活跃排序」 /// projects.updated_at):任务活跃为主信号(任务面是真实工作发生地),事件表已停写仅保留维度,
/// 展示,语义比 projects.updated_at(随元信息修改如改 description 也会刷新)更准确。 /// updated_at 兜底。供前端「最近活跃排序」展示与 system prompt 项目清单排序。
/// ///
/// Serialize 供 IPC 层直接序列化回前端(serde 字段名 snake_case,对齐 ProjectRecord)。 /// Serialize 供 IPC 层直接序列化回前端(serde 字段名 snake_case,对齐 ProjectRecord)。
#[derive(Debug, Clone, serde::Serialize)] #[derive(Debug, Clone, serde::Serialize)]
@@ -125,6 +126,7 @@ fn project_from_row(row: &Row<'_>) -> std::result::Result<ProjectRecord, rusqlit
idea_id: row.get("idea_id")?, idea_id: row.get("idea_id")?,
path: row.get("path")?, path: row.get("path")?,
stack: row.get("stack")?, stack: row.get("stack")?,
archived: row.get::<_, i32>("archived")? != 0,
created_at: row.get("created_at")?, created_at: row.get("created_at")?,
updated_at: row.get("updated_at")?, updated_at: row.get("updated_at")?,
}) })
@@ -200,20 +202,20 @@ impl_repo!(
from_row => |row| project_from_row(row), from_row => |row| project_from_row(row),
insert => |conn, rec| { insert => |conn, rec| {
conn.execute( conn.execute(
"INSERT INTO projects (id, name, description, status, idea_id, path, stack, created_at, updated_at) "INSERT INTO projects (id, name, description, status, idea_id, path, stack, archived, created_at, updated_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10)",
params![ params![
rec.id, rec.name, rec.description, rec.status.as_str(), rec.idea_id, rec.id, rec.name, rec.description, rec.status.as_str(), rec.idea_id,
rec.path, rec.stack, rec.created_at, rec.updated_at rec.path, rec.stack, rec.archived as i32, rec.created_at, rec.updated_at
], ],
) )
}, },
update => |conn, rec| { update => |conn, rec| {
conn.execute( conn.execute(
"UPDATE projects SET name = ?1, description = ?2, status = ?3, idea_id = ?4, path = ?5, stack = ?6, updated_at = ?7 WHERE id = ?8", "UPDATE projects SET name = ?1, description = ?2, status = ?3, idea_id = ?4, path = ?5, stack = ?6, archived = ?7, updated_at = ?8 WHERE id = ?9",
params![ params![
rec.name, rec.description, rec.status.as_str(), rec.idea_id, rec.name, rec.description, rec.status.as_str(), rec.idea_id,
rec.path, rec.stack, rec.updated_at, rec.id rec.path, rec.stack, rec.archived as i32, rec.updated_at, rec.id
], ],
) )
} }
@@ -226,7 +228,7 @@ impl ProjectRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, name, description, status, idea_id, path, stack, created_at, updated_at FROM projects WHERE deleted_at IS NULL ORDER BY created_at DESC") .prepare("SELECT id, name, description, status, idea_id, path, stack, archived, created_at, updated_at FROM projects WHERE deleted_at IS NULL AND archived = 0 ORDER BY created_at DESC")
.map_err(storage_err)?; .map_err(storage_err)?;
let rows = stmt let rows = stmt
.query_map([], |row| project_from_row(row)) .query_map([], |row| project_from_row(row))
@@ -241,30 +243,36 @@ impl ProjectRepo {
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 列出未删除项目,**按「最近活跃」排序**(LEFT JOIN project_events 取最新事件时间, /// 列出未删除项目,**按「最近活跃」排序**(三源取大:任务活跃为主信号)。
/// COALESCE 回退 updated_at)。
/// ///
/// 业务语义:项目「最近活跃」应反映业务事件(任务/灵感/状态推进),而非 projects.updated_at /// 业务语义:last_active_at = MAX(tasks.updated_at 最新值, project_events 最新 created_at,
/// (后者随元信息修改如改 description/path 也会刷新,不反映真实业务活跃)。本方法通过子查询 /// projects.updated_at)。**任务活跃为主信号**——ai-chat 会话/任务面是真实工作发生地,
/// 取每个项目 project_events 最新 created_at,无事件项目回退 updated_at,排序列统一可比。 /// 任务停更即项目停更;project_events 已停写(现网空表,保留维度向后兼容);
/// projects.updated_at 仅元信息修改(改 description/path 也会刷新,不反映真实业务活跃),
/// 仅作兜底。无任务无事件的项目回退 updated_at,排序列统一可比。
/// ///
/// 子查询而非 JOIN:project_events 每项目可能 0..N 条,JOIN 会展开需 DISTINCT,子查询 /// 子查询而非 JOIN:tasks/project_events 每项目可能 0..N 条,JOIN 会展开需 DISTINCT,
/// `(SELECT MAX(created_at) FROM project_events WHERE project_id = projects.id)` 每行一次 /// 标量子查询每行一次聚合,语义清晰无笛卡尔积风险(SQLite 多参 MAX 为标量取大,
/// 聚合,语义清晰无笛卡尔积风险。单用户桌面应用项目数小,无性能压力。 /// IFNULL 兜空子查询 NULL)。单用户桌面应用项目数小,无性能压力。
/// ///
/// 返回 `ProjectActivityRecord`(ProjectRecord + last_active_at 字段),供前端排序展示。 /// 返回 `ProjectActivityRecord`(ProjectRecord + last_active_at 字段),供前端排序展示
/// 与 system prompt「当前项目」清单(build_context_lists)按活跃度排序。
pub async fn list_active_with_activity(&self) -> Result<Vec<ProjectActivityRecord>> { pub async fn list_active_with_activity(&self) -> Result<Vec<ProjectActivityRecord>> {
let conn = self.conn.clone(); let conn = self.conn.clone();
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
// COALESCE(子查询最新事件, projects.updated_at):无事件项目回退 updated_at, // 三源取大(SQLite 多参 MAX=标量取大,IFNULL 兜空子查询):任务活跃为主信号,
// 保证所有项目有可比活跃时间。ORDER BY 该列 DESC,同时间 created_at DESC 兜底稳定。 // 事件表已停写仅保留维度,updated_at 兜底。ORDER BY 该列 DESC,同时间 created_at DESC
// 兜底稳定。
let mut stmt = guard let mut stmt = guard
.prepare( .prepare(
"SELECT id, name, description, status, idea_id, path, stack, created_at, updated_at, \ "SELECT id, name, description, status, idea_id, path, stack, archived, created_at, updated_at, \
COALESCE((SELECT MAX(created_at) FROM project_events WHERE project_id = projects.id), \ MAX(\
updated_at) AS last_active_at \ IFNULL((SELECT MAX(updated_at) FROM tasks WHERE project_id = projects.id AND deleted_at IS NULL), 0), \
FROM projects WHERE deleted_at IS NULL \ IFNULL((SELECT MAX(created_at) FROM project_events WHERE project_id = projects.id), 0), \
updated_at\
) AS last_active_at \
FROM projects WHERE deleted_at IS NULL AND archived = 0 \
ORDER BY last_active_at DESC, created_at DESC", ORDER BY last_active_at DESC, created_at DESC",
) )
.map_err(storage_err)?; .map_err(storage_err)?;
@@ -315,8 +323,8 @@ impl ProjectRepo {
// 动态拼 WHERE + 参数。WHERE 1=1 起点 + deleted_at IS NULL 固定基础条件, // 动态拼 WHERE + 参数。WHERE 1=1 起点 + deleted_at IS NULL 固定基础条件,
// 后续可选条件 AND 拼接(对齐 KnowledgeRepo::search 的 if-let 分支模式)。 // 后续可选条件 AND 拼接(对齐 KnowledgeRepo::search 的 if-let 分支模式)。
let mut sql = String::from( let mut sql = String::from(
"SELECT id, name, description, status, idea_id, path, stack, created_at, updated_at \ "SELECT id, name, description, status, idea_id, path, stack, archived, created_at, updated_at \
FROM projects WHERE deleted_at IS NULL", FROM projects WHERE deleted_at IS NULL AND archived = 0",
); );
// keyword 用 ?N 占位两次(name LIKE ? AND description LIKE ? 同 pattern), // keyword 用 ?N 占位两次(name LIKE ? AND description LIKE ? 同 pattern),
// 对齐 KnowledgeRepo::search 的 `params![pattern, pattern, ...]`。 // 对齐 KnowledgeRepo::search 的 `params![pattern, pattern, ...]`。
@@ -421,7 +429,7 @@ impl ProjectRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, name, description, status, idea_id, path, stack, created_at, updated_at FROM projects WHERE deleted_at IS NOT NULL ORDER BY updated_at DESC") .prepare("SELECT id, name, description, status, idea_id, path, stack, archived, created_at, updated_at FROM projects WHERE deleted_at IS NOT NULL ORDER BY updated_at DESC")
.map_err(storage_err)?; .map_err(storage_err)?;
let rows = stmt let rows = stmt
.query_map([], |row| project_from_row(row)) .query_map([], |row| project_from_row(row))
@@ -500,6 +508,54 @@ impl ProjectRepo {
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 归档/取消归档(归档功能 P2,V48)。
///
/// 纯元数据标记,仅作用于未软删项目(回收站优先级高于归档,先恢复再操作)。
/// archived=true 后项目退出主视图(list_active/list_active_with_activity/list_by_query
/// 恒带 archived = 0),仅在归档视图可见。对标 IdeaRepo::set_archived。返回是否命中。
/// 注:archived 不进 update_field 通用白名单,防工具层旁路改写(布尔语义走专用通道)。
pub async fn set_archived(&self, id: &str, archived: bool) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = now_millis_str();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE projects SET archived = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL",
params![if archived { 1 } else { 0 }, now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 列出归档项目(archived = 1 且未软删),按更新时间降序(最近归档在前)。
///
/// 归档视图数据源,与 list_active 对偶(恒带 archived = 1)。
/// 对标 IdeaRepo::list_archived。
pub async fn list_archived(&self) -> Result<Vec<ProjectRecord>> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut stmt = guard
.prepare("SELECT id, name, description, status, idea_id, path, stack, archived, created_at, updated_at FROM projects WHERE deleted_at IS NULL AND archived = 1 ORDER BY updated_at DESC")
.map_err(storage_err)?;
let rows = stmt
.query_map([], |row| project_from_row(row))
.map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
/// 彻底删除:事务级联删全部关联子表→projects(不可恢复) /// 彻底删除:事务级联删全部关联子表→projects(不可恢复)
/// ///
/// SQLite 已开 PRAGMA foreign_keys=ON 但表无 ON DELETE CASCADE,ALTER 改不了 FK 约束, /// SQLite 已开 PRAGMA foreign_keys=ON 但表无 ON DELETE CASCADE,ALTER 改不了 FK 约束,
@@ -718,6 +774,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
} }
@@ -789,4 +846,62 @@ mod tests {
assert_eq!(after.name, "原项目", "回收站项目内容不应被改写"); assert_eq!(after.name, "原项目", "回收站项目内容不应被改写");
assert_eq!(after.updated_at, current.updated_at, "updated_at 不应抖动"); assert_eq!(after.updated_at, current.updated_at, "updated_at 不应抖动");
} }
#[tokio::test]
async fn list_active_with_activity_orders_by_task_activity_over_created_at() {
// 回归(事故形状):项目 A created_at 较晚(创建序排前)但无任务/停更;项目 B
// created_at 较早但有一条 tasks.updated_at 最新 → 活跃序 B 排 A 前
// (任务活跃为主信号,治创建序误导 AI 把停摆项目当最高优先锚点)。
use crate::crud::task_repo::TaskRepo;
use crate::models::TaskRecord;
use df_types::types::TaskStatus;
let db = Database::open_in_memory().await.expect("open_in_memory");
let projects = ProjectRepo::new(&db);
let tasks = TaskRepo::new(&db);
// B 先建(早),A 后建(晚)——创建序会排 A 前
let mut early = prec("pb", "早建但活跃");
early.created_at = "1700000000000".to_string();
early.updated_at = "1700000000000".to_string();
let mut late = prec("pa", "晚建但停更");
late.created_at = "1700000010000".to_string();
late.updated_at = "1700000010000".to_string();
projects.insert(early).await.unwrap();
projects.insert(late).await.unwrap();
// B 挂一条最新任务;A 无任务 → B 活跃时间取任务 updated_at,A 回退自身 updated_at
let task = TaskRecord {
id: "tb".to_string(),
project_id: "pb".to_string(),
title: "B 的任务".to_string(),
description: String::new(),
status: TaskStatus::InProgress,
priority: 0,
branch_name: None,
assignee: None,
workflow_def_id: None,
base_branch: None,
review_rounds: 0,
output_json: None,
idea_id: None,
queue: "todo".to_string(),
parent_id: None,
content_json: None,
module_id: None,
archived: false,
created_at: "1700000000000".to_string(),
updated_at: "1790000000000".to_string(),
};
tasks.insert(task).await.unwrap();
let listed = projects.list_active_with_activity().await.unwrap();
assert_eq!(listed.len(), 2);
assert_eq!(listed[0].record.id, "pb", "任务活跃最新的早建项目应排前");
assert_eq!(listed[1].record.id, "pa", "无任务的晚建项目应排后");
assert_eq!(
listed[0].last_active_at, "1790000000000",
"活跃时间应取该项目任务 updated_at 最新值"
);
}
} }
@@ -319,6 +319,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
@@ -1,4 +1,5 @@
//! 任务横向关联 Repo:TaskLinkRepo(task_links 表,知识图谱 Phase 1 V29) //! 任务横向关联 Repo:TaskLinkRepo(task_links 表,知识图谱 Phase 1 V29)
//! 任务横向关联 Repo:TaskLinkRepo(task_links 表,知识图谱 Phase 1 V29)
//! //!
//! 表达任务间 depends_on / blocks / relates_to 关系,AI 拓扑排序编排调度的基础。 //! 表达任务间 depends_on / blocks / relates_to 关系,AI 拓扑排序编排调度的基础。
//! 对标设计 docs/02-架构设计/专项设计/项目知识图谱与任务队列系统-2026-06-26.md §2.2。 //! 对标设计 docs/02-架构设计/专项设计/项目知识图谱与任务队列系统-2026-06-26.md §2.2。
@@ -350,6 +351,7 @@ mod tests {
parent_id: None, parent_id: None,
content_json: None, content_json: None,
module_id: None, module_id: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
} }
@@ -375,6 +377,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
+140 -28
View File
@@ -1,4 +1,5 @@
//! 任务域 Repo:TaskRepo(含 advance_status_atomic 状态机收口) //! 任务域 Repo:TaskRepo(含 advance_status_atomic 状态机收口)
//! 任务域 Repo:TaskRepo(含 advance_status_atomic 状态机收口)
use std::sync::Arc; use std::sync::Arc;
@@ -15,6 +16,20 @@ use crate::models::TaskRecord;
use super::impl_repo; use super::impl_repo;
use super::{now_millis_str, storage_err, validate_column_name}; use super::{now_millis_str, storage_err, validate_column_name};
// ============================================================
// 列清单(防 COLS 漂移,对标 idea_repo.rs IDEA_COLS 先例)
// ============================================================
/// `tasks` 表 20 列(顺序与 TaskRecord 结构体一致,不含 deleted_at)。
///
/// 19→20 列(归档功能 P1 V46 加 archived)。此前散落 8 处内联列名串
/// (list_active/get_by_id_active/advance_status_atomic 回读/list_active_by_project/
/// list_by_query/get_children/move_task_queue/list_deleted),加列须处处同步,
/// 漏一处只在运行时 rusqlite 报错。集中一处 + from_row 按名取列,加列漏改测试期捕获。
pub(crate) const TASK_COLS: &str = "id, project_id, title, description, status, priority, \
branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, \
idea_id, queue, parent_id, content_json, module_id, archived, created_at, updated_at";
// ============================================================ // ============================================================
// from_row 辅助函数 // from_row 辅助函数
// ============================================================ // ============================================================
@@ -52,6 +67,8 @@ fn task_from_row(row: &Row<'_>) -> std::result::Result<TaskRecord, rusqlite::Err
content_json: row.get("content_json")?, content_json: row.get("content_json")?,
// 工程系统 V41:任务关联具体工程(module_id),18→19 列同步之一。 // 工程系统 V41:任务关联具体工程(module_id),18→19 列同步之一。
module_id: row.get("module_id")?, module_id: row.get("module_id")?,
// 归档功能 P1 V46:20 列同步之一(SQLite 无真 BOOLEAN,0→false/非0→true)。
archived: row.get::<_, i32>("archived")? != 0,
created_at: row.get("created_at")?, created_at: row.get("created_at")?,
updated_at: row.get("updated_at")?, updated_at: row.get("updated_at")?,
}) })
@@ -83,6 +100,10 @@ pub struct TaskQuery {
pub project_id: Option<String>, pub project_id: Option<String>,
/// 状态过滤(P1 下沉,命中 idx_tasks_status)。值集由上层 TaskStatus::is_valid 兜底校验。 /// 状态过滤(P1 下沉,命中 idx_tasks_status)。值集由上层 TaskStatus::is_valid 兜底校验。
pub status: Option<String>, pub status: Option<String>,
/// 状态排除集(1eff911c 缺陷修复,MCP list_tasks 无 status 过滤时默认排除
/// done/cancelled 终态,防存量终态任务挤占分页窗口)。与 status 互斥使用:
/// 同传时 status 等值优先,status_not_in 忽略(SQL 拼接顺序 status 先判)。
pub status_not_in: Option<Vec<String>>,
/// 优先级过滤(0=critical..3=low)。P3 基建就绪,视图暂不用。 /// 优先级过滤(0=critical..3=low)。P3 基建就绪,视图暂不用。
pub priority: Option<i32>, pub priority: Option<i32>,
/// 负责人过滤。P3 基建就绪,视图暂不用。 /// 负责人过滤。P3 基建就绪,视图暂不用。
@@ -138,26 +159,26 @@ impl_repo!(
from_row => |row| task_from_row(row), from_row => |row| task_from_row(row),
insert => |conn, rec| { insert => |conn, rec| {
conn.execute( conn.execute(
"INSERT INTO tasks (id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at) "INSERT INTO tasks (id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, archived, created_at, updated_at)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19)", VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17, ?18, ?19, ?20)",
params![ params![
rec.id, rec.project_id, rec.title, rec.description, rec.status.as_str(), rec.priority, rec.id, rec.project_id, rec.title, rec.description, rec.status.as_str(), rec.priority,
rec.branch_name, rec.assignee, rec.workflow_def_id, rec.base_branch, rec.branch_name, rec.assignee, rec.workflow_def_id, rec.base_branch,
rec.review_rounds, rec.output_json, rec.idea_id, rec.review_rounds, rec.output_json, rec.idea_id,
rec.queue, rec.parent_id, rec.content_json, rec.module_id, rec.queue, rec.parent_id, rec.content_json, rec.module_id,
rec.created_at, rec.updated_at rec.archived as i32, rec.created_at, rec.updated_at
], ],
) )
}, },
update => |conn, rec| { update => |conn, rec| {
conn.execute( conn.execute(
"UPDATE tasks SET project_id = ?1, title = ?2, description = ?3, status = ?4, priority = ?5, branch_name = ?6, assignee = ?7, workflow_def_id = ?8, base_branch = ?9, review_rounds = ?10, output_json = ?11, idea_id = ?12, queue = ?13, parent_id = ?14, content_json = ?15, module_id = ?16, updated_at = ?17 WHERE id = ?18", "UPDATE tasks SET project_id = ?1, title = ?2, description = ?3, status = ?4, priority = ?5, branch_name = ?6, assignee = ?7, workflow_def_id = ?8, base_branch = ?9, review_rounds = ?10, output_json = ?11, idea_id = ?12, queue = ?13, parent_id = ?14, content_json = ?15, module_id = ?16, archived = ?17, updated_at = ?18 WHERE id = ?19",
params![ params![
rec.project_id, rec.title, rec.description, rec.status.as_str(), rec.priority, rec.project_id, rec.title, rec.description, rec.status.as_str(), rec.priority,
rec.branch_name, rec.assignee, rec.workflow_def_id, rec.base_branch, rec.branch_name, rec.assignee, rec.workflow_def_id, rec.base_branch,
rec.review_rounds, rec.output_json, rec.idea_id, rec.review_rounds, rec.output_json, rec.idea_id,
rec.queue, rec.parent_id, rec.content_json, rec.module_id, rec.queue, rec.parent_id, rec.content_json, rec.module_id,
rec.updated_at, rec.id rec.archived as i32, rec.updated_at, rec.id
], ],
) )
} }
@@ -173,7 +194,7 @@ impl TaskRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at FROM tasks WHERE deleted_at IS NULL ORDER BY created_at DESC") .prepare(&format!("SELECT {TASK_COLS} FROM tasks WHERE deleted_at IS NULL AND archived = 0 ORDER BY created_at DESC"))
.map_err(storage_err)?; .map_err(storage_err)?;
let rows = stmt let rows = stmt
.query_map([], |row| task_from_row(row)) .query_map([], |row| task_from_row(row))
@@ -228,6 +249,34 @@ impl TaskRepo {
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 判定任务是否在回收站(soft_delete 已设 deleted_at)。
///
/// 专供推进链 CAS 失败后的**原因区分**:`advance_status_atomic` 的 CAS SQL 带
/// `deleted_at IS NULL` 条件,软删任务必然 affected==0 返回 None;调用方据此
/// 回查本方法,把「已变更(并发推进或旁路修改)」这个误导性错误,精确改写为
/// 「任务在回收站」,让 LLM/用户知道该先恢复而非重试。
///
/// 返回 true = 任务存在且 deleted_at 非 NULL(在回收站);false = 不存在或未删。
/// 单列轻查询(SELECT 1),不拉全记录,失败回查开销可忽略。
pub async fn is_deleted(&self, id: &str) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut stmt = guard
.prepare("SELECT 1 FROM tasks WHERE id = ?1 AND deleted_at IS NOT NULL")
.map_err(storage_err)?;
let found = stmt
.query_row(params![id], |_| Ok(()))
.optional()
.map_err(storage_err)?
.is_some();
Ok(found)
})
.await
.map_err(storage_err)?
}
/// 按 id 查未删除任务(`WHERE id AND deleted_at IS NULL`)。 /// 按 id 查未删除任务(`WHERE id AND deleted_at IS NULL`)。
/// ///
/// 专供 [`validate_parent_id`](super::task_validation) 的父任务存在性检查: /// 专供 [`validate_parent_id`](super::task_validation) 的父任务存在性检查:
@@ -240,7 +289,7 @@ impl TaskRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at FROM tasks WHERE id = ?1 AND deleted_at IS NULL") .prepare(&format!("SELECT {TASK_COLS} FROM tasks WHERE id = ?1 AND deleted_at IS NULL AND archived = 0"))
.map_err(storage_err)?; .map_err(storage_err)?;
let row = stmt let row = stmt
.query_row(params![id], |row| task_from_row(row)) .query_row(params![id], |row| task_from_row(row))
@@ -285,10 +334,10 @@ impl TaskRepo {
// 返回 None,杜绝回收站任务被推进(D-02 软删语义收口,一处关闭)。 // 返回 None,杜绝回收站任务被推进(D-02 软删语义收口,一处关闭)。
let sql = if bump_rounds { let sql = if bump_rounds {
"UPDATE tasks SET status = ?1, review_rounds = review_rounds + 1, updated_at = ?2 \ "UPDATE tasks SET status = ?1, review_rounds = review_rounds + 1, updated_at = ?2 \
WHERE id = ?3 AND status = ?4 AND deleted_at IS NULL" WHERE id = ?3 AND status = ?4 AND deleted_at IS NULL AND archived = 0"
} else { } else {
"UPDATE tasks SET status = ?1, updated_at = ?2 \ "UPDATE tasks SET status = ?1, updated_at = ?2 \
WHERE id = ?3 AND status = ?4 AND deleted_at IS NULL" WHERE id = ?3 AND status = ?4 AND deleted_at IS NULL AND archived = 0"
}; };
let affected = guard let affected = guard
.execute(sql, params![new_status, now, id, expected]) .execute(sql, params![new_status, now, id, expected])
@@ -298,7 +347,7 @@ impl TaskRepo {
} }
// 回读更新后的记录(含新 status / 累加后的 review_rounds / 新 updated_at)。 // 回读更新后的记录(含新 status / 累加后的 review_rounds / 新 updated_at)。
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at FROM tasks WHERE id = ?1") .prepare(&format!("SELECT {TASK_COLS} FROM tasks WHERE id = ?1"))
.map_err(storage_err)?; .map_err(storage_err)?;
let row = stmt let row = stmt
.query_row(params![id], |row| task_from_row(row)) .query_row(params![id], |row| task_from_row(row))
@@ -320,7 +369,7 @@ impl TaskRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at FROM tasks WHERE deleted_at IS NULL AND project_id = ?1 ORDER BY created_at DESC") .prepare(&format!("SELECT {TASK_COLS} FROM tasks WHERE deleted_at IS NULL AND archived = 0 AND project_id = ?1 ORDER BY created_at DESC"))
.map_err(storage_err)?; .map_err(storage_err)?;
let rows = stmt let rows = stmt
.query_map(params![pid], |row| task_from_row(row)) .query_map(params![pid], |row| task_from_row(row))
@@ -373,6 +422,7 @@ impl TaskRepo {
// 拷贝 query 字段进闭包('static 生命周期,spawn_blocking 要求) // 拷贝 query 字段进闭包('static 生命周期,spawn_blocking 要求)
let project_id = query.project_id.clone(); let project_id = query.project_id.clone();
let status = query.status.clone(); let status = query.status.clone();
let status_not_in = query.status_not_in.clone();
let priority = query.priority; let priority = query.priority;
let assignee = query.assignee.clone(); let assignee = query.assignee.clone();
let keyword = query.keyword.clone(); let keyword = query.keyword.clone();
@@ -389,7 +439,7 @@ impl TaskRepo {
// 真参数拿到 ?2 而 params_vec 只有 1 个元素 → rusqlite "needed 2, got 1"。 // 真参数拿到 ?2 而 params_vec 只有 1 个元素 → rusqlite "needed 2, got 1"。
// 用 params_vec.len()+1 保证 ?N 与参数位置严格对齐(N = 参数序号)。 // 用 params_vec.len()+1 保证 ?N 与参数位置严格对齐(N = 参数序号)。
// deleted_at IS NULL 恒带(常量条件无占位符),回收站任务不进结果(语义同 list_active)。 // deleted_at IS NULL 恒带(常量条件无占位符),回收站任务不进结果(语义同 list_active)。
let mut where_clauses: Vec<String> = vec!["deleted_at IS NULL".to_string()]; let mut where_clauses: Vec<String> = vec!["deleted_at IS NULL".to_string(), "archived = 0".to_string()];
let mut params_vec: Vec<Box<dyn rusqlite::ToSql>> = Vec::new(); let mut params_vec: Vec<Box<dyn rusqlite::ToSql>> = Vec::new();
if let Some(pid) = &project_id { if let Some(pid) = &project_id {
@@ -400,6 +450,22 @@ impl TaskRepo {
where_clauses.push(format!("status = ?{}", params_vec.len() + 1)); where_clauses.push(format!("status = ?{}", params_vec.len() + 1));
params_vec.push(Box::new(s.clone())); params_vec.push(Box::new(s.clone()));
} }
// status_not_in(1eff911c 修复):状态排除集,先拼占位符再统一绑值(两步分离,
// 避免边拼边绑的序号/值错位)。与 status 同传时 status 等值优先(互斥由调用方保证)。
if status.is_none() {
if let Some(excl) = &status_not_in {
if !excl.is_empty() {
let base = params_vec.len();
let placeholders: Vec<String> = (0..excl.len())
.map(|i| format!("?{}", base + i + 1))
.collect();
for v in excl {
params_vec.push(Box::new(v.clone()));
}
where_clauses.push(format!("status NOT IN ({})", placeholders.join(", ")));
}
}
}
if let Some(p) = priority { if let Some(p) = priority {
where_clauses.push(format!("priority = ?{}", params_vec.len() + 1)); where_clauses.push(format!("priority = ?{}", params_vec.len() + 1));
params_vec.push(Box::new(p)); params_vec.push(Box::new(p));
@@ -454,9 +520,7 @@ impl TaskRepo {
// 显式列出全部 19 列(含 V29 queue/parent_id/content_json 与 V41 module_id, // 显式列出全部 19 列(含 V29 queue/parent_id/content_json 与 V41 module_id,
// 不 SELECT deleted_at:TaskRecord 不带该字段,取了 from_row 会因未知列报错)。 // 不 SELECT deleted_at:TaskRecord 不带该字段,取了 from_row 会因未知列报错)。
let sql = format!( let sql = format!(
"SELECT id, project_id, title, description, status, priority, branch_name, \ "SELECT {TASK_COLS} \
assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, \
queue, parent_id, content_json, module_id, created_at, updated_at \
FROM tasks WHERE {} ORDER BY {} DESC{}", FROM tasks WHERE {} ORDER BY {} DESC{}",
where_clauses.join(" AND "), where_clauses.join(" AND "),
order_col, order_col,
@@ -511,7 +575,7 @@ impl TaskRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut where_clauses: Vec<String> = vec!["deleted_at IS NULL".to_string()]; let mut where_clauses: Vec<String> = vec!["deleted_at IS NULL".to_string(), "archived = 0".to_string()];
let mut params_vec: Vec<Box<dyn rusqlite::ToSql>> = Vec::new(); let mut params_vec: Vec<Box<dyn rusqlite::ToSql>> = Vec::new();
if let Some(ref pid) = project_id { if let Some(ref pid) = project_id {
@@ -581,7 +645,7 @@ impl TaskRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at FROM tasks WHERE deleted_at IS NULL AND parent_id = ?1 ORDER BY created_at ASC") .prepare(&format!("SELECT {TASK_COLS} FROM tasks WHERE deleted_at IS NULL AND archived = 0 AND parent_id = ?1 ORDER BY created_at ASC"))
.map_err(storage_err)?; .map_err(storage_err)?;
let rows = stmt let rows = stmt
.query_map(params![pid], |row| task_from_row(row)) .query_map(params![pid], |row| task_from_row(row))
@@ -618,7 +682,7 @@ impl TaskRepo {
let mut stmt = guard let mut stmt = guard
.prepare( .prepare(
"SELECT status, COUNT(*) AS cnt FROM tasks \ "SELECT status, COUNT(*) AS cnt FROM tasks \
WHERE deleted_at IS NULL AND parent_id = ?1 \ WHERE deleted_at IS NULL AND archived = 0 AND parent_id = ?1 \
GROUP BY status", GROUP BY status",
) )
.map_err(storage_err)?; .map_err(storage_err)?;
@@ -660,7 +724,7 @@ impl TaskRepo {
let affected = guard let affected = guard
.execute( .execute(
"UPDATE tasks SET status = ?1, updated_at = ?2 \ "UPDATE tasks SET status = ?1, updated_at = ?2 \
WHERE id = ?3 AND deleted_at IS NULL", WHERE id = ?3 AND deleted_at IS NULL AND archived = 0",
params![new_status, now, id], params![new_status, now, id],
) )
.map_err(storage_err)?; .map_err(storage_err)?;
@@ -716,11 +780,6 @@ impl TaskRepo {
let id = id.to_owned(); let id = id.to_owned();
let new_queue = new_queue.to_owned(); let new_queue = new_queue.to_owned();
let now = now_millis_str(); let now = now_millis_str();
// 显式列出全部 19 列(同 from_row 消费列,不 SELECT deleted_at:
// TaskRecord 不带该字段,取了 from_row 会因未知列报错)。
const TASK_COLS: &str = "id, project_id, title, description, status, priority, branch_name, \
assignee, workflow_def_id, base_branch, review_rounds, output_json, \
idea_id, queue, parent_id, content_json, module_id, created_at, updated_at";
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let mut guard = conn.blocking_lock(); let mut guard = conn.blocking_lock();
let tx = guard.transaction().map_err(storage_err)?; let tx = guard.transaction().map_err(storage_err)?;
@@ -729,7 +788,7 @@ impl TaskRepo {
let current: Option<TaskRecord> = { let current: Option<TaskRecord> = {
let mut stmt = tx let mut stmt = tx
.prepare(&format!( .prepare(&format!(
"SELECT {TASK_COLS} FROM tasks WHERE id = ?1 AND deleted_at IS NULL" "SELECT {TASK_COLS} FROM tasks WHERE id = ?1 AND deleted_at IS NULL AND archived = 0"
)) ))
.map_err(storage_err)?; .map_err(storage_err)?;
stmt.query_row(params![id], |row| task_from_row(row)) stmt.query_row(params![id], |row| task_from_row(row))
@@ -793,7 +852,7 @@ impl TaskRepo {
tokio::task::spawn_blocking(move || { tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock(); let guard = conn.blocking_lock();
let mut stmt = guard let mut stmt = guard
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, queue, parent_id, content_json, module_id, created_at, updated_at FROM tasks WHERE deleted_at IS NOT NULL ORDER BY updated_at DESC") .prepare(&format!("SELECT {TASK_COLS} FROM tasks WHERE deleted_at IS NOT NULL ORDER BY updated_at DESC"))
.map_err(storage_err)?; .map_err(storage_err)?;
let rows = stmt let rows = stmt
.query_map([], |row| task_from_row(row)) .query_map([], |row| task_from_row(row))
@@ -833,7 +892,7 @@ impl TaskRepo {
.execute( .execute(
// deleted_at IS NULL:回收站任务不可被 CAS 覆盖——MCP 拿旧 expected 也能 // deleted_at IS NULL:回收站任务不可被 CAS 覆盖——MCP 拿旧 expected 也能
// 命中已软删行,造成回收站内容被改且 updated_at 抖动,调用方无从感知。 // 命中已软删行,造成回收站内容被改且 updated_at 抖动,调用方无从感知。
"UPDATE tasks SET project_id = ?1, title = ?2, description = ?3, status = ?4, priority = ?5, branch_name = ?6, assignee = ?7, workflow_def_id = ?8, base_branch = ?9, review_rounds = ?10, output_json = ?11, idea_id = ?12, queue = ?13, parent_id = ?14, content_json = ?15, module_id = ?16, updated_at = ?17 WHERE id = ?18 AND updated_at = ?19 AND deleted_at IS NULL", "UPDATE tasks SET project_id = ?1, title = ?2, description = ?3, status = ?4, priority = ?5, branch_name = ?6, assignee = ?7, workflow_def_id = ?8, base_branch = ?9, review_rounds = ?10, output_json = ?11, idea_id = ?12, queue = ?13, parent_id = ?14, content_json = ?15, module_id = ?16, updated_at = ?17 WHERE id = ?18 AND updated_at = ?19 AND deleted_at IS NULL AND archived = 0",
params![ params![
rec.project_id, rec.title, rec.description, rec.status.as_str(), rec.priority, rec.project_id, rec.title, rec.description, rec.status.as_str(), rec.priority,
rec.branch_name, rec.assignee, rec.workflow_def_id, rec.base_branch, rec.branch_name, rec.assignee, rec.workflow_def_id, rec.base_branch,
@@ -859,7 +918,7 @@ impl TaskRepo {
validate_column_name(field, "tasks")?; validate_column_name(field, "tasks")?;
let conn = self.conn.clone(); let conn = self.conn.clone();
let sql = format!( let sql = format!(
"UPDATE tasks SET {} = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL", "UPDATE tasks SET {} = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL AND archived = 0",
field field
); );
let id = id.to_owned(); let id = id.to_owned();
@@ -875,6 +934,56 @@ impl TaskRepo {
.await .await
.map_err(storage_err)? .map_err(storage_err)?
} }
/// 归档/取消归档(归档功能 P1,V46)。
///
/// 纯元数据标记(对标 ConversationRepo::set_archived),仅作用于未软删任务:
/// 软删(回收站)优先级高于归档——回收站任务不可直接归档/取消归档,先恢复再操作。
/// archived=true 后任务冻结(活跃查询恒过滤,不可推进/跨池/字段更新),仅在归档视图可见。
/// 返回是否命中。
pub async fn set_archived(&self, id: &str, archived: bool) -> Result<bool> {
let conn = self.conn.clone();
let id = id.to_owned();
let now = now_millis_str();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let affected = guard
.execute(
"UPDATE tasks SET archived = ?1, updated_at = ?2 WHERE id = ?3 AND deleted_at IS NULL",
params![if archived { 1 } else { 0 }, now, id],
)
.map_err(storage_err)?;
Ok(affected > 0)
})
.await
.map_err(storage_err)?
}
/// 列出归档任务(archived = 1 且未软删),按更新时间降序(最近归档在前)。
///
/// 归档视图数据源。与 list_active 对偶:活跃查询恒带 archived = 0,
/// 本方法恒带 archived = 1,两集合互斥且并集 = 全部未删任务。
pub async fn list_archived(&self) -> Result<Vec<TaskRecord>> {
let conn = self.conn.clone();
tokio::task::spawn_blocking(move || {
let guard = conn.blocking_lock();
let mut stmt = guard
.prepare(&format!(
"SELECT {TASK_COLS} FROM tasks WHERE deleted_at IS NULL AND archived = 1 ORDER BY updated_at DESC"
))
.map_err(storage_err)?;
let rows = stmt
.query_map([], |row| task_from_row(row))
.map_err(storage_err)?;
let mut results = Vec::new();
for r in rows {
results.push(r.map_err(storage_err)?);
}
Ok(results)
})
.await
.map_err(storage_err)?
}
} }
#[cfg(test)] #[cfg(test)]
@@ -909,6 +1018,7 @@ mod tests {
parent_id: parent_id.map(|s| s.to_string()), parent_id: parent_id.map(|s| s.to_string()),
content_json: None, content_json: None,
module_id: None, module_id: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
} }
@@ -927,6 +1037,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
@@ -1029,6 +1140,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
@@ -103,6 +103,7 @@ mod tests {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
}) })
@@ -130,6 +131,7 @@ mod tests {
parent_id: parent_id.map(|s| s.to_string()), parent_id: parent_id.map(|s| s.to_string()),
content_json: None, content_json: None,
module_id: None, module_id: None,
archived: false,
created_at: "1700000000000".to_string(), created_at: "1700000000000".to_string(),
updated_at: "1700000000000".to_string(), updated_at: "1700000000000".to_string(),
} }
+468 -8
View File
@@ -1,6 +1,7 @@
//! 数据库迁移 — 建表 SQL 与版本管理 //! 数据库迁移 — 建表 SQL 与版本管理
//! 数据库迁移 — 建表 SQL 与版本管理
use crate::migrations_helpers::column_exists; use crate::migrations_helpers::{column_exists, table_exists};
use anyhow::Result; use anyhow::Result;
use rusqlite::Connection; use rusqlite::Connection;
@@ -53,7 +54,20 @@ pub fn run(conn: &Connection) -> Result<()> {
// 桌面侧栏/miniapp 列表按 origin 过滤子会话)。 // 桌面侧栏/miniapp 列表按 origin 过滤子会话)。
// V45 = 无人值守任务边界(ai_conversations.boundary_json TEXT,TaskBoundary JSON 快照, // V45 = 无人值守任务边界(ai_conversations.boundary_json TEXT,TaskBoundary JSON 快照,
// 发起时一次声明存续期生效,随会话同亡)。 // 发起时一次声明存续期生效,随会话同亡)。
let steps: [(i32, fn(&Connection) -> Result<()>); 45] = [ // V46/V47 = 任务/灵感归档(archived INTEGER NOT NULL DEFAULT 0,归档功能 P1,
// 对标 ai_conversations.archived V4;归档与软删正交:回收站=删除缓冲,归档=低频可见保检索)。
// V48 = 项目归档(projects.archived 列,归档功能 P2,同款三方对齐)。
// V49 = loop 生命周期标记(ai_conversations.loop_state TEXT,断点续跑底座:
// running/parked,重启扫描 running→parked 改写,侧栏「可恢复」标记)。
// V50 = 使用数据分析 schema 债补全(三项,详见 migrate_v50 文档):
// ai_conversations.status 会话终局态 / ai_tool_executions.duration_ms 纯执行耗时 /
// conversation_checkpoints.token_total 语义回填。
// V51 = 待发箱 ai_outbox 表(定时消息与待发箱持久化 P0 地基):队列/定时统一实体,
// 调度器按 status+due_at 索引到点派发。纯新增表,可逆=DROP TABLE ai_outbox。
// V52 = ai_messages.source 列(#958 消息溯源:外部渠道原始消息 ID JSON,NULL=桌面本端)。
// V53 = WorkingContext 持久化(ai_conversations.working_context_json TEXT,跨会话召回 L4
// 重启不失忆:save 随行写,会话载入与 parked 恢复读还原,NULL=无上下文行为同旧)。
let steps: [(i32, fn(&Connection) -> Result<()>); 53] = [
(1, migrate_v1), (1, migrate_v1),
(2, migrate_v2), (2, migrate_v2),
(3, migrate_v3), (3, migrate_v3),
@@ -99,6 +113,22 @@ pub fn run(conn: &Connection) -> Result<()> {
(43, migrate_v43), (43, migrate_v43),
(44, migrate_v44), (44, migrate_v44),
(45, migrate_v45), (45, migrate_v45),
// V46 = 任务归档(tasks.archived 列,归档功能 P1,对标 ai_conversations.archived V4);
// V47 = 灵感归档(ideas.archived 列,同款)。归档与软删正交:回收站=删除缓冲,归档=低频可见保检索。
(46, migrate_v46),
(47, migrate_v47),
// V48 = 项目归档(projects.archived 列,归档功能 P2,对标 V46/V47 同款)。
(48, migrate_v48),
// V49 = loop 生命周期标记(ai_conversations.loop_state,断点续跑底座)。
(49, migrate_v49),
// V50 = 使用数据分析 schema 债补全(status/duration_ms/token_total 三项)。
(50, migrate_v50),
// V51 = 待发箱 ai_outbox 表(定时消息与待发箱持久化 P0 地基)。
(51, migrate_v51),
// V52 = ai_messages 加 source 列(#958 消息溯源:飞书等外部渠道原始消息 ID)。
(52, migrate_v52),
// V53 = WorkingContext 持久化(ai_conversations.working_context_json,跨会话召回 L4)。
(53, migrate_v53),
]; ];
for (version, migrate_fn) in steps { for (version, migrate_fn) in steps {
@@ -1354,6 +1384,287 @@ fn migrate_v45(conn: &Connection) -> Result<()> {
Ok(()) Ok(())
} }
/// V46: 幂等补 tasks.archived 列(任务归档,归档功能 P1)
///
/// 归档语义对标 ai_conversations.archived(V4):纯元数据标记,与 status/queue/deleted_at 正交。
/// 「作废」走 status=cancelled,「完结」走 done/cancelled——归档是另一种动作:
/// 低频完结任务移出主视图但保留检索(回收站=删除缓冲不可见,归档=低频可见保检索)。
/// - `INTEGER NOT NULL DEFAULT 0`:老库行迁移后全为 0(未归档),行为零变化;
/// - 不进 update_field 白名单(settings.rs):归档走专用 set_archived(带埋点 + 布尔语义),
/// 对标 status 收口思路防旁路写;
/// - 活跃查询(list_active/list_by_query/count_by_query/list_active_by_project/get_children/
/// advance_status_atomic/move_task_queue)恒带 `archived = 0` 过滤,归档任务退出主视图与
/// 状态机(冻结:不可推进/不可跨池/不可字段更新,需先取消归档)。
/// - **幂等**:column_exists 守卫(同 V45 模式);**可逆**:`ALTER TABLE tasks DROP COLUMN archived`。
fn migrate_v46(conn: &Connection) -> Result<()> {
if !column_exists(conn, "tasks", "archived") {
conn.execute(
"ALTER TABLE tasks ADD COLUMN archived INTEGER NOT NULL DEFAULT 0",
[],
)?;
tracing::info!("v46: 补建 tasks.archived 列(任务归档)");
}
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [46])?;
tracing::info!("迁移 v46 完成: tasks 加 archived 列");
Ok(())
}
/// V47: 幂等补 ideas.archived 列(灵感归档,归档功能 P1)
///
/// 对标 V46 tasks.archived:rejected/deferred/已晋升灵感一键归档移出主视图,保留检索。
/// 灵感无状态机收口(status 走 idea_repo 通用路径),但归档同样不进 update_field 白名单,
/// 走专用 set_archived(布尔语义 + 防旁路)。活跃查询恒带 `archived = 0`。
/// - **幂等**:column_exists 守卫;**可逆**:`ALTER TABLE ideas DROP COLUMN archived`。
fn migrate_v47(conn: &Connection) -> Result<()> {
if !column_exists(conn, "ideas", "archived") {
conn.execute(
"ALTER TABLE ideas ADD COLUMN archived INTEGER NOT NULL DEFAULT 0",
[],
)?;
tracing::info!("v47: 补建 ideas.archived 列(灵感归档)");
}
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [47])?;
tracing::info!("迁移 v47 完成: ideas 加 archived 列");
Ok(())
}
/// V48: 幂等补 projects.archived 列(项目归档,归档功能 P2)
///
/// 对标 V46 tasks / V47 ideas 同款:归档与软删正交(回收站=删除缓冲,归档=低频可见保检索)。
/// archived 不进 update_field 通用白名单,走专用 set_archived(布尔语义 + 防旁路)。
/// 活跃查询(list_active/list_active_with_activity/list_by_query)恒带 `archived = 0`。
/// - **幂等**:column_exists 守卫;**可逆**:`ALTER TABLE projects DROP COLUMN archived`。
fn migrate_v48(conn: &Connection) -> Result<()> {
if !column_exists(conn, "projects", "archived") {
conn.execute(
"ALTER TABLE projects ADD COLUMN archived INTEGER NOT NULL DEFAULT 0",
[],
)?;
tracing::info!("v48: 补建 projects.archived 列(项目归档)");
}
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [48])?;
tracing::info!("迁移 v48 完成: projects 加 archived 列");
Ok(())
}
/// V49: 幂等补 ai_conversations.loop_state 列(断点续跑底座)
///
/// loop_state TEXT NULL:None=非任务会话/已收敛;"running"=任务 loop 在跑(spawn 时置起);
/// "parked"=重启扫描时 running 残留改写(侧栏标「任务中断可恢复」+ 一键恢复)。
/// 对齐无人值守长任务设计 §4.5 拍板 A(不自动续跑,标记+一键恢复)。
/// - **幂等**:column_exists 守卫;**可逆**:`ALTER TABLE ai_conversations DROP COLUMN loop_state`。
fn migrate_v49(conn: &Connection) -> Result<()> {
if !column_exists(conn, "ai_conversations", "loop_state") {
conn.execute(
"ALTER TABLE ai_conversations ADD COLUMN loop_state TEXT",
[],
)?;
tracing::info!("v49: 补建 ai_conversations.loop_state 列(断点续跑标记)");
}
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [49])?;
tracing::info!("迁移 v49 完成: ai_conversations 加 loop_state 列");
Ok(())
}
/// V50: 使用数据分析 schema 债补全(2026-10-04 实证三笔,归「V49 schema 迁移批」)。
///
/// 一次迁移承载三项相关改动,均为纯粹加列/回填,不改既有列语义:
///
/// ① ai_conversations.status TEXT NOT NULL DEFAULT 'active' —— 会话终局态。
/// 原仅靠侧栏内消息尾部推断「是否收尾」(17/89 会话停在 tool 结果无 assistant 收尾),
/// 无法直接统计未完成率。新增列建模终局:
/// active=进行中/默认(含正常完成,完成态精确写入待收尾单点接驳),incomplete=停靠/
/// 断流等未完成终局,error=循环错误终局。与 loop_state(运行期 running/parked)正交:
/// loop_state 管「能否续跑」,status 管「会话如何结束」。
/// ② ai_tool_executions.duration_ms INTEGER —— 纯执行耗时(毫秒)。
/// 原审计只有 requested_at→executed_at 区间,该区间在审批路径混入「等待人工批准」
/// 时长,耗时统计不可信。本列专记工具真正执行的墙钟毫秒(执行前→执行后),与审批
/// 等待解耦;NULL=老数据/未测路径(向后兼容)。
/// ③ conversation_checkpoints.token_total 回填 —— 原写入侧误用「消息条数」当 token 总量
/// (p50=120 vs 真实数万级,差 3 个量级)。写入侧已修为真实 token 累加(见 save 侧);
/// 本迁移对历史行按 snapshot 内逐条 prompt_tokens+completion_tokens 重算回填,使存量
/// 可比。仅当重算结果 > 0 才写(N 老快照无消息级 token 时保留原值不误清零)。
///
/// - **幂等**:①②`column_exists` 守卫,重复执行不报 duplicate column;③重算确定(同输入同输出),
/// 反复执行结果一致,且经 schema_version 版本号闸只跑一次。
/// - **可逆**:①`ALTER TABLE ai_conversations DROP COLUMN status`;
/// ②`ALTER TABLE ai_tool_executions DROP COLUMN duration_ms`;
/// ③无回退(DROP COLUMN 不会还原被覆盖的 token_total 旧值)——如需回退重建快照表
/// (老值本就失真无保留价值)。
fn migrate_v50(conn: &Connection) -> Result<()> {
// ① 会话终局态列(NOT NULL 常量默认,SQLite ALTER 合法)。
if table_exists(conn, "ai_conversations") && !column_exists(conn, "ai_conversations", "status") {
conn.execute(
"ALTER TABLE ai_conversations ADD COLUMN status TEXT NOT NULL DEFAULT 'active'",
[],
)?;
tracing::info!("v50: 补建 ai_conversations.status 列(会话终局态)");
}
// ② 纯执行耗时列(NULL=老数据/未测路径)。
if table_exists(conn, "ai_tool_executions")
&& !column_exists(conn, "ai_tool_executions", "duration_ms")
{
conn.execute(
"ALTER TABLE ai_tool_executions ADD COLUMN duration_ms INTEGER",
[],
)?;
tracing::info!("v50: 补建 ai_tool_executions.duration_ms 列(纯执行耗时)");
}
// ③ 存量 checkpoint token_total 按 snapshot 重算回填。
backfill_checkpoint_token_total(conn)?;
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [50])?;
tracing::info!("迁移 v50 完成: status/duration_ms 加列 + checkpoint token_total 回填");
Ok(())
}
/// V51: 待发箱 ai_outbox 表(定时消息与待发箱持久化 P0 地基)。
///
/// 对标 docs/02-架构设计/专项设计/定时消息与待发箱持久化-设计-2026-10-10.md §3.1。
/// 队列(931)与定时消息(935)本质是同一实体、不同触发条件,统一落本表:
/// 队列项 due_at=NULL(会话空闲即发),定时项 due_at=epoch_ms(到点且空闲即发)。
///
/// 选独立表而非 app_settings KV:outbox 需按 status+due_at 索引扫描、按 id 逐条 CAS 更新,
/// KV 整块 JSON 读改写有并发改期/取消互相覆盖的风险且不可索引。
///
/// - **幂等**:table_exists 守卫建表(同 V29/V30/V31 模式),索引 CREATE INDEX IF NOT EXISTS
/// 原生幂等;版本号 INSERT OR IGNORE 防崩溃重跑 PK 冲突。全链重跑/断点续跑均安全。
/// - **可逆**(回退方式,当前不执行):本表纯新增、不触碰旧数据,回退即
/// `DROP TABLE ai_outbox`(同时关调度器开关 df-outbox-enabled=false),无需数据搬迁。
/// - **状态落点**:pending|sending|sent|failed|canceled 由调度器按项逐条 CAS 写,不依赖全局
/// 内存推算(V21 教训:全局 COUNT 幂等窗口中途崩溃致永久残缺)。崩溃残留的 sending 由
/// 调度器启动时 reset_stale_sending 复位为 pending。
/// 迁移链顶点版本(单一真值来源):**新增迁移时必须同步 +1**,否则下方迁移测试会失败
/// (测试用它断言「全量迁移后 schema_version == 顶点」「重跑仍到顶点」——既防链中断,也免多处写死)。
pub const LATEST_SCHEMA_VERSION: i32 = 53;
fn migrate_v51(conn: &Connection) -> Result<()> {
if !table_exists(conn, "ai_outbox") {
conn.execute(
"CREATE TABLE ai_outbox (
id TEXT PRIMARY KEY,
conversation_id TEXT NOT NULL,
kind TEXT NOT NULL DEFAULT 'queue',
text TEXT NOT NULL,
skill TEXT,
model_override TEXT,
parts_json TEXT,
spans_json TEXT,
due_at INTEGER,
status TEXT NOT NULL DEFAULT 'pending',
attempts INTEGER NOT NULL DEFAULT 0,
last_error TEXT,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
sent_at INTEGER
)",
[],
)?;
tracing::info!("v51: 建 ai_outbox 表(待发箱)");
}
// 索引覆盖两条最高频扫描:调度器 list_due(status+due_at)/ 前端按会话取视图(conversation_id+status)。
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_ai_outbox_due ON ai_outbox(status, due_at)",
[],
)?;
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_ai_outbox_conv ON ai_outbox(conversation_id, status)",
[],
)?;
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [51])?;
tracing::info!("迁移 v51 完成: ai_outbox 表 + 两索引(定时消息与待发箱持久化)");
Ok(())
}
/// V52: ai_messages 加 source 列(#958 消息溯源)。TEXT JSON(MessageSource 序列化),
/// NULL=桌面本端。纯加列零回退风险(老数据恒 NULL=既有语义)。
fn migrate_v52(conn: &Connection) -> Result<()> {
if !column_exists(conn, "ai_messages", "source") {
conn.execute("ALTER TABLE ai_messages ADD COLUMN source TEXT", [])?;
tracing::info!("v52: ai_messages 加 source 列(消息溯源)");
}
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [52])?;
tracing::info!("迁移 v52 完成");
Ok(())
}
/// V53: WorkingContext 持久化 — ai_conversations 加 working_context_json TEXT 列。
///
/// 跨会话召回 L4(重启不失忆):WorkingContext(自动目标/决策/未解项/当前步骤/脏标记版本)
/// 此前纯内存,重启后压缩重锚定回填的决策与未解项全丢,恢复的会话半失忆续跑。本列随 save
/// 每轮随行写、会话载入与 parked 恢复读还原;NULL=无上下文(老会话/全空),行为同旧。
/// 目标快照裁决语义另批,本列只做存储载体。
/// - **幂等**:column_exists 守卫(同 V44/V45/V49 模式),崩溃重跑/全链两遍不炸;
/// - **可逆**:`ALTER TABLE ai_conversations DROP COLUMN working_context_json`(SQLite 3.35+)。
fn migrate_v53(conn: &Connection) -> Result<()> {
if !column_exists(conn, "ai_conversations", "working_context_json") {
conn.execute(
"ALTER TABLE ai_conversations ADD COLUMN working_context_json TEXT",
[],
)?;
tracing::info!("v53: 补建 ai_conversations.working_context_json 列(WorkingContext 持久化)");
}
conn.execute("INSERT OR IGNORE INTO schema_version (version) VALUES (?)", [53])?;
tracing::info!("迁移 v53 完成: ai_conversations 加 working_context_json 列");
Ok(())
}
/// 回填 conversation_checkpoints.token_total:按 snapshot JSON 内逐条消息的
/// prompt_tokens+completion_tokens 求真实总量。
///
/// 幂等:重算确定,重复执行相同结果;仅当重算值 > 0 且与现值不同才 UPDATE
/// (老快照无消息级 token → 重算 0 → 保留原值,不误清零)。表/列不存在时静默跳过
/// (极老库未跑到 V37,或手工删表)。
fn backfill_checkpoint_token_total(conn: &Connection) -> Result<()> {
// 表存在性探测:未跑到 V37 的极老库无此表,PRAGMA 返回空即跳过。
if !column_exists(conn, "conversation_checkpoints", "token_total") {
return Ok(());
}
// 收集待回填行到内存再逐条 UPDATE:避免同一连接上「先 prepare 读游标、再同连接 execute 写」
// 造成的隐式锁/游标失效,分批小表可接受。
let rows: Vec<(String, String, i64)> = {
let mut stmt = conn.prepare(
"SELECT id, snapshot, token_total FROM conversation_checkpoints",
)?;
let mapped = stmt.query_map([], |r| {
Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?, r.get::<_, i64>(2)?))
})?;
mapped.filter_map(|r| r.ok()).collect()
};
let mut updated = 0usize;
for (id, snapshot, old_total) in rows {
let recomputed = sum_snapshot_tokens(&snapshot);
if recomputed > 0 && recomputed != old_total {
conn.execute(
"UPDATE conversation_checkpoints SET token_total = ?1 WHERE id = ?2",
rusqlite::params![recomputed, id],
)?;
updated += 1;
}
}
if updated > 0 {
tracing::info!("v50: checkpoint token_total 回填 {updated} 行(消息条数→真实 token)");
}
Ok(())
}
/// 从消息 JSON 数组求 token 总量:逐条取 prompt_tokens + completion_tokens(UInt64)
/// 求和。字段缺失/类型不符按 0 计(老消息无消息级 token → 贡献 0)。解析失败返回 0。
///
/// 用 `serde_json::Value` 裸取(df-storage 不依赖 df-ai-core 的 ChatMessage,同 V21 迁移模式)。
fn sum_snapshot_tokens(snapshot: &str) -> i64 {
let Ok(msgs) = serde_json::from_str::<Vec<serde_json::Value>>(snapshot) else {
return 0;
};
msgs.iter()
.map(|m| {
let p = m.get("prompt_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
let c = m.get("completion_tokens").and_then(|v| v.as_i64()).unwrap_or(0);
p + c
})
.sum()
}
/// V21 建表 SQL — 消息拆分存储 ai_messages 表 /// V21 建表 SQL — 消息拆分存储 ai_messages 表
/// ///
/// 与 V9_SQL 中的 ai_messages 镜像(V9 给新库,此 const 给老库 V21 迁移用 IF NOT EXISTS)。 /// 与 V9_SQL 中的 ai_messages 镜像(V9 给新库,此 const 给老库 V21 迁移用 IF NOT EXISTS)。
@@ -2357,8 +2668,23 @@ mod tests {
column_exists(&conn, "tasks", "module_id"), column_exists(&conn, "tasks", "module_id"),
"tasks.module_id 列缺失(V41 加)" "tasks.module_id 列缺失(V41 加)"
); );
// tasks.archived(V46 归档功能 P1)
assert!(
column_exists(&conn, "tasks", "archived"),
"tasks.archived 列缺失(V46 加)"
);
// ideas.archived(V47 归档功能 P1)
assert!(
column_exists(&conn, "ideas", "archived"),
"ideas.archived 列缺失(V47 加)"
);
// projects.archived(V48 项目归档 P2)
assert!(
column_exists(&conn, "projects", "archived"),
"projects.archived 列缺失(V48 加)"
);
// 3. schema_version 应推进到 45(全量迁移成功落版本号) // 3. schema_version 应推进到 48(全量迁移成功落版本号)
let max_version: i64 = conn let max_version: i64 = conn
.query_row( .query_row(
"SELECT COALESCE(MAX(version), 0) FROM schema_version", "SELECT COALESCE(MAX(version), 0) FROM schema_version",
@@ -2367,9 +2693,9 @@ mod tests {
) )
.expect("查 schema_version 应成功"); .expect("查 schema_version 应成功");
assert_eq!( assert_eq!(
max_version, 45, max_version, LATEST_SCHEMA_VERSION as i64,
"全量迁移后 schema_version 应为 45(实际 {}),说明某条 migrate_vN 链路断在中间", "全量迁移后 schema_version 应为顶点 {}(实际 {}),说明某条 migrate_vN 链路断在中间",
max_version LATEST_SCHEMA_VERSION, max_version
); );
// 4. V40 抽查:project_modules 表 description / status 列存在(老库 ALTER 补,新库 V40 也跑) // 4. V40 抽查:project_modules 表 description / status 列存在(老库 ALTER 补,新库 V40 也跑)
@@ -2445,7 +2771,7 @@ mod tests {
|r| r.get(0), |r| r.get(0),
) )
.unwrap(); .unwrap();
assert_eq!(max_v, 45, "首轮应推进到 45"); assert_eq!(max_v, LATEST_SCHEMA_VERSION as i64, "首轮应推进到 {}", LATEST_SCHEMA_VERSION);
// 清空版本表强制全链第二遍(每步 execute 第二次) // 清空版本表强制全链第二遍(每步 execute 第二次)
conn.execute("DELETE FROM schema_version", []).unwrap(); conn.execute("DELETE FROM schema_version", []).unwrap();
@@ -2457,7 +2783,7 @@ mod tests {
|r| r.get(0), |r| r.get(0),
) )
.unwrap(); .unwrap();
assert_eq!(max_v2, 45, "重跑后应重新推进到 45"); assert_eq!(max_v2, LATEST_SCHEMA_VERSION as i64, "重跑后应重新推进到 {}", LATEST_SCHEMA_VERSION);
} }
// ============================================================ // ============================================================
@@ -2563,4 +2889,138 @@ mod tests {
.unwrap(); .unwrap();
assert!(got.is_some(), "boundary_json 应可写可读"); assert!(got.is_some(), "boundary_json 应可写可读");
} }
/// V50: 三列/回填一次承载,幂等重跑不炸。
/// ① ai_conversations.status 补建默认 'active';
/// ② ai_tool_executions.duration_ms 补建(可空);
/// ③ conversation_checkpoints.token_total 按 snapshot 重算(消息条数→真实 token)。
#[test]
fn v50_adds_columns_and_backfills_token_total() {
let conn = setup_legacy_db();
// 前置:最小库无 status/duration_ms 列(V45 之后建,老库缺)
assert!(!column_exists(&conn, "ai_conversations", "status"));
// V50 需要 checkpoint / tool_executions 表(正常由 V1/V9 建;此处手工建以隔离本迁移被测量)
conn.execute_batch(
"CREATE TABLE IF NOT EXISTS conversation_checkpoints (\
id TEXT PRIMARY KEY, conv_id TEXT NOT NULL, snapshot TEXT NOT NULL,\
token_total INTEGER NOT NULL, label TEXT, created_at TEXT NOT NULL);
INSERT INTO conversation_checkpoints (id, conv_id, snapshot, token_total, created_at)\
VALUES ('ck1','c1',\
'[{\"role\":\"assistant\",\"prompt_tokens\":150000,\"completion_tokens\":20000}]',\
1,'100');
CREATE TABLE IF NOT EXISTS ai_tool_executions (\
id TEXT PRIMARY KEY, tool_call_id TEXT NOT NULL, tool_name TEXT NOT NULL,\
arguments TEXT NOT NULL, requested_at TEXT NOT NULL);",
)
.expect("建 checkpoint/tool_executions 表 + 塞一条失真行");
migrate_v50(&conn).expect("v50 首次迁移应成功");
assert!(column_exists(&conn, "ai_conversations", "status"), "应有 status 列");
assert!(column_exists(&conn, "ai_tool_executions", "duration_ms"), "应有 duration_ms 列");
// ① 默认值 active 生效
conn.execute(
"INSERT INTO ai_conversations (id, messages, created_at, updated_at) VALUES ('c1','[]','0','0')",
[],
)
.unwrap();
let status: String = conn
.query_row("SELECT status FROM ai_conversations WHERE id = 'c1'", [], |r| r.get(0))
.unwrap();
assert_eq!(status, "active", "新列默认值应为 active");
// ③ 失真 token_total(1)被回填为真实和(150000+20000)
let tt: i64 = conn
.query_row("SELECT token_total FROM conversation_checkpoints WHERE id = 'ck1'", [], |r| r.get(0))
.unwrap();
assert_eq!(tt, 170_000, "应回填为 snapshot 内 token 真实和");
let v: i64 = conn
.query_row("SELECT MAX(version) FROM schema_version", [], |r| r.get(0))
.unwrap();
assert_eq!(v, 50, "应写入版本号 50");
// 幂等:列已存在时重跑走 column_exists 短路;回填重算确定(值不变)
migrate_v50(&conn).expect("列已存在时重跑应幂等不报错");
let tt2: i64 = conn
.query_row("SELECT token_total FROM conversation_checkpoints WHERE id = 'ck1'", [], |r| r.get(0))
.unwrap();
assert_eq!(tt2, 170_000, "重跑回填结果应一致");
}
/// V50 回填不误清零:老快照无消息级 token(重算=0)时保留原值(不 UPDATE)。
#[test]
fn v50_backfill_keeps_legacy_when_no_message_tokens() {
let conn = setup_legacy_db();
conn.execute_batch(
"CREATE TABLE IF NOT EXISTS conversation_checkpoints (\
id TEXT PRIMARY KEY, conv_id TEXT NOT NULL, snapshot TEXT NOT NULL,\
token_total INTEGER NOT NULL, label TEXT, created_at TEXT NOT NULL);
INSERT INTO conversation_checkpoints (id, conv_id, snapshot, token_total, created_at)\
VALUES ('ck2','c2','[{\"role\":\"user\",\"content\":\"hi\"}]', 42, '100');",
)
.expect("建表塞老快照");
migrate_v50(&conn).expect("v50 应成功");
let tt: i64 = conn
.query_row("SELECT token_total FROM conversation_checkpoints WHERE id = 'ck2'", [], |r| r.get(0))
.unwrap();
assert_eq!(tt, 42, "无消息级 token 的老快照应保留原值不误清零");
}
/// sum_snapshot_tokens 纯函数:字段缺失/非法 JSON 按 0 计。
#[test]
fn sum_snapshot_tokens_handles_missing_and_bad_json() {
assert_eq!(sum_snapshot_tokens("not json"), 0, "非法 JSON 应返回 0");
assert_eq!(sum_snapshot_tokens("[]"), 0, "空数组应为 0");
assert_eq!(
sum_snapshot_tokens(r#"[{"prompt_tokens":10},{"completion_tokens":5},{"foo":1}]"#),
15,
"逐条相加,缺字段按 0"
);
}
/// V53 幂等:最小老库补建 working_context_json 列;重跑(含版本号回退模拟崩溃断点)
/// 不炸不报 duplicate column;列可写可读(NULL 语义=无 WorkingContext)。
#[test]
fn v53_working_context_json_idempotent() {
let conn = setup_legacy_db();
assert!(
!column_exists(&conn, "ai_conversations", "working_context_json"),
"迁移前应无 working_context_json 列"
);
migrate_v53(&conn).expect("v53 首次迁移应成功");
assert!(column_exists(&conn, "ai_conversations", "working_context_json"), "应有 working_context_json 列");
let v: i64 = conn
.query_row("SELECT MAX(version) FROM schema_version", [], |r| r.get(0))
.unwrap();
assert_eq!(v, 53, "应写入版本号 53");
// 模拟崩溃断点:版本号回退但列已建 → 重跑走 column_exists 短路
conn.execute("DELETE FROM schema_version WHERE version = 53", []).unwrap();
migrate_v53(&conn).expect("列已存在时重跑应幂等跳过 ALTER 不报错");
let v2: i64 = conn
.query_row("SELECT MAX(version) FROM schema_version", [], |r| r.get(0))
.unwrap();
assert_eq!(v2, 53, "重跑后版本号恢复 53");
// 列可写可读;老会话行(不带该列插入)读回 NULL
conn.execute(
"INSERT INTO ai_conversations (id, messages, created_at, updated_at, working_context_json) \
VALUES ('c1', '[]', '0', '0', '{\"goals\":[],\"decisions\":[]}')",
[],
)
.unwrap();
conn.execute(
"INSERT INTO ai_conversations (id, messages, created_at, updated_at) VALUES ('c2', '[]', '0', '0')",
[],
)
.unwrap();
let got: Option<String> = conn
.query_row("SELECT working_context_json FROM ai_conversations WHERE id = 'c1'", [], |r| r.get(0))
.unwrap();
assert!(got.is_some(), "working_context_json 应可写可读");
let legacy: Option<String> = conn
.query_row("SELECT working_context_json FROM ai_conversations WHERE id = 'c2'", [], |r| r.get(0))
.unwrap();
assert!(legacy.is_none(), "老会话行应为 NULL(无 WorkingContext)");
}
} }
@@ -14,6 +14,19 @@ pub(crate) fn column_exists(conn: &Connection, table: &str, col: &str) -> bool {
false false
} }
/// 检测表是否存在(幂等迁移辅助)。
///
/// [`column_exists`] 对不存在的表与缺列都返回 false,无法区分——迁移里「加列」需先确认
/// 表在(否则 ALTER 报 no such table)。sqlite_master 精确查名(非 LIKE,避免前缀误命中)。
pub(crate) fn table_exists(conn: &Connection, table: &str) -> bool {
conn.query_row(
"SELECT COUNT(*) > 0 FROM sqlite_master WHERE type = 'table' AND name = ?1",
[table],
|r| r.get::<_, bool>(0),
)
.unwrap_or(false)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@@ -74,4 +87,16 @@ mod tests {
assert!(!column_exists(&conn, "probe_t", "prompt")); assert!(!column_exists(&conn, "probe_t", "prompt"));
assert!(!column_exists(&conn, "probe_t", "model_configs")); assert!(!column_exists(&conn, "probe_t", "model_configs"));
} }
/// table_exists 精确匹配表名,不误命中前缀/近似名(与 column_exists 区分表缺失与缺列)。
#[test]
fn table_exists_matches_exact_name() {
let conn = Connection::open_in_memory().expect("open in-memory db");
conn.execute_batch("CREATE TABLE probe_t (id TEXT);")
.expect("create probe table");
assert!(table_exists(&conn, "probe_t"));
assert!(!table_exists(&conn, "probe"), "不误命中前缀");
assert!(!table_exists(&conn, "probe_t2"), "不误命中扩展名");
assert!(!table_exists(&conn, "ghost"), "未建表返回 false");
}
} }
+65 -1
View File
@@ -23,6 +23,10 @@ pub struct IdeaRecord {
pub ai_analysis: Option<String>, // AI 分析结果 JSON pub ai_analysis: Option<String>, // AI 分析结果 JSON
pub scores: Option<String>, // 多维评分 JSON (feasibility/impact/urgency/overall) pub scores: Option<String>, // 多维评分 JSON (feasibility/impact/urgency/overall)
pub related_ids: Option<String>, // 关联灵感 id JSON 数组(同 tags 模式,为关联关系 UI 打底) pub related_ids: Option<String>, // 关联灵感 id JSON 数组(同 tags 模式,为关联关系 UI 打底)
/// 归档标记(归档功能 P1,V47 列)。纯元数据,archived=true 的灵感退出主视图,仅在归档视图可见。
/// DEFAULT 0 保证老库行非 NULL;#[serde(default)] 兼容旧前端 JSON(缺省 false)。
#[serde(default)]
pub archived: bool,
pub created_at: String, pub created_at: String,
pub updated_at: String, pub updated_at: String,
} }
@@ -60,6 +64,11 @@ pub struct ProjectRecord {
pub path: Option<String>, pub path: Option<String>,
/// 技术栈 JSON 数组字符串(如 ["rust","vue","tauri"],由探测填充,可空) /// 技术栈 JSON 数组字符串(如 ["rust","vue","tauri"],由探测填充,可空)
pub stack: Option<String>, pub stack: Option<String>,
/// 项目归档标记(归档功能 P2,V48)。true=归档态,退出主视图(list_active/
/// list_active_with_activity/list_by_query 恒带 archived = 0),仅在归档视图可见。
/// 与软删正交:回收站=删除缓冲,归档=低频可见保检索。#[serde(default)] 兼容旧前端 JSON。
#[serde(default)]
pub archived: bool,
pub created_at: String, pub created_at: String,
pub updated_at: String, pub updated_at: String,
} }
@@ -123,9 +132,13 @@ pub struct TaskRecord {
/// 一个项目可含多个工程(Monorepo/微服务),本字段让任务落到具体工程,是后续 /// 一个项目可含多个工程(Monorepo/微服务),本字段让任务落到具体工程,是后续
/// IPC / 前端 / AI 按工程筛选任务的数据基础。外键 REFERENCES project_modules(id) /// IPC / 前端 / AI 按工程筛选任务的数据基础。外键 REFERENCES project_modules(id)
/// ON DELETE SET NULL(module 删除时任务自动解关联)。可空(老任务无关联 → None)。 /// ON DELETE SET NULL(module 删除时任务自动解关联)。可空(老任务无关联 → None)。
/// #[serde(default)] 兼容旧前端 JSON(无该字段时为 None)。
#[serde(default)] #[serde(default)]
pub module_id: Option<String>, pub module_id: Option<String>,
/// 归档标记(归档功能 P1,V46 列)。与 status/queue/deleted_at 正交的纯元数据:
/// archived=true 的任务退出主视图/状态机(冻结,不可推进/跨池/字段更新),仅在归档视图可见。
/// DEFAULT 0 保证老库行非 NULL;#[serde(default)] 兼容旧前端 JSON(缺省 false)。
#[serde(default)]
pub archived: bool,
pub created_at: String, pub created_at: String,
pub updated_at: String, pub updated_at: String,
} }
@@ -377,6 +390,21 @@ pub struct AiConversationRecord {
/// 无人值守任务边界快照(V45):TaskBoundary JSON;None=未声明(现状逐次审批)。 /// 无人值守任务边界快照(V45):TaskBoundary JSON;None=未声明(现状逐次审批)。
/// 生命周期=会话存续(发起时一次声明,随会话同亡),save 随行写/switch 读还原。 /// 生命周期=会话存续(发起时一次声明,随会话同亡),save 随行写/switch 读还原。
pub boundary_json: Option<String>, pub boundary_json: Option<String>,
/// loop 生命周期标记(V49,断点续跑底座):None=非任务会话/已收敛清理;
/// Some("running")=任务 loop 在跑(spawn 时置起);Some("parked")=进程重启时检测到
/// running 残留 → 启动扫描改写,侧栏标「任务中断可恢复」+ 一键恢复。
/// 纯元数据标记不进 update_full 通用 UPDATE,走专用 set_loop_state。
#[serde(default)]
pub loop_state: Option<String>,
/// 会话终局态(V50):active=进行中/默认(含正常完成),incomplete=停靠/断流等未完成终局,
/// error=循环错误终局。与 loop_state(运行期 running/parked,管「能否续跑」)正交——
/// 本列回答「会话如何结束」,供未完成率等统计。不进 update_full,走专用 set_status 单点写。
pub status: String,
/// WorkingContext 持久化(V53,跨会话召回 L4 重启不失忆):WorkingContext JSON 序列化,
/// 存储层按不透明 TEXT 存取(结构归 df-ai)。NULL=无上下文(老会话/全空),行为同旧;
/// save 随行写/会话载入与 parked 恢复读还原,随会话级联(对齐 boundary_json 模式)。
#[serde(default)]
pub working_context_json: Option<String>,
} }
/// AI 工具执行审计记录 /// AI 工具执行审计记录
@@ -400,6 +428,11 @@ pub struct AiToolExecutionRecord {
pub requested_at: String, pub requested_at: String,
pub executed_at: Option<String>, pub executed_at: Option<String>,
pub decided_by: Option<String>, // human/auto pub decided_by: Option<String>, // human/auto
/// 纯执行耗时(毫秒,V50):工具真正执行的墙钟时长(执行前→执行后),与审批等待解耦。
/// None=老数据 / 未测路径(缓存命中/未真执行)。审计面板耗时统计用本列,替代
/// requested_at→executed_at(后者在审批路径混入等待人工批准时长)。
#[serde(default, skip_serializing_if = "Option::is_none")]
pub duration_ms: Option<i64>,
} }
/// 消息记录(ai_messages 表,消息拆分存储)。 /// 消息记录(ai_messages 表,消息拆分存储)。
@@ -451,6 +484,37 @@ pub struct AiMessageRecord {
/// 本轮 prompt 是否估算值(round_usage.prompt_tokens==0 → estimated_prompt 兜底打标)。 /// 本轮 prompt 是否估算值(round_usage.prompt_tokens==0 → estimated_prompt 兜底打标)。
/// reload 逐条回显「估算」标注;老消息 NULL → None(向前兼容)。 /// reload 逐条回显「估算」标注;老消息 NULL → None(向前兼容)。
pub is_estimated: Option<bool>, pub is_estimated: Option<bool>,
/// 消息来源 JSON(#958 溯源,MessageSource 序列化;V52 加列)。
/// NULL=桌面本端(既有语义);非空=外部渠道(飞书 message_id 等,可跳原消息追溯)
pub source: Option<String>,
}
/// 待发箱记录(ai_outbox 表,V51)。
///
/// 队列与定时消息统一实体:队列项 due_at=NULL(会话空闲即发),定时项 due_at=epoch_ms(到点且空闲即发)。
/// 表是唯一真相源(前端内存队列退化为该会话 outbox 的只读视图)。设计见
/// docs/02-架构设计/专项设计/定时消息与待发箱持久化-设计-2026-10-10.md §3.1。
///
/// kind:queue(尽快)| scheduled(定时);预留 task(937 对话内定时任务)。
/// status:pending|sending|sent|failed|canceled。时间戳均 epoch 毫秒(存储层不转字符串本表特例:
/// due_at 需按数值比较排序,created_at/updated_at/sent_at 同为毫秒整数便于调度器直接比较)。
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AiOutboxRecord {
pub id: String,
pub conversation_id: String,
pub kind: String,
pub text: String,
pub skill: Option<String>,
pub model_override: Option<String>,
pub parts_json: Option<String>,
pub spans_json: Option<String>,
pub due_at: Option<i64>,
pub status: String,
pub attempts: i64,
pub last_error: Option<String>,
pub created_at: i64,
pub updated_at: i64,
pub sent_at: Option<i64>,
} }
// ============================================================ // ============================================================
@@ -32,6 +32,7 @@ fn project(id: &str) -> ProjectRecord {
idea_id: None, idea_id: None,
path: None, path: None,
stack: None, stack: None,
archived: false,
created_at: now_ts(), created_at: now_ts(),
updated_at: now_ts(), updated_at: now_ts(),
} }
@@ -56,6 +57,7 @@ fn task(id: &str, project_id: &str) -> TaskRecord {
parent_id: None, parent_id: None,
content_json: None, content_json: None,
module_id: None, module_id: None,
archived: false,
created_at: now_ts(), created_at: now_ts(),
updated_at: now_ts(), updated_at: now_ts(),
} }
+41
View File
@@ -11,12 +11,25 @@
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
/// 当前 Hello 握手协议版本。
///
/// 协商规则:旧客户端 Hello 缺 `proto` 字段 → 视为 v1(向后兼容);
/// 高于对端支持版本时由 peer 记录告警,为后续破坏性变更预留通知位。
pub const HELLO_PROTO_VERSION: u32 = 1;
/// `proto` 缺省值:旧客户端不带该字段时按 v1 处理(向后兼容)。
fn default_proto() -> u32 {
HELLO_PROTO_VERSION
}
/// 握手身份宣告帧(对齐 u-relay::relay::Hello) /// 握手身份宣告帧(对齐 u-relay::relay::Hello)
/// ///
/// 桌面端 WS 连 `/ws/agent` 后,首条消息发此 JSON,relay 校验 kind 与 token。 /// 桌面端 WS 连 `/ws/agent` 后,首条消息发此 JSON,relay 校验 kind 与 token。
/// 字段名/serde rename 与 u-relay 严格一致(`#[serde(rename_all="snake_case")]`), /// 字段名/serde rename 与 u-relay 严格一致(`#[serde(rename_all="snake_case")]`),
/// 否则 relay 反序列化失败 → 握手被拒。 /// 否则 relay 反序列化失败 → 握手被拒。
/// ///
/// `proto` 为协议版本,供 relay 端做版本协商(缺字段按 v1,新增破坏性变更时递增)。
///
/// 注意:本 crate 不依赖 u-relay(避免 lib→server 反向依赖),自行镜像协议结构。 /// 注意:本 crate 不依赖 u-relay(避免 lib→server 反向依赖),自行镜像协议结构。
/// 协议字段如有变更须同步 u-relay::relay::Hello 与本类型。 /// 协议字段如有变更须同步 u-relay::relay::Hello 与本类型。
/// ///
@@ -30,6 +43,9 @@ pub struct Hello {
pub channel_id: String, pub channel_id: String,
/// 配对 token(env U_RELAY_TOKEN 同源) /// 配对 token(env U_RELAY_TOKEN 同源)
pub token: String, pub token: String,
/// 协议版本(缺省按 v1,兼容旧客户端)
#[serde(default = "default_proto")]
pub proto: u32,
} }
/// Hello.kind 传输表示(serde 字符串,与 u-relay::ClientKindWire 一致) /// Hello.kind 传输表示(serde 字符串,与 u-relay::ClientKindWire 一致)
@@ -155,3 +171,28 @@ pub enum TunnelMessage {
/// 云后端转发的指令(上行,发往桌面端) /// 云后端转发的指令(上行,发往桌面端)
Command(TunnelCommand), Command(TunnelCommand),
} }
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn hello_proto_defaults_to_v1_when_absent() {
// 旧客户端不发 proto → 反序列化按 v1,保证向后兼容
let json = r#"{"kind":"agent","channel_id":"c1","token":"t"}"#;
let h: Hello = serde_json::from_str(json).expect("缺 proto 应可解析");
assert_eq!(h.proto, HELLO_PROTO_VERSION);
}
#[test]
fn hello_serializes_proto_field() {
let h = Hello {
kind: HelloKind::Agent,
channel_id: "c1".into(),
token: "t".into(),
proto: HELLO_PROTO_VERSION,
};
let v: serde_json::Value = serde_json::to_value(&h).unwrap();
assert_eq!(v["proto"], HELLO_PROTO_VERSION);
}
}
+11 -5
View File
@@ -38,7 +38,7 @@ use tokio_tungstenite::tungstenite::protocol::CloseFrame;
use tokio_tungstenite::tungstenite::Message as WsMessage; use tokio_tungstenite::tungstenite::Message as WsMessage;
use crate::error::{Result, TunnelError}; use crate::error::{Result, TunnelError};
use crate::events::{Hello, HelloKind, RelayControlError, TunnelEvent}; use crate::events::{Hello, HelloKind, RelayControlError, TunnelEvent, HELLO_PROTO_VERSION};
/// 收到 commander→agent 指令 payload 时的回调类型(Boxed Future,在收发循环 task 内 await) /// 收到 commander→agent 指令 payload 时的回调类型(Boxed Future,在收发循环 task 内 await)
/// ///
@@ -105,9 +105,13 @@ const HANDSHAKE_PROBE: Duration = Duration::from_secs(3);
/// 包 `tokio::time::timeout` 在此窗口内未完成 → 返 TunnelError::Connect 显式错误。 /// 包 `tokio::time::timeout` 在此窗口内未完成 → 返 TunnelError::Connect 显式错误。
const CONNECT_TIMEOUT: Duration = Duration::from_secs(15); const CONNECT_TIMEOUT: Duration = Duration::from_secs(15);
/// 心跳间隔(应用层 Ping,补协议层 keepalive,防 NAT 连接表超时回收) /// 心跳间隔(WS 协议层 Ping,补 TCP keepalive,防 NAT/反代空闲回收连接)。
/// u-relay 心跳走应用层 control 帧 ping→pong,relay 本地直回不转对端 ///
const HEARTBEAT_INTERVAL: Duration = Duration::from_secs(25); /// 上界约束:链路最短空闲超时在反代侧(生产 nginx `proxy_read_timeout`,配置样例见
/// u-relay wire 协议文档 §5.2,最小 300s),间隔须显著小于它并留余量——容忍数次心跳
/// 丢失或网络抖动仍不被判定空闲断链。取 30s:满足约束,且与 commander 参考实现
/// (u-ppt 30s ping)节拍一致,避免两端看门狗口径不一导致误重连。
const HEARTBEAT_INTERVAL: Duration = Duration::from_secs(30);
/// 重连初始退避 /// 重连初始退避
const RECONNECT_BASE: Duration = Duration::from_secs(1); const RECONNECT_BASE: Duration = Duration::from_secs(1);
@@ -201,7 +205,8 @@ impl TunnelClient for WsTunnelClient {
))); )));
} }
}; };
tracing::info!(%url, "WS 连接已建立,开始 Hello 握手"); // 版本协商(本端宣告):Hello 携带 proto,供 relay 侧比对决定兼容性
tracing::info!(%url, proto = HELLO_PROTO_VERSION, "WS 连接已建立,开始 Hello 握手");
let (mut sink, mut stream) = ws_stream.split(); let (mut sink, mut stream) = ws_stream.split();
@@ -210,6 +215,7 @@ impl TunnelClient for WsTunnelClient {
kind: HelloKind::Agent, kind: HelloKind::Agent,
channel_id: channel_id.to_string(), channel_id: channel_id.to_string(),
token: token.to_string(), token: token.to_string(),
proto: HELLO_PROTO_VERSION,
}; };
let hello_json = serde_json::to_string(&hello)?; let hello_json = serde_json::to_string(&hello)?;
sink.send(WsMessage::Text(hello_json)).await.map_err(|e| { sink.send(WsMessage::Text(hello_json)).await.map_err(|e| {
+4
View File
@@ -315,6 +315,10 @@ impl ProjectStatus {
/// ///
/// Input Augmentation 层 ProjectResolver/TaskResolver 把记录的 String status 投影成 /// Input Augmentation 层 ProjectResolver/TaskResolver 把记录的 String status 投影成
/// 类型化枚举用。未知字符串返 None。 /// 类型化枚举用。未知字符串返 None。
///
/// ⚠️ TYPES-2:本函数返 None 是显式信号,但若调用方用 `unwrap_or_default()` 兜底,
/// 未知值会**静默归 Planning**(ProjectStatus::default),掩盖数据异常。建议调用方对
/// None 分支记 warn 审计(而非静默兜底),便于发现脏 status。
pub fn from_db_str(s: &str) -> Option<Self> { pub fn from_db_str(s: &str) -> Option<Self> {
Some(match s { Some(match s {
"planning" => ProjectStatus::Planning, "planning" => ProjectStatus::Planning,
+18
View File
@@ -30,6 +30,10 @@ use df_types::types::{NodeId, NodeStatus};
pub struct StateMachine { pub struct StateMachine {
/// 节点状态映射(Arc 共享,clone 浅拷贝同一 HashMap) /// 节点状态映射(Arc 共享,clone 浅拷贝同一 HashMap)
states: Arc<Mutex<HashMap<NodeId, NodeStatus>>>, states: Arc<Mutex<HashMap<NodeId, NodeStatus>>>,
/// 取消信号唤醒器(FR-P4):set_cancelled 时 notify_waiters,
/// 阻塞节点(HumanNode)用 notified() 被动等待,替代 500ms 轮询。
/// 与 states 同 Arc 共享,clone 浅拷贝同一 Notify。
cancel_notify: Arc<tokio::sync::Notify>,
} }
impl StateMachine { impl StateMachine {
@@ -37,9 +41,19 @@ impl StateMachine {
pub fn new() -> Self { pub fn new() -> Self {
Self { Self {
states: Arc::new(Mutex::new(HashMap::new())), states: Arc::new(Mutex::new(HashMap::new())),
cancel_notify: Arc::new(tokio::sync::Notify::new()),
} }
} }
/// 获取取消信号等待器(FR-P4)
///
/// 返回的 `Notified` future 在 `set_cancelled` 被调用时 resolve。
/// 调用方应配合 `is_cancelled` 做条件判断(Notify 无状态,可能错过
/// 早于 await 的 notify,故 select! 分支 resolve 后仍须复查 is_cancelled)。
pub fn cancel_notified(&self) -> tokio::sync::futures::Notified<'_> {
self.cancel_notify.notified()
}
/// 获取节点状态 /// 获取节点状态
/// ///
/// 锁中毒时降级返回 `NodeStatus::Pending`(保守默认:视为未启动, /// 锁中毒时降级返回 `NodeStatus::Pending`(保守默认:视为未启动,
@@ -139,6 +153,10 @@ impl StateMachine {
); );
} }
} }
// FR-P4:唤醒所有等待取消信号的阻塞节点(HumanNode notified 分支)。
// notify_waiters 不持锁调用,避免与 states 锁耦合;Notify 无状态,
// 等待方 resolve 后仍复查 is_cancelled 兜底(防 notify 早于 await 丢失)。
self.cancel_notify.notify_waiters();
} }
/// 获取所有状态快照(clone 返回,调用方持独立副本) /// 获取所有状态快照(clone 返回,调用方持独立副本)
@@ -0,0 +1,36 @@
# DevFlow 团队级基建 — 部署与运维
> 2026-10-05 落地。架构定位见《数据存储架构-定稿-2026-10-05》(团队级 = 汇聚节点 + 对象存储)。
## 资源清单
| 资源 | 位置 | 版本/规格 | 用途 |
|---|---|---|---|
| PG 容器 `df-pg` | ecs-os(39.99.131.191) | PostgreSQL 18.6-alpine | devflow 团队库(服务端数据面) |
| OSS 私有桶 `devflow-data` | 阿里云 cn-hongkong | 私有 ACL | 团队文件存储(温层,内容寻址) |
| 域名 `df-oss.1216.top` | CNAME → devflow-data.oss-cn-hongkong.aliyuncs.com | HTTPS(泛域名证书已绑) | 桶访问入口 |
## PG(df-pg)
- 连接:`127.0.0.1:5432`(仅本机/内网,**不暴露公网**);服务端将来部署 ecs-os 时走 localhost
- 库:`devflow`(容器首启自动创建);用户 postgres
- 凭据:`/data/docker-apps/pg/.pg-credentials`(600 权限)
- compose:`/data/docker-apps/pg/docker-compose.yml`(服务名=容器名 df-pg,数据卷自包含 `/data/docker-apps/pg/data`)
- 备份:cron 每日 03:30 `/data/scripts/pg-backup.sh` → `/data/backup/pg/` 保留 7 份
## OSS(devflow-data)
- 访问:私有桶,读写须签名(oss2 AK:ops-kit/app.yml aliyun.accounts[4],即 zhub-site 归属主账号)
- HTTPS:泛域名 `*.1216.top` 证书(suke-dev /etc/letsencrypt/live/1216.top)已绑 CNAME;归属验证 TXT 已用毕删除
- 服务端用法:oss2 SDK 签发临时 URL 供客户端下载;上传走 SDK 直传
- 续期注意:证书到期随泛域名统一续期(Let's Encrypt DNS-01),续期后需重新 put_bucket_cname(参考 cert-manage)
## 升级记录
- 2026-10-05 建 PG 16 当日即升 18.6(库空无损);compose 风格对齐 ecs-os 惯例(注释头/服务名=容器名/卷自包含);旧顶层卷 /data/pg-data 已清
- 选型记录:文件桶定阿里云 OSS(七牛私有桶缺主账号 SK,qshell 双账号 uppt/lakeapp-suke 已确认;七牛保留公开 CDN 分工 img.1216.top)
## 关联
- 架构:《数据存储架构-定稿-2026-10-05》§六 团队级/平台级
- 服务索引:kms/服务索引-全机器.md(df-oss/df-pg 两行)
@@ -0,0 +1,246 @@
# DevFlow 签名与发布 SOP
> 2026-10-10 制定。适用对象:上游发布者(你)。目标:让「签名 → 打包 → 发布清单 → 客户端自动升级」这条链**可重复执行、可验证、密钥可控**。
> 相关文档:《版本更新与自动升级-设计-2026-10-09.md》(链路设计)、《团队级基建-部署与运维-2026-10-05.md》(分发设施)。
---
## 一、密钥体系一句话
DevFlow 用 **minisign / ed25519** 非对称签名:
| 角色 | 文件 | 位置 | 作用 |
|---|---|---|---|
| 私钥(绝密) | `devflow.key` | `%USERPROFILE%\.tauri\` | 签名安装包,**泄露=可伪造升级** |
| 公钥(公开) | `devflow.key.pub` | 同上 | 验签,**同时硬编码进客户端** |
| 公钥副本 | `pubkey` 字段 | `src-tauri/tauri.conf.json` `plugins.updater.pubkey` | 编译进 exe,客户端用它验签 |
**核心约束**:客户端 exe 里的公钥是**编译期固化**的。换密钥 = 换公钥 = 已分发的老客户端**永远无法自动升级到新密钥签的包**(只能手动重装)。所以密钥轮换是**断链级操作**,非必要不做。
---
## 二、密钥文件格式(排障必知)
Tauri 私钥文件是 **base64 文本包 base64** 的两层结构:
```
外层 base64 → 解码得文本,两行:
L1 untrusted comment: ...
L2 <内层 base64> → 解码得 payload 字节
```
payload 布局:
| 长度 | 类型 | 含义 |
|---|---|---|
| 106 B | 明文(未加密) | `sig_alg(2)+keynum(8)+sk(64)+ck(32)` |
| 158 B | 加密(Scrypt KDF 段) | `sig_alg(2)+kdf_alg(2)+ck_alg(2)+salt(32)+opslimit(8)+memlimit(8)+keynum(8)+sk(64)+ck(32)` |
> ⚠️ **`cargo tauri signer generate` 产出的都是 158 B(带 KDF 段)**——即使你「没设密码」,结构上仍是加密布局,只是口令为空。所以**不能用文件大小判断是否设了密码**,必须用「空密码能否签名」实测(见 §六 自检)。
**公钥** payload 固定 42 B:`sig_alg(2)+keynum(8)+key(32)`。`keynum = payload[2:10]`,是**密钥指纹**,用于快速比对「配置公钥 ↔ 私钥 ↔ 签名」三者是否同一把钥匙。
---
## 三、密钥生命周期操作
### 3.1 生成(首次或轮换)
```powershell
# 无密码(不推荐,当前生产就是这个状态)
cargo tauri signer generate -w "$env:USERPROFILE\.tauri\devflow.key"
# 带密码(推荐)
cargo tauri signer generate -p "<强口令>" -w "$env:USERPROFILE\.tauri\devflow.key"
```
生成后把 `.pub` 的内容填进 `src-tauri/tauri.conf.json` 的 `plugins.updater.pubkey`,然后**重新编译客户端**才生效。
### 3.2 备份(强制)
私钥丢失 = 无法再签发升级包 = 只能原地冻版。**生成后立刻**:
```
1. 复制 devflow.key / devflow.key.pub 到离线介质(U 盘 / 加密卷)
2. 记录密钥指纹(keynum),见 §六
3. 关键:口令单独记录(不要和私钥同处存储)
```
### 3.3 轮换(断链级,慎用)
```
代价:已分发的所有客户端自动升级链断裂 → 必须手动重装
步骤:
1. 生成新密钥对 → 旧密钥改名归档(不要删,历史包验签要用)
2. 更新 tauri.conf.json pubkey → 重新编译 → 发布「桥接版」
(桥接版仍用旧私钥签,老客户端能升上来;桥接版内含新公钥,之后再用新私钥签)
3. 桥接版铺开 ≥1 个版本周期后,才切新私钥签发
```
> 单管理员自用场景可简化:接受「一次手动重装」,直接换。对外分发则**必须走桥接版**。
---
## 四、发布流程(一键脚本)
```powershell
cd E:\wk-lab\devflow
.\scripts\publish-release.ps1 # 完整发布(含 tauri build)
.\scripts\publish-release.ps1 -Notes "修复xx" # 手动指定更新说明
.\scripts\publish-release.ps1 -SkipBuild # 复用已有产物,只生成清单+上传
```
脚本六步(每步失败即 `exit 1`,不静默):
| 步 | 动作 | 失败含义 |
|---|---|---|
| 1 | **版本三源校验**(Cargo.toml / tauri.conf.json / package.json) | 三处版本号不一致,先对齐(见 §七 坑 2) |
| 2 | 签名私钥就位检查(env 优先,回退 `~/.tauri/devflow.key`) | 私钥缺失 |
| 3 | `cargo tauri build`(产出 `devflow_<ver>_x64-setup.exe` + `.exe.sig`) | 编译/打包失败 |
| 4 | 生成 `latest.json`(**UTF-8 无 BOM**) | — |
| 5 | 上传 `ssh-proxy` → `suke-dev:/var/www/app-site/download/devflow/` | SSH/上传失败 |
| 6 | 远端 `ls` + `cat latest.json` 回显验证 | 上传未落盘 |
**发布清单契约**(`latest.json`):
```json
{
"version": "0.1.2",
"notes": "...",
"pub_date": "2026-10-10T14:00:00Z",
"platforms": {
"windows-x86_64": {
"signature": "<.exe.sig 全文>",
"url": "https://app.1216.top/download/devflow/devflow_0.1.2_x64-setup.exe"
}
}
}
```
`app.1216.top` 已配 `no-cache`,改完即时生效,无需刷 CDN。
---
## 五、手工签名(脚本之外,补签/演练用)
```powershell
# 无密码密钥
cargo tauri signer sign -f "$env:USERPROFILE\.tauri\devflow.key" -p "" --app-version 0.1.2 <file>
# 带密码密钥(口令走参数或环境变量,勿写进脚本)
cargo tauri signer sign -f "$env:USERPROFILE\.tauri\devflow.key" -p "<口令>" --app-version 0.1.2 <file>
# 环境变量方式(CI 友好)
$env:TAURI_SIGNING_PRIVATE_KEY = "$env:USERPROFILE\.tauri\devflow.key"
$env:TAURI_SIGNING_PRIVATE_KEY_PASSWORD = "<口令>"
```
> **务必带 `--app-version`**:它把版本号写进签名的 trusted comment 并被签名覆盖。若客户端/服务端开启 `requireSignedVersion`,清单里改版本号就会被拒——这是防「篡改清单把新版号配旧包」的关键绑定。`cargo tauri build` 会自动带上,手工签要自己带。
---
## 六、验证(发布后必做)
### 6.1 一键验签(推荐)
```powershell
node temp/verify_sig.mjs
```
输出:配置公钥 keynum / 每个 exe 的 keynum 匹配性 / **ed25519 实际验签结果** / `latest.json` 与 `.sig` 内容一致性。
验签数学:payload 首 2 字节 `ED` 表示 **blake2b512 预哈希**模式,即 `verify(blake2b512(file), sig)`;`Ed` 表示直接对原文。
### 6.2 密钥指纹自检
```powershell
python temp/sign_state_check.py # 三源 keynum 对照
python temp/key_layout_final.py # 私钥格式(106/158B)判定
```
判据:**配置公钥 keynum == 预期私钥对应 keynum**。不等 = 拿错钥匙,签出来的包客户端一律拒。
### 6.3 「私钥是否设了密码」实测(唯一可信方法)
```powershell
$probe = "$env:TEMP\pw_probe.txt"; "x" | Set-Content $probe -NoNewline
cargo tauri signer sign -f "$env:USERPROFILE\.tauri\devflow.key" -p "" "$probe" ; $LASTEXITCODE
# exit=0 → 无密码保护;exit≠0 且报 incorrect password → 有密码
```
---
## 七、踩坑记录(实证)
**坑 1 — `latest.json` 的 UTF-8 BOM**
Windows PowerShell 5.1 的 `Out-File -Encoding utf8` / `Set-Content -Encoding utf8` **会写 BOM**(`EF BB BF`)。JSON 解析器遇到 BOM 报 `expected value at line 1 column 1`,表现为「签名没错、包没错,updater 就是拉不到清单」。
✅ 修正:`[System.IO.File]::WriteAllText($path, $json, (New-Object System.Text.UTF8Encoding($false)))`。已固化进 `publish-release.ps1`。
**坑 2 — 版本号三源分叉**
版本号存在于三处:`src-tauri/Cargo.toml`(Rust crate / `env!("CARGO_PKG_VERSION")`)、`src-tauri/tauri.conf.json`(打包器实际使用)、`package.json`(前端)。
任一处漏改 → 设置页显示与实际安装版本不符(灵异现象)。
✅ 修正:`publish-release.ps1` 第 1 步**硬校验三源一致**,不一致直接拒发。2026-10-10 已修 `package.json` 0.1.1 → 0.1.2。
**坑 3 — 发布脚本硬编码版本号**
`swap-install.ps1` 曾硬编码 `devflow_0.1.1_x64-setup.exe`。升版忘改 → 若旧包残留同目录,**静默装回旧版**(伪成功,最危险)。
✅ 修正:改 glob 探测 + `Sort-Object LastWriteTime -Descending | Select-Object -First 1`。
**坑 4 — 用文件大小判密码**
`cargo tauri signer generate` 产出的私钥**一律 158 B**(带 Scrypt KDF 段),无密码时只是口令为空。按大小判断会误判。
✅ 修正:用 §6.3 空密码实签判定。
**坑 5 — 密钥文件被误覆盖**
2026-10-10 曾发生野密钥覆盖生产密钥,靠备份 `*.bak-20261010-213046` 恢复。
✅ 教训:**动密钥前先归档**(`restore-key.ps1` 的归档-恢复-校验三步式可复用);密钥目录加 ACL 防误改(见 §八)。
---
## 八、安全加固(已实施 2026-10-10)
**威胁模型**:任何能读到 `devflow.key` 的本地进程,都能签发被客户端信任的更新包 → 推恶意升级。
已实施的缓解(见 `temp/harden_key_acl.ps1`):
```
%USERPROFILE%\.tauri\ 阻断继承,仅 (当前用户 / SYSTEM)
%USERPROFILE%\.tauri\devflow.key* 阻断继承,仅 (当前用户 读写删 / SYSTEM 全控)
```
**仍未消除的风险(重要)**:私钥当前**无口令保护**,文件本身即等价于签名能力;ACL 只挡了「非授权本地用户」,挡不住「以当前用户身份运行的程序」。
**根治方案(需拍板,有代价)**:
| 方案 | 强度 | 代价 |
|---|---|---|
| A. 私钥加口令 | 中 | 必须**重新生成密钥对**(`signer` 无 reencrypt 子命令)→ 公钥变 → **换公钥** → 见 §3.3 断链(老客户端需手动重装) |
| B. 私钥迁离线介质 | 高 | 每次发布需插盘;CI 不可用 |
| C. A+B 组合 | 最高 | 同上,推荐对外分发场景 |
| D. 仅 ACL(现状) | 低 | 零代价,只防误改/误读 |
> 现状为 D。若 DevFlow 仅自用,D 可接受;一旦对外分发,建议升到 C。
---
## 九、应急:私钥泄露怎么办
```
1. 立即停发:删除远端 latest.json(客户端拉不到清单即停止升级链)
2. 评估:泄露时间窗内是否有未授权包被签(检查远端目录文件时间戳)
3. 轮换:按 §3.3 生成新密钥对 → 发布桥接版(旧私钥签)→ 铺开后切新私钥
4. 通告:对外分发场景需告知用户手动重装
```
**注意:`删除 latest.json` 是唯一的「远端急停」手段**——客户端只在 check 时拉清单,清单不在 = 无法升级。
---
## 十、日常检查清单
发布前:
- [ ] 版本三源一致(脚本第 1 步自动校验)
- [ ] 私钥就位且指纹与配置公钥一致(`node temp/verify_sig.mjs` 的 keynum 对照)
- [ ] 密钥已备份(首次/轮换后)
发布后:
- [ ] `node temp/verify_sig.mjs` 全绿(验签 + latest.json 对账)
- [ ] 远端 `curl https://app.1216.top/download/devflow/latest.json` 可取且 version 正确
- [ ] 老客户端实测一次「检查更新 → 下载 → 安装 → 重启后版本号变新」(B3 端到端)
+30 -2
View File
@@ -4,7 +4,7 @@
> 上级索引:[../INDEX.md](../INDEX.md) > 上级索引:[../INDEX.md](../INDEX.md)
> 命名规范:见本目录 [文档记录规范-2026-06-14.md](./滚动规范/文档记录规范-2026-06-14.md) §九「文档命名规范」 > 命名规范:见本目录 [文档记录规范-2026-06-14.md](./滚动规范/文档记录规范-2026-06-14.md) §九「文档命名规范」
本目录是 DevFlow 的**架构设计 / 设计决策 / 设计方案 / 构想**文档集合(不含实现流水,流水走 `PROGRESS.md`;不含审查报告,审查走 `05-代码审查/`)。 本目录是 DevFlow 的**架构设计 / 设计决策 / 设计方案 / 构想**文档集合(不含实现流水,流水走 [`../07-项目管理/PROGRESS.md`](../07-项目管理/PROGRESS.md);不含审查报告,审查走 `05-代码审查/`)。
按**性质**分四类,每类一个子目录: 按**性质**分四类,每类一个子目录:
@@ -23,7 +23,7 @@
| 文档 | 状态 | 核心内容 | | 文档 | 状态 | 核心内容 |
|---|---|---| |---|---|---|
| [业务系统设计-2026-06-12.md](./滚动规范/业务系统设计-2026-06-12.md) | 🏗 ARCHITECTURE 载体 | 项目无独立 ARCHITECTURE.md,本文档为实质载体:产品定位 / crate 结构 / 数据模型 / Phase 规划 | | [业务系统设计-2026-06-12.md](./滚动规范/业务系统设计-2026-06-12.md) | 📐 设计 | 产品定位 / crate 结构 / 数据模型 / Phase 规划。项目架构总纲见根 [ARCHITECTURE.md](../../ARCHITECTURE.md) |
| [Phase1架构决策-2026-06-12.md](./滚动规范/Phase1架构决策-2026-06-12.md) | 📐 ADR | Phase 1 引擎骨架阶段架构级选型(ADR-001 引擎不绑定业务 等) | | [Phase1架构决策-2026-06-12.md](./滚动规范/Phase1架构决策-2026-06-12.md) | 📐 ADR | Phase 1 引擎骨架阶段架构级选型(ADR-001 引擎不绑定业务 等) |
| [功能决策记录-2026-06-14.md](./滚动规范/功能决策记录-2026-06-14.md) | 📋 滚动 | 功能**需求规格 + 设计决策规格**(为什么这么定 / 要做什么)。最高频引用(67 入链) | | [功能决策记录-2026-06-14.md](./滚动规范/功能决策记录-2026-06-14.md) | 📋 滚动 | 功能**需求规格 + 设计决策规格**(为什么这么定 / 要做什么)。最高频引用(67 入链) |
| [功能决策记录-归档-2026-06-14.md](./滚动规范/功能决策记录-归档-2026-06-14.md) | 🗄 归档只读 | 从主文档归档:纯流水 / 老 Sprint 决策 / UX 微调 / 已被取代细节 | | [功能决策记录-归档-2026-06-14.md](./滚动规范/功能决策记录-归档-2026-06-14.md) | 🗄 归档只读 | 从主文档归档:纯流水 / 老 Sprint 决策 / UX 微调 / 已被取代细节 |
@@ -91,6 +91,34 @@
| [后续批次测试用例设计-2026-07-01.md](./专项设计/后续批次测试用例设计-2026-07-01.md) | 📐 测试 | 多Agent并行/worktree隔离/仲裁合并/模板系统/Git-CI/审批政策 的单元+集成+端到端测试用例 | | [后续批次测试用例设计-2026-07-01.md](./专项设计/后续批次测试用例设计-2026-07-01.md) | 📐 测试 | 多Agent并行/worktree隔离/仲裁合并/模板系统/Git-CI/审批政策 的单元+集成+端到端测试用例 |
| [测试落地实施方案-2026-07-01.md](./专项设计/测试落地实施方案-2026-07-01.md) | 📐 测试 | 各测试用例的落地方式(Rust 单元/集成/vitest/E2E 手动)、文件位置、Mock 策略、CI 自动化路径 | | [测试落地实施方案-2026-07-01.md](./专项设计/测试落地实施方案-2026-07-01.md) | 📐 测试 | 各测试用例的落地方式(Rust 单元/集成/vitest/E2E 手动)、文件位置、Mock 策略、CI 自动化路径 |
| [devflow集成三方IM-飞书企微钉钉Web-规划设计-2026-10-04.md](./专项设计/devflow集成三方IM-飞书企微钉钉Web-规划设计-2026-10-04.md) | 📐 规划(待评审) | 飞书/企微/钉钉/自定义Web 远程接入 AI Chat:新增 df-im-gateway(WS Commander)复用 u-relay+df-tunnel,桌面端零改动。含必要性分析(§三A)、破坏性分析(§三B)、演进预留(§十 S1-S4:多人/引擎上云/新端)。P0网关→P1 Web→P2飞书→P3钉钉→P4企微 | | [devflow集成三方IM-飞书企微钉钉Web-规划设计-2026-10-04.md](./专项设计/devflow集成三方IM-飞书企微钉钉Web-规划设计-2026-10-04.md) | 📐 规划(待评审) | 飞书/企微/钉钉/自定义Web 远程接入 AI Chat:新增 df-im-gateway(WS Commander)复用 u-relay+df-tunnel,桌面端零改动。含必要性分析(§三A)、破坏性分析(§三B)、演进预留(§十 S1-S4:多人/引擎上云/新端)。P0网关→P1 Web→P2飞书→P3钉钉→P4企微 |
| [三方IM-Rust接入与SDK调研-2026-10-04.md](./专项设计/三方IM-Rust接入与SDK调研-2026-10-04.md) | ✅ 调研完成 | 三平台 Rust SDK 选型:飞书 larksuite-oapi-sdk-rs / 钉钉 dingtalk-stream / 企微 wecom-aibot-rust-sdk。**重大发现:企微智能机器人走 WS 长连接免公网回调,三平台统一形态**。含统一 adapter 映射表与 5 项落地风险清单 |
| [上下文管理演进-任务技术设计-2026-07-20.md](./专项设计/上下文管理演进-任务技术设计-2026-07-20.md) | 📐 设计 | 上下文管理演进任务技术设计 |
| [上下文管理演进-执行计划-2026-07-20.md](./专项设计/上下文管理演进-执行计划-2026-07-20.md) | 📐 计划 | 上下文管理演进执行计划 |
| [上下文管理演进与发散思考-2026-07-20.md](./专项设计/上下文管理演进与发散思考-2026-07-20.md) | 📐 思考 | 上下文管理演进与发散思考 |
| [aichat-plan-driven-设计-2026-08-01.md](./专项设计/aichat-plan-driven-设计-2026-08-01.md) | 📐 设计 | AI Chat plan-driven 模式设计 |
| [devflow集成u-claw-可行性研究-2026-09-28.md](./专项设计/devflow集成u-claw-可行性研究-2026-09-28.md) | 📐 研究 | devflow 集成 u-claw 可行性研究 |
| [跨会话记忆四层架构-设计-2026-09-29.md](./专项设计/跨会话记忆四层架构-设计-2026-09-29.md) | 📐 设计 | 跨会话记忆四层架构 |
| [aichat上下文管理-派工与决策单-2026-09-29.md](./专项设计/aichat上下文管理-派工与决策单-2026-09-29.md) | 📋 决策单 | aichat 上下文管理派工与决策 |
| [DRY技术债走查报告-2026-10-01.md](./专项设计/DRY技术债走查报告-2026-10-01.md) | 📋 报告 | 全库 DRY 技术债走查 |
| [并行化与异步唤醒-设计-2026-10-04.md](./专项设计/并行化与异步唤醒-设计-2026-10-04.md) | 📐 设计 | 并行化与异步唤醒 |
| [无人值守长任务-设计-2026-10-04.md](./专项设计/无人值守长任务-设计-2026-10-04.md) | 📐 设计 | 无人值守长任务 |
| [上下文三层记忆-设计-2026-10-04.md](./专项设计/上下文三层记忆-设计-2026-10-04.md) | 📐 设计 | 上下文三层记忆 |
| [跨会话召回最小闭环-设计-2026-10-04.md](./专项设计/跨会话召回最小闭环-设计-2026-10-04.md) | 📐 设计 | 跨会话召回最小闭环 |
| [context-helpers拆分备勘-设计-2026-10-04.md](./专项设计/context-helpers拆分备勘-设计-2026-10-04.md) | 📐 设计 | context-helpers 拆分备勘 |
| [namespace淘汰治理-设计-2026-10-04.md](./专项设计/namespace淘汰治理-设计-2026-10-04.md) | 📐 设计 | namespace 淘汰治理 |
| [子代理完成卡已消化降级-设计-2026-10-04.md](./专项设计/子代理完成卡已消化降级-设计-2026-10-04.md) | 📐 设计 | 子代理完成卡消化降级 |
| [uclaw两侧勘察补全-2026-10-04.md](./专项设计/uclaw两侧勘察补全-2026-10-04.md) | 📋 勘察 | u-claw 两侧勘察补全 |
| [u-relay-wire-protocol-2026-10-04.md](./专项设计/u-relay-wire-protocol-2026-10-04.md) | 📐 协议 | u-relay 线协议规范 |
| [数据存储架构-定稿-2026-10-05.md](./专项设计/数据存储架构-定稿-2026-10-05.md) | 📐 定稿 | 数据存储架构定稿 |
| [知识库模型重审-精细化设计-2026-10-05.md](./专项设计/知识库模型重审-精细化设计-2026-10-05.md) | 📐 设计 | 知识库模型精细化重审 |
| [飞书直连接入-形态修正与问题记录-2026-10-05.md](./专项设计/飞书直连接入-形态修正与问题记录-2026-10-05.md) | 📋 记录 | 飞书直连接入形态修正与问题记录 |
| [版本更新与自动升级-设计-2026-10-09.md](./专项设计/版本更新与自动升级-设计-2026-10-09.md) | 📐 设计 | 版本更新与自动升级 |
| [u-relay-运维手册-2026-10-10.md](./专项设计/u-relay-运维手册-2026-10-10.md) | 📖 手册 | u-relay 运维手册 |
| [上下文分层组装器-落地方案-2026-10-10.md](./专项设计/上下文分层组装器-落地方案-2026-10-10.md) | 📐 方案 | 上下文分层组装器落地方案 |
| [定时消息与待发箱持久化-设计-2026-10-10.md](./专项设计/定时消息与待发箱持久化-设计-2026-10-10.md) | 📐 设计 | 定时消息与待发箱持久化 |
| [对话内定时延时任务-设计-2026-10-10.md](./专项设计/对话内定时延时任务-设计-2026-10-10.md) | 📐 设计 | 对话内定时延时任务 |
| [消息渲染管线重构-设计-2026-10-10.md](./专项设计/消息渲染管线重构-设计-2026-10-10.md) | 📐 设计 | 消息渲染管线重构 |
| [输入框列表预览与自动编号-设计-2026-10-10.md](./专项设计/输入框列表预览与自动编号-设计-2026-10-10.md) | 📐 设计 | 输入框列表预览与自动编号 |
--- ---
@@ -0,0 +1,191 @@
# 上下文根治设计(2026-10-06)—— 治 223:1 结构性浪费
> 铁证:84cb104c 会话 2.2 天烧 4.3 亿 tokens,prompt:completion=223:1,日均 1.9 亿。
> 本质:每轮全量重发历史——轮数线性,token 爆炸,首 token 延迟同步恶化。不是 bug,是组装策略必然。
> 原则:分层组装取代全量重发;历史不删只降级(压缩可逆);对用户透明,对 agent 工具透明。
## 一、分层上下文(治"全量")
- **L1 常驻(固定小)**:system_prompt + pinned_goals + 任务状态卡。目标 <8K,永不随轮次增长。
- **L2 工作集(滑动窗口)**:最近 N 轮原文 + 当前任务活跃文件/结果。按"信息增益"滑动,非固定条数。
- **L3 检索层(按需召回)**:窗口外历史不进 prompt,转为可检索——关键词/语义命中才临时注入(复用 knowledge_inject 的检索管线,扩到对话历史)。
- 无破坏性保证:所有降级内容原文全在库(L3 可召回+压缩可逆 C-r1),agent 能力不降级,丢的是"每轮都带全部家当"的浪费。
## 二、工具结果瘦身(治最大头,先行)
9386 条消息中 tool 4379 条——单条最大 8K+,是上下文膨胀主力。
- 超阈值(如 2K)的工具结果:正文落盘(外置文件/独立表),上下文只留「头部 200 字摘要 + 句柄路径」。
- 与 D2a(过期外置)同构一套机制,两处受益。
- 句柄可被 agent 主动展开(read 句柄文件)——需要时找得回,不需要时不占位。
## 三、压缩升级(从"水位兜底"到"常规操作")
- 现状:120K 水位才触发,1M 窗口下形同虚设。
- 改:压缩不再是水位兜底,而是 L2→L3 的常规滑动——每轮组装时,窗口外的轮次自动结构化摘要(记忆卡形态,信息增益判据保留),原文留库可按段重载(C-r1)。
## 四、量化预期
- prompt 均值 71K → 目标 ≤20K(工作集+召回),单轮成本/延迟降 70%+
- 223:1 → 目标 ≤50:1(继续随 C-m 记忆卡质量优化)
- 长会话不再有"轮数税"——206 轮与 6 轮的单轮成本趋同。
## 五、实施序(每阶段独立可交付,无破坏)
- P1(S):工具结果瘦身(阈值落盘+句柄)——最大头先行,零行为变化
- P2(M):L1/L2/L3 组装器重构(build_for_request 改造,窗口+检索)
- P3(M):压缩常规化+记忆卡产出(C-m1)+按段重载(C-r1)
- P4(S):量化验收(223:1→≤50:1 看板)+回归(agent 能力不降级测试集)
## 六、与旧 C 批次关系
本设计整合取代 C1-C6/C-m1-m3 的发散条目:C-r0(验证后台化)独立不变;C-m2(判据挂 pinned_goals)/C-r1(可逆)并入 P2/P3。旧条目销账以本档为准。
## 七、修订(2026-10-06 用户定调):从「分层组装」升级为「上下文调度系统」
> 定调:同会话连续工作是**常态**,设计目标不是控制会话长度,而是支撑无限长会话的资源调度——OS 虚拟内存同构:窗口有限,会话地址空间无限。
> 修正:撤回"百轮会话收敛拆分"建议;一~六节的 L1/L2/L3 保留为**页管理基建**,本节新增调度层。
### 调度系统设计(新增)
- **页式单位**:上下文按页管理——轮次页/工具结果页/文件页。页=换入换出与压缩的最小单位。
- **热度调度**:页带热度(最近访问/被引用频次)。热页驻工作集(L2);冷页压缩归档(L3,结构化摘要+原文留库);**压缩不丢**=冷页永远可召回。
- **缺页召回**:agent 引用历史(关键词/句柄/goal 关联)→检索命中→临时换入本轮上下文,用完即出——召回不计入常驻。
- **任务聚类切换**(切换能力的核心):工作集按**任务/主题聚类**重组,非纯时间窗口——用户在多任务间切换时,上下文随任务维度切换(当前任务的热页+该任务关联页),并行任务互不污染。任务边界=页的聚类标签(pinned_goals/主题标记)。
- **调度目标函数**:token 花在刀刃=按"任务相关性×信息增益"分配窗口配额;调度决策零 LLM 调用(规则引擎,成本为零)。
### 修订后的一句话
**长会话=常态运行环境;上下文调度器=它的虚拟内存管理器;压缩可召回=它的 swap;任务聚类切换=它的进程调度。**
## 八、再修订(2026-10-06 用户定调):跨会话调度与召回也是常态
> 定调:调度地址空间=会话内 + **跨会话**。会话 A 的产出/结论,会话 B 工作中可召回引用——跨会话不是边界,是同一个页池的不同命名空间。
### 跨会话调度设计(新增)
- **页命名空间分级**:会话内页(默认私有) / **跨会话共享页**(结论页/知识页/任务产出页,显式或按策略晋升共享)。
- **跨会话召回**:检索半径覆盖全页池——会话 B 缺页召回时,共享页与会话 A 的可召回页同池命中,带来源标注(「来自会话 X 的结论」)。
- **与 knowledge 库关系**:知识条目=最高等级的共享页(已晋升、已蒸馏);本设计的跨会话召回把"未蒸馏的对话产出"也纳入召回半径——knowledge 是终点,页池是全程。
- **晋升机制**:会话内页→(蒸馏/用户标记/高频跨任务引用)→共享页→(结构化)→knowledge 条目。晋升是单向门,防未蒸馏噪音污染共享池。
### 多人场景(预留位,下阶段设计)
- 页带 owner 与 ACL 位;共享池分个人池/团队池;多人可召回引用=团队池+权限过滤。
- 本期只预留页结构字段(owner/acl/scope),不做多人实现——接口先行,机制后置。
## 九、完整审视(2026-10-06 二次评审)——四个真问题与修正
### 问题1:机制并存未收敛(最重)
压缩机制(现有)/D2a 工具结果外置/L3 检索召回/knowledge 条目/页池——**五个名字,一套事**(都是"内容降级+可找回")。三次定调追加未合并同类项。
**修正**:统一收敛——**唯一存储=页池,唯一调度器,唯一召回入口**;压缩/D2a/knowledge 都是页池上的策略与晋升形态,不再作为平行系统各自演化。旧机制改造为页池前身,不新建平行通路。
### 问题2:实施范围失控风险
页池+晋升+跨会话+多人——北极星画完了,但一个桌面 app 一次吃下=过度设计。
**修正:v1 范围锁死**=单会话页池+极简调度器(recency+引用计数热度,goal 标签聚类,关键词缺页召回)+knowledge 照旧(它已覆盖跨会话召回大部分需求)。跨会话命名空间/晋升/多人=**v2+,触发条件=单会话版实测后仍有跨会话痛点**。基建等真实触发。
### 问题3:缺评估设计与调度可观测
"agent 能力不降级"怎么证?P4 的回归测试集不存在。
**修正**:①建任务回归集(固定任务清单,改造前后跑对比:完成质量/轮数/成本)②调度器可观测(页命中/召回率/换出量进日志与面板)③误调度代价显性化(热页误换出→下轮回查次数)。
### 问题4:与模型平台原生能力对齐未检查
模型侧可能原生提供上下文编辑/记忆工具(如 Claude 的 context editing/memory tool)——自研调度若与平台能力重复,应优先用平台原生(维护成本低、演化跟随厂商)。
**修正**:实施前先做平台能力对齐检查(一次调研任务),能用原生的不自研;自研部分聚焦平台没有的(任务聚类切换/跨会话页池)。
### 审视后的 v1 锁定范围(修正结论)
P1 工具结果瘦身 + P2 单会话页池与极简调度 + P3 压缩常规化(页池前身改造) + P4 评估集与量化。
七/八节的调度系统与跨会话设计降级为**北极星方向**;实施以本节 v1 范围为准。
## 十、知识引用注入的统筹与分工(2026-10-06 补充:第九节收敛的正向展开)
> 上下文生态的注入类型盘点:工具结果句柄/对话历史页/压缩摘要与记忆卡/knowledge 条目/pinned_goals/技能注入/规范注入——统筹不当=五套并行的重演。
### 分工矩阵(按蒸馏程度×生命周期×注入方式)
| 类型 | 蒸馏程度 | 生命周期 | 注入方式 | 角色 |
|---|---|---|---|---|
| 工具结果句柄 | 原始 | 任务内 | agent 主动 read 展开 | 工作证据 |
| 对话历史页 | 轻摘要 | 会话内→跨会话(v2) | 检索召回临时换入 | 过程记忆 |
| 压缩摘要/记忆卡 | 中蒸馏 | 会话长期 | 工作集常驻(摘要形态) | 阶段结论 |
| knowledge 条目 | 高蒸馏 | 永久 | system_prompt 检索注入 | 沉淀规范 |
| pinned_goals/任务态 | 实时 | 任务期 | L1 常驻 | 当前意图 |
### 统筹四原则
1. **单一来源**:每种内容只有一个当前形态的家(原始在页池/蒸馏在 knowledge),晋升单向门,禁双存双写。
2. **注入方式随蒸馏程度**:越原始越被动(agent 主动拉/句柄展开),越蒸馏越主动(系统注入 prompt)——检索系统替 agent 省的每一分注意力,都来自上游的蒸馏质量。
3. **召回入口统一**:全部类型注册到同一检索面,agent 视角只有一个记忆系统(分类型返回,来源标注)——agent 不需要知道背后几套存储。
4. **三层配额防炸弹**:prompt 注入配额(既有 ≤20 条/1.5K)+每轮召回页配额(新增,如 ≤5 页)+句柄展开按需——总量有闸,信息炸弹在结构上不可能。
### 与既有机制对账
knowledge_inject(检索注入)=统筹第 4 行的既有实现,扩为统一检索面的注册方之一;compress(压缩)=第 3 行的产生器;D2a/P1(句柄)=第 1 行的产生器;审计 subscriber(H4)=统筹的观测面。**无新系统,只有分工明确化。**
## 十一、自主性原则(2026-10-06 用户定调:海纳川流,增强非依赖)
> 定调:CLAUDE.md/看板/外部规范/memory 等=**可选输入源**(海纳川流,有则增强);devflow 核心运作(知识/纪律/调度/领单)必须**内生**——外部全删,系统照常运作。
### 依赖矩阵重排
- **内生核心(必需,无外部也运作)**:知识库条目(自己的注入面)/引擎层机制(B5b 看板轮询领单=运行时化,非提示词)/仓库 pre-commit hook(跟仓库走)/状态机与调度器。
- **增强输入(可选,缺失不影响)**:CLAUDE.md 注入/共享技能(cm)/看板摘要/memory 召回/走查报告。
- **迁移**:CLAUDE.md 领单纪律降级为过渡形态,正解=①治理规则做成知识库条目(自己的记忆) ②领单引擎化(B5b)。pre-commit hook 从技能层迁仓库层(更内生)。
## 十二、顶层定调:广义知识库=知识载体总集(2026-10-06 用户定调)
> 定调:知识库=宏观概念,**包含 kms/memory/skill/页池/knowledge/外部输入的一切知识载体**。技能与记忆的"海纳+有效高质量引用"同属上下文知识库管理的一环。上下文调度系统的地址空间=全部载体。
### 载体谱系(蒸馏程度×形态×引用方式的光谱)
| 载体 | 形态 | 蒸馏/时效 | 引用方式 | 调度角色 |
|---|---|---|---|---|
| 会话页池 | 对话/工具原始页 | 原始/热 | 缺页召回换入 | 工作记忆 |
| 压缩摘要·记忆卡 | 结构化摘要 | 中蒸馏/温 | 工作集常驻 | 阶段结论 |
| knowledge 条目 | 高蒸馏条目 | 高蒸馏/持久 | system_prompt 注入 | 规范与沉淀 |
| pinned_goals | 意图态 | 实时 | L1 常驻 | 当前意图 |
| **memory 文件** | 跨会话个人记忆 | 中蒸馏/持久 | 项目召回+按需 Read | 跨会话自我认知 |
| **kms 文档** | 文档级知识 | 沉淀/冷 | 按需 Read | 领域参考书 |
| **skill** | **可执行流程知识** | 沉淀/持久 | **触发执行(不是读,是跑)** | 能力包 |
| 外部输入(CLAUDE.md/看板/报告) | 增强流 | 实时 | 注入/摄入 | 海纳川流 |
### 统筹原则扩展(在十节四原则之上)
5. **载体按形态分工**:陈述性知识走引用(页/条目/kms/memory),程序性知识走执行(skill)——"知道什么"引用,"会做什么"触发,引用与执行分治。
6. **海纳=摄入管线统一**:外部输入与内部产出进同一摄入管线(页池→蒸馏→晋升),skill 的沉淀也走此管线的"执行验证通过"分支——摄入不分内外,质量 gate 统一。
7. **引用质量=相关性×蒸馏度×时机**:高质量引用不是"多给",是"对的载体在对的时机以对的方式"——调度器的目标函数升级为此三元组。
### 对实施的影响
统一检索面(P2 细化)的注册方扩展至全载体(memory/kms/skill 的索引也注册);skill 特殊——注册的是"能力描述",命中时走执行而非注入。P2 工作量小幅上调,架构位置不变。
## 十三、devflow.md:项目认知投影文件(对标 CLAUDE.md,本土化超越)
> 定调:类似而非照搬,青出于蓝。CLAUDE.md=用户手写的静态注入;devflow.md=**agent 自主维护的活的项目认知投影**——从"每次启动读一遍死文件"升级为"分层分区的活上下文"。
### 与 CLAUDE.md 的能力对照
| 维度 | CLAUDE.md | devflow.md |
|---|---|---|
| 维护者 | 用户手写 | **agent 自动维护为主,用户可写覆盖** |
| 形态 | 纯文本一次读 | **结构化分区,分区独立刷新** |
| 时效 | 启动读一次静态 | 启动读取+**运行中事件驱动刷新**(警示/状态可中途生效) |
| 内容 | 手写规范 | 规范(迁移自 CLAUDE.md)+**状态投影**(HEAD/看板摘要/活跃任务,自动)+**知识投影**(按当前任务相关性从页池/knowledge 拉) |
| 兼容 | — | **海纳:项目存在 CLAUDE.md 则自动并入投影**(增强非依赖,呼应第十一节) |
### 四分区设计
1. **规范纪律区**(低频):领单纪律/提交规范/红线——从现 CLAUDE.md 迁移;用户手写优先,agent 建议追加需用户确认。
2. **项目状态区**(高频自动):HEAD/分支/未提交数/看板摘要/活跃任务——每次刷新重写,机器事实源(防凭记忆汇报的文件化)。
3. **知识投影区**(任务相关):当前任务关联的 knowledge 条目/记忆卡/页池热页摘要——调度器按相关性投喂,会话切换即换。
4. **警示区**(事件驱动):挂机必读/外部干预告知/退回说明——事件触发插入,读后标记已消费。
### 机制要点
- 读取:会话启动读 devflow.md(存在则并入;CLAUDE.md 同目录存在也并入,优先级 devflow.md > CLAUDE.md)——**两者都没有,按知识库内生纪律照常运作**(第十一节自主性)。
- 刷新:状态区/警示区事件驱动重写(写者唯一=状态投影器);运行中会话的刷新对下一轮组装生效。
- 边界:devflow.md 是**投影不是真相**——真相在引擎/DB/页池;文件损坏可随时重建(从真相源再生成)。
## 十四、devflow.md 多角色论证收敛(2026-10-06 /swarm:怀疑者+行业对标+实施务实+用户代言四方)
### 三路论证共识(全部指向同一结论)
**运行中文件刷新(原十三节设计)砍除**——①架构矛盾:system_prompt/volatile_tail 是 loop 不变量(启动组装一次,mod.rs:138/:441),中途改文件对运行中 loop 零生效;②token 代价:兑现刷新需破坏 provider prompt 缓存前缀,每轮重算 system 段=token 实涨(违背知识管理高效原则);③行业零先例(全行业规则文件均会话级加载),高风险无容错。
**替代(零新机制)**:中途更新走既有 insert 消息范式(insert_stall_warning/insert_goal_verify_nudge/inject_runtime_message——往 conv.messages 尾插,build_for_request 每轮自然重组,天然下一轮生效,G2 已在用)。
### 行业对标关键结论
验证有效:agent 生成规则(Cursor)/按相关性加载(Agent Requested+repo map)/跨会话持久。已证为坑:init fossilization/只增不删/多源漂移/静默截断。
**真空白护城河(坚持)**:状态机器投影(五家工具无一把工作流状态写入记忆文件)+警示事件驱动插入——但载体不是文件,是 insert 消息范式+DB 直注。
**同质化区(兼容)**:纪律区做得像 CLAUDE.md 越利于兼容——创新在投影/警示/领单,不在文件格式标新立异。
### 定稿:v1 MVP 切片(S-M)
1. **启动读取**:devflow.md(两分区:规范纪律+领单)+CLAUDE.md 并入→build_runtime_prompt 第七段纯追加(两条组装路径都接);文件读取微秒级,**注入的 token 才是成本——内容按相关性裁剪,禁全量**。
2. **警示区**:事件→insert 消息范式(零新机制);非文件分区。
3. **状态**:不文件化——DB 直注(build_context_lists 既有,比文件投影更新鲜)+H3 git 实测(已有)。
4. **知识投影**:knowledge_inject 既有+补相关性元数据(行业经验:规则死于描述,元数据决定拉取质量)。
### v2+(北极星,触发制)
agent 自主维护协议/知识投影动态化/跨会话投影——v1 实测后按真实痛点渐进。
### token 高效三律([补 1] 原则落定)
①不破坏 prompt 缓存前缀 ②投影内容按相关性裁剪禁全量注入 ③文件读取不花钱,注入 token 才花钱——一切设计以"注入的 token 换来的决策价值"为核算单位。
File diff suppressed because one or more lines are too long
@@ -0,0 +1,8 @@
$queries = @('feishu', 'lark', 'larksuite')
foreach ($q in $queries) {
Write-Output "===== query: $q ====="
$r = Invoke-RestMethod -Uri "https://crates.io/api/v1/crates?q=$q&per_page=20" -Headers @{ 'User-Agent' = 'devflow-research' }
$r.crates | ForEach-Object {
"{0,-22} v{1,-10} dl={2,-8} upd={3} repo={4}" -f $_.id, $_.max_version, $_.downloads, ($_.updated_at -split 'T')[0], $_.repository
}
}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,168 @@
# DingTalk Stream SDK for Rust
[![Crates.io](https://img.shields.io/crates/v/dingtalk-stream.svg)](https://crates.io/crates/dingtalk-stream)
[![docs.rs](https://docs.rs/dingtalk-stream/badge.svg)](https://docs.rs/dingtalk-stream)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Rust](https://img.shields.io/badge/rust-%3E%3D1.85.0-orange.svg)](https://www.rust-lang.org)
钉钉 Stream 模式 SDK 的 Rust 实现,完整对齐 Python 版 [dingtalk-stream-sdk-python](https://github.com/open-dingtalk/dingtalk-stream-sdk-python)。
> 对齐的 Python SDK 版本: commit [`6381a7b`](https://github.com/open-dingtalk/dingtalk-stream-sdk-python/commit/6381a7bacb0353e1e0cd425688a02ff785fda44c)
## 功能特性
- 基于 WebSocket 的钉钉 Stream 协议连接(自动重连、60s Keepalive)
- 聊天机器人消息收发(文本、Markdown、卡片)
- 交互式卡片(Markdown / AI 流式 / 按钮 / 轮播 / RPA 插件)
- Graph API 支持
- 事件订阅
- OAuth2 Token 自动管理(带缓存与 5 分钟提前刷新)
- 统一 async/await(基于 tokio)
## 快速开始
将依赖添加到 `Cargo.toml`:
```toml
[dependencies]
dingtalk-stream = "0.1"
```
最小示例:
```rust
use async_trait::async_trait;
use dingtalk_stream::*;
struct MyHandler;
#[async_trait]
impl CallbackHandler for MyHandler {
async fn process(&self, message: &messages::frames::MessageBody) -> (u16, String) {
println!("Received: {}", message.data);
(AckMessage::STATUS_OK, "OK".to_owned())
}
}
fn main() {
let credential = Credential::new("your_client_id", "your_client_secret");
let handler = MyHandler;
let mut client = DingTalkStreamClient::builder(credential)
.register_callback_handler(ChatbotMessage::TOPIC, handler)
.build();
client.start_forever().unwrap();
}
```
更多示例见 [`examples/`](examples/) 目录。
## 项目结构
```
src/
├── lib.rs # 公共 API 导出
├── client.rs # DingTalkStreamClient + Builder
├── credential.rs # Credential(Debug 遮蔽 secret)
├── error.rs # 统一错误类型
├── handlers/ # Handler traits (Callback/Event/System/Chatbot/Graph)
├── messages/ # 消息类型 (Frames/Chatbot/CardCallback/Graph)
├── card/ # 卡片系统 (Replier/Instances/Templates)
└── transport/ # 传输层 (HTTP/Token/WebSocket)
```
## 文档
- [整体架构图](docs/architecture.md)
- [连接生命周期时序图](docs/connection-lifecycle.md)
- [卡片操作时序图](docs/card-operations.md)
## 端到端测试
1. 复制配置模板并填入凭证:
```bash
cp examples/secret.example.toml examples/secret.toml
# 编辑 examples/secret.toml,填入 client_id 和 client_secret
```
2. 运行测试机器人:
```bash
cargo run --example e2e_test
```
3. 带媒体下载测试:
```bash
cargo run --example e2e_test -- --download-dir /tmp/dingtalk_downloads
```
4. 通过钉钉向机器人发送消息测试:
- `ping` → 回复 pong
- `echo <text>` → 回声
- `card` → 发送 Markdown 卡片
- `info` → 显示消息详情
## 与 Python SDK 的设计差异
| Python | Rust | 说明 |
|---|---|---|
| sync + async 双版本 | 统一 async | 消除代码重复 |
| 类继承 | Trait + 组合 | Rust 惯用模式 |
| ThreadPoolExecutor | tokio::spawn | 更高效的异步调度 |
| dict 动态类型 | 强类型 struct + serde | 编译期类型安全 |
## Rust SDK 独有功能
以下功能为 Rust SDK 独有,Python SDK 中不存在。同步 Python SDK 时请勿删除(代码中均有 `Rust SDK exclusive` 标注)。
### 扩展消息类型
Rust SDK 支持解析钉钉机器人可接收的全部 6 种消息类型,Python SDK 仅支持 text / richText / picture:
| 消息类型 | 结构体 | 场景限制 |
|---------|--------|---------|
| audio 语音 | `AudioContent` | 仅单聊 |
| file 文件 | `FileContent` | 仅单聊 |
| video 视频 | `VideoContent` | 仅单聊 |
```rust
let msg = ChatbotMessage::from_value(&json)?;
if let Some(fc) = &msg.file_content {
println!("文件名: {:?}, 下载码: {:?}", fc.file_name, fc.download_code);
}
```
### 统一下载码提取
`get_all_download_codes()` 一次性获取消息中所有媒体的下载码,覆盖 picture / richText 图片 / audio / file / video:
```rust
for (media_type, download_code) in msg.get_all_download_codes() {
let url = replier.get_image_download_url(&download_code).await?;
let bytes = replier.download_bytes(&url).await?;
}
```
### 文件下载
| 方法 | 说明 |
|------|------|
| `download_bytes(url)` | 下载文件字节内容 |
| `download_bytes_with_limit(url, max_size)` | 带大小限制的流式下载(Content-Length 预检 + 累计检查) |
### 主动单聊消息
`send_oto_message()` 通过 OpenAPI 向指定用户发送单聊消息(对应 `POST /v1.0/robot/oToMessages/batchSend`):
```rust
replier.send_oto_message(user_id, "sampleText", r#"{"content":"hello"}"#).await?;
```
## 环境要求
- Rust ≥ 1.85.0(edition 2024)
- tokio 异步运行时
## License
MIT
@@ -0,0 +1,285 @@
# Feishu SDK Rust
[![Crates.io](https://img.shields.io/crates/v/feishu-sdk.svg)](https://crates.io/crates/feishu-sdk)
[![Documentation](https://docs.rs/feishu-sdk/badge.svg)](https://docs.rs/feishu-sdk)
[![License](https://img.shields.io/crates/l/feishu-sdk.svg)](https://github.com/your-repo/feishu-sdk-rust)
> Notice: This is a community-maintained crate and is not an official Feishu/Lark SDK.
Rust SDK for Feishu/Lark Open Platform API. Provides a complete, type-safe interface to interact with Feishu APIs.
## Features
- **Complete API Coverage**: 1448+ operations from Feishu Open Platform
- **Token Management**: Automatic app_access_token and tenant_access_token handling
- **Event Processing**: Event subscription and card callback handling
- **WebSocket Support**: Long connection for real-time messaging
- **HTTP Server**: Built-in Salvo server for webhook handling
- **Type Safety**: Generated operation constants and typed APIs
- **Async/Await**: Full async support with tokio runtime
- **Pluggable Components**: Customizable cache, logger, serializer, and HTTP client
- **Code Generation**: Operation wrappers + typed model stubs from endpoint catalog
## Installation
Add to your `Cargo.toml`:
```toml
[dependencies]
feishu-sdk = "0.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
```
### Optional Features
```toml
[dependencies]
feishu-sdk = { version = "0.1", features = ["server", "websocket"] }
```
- `server`: Enable HTTP server for event handling (Salvo)
- `websocket`: Enable WebSocket client for long connections
## Quick Start
### Basic Usage
```rust
use feishu_sdk::core::{Config, FEISHU_BASE_URL};
use feishu_sdk::Client;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let config = Config::builder("app_id", "app_secret")
.base_url(FEISHU_BASE_URL)
.build();
let client = Client::new(config)?;
// Get chat list
let resp = client
.operation("im.v1.chat.list")
.query_param("page_size", "20")
.send()
.await?;
println!("Status: {}", resp.status);
println!("Body: {}", String::from_utf8_lossy(&resp.body));
Ok(())
}
```
### Send Message
```rust
use feishu_sdk::core::Config;
use feishu_sdk::Client;
async fn send_message(client: &Client, chat_id: &str) -> Result<(), Box<dyn std::error::Error>> {
let message = serde_json::json!({
"receive_id": chat_id,
"msg_type": "text",
"content": r#"{"text":"Hello!"}"#
});
let resp = client
.operation("im.v1.messages.create")
.path_param("receive_id_type", "chat_id")
.body_json(&message)?
.send()
.await?;
println!("Message sent: {:?}", resp.json_value()?);
Ok(())
}
```
### Event Handling Server
```rust
use feishu_sdk::core::{noop_logger, Config};
use feishu_sdk::event::{EventDispatcher, EventDispatcherConfig};
use feishu_sdk::server::FeishuServer;
#[tokio::main]
async fn main() {
let event_config = EventDispatcherConfig::new()
.verification_token("your_token");
let dispatcher = EventDispatcher::new(event_config, noop_logger());
// Register event handlers
dispatcher.on_event("im.message.receive_v1", |event| {
Box::pin(async move {
println!("Received message: {:?}", event);
Ok(None)
})
});
// Start server
let server = FeishuServer::new(dispatcher).port(8080);
server.run().await;
}
```
### WebSocket Client
```rust
use feishu_sdk::core::noop_logger;
use feishu_sdk::ws::{WebSocketClient, WsConfig};
use std::time::Duration;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let config = WsConfig::new("wss://ws.feishu.cn/...")
.heartbeat_interval(Duration::from_secs(30))
.reconnect_delay(Duration::from_secs(5));
let client = WebSocketClient::new(config, noop_logger()).await?;
// Check connection status
if client.is_connected().await {
println!("Connected to WebSocket");
}
Ok(())
}
```
## Configuration
### Basic Configuration
```rust
use feishu_sdk::core::{Config, LogLevel, FEISHU_BASE_URL, LARK_BASE_URL};
use std::time::Duration;
let config = Config::builder("app_id", "app_secret")
.base_url(FEISHU_BASE_URL) // or LARK_BASE_URL for international
.log_level(LogLevel::Debug) // Set log level
.request_timeout(Duration::from_secs(30)) // Request timeout
.enable_token_cache(true) // Enable token caching
.build();
```
### Marketplace App Configuration
```rust
let config = Config::builder("app_id", "app_secret")
.marketplace_app() // Enable marketplace mode
.app_ticket("your_app_ticket") // Required for marketplace apps
.build();
```
### Helpdesk Configuration
```rust
let config = Config::builder("app_id", "app_secret")
.helpdesk("helpdesk_id", "helpdesk_token")
.build();
```
### Client Options
```rust
use feishu_sdk::client::{ClientBuilder, with_http_client, with_log_level, with_request_timeout, with_serializer};
use feishu_sdk::core::{Config, LogLevel};
let client = ClientBuilder::new(Config::builder("app_id", "app_secret").build())
.with(with_log_level(LogLevel::Debug))
.with(with_request_timeout(Duration::from_secs(60)))
.with(with_http_client(custom_http_client))
.with(with_serializer(custom_serializer))
.build()?;
```
## API Reference
### Operation IDs
Operations use the format: `{service}.{version}.{resource}.{method}`
Examples:
- `im.v1.messages.create` - Send message
- `im.v1.chat.list` - List chats
- `contact.v3.user.get` - Get user info
- `auth.v3.tenant_access_token.internal.post` - Get tenant token
### Typed API Wrappers
```rust
use feishu_sdk::api::all_services;
// Get chat info
let builder = all_services::im::v1::chat::get(&client, "oc_xxx")
.query_param("user_id_type", "open_id");
// List users
let builder = all_services::contact::v3::user::list(&client)
.query_param("department_id", "0")
.query_param("page_size", "50");
```
## Examples
See the `examples/` directory:
- `get_token.rs` - Get tenant access token
- `send_message.rs` - Send a message
- `get_user.rs` - Get user information
- `im_v1_message.rs` - IM message operations
- `drive_v1_file.rs` - Drive file operations
- `calendar_v4.rs` - Calendar operations
- `docx_v1.rs` - Docx document operations
- `sheets_v3.rs` - Sheets operations
- `bitable_v1.rs` - Bitable operations
- `approval_v4.rs` - Approval operations
- `event_server.rs` - Event handling server
- `card_handler.rs` - Card callback server
- `websocket_client.rs` - WebSocket client
- `full_app.rs` - End-to-end app example (client + event + card)
Run examples:
```bash
# Set environment variables
export FEISHU_APP_ID=your_app_id
export FEISHU_APP_SECRET=your_app_secret
# Run example
cargo run --example get_token
# Run server example (requires server feature)
cargo run --features server --example event_server
# Run websocket example (requires websocket feature)
cargo run --features websocket --example websocket_client
```
## Documentation
- Usage guide: [`docs/USAGE.md`](docs/USAGE.md)
- FAQ: [`docs/FAQ.md`](docs/FAQ.md)
- Migration guide: [`docs/MIGRATION.md`](docs/MIGRATION.md)
- Best practices: [`docs/BEST_PRACTICES.md`](docs/BEST_PRACTICES.md)
## Error Handling
```rust
use feishu_sdk::core::Error;
match client.operation("im.v1.messages.create").send().await {
Ok(resp) => println!("Success: {:?}", resp.json_value()?),
Err(Error::Api(api_error)) => {
println!("API Error: code={}, msg={}", api_error.code, api_error.msg);
}
Err(Error::RequestTimeout(_)) => println!("Request timed out"),
Err(e) => println!("Error: {}", e),
}
```
## License
MIT License
@@ -0,0 +1,10 @@
$ErrorActionPreference = 'Stop'
$zip = "$env:TEMP\larksdk.zip"
$dir = "$env:TEMP\larksdk"
curl.exe -sL "https://goproxy.cn/github.com/larksuite/oapi-sdk-go/v3/@v/v3.12.0.zip" -o $zip
if (Test-Path $dir) { Remove-Item -Recurse -Force $dir }
Expand-Archive -Path $zip -DestinationPath $dir -Force
Write-Host "=== ws 相关文件 ==="
Get-ChildItem -Recurse $dir -Filter *.go | Where-Object { $_.FullName -match '\\ws\\|ws\\.go|longconn' } | ForEach-Object { $_.FullName.Replace($dir, '') }
Write-Host "`n=== 顶层目录 ==="
Get-ChildItem $dir | ForEach-Object { Get-ChildItem $_.FullName -Directory | Select-Object -ExpandProperty Name } | Select-Object -Unique
@@ -0,0 +1,3 @@
# 从 larkrs-tree.json 提取关键源文件路径(ws/card/event/channel)
$j = Get-Content 'E:\wk-lab\devflow\docs\02-架构设计\专项设计\.research\larkrs-tree.json' -Raw | ConvertFrom-Json
$j.tree | Where-Object { $_.path -match '(ws|card|event|channel)' -and $_.path -match '\.rs$' } | ForEach-Object { $_.path }
File diff suppressed because one or more lines are too long
@@ -0,0 +1,117 @@
=== maemual @ 2024-03-22T06:42:29Z ===
近期卡片新特性较多,人力比较紧张,还在推进中。
可以继续使用旧版的卡片回传交互使用。
=== Martin2877 @ 2024-06-21T02:13:35Z ===
卡片回调一直报这个错误 `json: cannot unmarshal string into Go struct field CardAction.Header of type []string]`
=== sbaby1997 @ 2024-07-31T13:45:20Z ===
这个问题非常影响使用呀!
=== maemual @ 2024-07-31T13:53:03Z ===
@sbaby1997 升级最新版 v3.3.0,然后卡片回传交互选“卡片回传交互(旧)”
=== Martin2877 @ 2024-08-02T01:54:18Z ===
All right. 鉴于官方还没修,我这里提出一个临时解决方法,大家可以参考:
在 `package larkcard` 文件中
1. 修改 `model.go` 中的 `CardAction`
```go
type CardAction struct {
*larkevent.EventReq
Schema string `json:"schema"`
Header struct {
EventID string `json:"event_id"`
Token string `json:"token"`
CreateTime string `json:"create_time"`
EventType string `json:"event_type"`
TenantKey string `json:"tenant_key"`
AppID string `json:"app_id"`
} `json:"header"`
Challenge string `json:"challenge"`
Token string `json:"token"`
Type string `json:"type"`
Event struct {
Operator struct {
TenantKey string `json:"tenant_key"`
UserID string `json:"user_id"`
OpenID string `json:"open_id"`
UnionID string `json:"union_id"`
} `json:"operator"`
Token string `json:"token"`
Action struct {
Tag string `json:"tag"`
Timezone string `json:"timezone"`
FormValue map[string]string `json:"form_value"`
Name string `json:"name"`
} `json:"action"`
Host string `json:"host"`
Context struct {
OpenMessageID string `json:"open_message_id"`
OpenChatID string `json:"open_chat_id"`
} `json:"context"`
} `json:"event"`
}
```
2. 修改`card.go`中的 `VerifySign`方法,使用 eventEncryptKey
```go
if h.eventEncryptKey == "" {
return nil
}
...
targetSign := Signature(requestTimestamp, requestNonce,h.eventEncryptKey,string(req.Body))
```
3. 修改`card.go`中的 `Signature` 方法,使用 sha256加密
```go
h := sha256.New()
```
=== crazykun @ 2024-09-14T02:43:54Z ===
新的改成event事件承载了
[v3代码示例](https://github.com/larksuite/oapi-sdk-go/blob/v3_main/sample/event/event.go#L171)
```
.OnP2CardActionTrigger(func(ctx context.Context, event *callback.CardActionTriggerEvent) (*callback.CardActionTriggerReponse, error) {
fmt.Println("receive card action")
fmt.Println(larkcore.Prettify(event))
return nil, nil
})
```
=== kj648 @ 2025-01-23T11:52:59Z ===
对于卡片消息回调的 Value 应该定义成interface{} . 至少目前的卡片消息回调, 回调事件的值 支持 字符串,如果用字符串传进来 就直接cannot unmarshal string into Go struct XXXXXXX 。为了不出发这个error 只能强制使用 对象结构作为value
event *callback.CardActionTriggerEvent
```go
type CallBackAction struct {
Value map[string]interface{} `json:"value"`
Tag string `json:"tag"`
Option string `json:"option"`
Timezone string `json:"timezone"`
Name string `json:"name"`
FormValue map[string]interface{} `json:"form_value"`
InputValue string `json:"input_value"`
Options []string `json:"options"`
Checked bool `json:"checked"`
}
```
=== xiaozhi747 @ 2025-03-14T09:53:22Z ===
还没修复么?
=== maemual @ 2025-03-14T10:12:24Z ===
@xiaozhi747 早就支持了新版卡片回调,建议按照 README 文档使用。
=== eyotang @ 2025-11-24T04:13:19Z ===
> 卡片回调一直报这个错误 `json: cannot unmarshal string into Go struct field CardAction.Header of type []string]`
v3.4.3也有这个问题,请问怎么解决的?header是map[string]string,但是sdk里面定义的是map[string][]string
@@ -0,0 +1 @@
{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130","repository_url":"https://api.github.com/repos/larksuite/oapi-sdk-go","labels_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/labels{/name}","comments_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/comments","events_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/events","html_url":"https://github.com/larksuite/oapi-sdk-go/issues/130","id":2201794740,"node_id":"I_kwDOEiHOJs6DPLi0","number":130,"title":"使用新版卡片回调消息格式不匹配,反序列化错误","user":{"login":"lxitgto","id":11627541,"node_id":"MDQ6VXNlcjExNjI3NTQx","avatar_url":"https://avatars.githubusercontent.com/u/11627541?v=4","gravatar_id":"","url":"https://api.github.com/users/lxitgto","html_url":"https://github.com/lxitgto","followers_url":"https://api.github.com/users/lxitgto/followers","following_url":"https://api.github.com/users/lxitgto/following{/other_user}","gists_url":"https://api.github.com/users/lxitgto/gists{/gist_id}","starred_url":"https://api.github.com/users/lxitgto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lxitgto/subscriptions","organizations_url":"https://api.github.com/users/lxitgto/orgs","repos_url":"https://api.github.com/users/lxitgto/repos","events_url":"https://api.github.com/users/lxitgto/events{/privacy}","received_events_url":"https://api.github.com/users/lxitgto/received_events","type":"User","user_view_type":"public","site_admin":false},"labels":[],"state":"closed","locked":false,"assignees":[],"milestone":null,"comments":10,"created_at":"2024-03-22T06:30:32Z","updated_at":"2025-11-24T04:13:19Z","closed_at":"2025-03-14T10:12:28Z","assignee":null,"author_association":"NONE","issue_field_values":[],"type":null,"active_lock_reason":null,"sub_issues_summary":{"total":0,"completed":0,"percent_completed":0},"issue_dependencies_summary":{"blocked_by":0,"total_blocked_by":0,"blocking":0,"total_blocking":0},"body":"使用新版卡片回调消息格式不匹配,反序列化错误。\r\n\r\n`CardAction struct`格式为v1格式,请问什么时候支持新版消息格式?","closed_by":{"login":"maemual","id":2352843,"node_id":"MDQ6VXNlcjIzNTI4NDM=","avatar_url":"https://avatars.githubusercontent.com/u/2352843?v=4","gravatar_id":"","url":"https://api.github.com/users/maemual","html_url":"https://github.com/maemual","followers_url":"https://api.github.com/users/maemual/followers","following_url":"https://api.github.com/users/maemual/following{/other_user}","gists_url":"https://api.github.com/users/maemual/gists{/gist_id}","starred_url":"https://api.github.com/users/maemual/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maemual/subscriptions","organizations_url":"https://api.github.com/users/maemual/orgs","repos_url":"https://api.github.com/users/maemual/repos","events_url":"https://api.github.com/users/maemual/events{/privacy}","received_events_url":"https://api.github.com/users/maemual/received_events","type":"User","user_view_type":"public","site_admin":false},"reactions":{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/reactions","total_count":0,"+1":0,"-1":0,"laugh":0,"hooray":0,"confused":0,"heart":0,"rocket":0,"eyes":0},"timeline_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/timeline","performed_via_github_app":null,"state_reason":"completed","pinned_comment":null}
@@ -0,0 +1 @@
{"total_count":1,"incomplete_results":false,"items":[{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130","repository_url":"https://api.github.com/repos/larksuite/oapi-sdk-go","labels_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/labels{/name}","comments_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/comments","events_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/events","html_url":"https://github.com/larksuite/oapi-sdk-go/issues/130","id":2201794740,"node_id":"I_kwDOEiHOJs6DPLi0","number":130,"title":"使用新版卡片回调消息格式不匹配,反序列化错误","user":{"login":"lxitgto","id":11627541,"node_id":"MDQ6VXNlcjExNjI3NTQx","avatar_url":"https://avatars.githubusercontent.com/u/11627541?v=4","gravatar_id":"","url":"https://api.github.com/users/lxitgto","html_url":"https://github.com/lxitgto","followers_url":"https://api.github.com/users/lxitgto/followers","following_url":"https://api.github.com/users/lxitgto/following{/other_user}","gists_url":"https://api.github.com/users/lxitgto/gists{/gist_id}","starred_url":"https://api.github.com/users/lxitgto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lxitgto/subscriptions","organizations_url":"https://api.github.com/users/lxitgto/orgs","repos_url":"https://api.github.com/users/lxitgto/repos","events_url":"https://api.github.com/users/lxitgto/events{/privacy}","received_events_url":"https://api.github.com/users/lxitgto/received_events","type":"User","user_view_type":"public","site_admin":false},"labels":[],"state":"closed","locked":false,"assignees":[],"milestone":null,"comments":10,"created_at":"2024-03-22T06:30:32Z","updated_at":"2025-11-24T04:13:19Z","closed_at":"2025-03-14T10:12:28Z","assignee":null,"author_association":"NONE","issue_field_values":[],"type":null,"active_lock_reason":null,"sub_issues_summary":{"total":0,"completed":0,"percent_completed":0},"issue_dependencies_summary":{"blocked_by":0,"total_blocked_by":0,"blocking":0,"total_blocking":0},"body":"使用新版卡片回调消息格式不匹配,反序列化错误。\r\n\r\n`CardAction struct`格式为v1格式,请问什么时候支持新版消息格式?","reactions":{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/reactions","total_count":0,"+1":0,"-1":0,"laugh":0,"hooray":0,"confused":0,"heart":0,"rocket":0,"eyes":0},"timeline_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/130/timeline","performed_via_github_app":null,"state_reason":"completed","pinned_comment":null,"score":1.0}],"search_type":"lexical"}
@@ -0,0 +1 @@
{"total_count":1,"incomplete_results":false,"items":[{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/244","repository_url":"https://api.github.com/repos/larksuite/oapi-sdk-go","labels_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/244/labels{/name}","comments_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/244/comments","events_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/244/events","html_url":"https://github.com/larksuite/oapi-sdk-go/pull/244","id":5270680313,"node_id":"PR_kwDOEiHOJs8AAAABBPl61Q","number":244,"title":"fix(ws): make WebSocket client lifecycle cancellable","user":{"login":"SadMemories","id":44433408,"node_id":"MDQ6VXNlcjQ0NDMzNDA4","avatar_url":"https://avatars.githubusercontent.com/u/44433408?v=4","gravatar_id":"","url":"https://api.github.com/users/SadMemories","html_url":"https://github.com/SadMemories","followers_url":"https://api.github.com/users/SadMemories/followers","following_url":"https://api.github.com/users/SadMemories/following{/other_user}","gists_url":"https://api.github.com/users/SadMemories/gists{/gist_id}","starred_url":"https://api.github.com/users/SadMemories/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SadMemories/subscriptions","organizations_url":"https://api.github.com/users/SadMemories/orgs","repos_url":"https://api.github.com/users/SadMemories/repos","events_url":"https://api.github.com/users/SadMemories/events{/privacy}","received_events_url":"https://api.github.com/users/SadMemories/received_events","type":"User","user_view_type":"public","site_admin":false},"labels":[],"state":"closed","locked":false,"assignees":[],"milestone":null,"comments":0,"created_at":"2026-08-27T18:07:43Z","updated_at":"2026-08-28T09:41:33Z","closed_at":"2026-08-28T09:41:33Z","assignee":null,"author_association":"COLLABORATOR","issue_field_values":[],"type":null,"active_lock_reason":null,"draft":false,"pull_request":{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/pulls/244","html_url":"https://github.com/larksuite/oapi-sdk-go/pull/244","diff_url":"https://github.com/larksuite/oapi-sdk-go/pull/244.diff","patch_url":"https://github.com/larksuite/oapi-sdk-go/pull/244.patch","merged_at":"2026-08-28T09:41:33Z"},"body":"## Summary\n- make `Client.Start` stop deterministically for caller cancellation, explicit `Close`, and terminal connection errors\n- preserve public bootstrap error types and synchronous lifecycle callback delivery\n- detect half-open connections with read deadlines, and bound WebSocket writes so read or write failures enter the reconnect flow\n- remove the bootstrap dependency on the global HTTP client: add `WithHttpClient` and `WithWebSocketDialer`, with a bounded default bootstrap HTTP client\n- keep `Close` callback-safe and non-blocking; add `CloseAndWait(ctx)` for callers that must wait for connection teardown, `OnDisconnected`, and SDK workers\n- restore legacy lifecycle log wording and per-connection `conn_id` output\n\n## Fixed issues\n\nFixes #189\nFixes #201\nFixes #204\nFixes #224\n\n## Notes\n- `CloseAndWait(ctx)` must be called outside lifecycle callbacks; callbacks should call `Close()` to avoid waiting for the coordinator that is currently executing them.\n- Ping may start before `OnReady` / `OnReconnected` returns, consistent with the accepted behavior for this PR.\n\n## Validation\n- `go test -count=1 -race ./ws`\n- `go test ./...`\n- IM sample: Code 0\n- event and card mocks: HTTP 200\n- WebSocket sample: connected, ping success, receive pong","reactions":{"url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/244/reactions","total_count":0,"+1":0,"-1":0,"laugh":0,"hooray":0,"confused":0,"heart":0,"rocket":0,"eyes":0},"timeline_url":"https://api.github.com/repos/larksuite/oapi-sdk-go/issues/244/timeline","performed_via_github_app":null,"state_reason":null,"score":1.0}],"search_type":"lexical"}
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
{"message":"Not Found","documentation_url":"https://docs.github.com/rest/git/trees#get-a-tree","status":"404"}
@@ -0,0 +1,46 @@
use std::collections::HashMap;
use larksuite_oapi_sdk_rs::{CardAction, CardActionHandler, EventReq};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let handler =
CardActionHandler::new("verification_token", "", |action: CardAction| async move {
println!("card clicked by open_id={}", action.open_id);
println!("action value: {:?}", action.action.value);
Ok(serde_json::json!({
"toast": {
"type": "info",
"content": "Handled by Rust"
}
}))
})
.skip_sign_verify();
let payload = serde_json::json!({
"open_id": "ou_xxx",
"open_message_id": "om_xxx",
"open_chat_id": "oc_xxx",
"tenant_key": "tenant_xxx",
"token": "verification_token",
"action": {
"tag": "button",
"value": {
"action": "acknowledge",
"ticket_id": "INC-1001"
}
}
});
let req = EventReq {
headers: HashMap::new(),
body: serde_json::to_vec(&payload)?,
request_uri: "/webhook/card".to_string(),
};
let resp = handler.handle(req).await;
println!("response status: {}", resp.status_code);
println!("response body: {}", String::from_utf8_lossy(&resp.body));
Ok(())
}
@@ -0,0 +1,132 @@
# Card protocol alignment
Message cards are an embedded JSON protocol: IM and CardKit APIs transport a
card as opaque content, so REST route parity cannot establish card-schema
coverage. This guide defines the required reference and verification workflow
for every change to the SDK's card support.
## Source roles
Use each reference for the thing it can prove:
1. **Official Lark card documentation** is the wire-contract authority. It
defines Card JSON 1.0 and 2.0 roots, component tags, field names, value
domains, constraints, client-version requirements, and deprecations.
2. **The Lark CLI card references and fixtures** are an operational
cross-check. They supply representative valid payloads and callback/update
lifecycle details, including `card.action.trigger`.
3. **The official Go SDK** is an implementation cross-check and compatibility
reference. Its card model may deliberately represent a different layer or
expose a different surface from a particular documentation page; an apparent
gap is a discrepancy to investigate, not evidence that the Go SDK is
obsolete or incomplete.
4. **The existing Rust SDK** is a regression baseline only. An existing
serializer or a test that repeats its output is not evidence of protocol
correctness.
When sources disagree, record the discrepancy in the protocol inventory and
investigate its version, schema, and transport context. The official
documentation defines the wire contract, while current Go SDK and CLI behavior
are required implementation evidence; do not assume that either implementation
is stale, or silently preserve a conflicting spelling.
## Required workflow
Before implementing or accepting an embedded-protocol change:
1. Classify it. REST resources and typed events use the Go compatibility
workflow; message cards use this workflow. A change touching both must pass
both sets of checks.
2. Refresh the pinned Go SDK and Lark CLI references, and record the source
revision plus official-documentation URLs and access date in the protocol
inventory.
3. Update the version-specific inventory before implementation. Card JSON 1.0
and 2.0 have separate roots and must have separate component, field,
enumeration, validation, and client-version entries.
4. Add or update exact JSON fixtures for every changed wire shape. Fixtures
must cover the tag, required fields, optional fields, enum spelling, and at
least one invalid cross-field combination where the protocol defines one.
5. Cross-check each changed card component against the CLI and Go references.
Record an absence from either reference as an absence, not as permission to
omit a documented capability.
6. Run the aggregate reference-alignment check and the focused Rust tests. A
new card feature cannot merge with only a builder unit test that asserts the
implementation's own JSON.
## Callback and update coverage
For callback-bearing Card JSON 1.0 components, the inventory and fixtures must
also cover the corresponding `card.action.trigger` fields, callback value or
form-value shape, and the three-second acknowledgement/update response. The
SDK may expose outbound card construction and inbound event types independently,
but their protocol relationship must be tested together.
Card JSON 2.0 and CardKit are separate tracks. CardKit's typed Card JSON 2.0
document transport and ordered text-content streaming have their own manifest
and transport fixtures; they are not optional extensions of a Card JSON 1.0
fixture. CardKit settings, element mutation, and batch-action helpers remain
separate protocol surfaces with their own exact transport fixtures.
## Gate status
The Card JSON 1.0 inventory and fixture corpus extend the deterministic
`reference-alignment-check` command with card fixtures and validation. The
aggregate command receives pinned Go and Lark CLI checkout paths, verifies
their revisions, and checks recorded source artifacts before it runs the Rust
fixture tests. Card checks remain one part of this aggregate gate, alongside
REST, event, and future protocol alignment; they do not get a separate command
or CI job.
The Card JSON 2.0 inventory is separate from the 1.0 corpus. It records the
incompatible root, every documented handwritten component, the root and
cross-field constraints that the typed AST must enforce, and exact valid and
invalid wire fixtures. `partial` means that a component has a typed wire shape
but is not yet eligible to be marked complete: its optional fields, validation,
and exact fixtures must still be covered. The documented Card-Builder-only
recycling container is recorded as a known divergence because it has no
handwritten Card JSON tag.
Card Builder is not a Card JSON AST. The SDK does not replicate the visual
editor or represent builder-only components. It supports delivery of a published
template through `card::template::TemplateMessage<T>`. `T` is a caller-defined
`Serialize` variable object, validated before delivery; `TemplateMessage` uses
`JsonValue` by default as the dynamic JSON escape hatch. The template-delivery
manifest keeps that envelope, its IM transport, and the builder-only
recycling-container example in the same aggregate alignment check.
`implemented` is reserved for a component whose field-level inventory, valid
fixture, invalid constraint fixture where applicable, and direct serde
round-trip coverage are all present. A complete component group does not imply
that the incompatible Card JSON 2.0 root or unrelated interaction and update
surfaces are complete.
## CardKit document, mutation, and instance coverage
`card::cardkit::CardDocument` accepts only a locally validated Card JSON 2.0
document and emits CardKit's escaped `type: "card_json"` payload. The typed
`LarkClient::cardkit_cards()` helper covers entity creation and creates an
ordered update session for full replacement and full-content updates on one
text or Markdown element.
Creation returns a CardKit entity ID, not an IM message. Send it through an IM
`interactive` request with `card::cardkit::CardEntityMessage`, which emits the
separate `{"type":"card","data":{"card_id":"..."}}` content envelope.
Do not reuse the CardKit `card_json` REST envelope for this step.
Each CardKit update carries a caller-provided idempotency key and a positive
sequence. One `CardKitUpdateSession` owns the sequence across document and
content, settings, element, and batch updates, advancing it only after a
successful request; a failed update can therefore be retried with the same
sequence and key. Content updates intentionally send the complete current
value: Lark renders a prefix extension as a typewriter update and otherwise
replaces the element content.
The typed mutation surface retains Card JSON 2.0 settings and full elements,
validates partial-element and insertion constraints, and encodes the nested
objects required by CardKit batch actions exactly once. It also covers template
instance creation and variable updates (a name-keyed object with open JSON
values). The pinned Go CardKit source does not expose Card Instance resources,
so those endpoints are normatively tracked against the official documentation
rather than unrelated Go template models. The raw generated resource methods
remain available for any future CardKit endpoint not yet given a typed
composition layer.
@@ -0,0 +1,105 @@
# Cards
The SDK keeps Card JSON 1.0, Card JSON 2.0, CardKit, and published Card Builder
templates as separate protocols. Choose the smallest surface that matches the
message lifecycle you need.
## Send a Card JSON document
Build a versioned Card JSON value, then create a `CardDocument` before passing
it to IM or channel message helpers. `CardDocument::new` validates the document;
outbound interactive-card helpers deliberately reject arbitrary serializable
values.
[`examples/card_send.rs`](../examples/card_send.rs) sends a validated Card JSON
1.0 document through the IM create-message API:
```sh
APP_ID=... APP_SECRET=... CHAT_ID=oc_... cargo run --example card_send
```
Use `card::v1` for Card JSON 1.0. Use `card::v2` for Card JSON 2.0; its root,
shared-card requirements, and element-identifier rules are incompatible with
1.0.
## Author Card JSON 2.0 layouts safely
`CardDocument::new` remains the final strict protocol gate. For dynamic
authoring, `CardDocument::new_with_diagnostic` preserves the original
`ValidationError` and also returns a stable code, a JSON Pointer, the violated
constraint, and relevant legal values. Do not parse its display string.
`ColumnSet` has constructors for the three legal width branches. They emit the
wire shape required by Card JSON 2.0 and leave document validation enabled:
```rust
use larksuite_oapi_sdk_rs::card::v2::{
AutoColumn, Body, Card, CardDocument, ColumnSet, ColumnWeight, Config,
Element, FixedColumn, FixedColumnWidth, Markdown, WeightedColumn,
};
let automatic = Element::ColumnSet(ColumnSet::automatic([
AutoColumn::new().element(Element::Markdown(Markdown::new("auto"))),
]));
let fixed = Element::ColumnSet(ColumnSet::fixed([
FixedColumn::new(FixedColumnWidth::pixels(160)?)
.element(Element::Markdown(Markdown::new("fixed"))),
]));
let weighted = Element::ColumnSet(ColumnSet::weighted([
WeightedColumn::new(ColumnWeight::new(3)?)
.element(Element::Markdown(Markdown::new("weighted"))),
]));
let card = Card::new()
.config(Config::new().update_multi())
.body(Body::new().element(automatic).element(fixed).element(weighted));
let _document = CardDocument::new(card)?;
# Ok::<(), larksuite_oapi_sdk_rs::card::v2::ValidationError>(())
```
The `Padding`, `Margin`, `Spacing::pixels`, `FixedColumnWidth`, and
`ColumnWeight` constructors encode the documented finite pixel/range grammar.
Existing string-oriented builders remain supported for compatibility and are
still validated by `CardDocument`.
## Stream a CardKit document
CardKit accepts a validated Card JSON 2.0 `card::cardkit::CardDocument`.
`LarkClient::cardkit_cards()` creates the entity; send that entity once with a
`card::cardkit::CardEntityMessage` through IM before beginning ordered updates.
Keep one update session for full-document, settings, element, and content
updates; each successful mutation advances its sequence. CardKit entities have
the documented 14-day lifetime and may be sent only once.
[`examples/cardkit_stream.rs`](../examples/cardkit_stream.rs) creates a
streaming CardKit card and replaces a Markdown element's complete content:
```sh
APP_ID=... APP_SECRET=... CHAT_ID=oc_... cargo run --example cardkit_stream
```
## Send a published Card Builder template
Card Builder's editor and builder-only components are not a Card JSON AST.
Send a published template with `TemplateMessage<T>`, where `T: Serialize` is
your application's template-variable object. The SDK requires that it serialize
to a JSON object; `TemplateMessage<JsonValue>` remains available for dynamic
bindings.
[`examples/card_template_send.rs`](../examples/card_template_send.rs) sends a
typed published template:
```sh
APP_ID=... APP_SECRET=... CHAT_ID=oc_... CARD_TEMPLATE_ID=... \
cargo run --example card_template_send
```
## Handle callbacks
Card callbacks are inbound events. Use
[`examples/card_action_handler.rs`](../examples/card_action_handler.rs) for a
typed `CardActionHandler` that returns a callback response. Application-level
authorization, routing, deduplication, and state management remain the
application's responsibility.
@@ -0,0 +1,30 @@
use larksuite_oapi_sdk_rs::channel::{Channel, ChannelPolicy};
use larksuite_oapi_sdk_rs::{EventDispatcher, LarkClient, LarkError};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let app_id = std::env::var("APP_ID").expect("APP_ID env var required");
let app_secret = std::env::var("APP_SECRET").expect("APP_SECRET env var required");
let verification_token = std::env::var("VERIFICATION_TOKEN").unwrap_or_default();
let encrypt_key = std::env::var("ENCRYPT_KEY").unwrap_or_default();
let client = LarkClient::builder(app_id, app_secret).build()?;
let dispatcher = EventDispatcher::new(verification_token, encrypt_key);
Channel::builder(&client, dispatcher)
.policy(ChannelPolicy::default().require_mention(false))
.on_message(|message| async move {
println!("message_id: {}", message.message_id);
println!("raw content: {}", message.content);
println!("normalized text: {}", message.text.unwrap_or_default());
for resource in message.resources {
println!("resource: {} {}", resource.resource_type, resource.file_key);
}
Ok::<(), LarkError>(())
})
.build()
.start()
.await?;
Ok(())
}
@@ -0,0 +1,158 @@
# Events, WebSockets, and channels
Webhook handlers use `EventDispatcher` for typed event callbacks, signature
verification, and encrypted payload processing. The optional `ws` feature adds
long connections for event delivery; see
[`examples/ws_client.rs`](../examples/ws_client.rs).
Trusted user channels match the upstream Go SDK behavior. Configure the channel
before obtaining its control handle, start the client, then bind a user token
after the connection becomes ready. See
[`examples/ws_client.rs`](../examples/ws_client.rs) for the connection lifecycle
and [`examples/channel_send.rs`](../examples/channel_send.rs) for higher-level
messages.
```rust,no_run
use std::collections::HashMap;
use std::time::Duration;
# use larksuite_oapi_sdk_rs::{EventDispatcher, LarkClient};
# #[tokio::main]
# async fn main() -> Result<(), Box<dyn std::error::Error>> {
# let client = LarkClient::builder("APP_ID", "APP_SECRET").build()?;
# let dispatcher = EventDispatcher::new("VERIFICATION_TOKEN", "ENCRYPT_KEY");
let ws_client = client
.ws_client(dispatcher)
.channel_tag("trusted_channel")
.websocket_query_params(HashMap::from([("env".into(), "staging".into())]));
let control = ws_client.control();
let task = tokio::spawn(ws_client.start());
while control.connection_id().is_none() {
tokio::time::sleep(Duration::from_millis(50)).await;
}
control.attach_user("USER_ACCESS_TOKEN").await?;
// Later, before dropping the user session:
// control.detach_user("USER_ACCESS_TOKEN").await?;
control.close_and_wait().await;
task.await??;
# Ok(())
# }
```
`WsClientControl::close` requests shutdown without waiting, which is safe from
lifecycle callbacks. `close_and_wait` is for external teardown and waits until
the running client has stopped. The client honors a finite reconnect count from
the gateway bootstrap configuration, treats an inbound-silent connection as
failed after two ping intervals plus five seconds, and reconnects after either
that liveness timeout or a timed-out connection write. It bounds each
connection write to ten seconds by default and accepts `write_timeout` for a
different bound. Event callbacks run independently of the receive loop, so a
slow callback does not mask a silent gateway. Use
`websocket_connector` when a custom proxy, TLS, or test dial transport is
needed; it affects only the WebSocket gateway dial, not the bootstrap HTTP
client.
## Calendar share-token joins
Calendar invitations provide an opaque share token through their link, QR
code, or RSVP card. Join with that token; the endpoint deliberately has no
event-ID alternative.
```rust,no_run
use larksuite_oapi_sdk_rs::service::calendar::v4::JoinCalendarEventReqBody;
use larksuite_oapi_sdk_rs::{LarkClient, RequestOption};
# async fn example() -> Result<(), larksuite_oapi_sdk_rs::LarkError> {
let client = LarkClient::builder("APP_ID", "APP_SECRET").build()?;
client
.calendar()
.calendar
.join_event(
&JoinCalendarEventReqBody::new("CALENDAR_SHARE_TOKEN"),
&RequestOption::default(),
)
.await?;
# Ok(())
# }
```
Use the optional `axum` feature for Axum HTTP adapters. The optional `messaging`
feature provides outbound send, reply, update, upload, and download operations
without WebSocket dependencies. The optional `channel` feature adds inbound
message normalization and runtime policy controls and enables both `messaging`
and `ws`.
## Replies
`ChannelMessaging::send` preserves convenience behavior: a reply whose target has
disappeared may retry as a top-level send. For workflows that must never escape
a thread, use `ChannelMessaging::reply` or `ChannelMessaging::reply_in_thread`.
Both methods only call the reply endpoint and return its error unchanged. Set
`uuid` to opt into Lark's one-hour idempotency window; automatic split messages
derive distinct UUIDs, so the supplied UUID must leave room for a `-N` suffix.
```rust,no_run
use larksuite_oapi_sdk_rs::channel::SendInput;
use larksuite_oapi_sdk_rs::{LarkClient, RequestOption};
# async fn run() -> Result<(), Box<dyn std::error::Error>> {
let client = LarkClient::builder("APP_ID", "APP_SECRET").build()?;
let messaging = client.channel_messaging();
messaging
.reply_in_thread(
"om_parent_message",
&SendInput {
markdown: Some("A reply that stays in the topic".into()),
uuid: Some("order-42-status".into()),
..Default::default()
},
&RequestOption::default(),
)
.await?;
# Ok(())
# }
```
## Message updates
Use `ChannelMessaging::edit_text` to update a text message. Use
`ChannelMessaging::edit_card` only for validated interactive-card documents; it
uses the separate card patch operation. See
[`examples/card_send.rs`](../examples/card_send.rs) for Card JSON delivery and
[`examples/card_action_handler.rs`](../examples/card_action_handler.rs) for
interactive callback responses.
```rust,no_run
use larksuite_oapi_sdk_rs::card::v1::{Card, CardDocument, Div, Element, Header, Text};
use larksuite_oapi_sdk_rs::{LarkClient, RequestOption};
# async fn run() -> Result<(), Box<dyn std::error::Error>> {
let client = LarkClient::builder("APP_ID", "APP_SECRET").build()?;
let messaging = client.channel_messaging();
let card = CardDocument::new(
Card::new()
.header(Header::new("Done"))
.element(Element::Div(Div::new(Text::lark_md("Complete")))),
)?;
messaging
.edit_text("om_text_message", "Updated status", &RequestOption::default())
.await?;
messaging
.edit_card(
"om_card_message",
&card,
&RequestOption::default(),
)
.await?;
# Ok(())
# }
```
## Cards
Use `card::v1` or `card::v2` to build versioned Card JSON, and validate it with
their `CardDocument` types before sending. `CardActionHandler` handles callback
payloads. See the [Cards guide](cards.md) for CardKit and published Card Builder
templates.
@@ -0,0 +1,219 @@
# larksuite-oapi-sdk-rs
[![Crates.io](https://img.shields.io/crates/v/larksuite-oapi-sdk-rs)](https://crates.io/crates/larksuite-oapi-sdk-rs)
[![Docs](https://docs.rs/larksuite-oapi-sdk-rs/badge.svg)](https://docs.rs/larksuite-oapi-sdk-rs)
[![CI](https://github.com/adamcavendish/larksuite-oapi-sdk-rs/actions/workflows/ci.yml/badge.svg)](https://github.com/adamcavendish/larksuite-oapi-sdk-rs/actions/workflows/ci.yml)
[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
Non-official, Tokio-native Rust SDK for Lark / Feishu (飞书) Open Platform
APIs, bots, events, WebSockets, and interactive cards. It follows the official
[Go SDK](https://github.com/larksuite/oapi-sdk-go) closely while providing
idiomatic Rust configuration, typed REST resources, event handling, and optional
WebSocket, Axum, and channel integration.
## Build with Lark / Feishu
- **Build a bot:** receive typed message events and reply through the IM API.
- **Receive events:** use WebSocket long connections without a public callback
URL, or verify and dispatch HTTP webhooks.
- **Automate Open Platform workflows:** use typed REST resources for messages,
calendars, documents, Base, approvals, and more.
New to the platform? Start with the
[build a Lark / Feishu bot tutorial](docs/tutorial-build-a-feishu-bot.md). It
creates a bot that replies to received messages over a WebSocket long
connection.
## Highlights
- Tenant, user, app, and marketplace token acquisition with configurable
caching and retry behavior.
- Typed REST resources with query inputs, upload/download helpers, and lazy
iterators for supported paginated endpoints.
- Raw requests and a Go-compatibility bridge for endpoints not yet promoted to
dedicated Rust resources.
- Webhook event dispatch with signature verification and encrypted payload
decryption.
- Optional WebSocket long connections, Axum handlers, and higher-level channel
message helpers.
## Install
```toml
[dependencies]
larksuite-oapi-sdk-rs = "0.3.12"
```
The minimum supported Rust version is 1.95.0.
## Quick Start
Build a client once, then call a typed resource. The SDK handles tenant-token
acquisition and request authentication for this endpoint.
```rust,no_run
use larksuite_oapi_sdk_rs::service::im::v1::CreateMessageReqBody;
use larksuite_oapi_sdk_rs::{LarkClient, RequestOption};
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let client = LarkClient::builder("APP_ID", "APP_SECRET").build()?;
let body = CreateMessageReqBody {
receive_id: Some("CHAT_ID".to_string()),
msg_type: Some("text".to_string()),
content: Some(r#"{"text":"Hello from Rust"}"#.to_string()),
uuid: None,
};
let response = client
.im()
.message
.create("chat_id", &body, &RequestOption::default())
.await?;
println!("success: {}", response.success());
Ok(())
}
```
Use `LarkClient::builder` to set a base URL, timeout, retry limit, token cache,
custom headers, marketplace mode, helpdesk credentials, or a JWT client
assertion provider. See [`examples/client_config.rs`](examples/client_config.rs)
for the complete setup.
## Common Flows
| Need | Start here |
| --- | --- |
| Build a first Lark / Feishu bot | [Tutorial: build a Lark / Feishu bot](docs/tutorial-build-a-feishu-bot.md) |
| Call a typed REST endpoint | [`examples/send_message.rs`](examples/send_message.rs) or [generated examples](examples/README.md) |
| Call an uncovered endpoint | [`examples/raw_api.rs`](examples/raw_api.rs) |
| Control VC meeting bots | [VC meeting bots](docs/vc-bots.md) |
| Export Spark app source | [Spark app export](docs/spark-app-export.md) |
| Preview or apply Spark database schema changes | [Spark database migration](docs/spark-db-migration.md) |
| Read or modify Mail conversations | [Mail user-mailbox threads](docs/mail-v1-threads.md) |
| Resolve Drive object metadata by token | [Drive token lookup](docs/drive-token-lookup.md) |
| Manage Unified KMS autonomous keys | [Unified KMS](docs/unified-kms.md) |
| Discover or manage BaseApps | [`examples/base_v3_app_read.rs`](examples/base_v3_app_read.rs) and [Base v3 application mode](docs/base-v3-apps.md) |
| Manage Base dashboards or dashboard blocks | [Base v3 dashboards](docs/base-v3-dashboards.md) |
| Read or update Base dashboard or form sharing | [Base v3 sharing](docs/base-v3-sharing.md) |
| Manage Base v3 forms and questions | [Base v3 forms](docs/base-v3-forms.md) |
| Manage Base workflows or visible fields | [Base workflows](docs/base-v3-workflows.md) and [Base visible fields](docs/base-visible-fields.md) |
| Configure a Base field extension or share records | [Base v3 field extensions and record share links](docs/base-v3-field-extensions.md) |
| Read or update modern Docx content | [`examples/docs_ai_fetch.rs`](examples/docs_ai_fetch.rs) and [Docs AI document content](docs/docs-ai.md) |
| Read, edit, or render Slides AI presentations | [`examples/slides_ai_read.rs`](examples/slides_ai_read.rs), [`examples/slides_ai_render.rs`](examples/slides_ai_render.rs), and [Slides AI presentation content](docs/slides-ai.md) |
| Exchange or refresh OAuth tokens | [`examples/authen_oauth.rs`](examples/authen_oauth.rs) and [OAuth SSO guide](docs/oauth-sso.md) |
| Join a shared calendar event | [Events, WebSockets, and channels](docs/events-and-channels.md#calendar-share-token-joins) |
| Receive webhook events | [`examples/event_handler.rs`](examples/event_handler.rs) |
| Send a validated interactive card | [`examples/card_send.rs`](examples/card_send.rs) and [Cards guide](docs/cards.md) |
| Stream a CardKit document | [`examples/cardkit_stream.rs`](examples/cardkit_stream.rs) and [Cards guide](docs/cards.md) |
| Send a published Card Builder template | [`examples/card_template_send.rs`](examples/card_template_send.rs) and [Cards guide](docs/cards.md) |
| Handle interactive card callbacks | [`examples/card_action_handler.rs`](examples/card_action_handler.rs) |
| Run a WebSocket event client | [`examples/ws_client.rs`](examples/ws_client.rs) with `ws` enabled |
| Send channel messages | [`examples/channel_send.rs`](examples/channel_send.rs) with `messaging` enabled |
| Normalize channel messages | [`examples/channel_normalize.rs`](examples/channel_normalize.rs) with `channel` enabled |
| Search bots or work with OKRs | [`examples/bot_search.rs`](examples/bot_search.rs) and [`examples/okr_v2.rs`](examples/okr_v2.rs) |
## Guides
- [Tutorial: build a Lark / Feishu bot](docs/tutorial-build-a-feishu-bot.md):
create a WebSocket-connected bot that receives and replies to messages.
- [Go service and event compatibility](docs/go-compatibility.md): raw
requests, the compatibility bridge, and deterministic Go-to-Rust checks.
- [Cards](docs/cards.md): versioned Card JSON, CardKit streaming, published
Card Builder templates, and callbacks.
- [Card protocol alignment](docs/card-protocol.md): source hierarchy and
verification workflow for Card JSON and card callbacks.
- [Events, WebSockets, and channels](docs/events-and-channels.md): webhook
dispatch, long connections, trusted user channels, replies, and updates.
- [OAuth SSO](docs/oauth-sso.md): Feishu and Lark browser login, PKCE, token
refresh, and application-owned session handling.
- [VC meeting bots](docs/vc-bots.md): event queries and outbound meeting join,
leave, message, countdown, and active-meeting operations.
- [Spark app export](docs/spark-app-export.md): streaming source archives,
response validation, and app-type-dependent source snapshots.
- [Spark database migration](docs/spark-db-migration.md): explicit preview,
irreversible apply, and caller-owned asynchronous status polling.
- [Mail user-mailbox threads](docs/mail-v1-threads.md): conversation reads
and batch label, folder, and Trash operations.
- [Drive token lookup](docs/drive-token-lookup.md): underlying object metadata
resolution without implicit download or export.
- [Unified KMS](docs/unified-kms.md): tenant-scoped autonomous-key lifecycle,
deletion plans, recovery, and import material.
- [Base v3 application mode](docs/base-v3-apps.md): user-token workspace,
BaseApp, page, and block operations.
- [Base v3 dashboards](docs/base-v3-dashboards.md): dashboard and dashboard
block lifecycle, chart data, and layout operations.
- [Base v3 sharing](docs/base-v3-sharing.md): dashboard and form sharing
reads and partial updates with user or tenant credentials.
- [Base v3 forms](docs/base-v3-forms.md): form lifecycle and form-question
operations, including explicit destructive-delete semantics.
- [Base v3 workflows](docs/base-v3-workflows.md): workflow lifecycle with
caller-owned definition bodies.
- [Base visible fields](docs/base-visible-fields.md): full-list view and form
field visibility and ordering.
- [Base v3 field extensions and record share links](docs/base-v3-field-extensions.md):
typed extension configuration, cell updates, and batch record links.
- [Docs AI document content](docs/docs-ai.md): modern Docx content fetches,
updates, asynchronous-task reads, and version-history operations.
- [Slides AI presentation content](docs/slides-ai.md): XML presentation and
slide operations with version-history support.
- [Documentation index](docs/README.md): detailed guides and runnable examples.
## Migration from 0.2
Version 0.3 makes API-breaking corrections so the public Rust surface matches
the upstream Lark contracts more closely.
- Replace `Client` with `LarkClient` and `ClientBuilder` with
`LarkClientBuilder`.
- Replace `client.wiki()` with `client.wiki_v1()` or `client.wiki_v2()`.
- Replace `do_req`, `do_req_typed`, and verb helpers with `raw_request*` and an
`ApiReq` when a dedicated typed resource is unavailable.
- Treat Hire v1 `IdNameObject.name` as `Option<I18n>` with `zh_cn` and `en_us`
fields.
- Handle JSON construction failures as `LarkError::Json`; legacy card element
helpers are fallible for the same reason.
- Use `JsonValue` for intentionally dynamic API data instead of public
`serde_json::Value` model fields.
Typed REST responses and incoming event payloads are `#[non_exhaustive]`.
Deserialize them and read their public fields as before, but do not use struct
literals to create server-owned values.
## Cargo Features
| Feature | Description |
| --- | --- |
| `ws` | WebSocket long-connection event client with protobuf framing. |
| `messaging` | High-level outbound messaging and resource operations without WebSocket dependencies. |
| `channel` | Inbound normalization and runtime policy helpers; enables `messaging` and `ws`. |
| `axum` | Axum adapters for webhook and card-action handlers. |
Cargo unifies enabled features across the dependency graph. Enabling `channel`
therefore enables both `messaging` and `ws`; enabling `messaging` and `ws`
separately does not enable the `channel` runtime API.
```toml
[dependencies]
larksuite-oapi-sdk-rs = { version = "0.3.12", features = ["ws", "axum"] }
```
## API Coverage
The SDK provides dedicated resources across IM, Calendar, Drive, Docs, Sheets,
Bitable, Contacts, Approval, VC, Wiki, Hire, and the broader Lark/Feishu
OpenAPI surface. Generated smoke tests verify request paths, query inputs,
bodies, uploads, downloads, and response handling. See the
[changelog](CHANGELOG.md) for release detail and the
[examples index](examples/README.md) for runnable calls.
## Requirements
- Rust 1.95.0 or newer
- Rust edition 2024
## License
Licensed under [MIT](LICENSE).
File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More