新增: 批次工作落地(推进链/评估闭环/事件总线/并发/加固) + 技术债清理 + 文档整理
后端: - 工作流推进链(D-03):advance_task/状态机/闸门走 df-nodes Node trait,conditions 条件引擎扩展 - 想法评估闭环:启发式评分+对抗评估,df-ideas/scoring + df-storage/idea_eval_repo + idea 前端打通 - 全局事件数据总线:df-ai/context+context_helpers+augmentation 跨模块解耦 - AI planner/plan_hint/intent:aichat B 路线并行多轮基础 - patch_file 加固(TD-03/04):读改写整体锁防 lost update,expected_hash 合约闭环 - 压缩超时兜底(F-15 卡死根治) - F-09 多会话并发:LlmConcurrency per-conv + streamingGuard 前端守护 + verify 脚本 - 知识注入 DRY/skills/audit 扩展 清理: - aichat 技术债(误报 allow/死导入/过时注释 30 项) - URGENT.md 删除(11 项加急全解决/迁 todo) - 文档整理(todo/待决策/待审查/ARCHITECTURE/INDEX + 总线/技术债审查新文档)
This commit is contained in:
299
crates/df-storage/src/crud/idea_eval_repo.rs
Normal file
299
crates/df-storage/src/crud/idea_eval_repo.rs
Normal file
@@ -0,0 +1,299 @@
|
||||
//! 灵感评估历史 Repo:IdeaEvalRepo(追加型审计表 idea_evaluations,V22)
|
||||
//!
|
||||
//! 每次灵感 AI 评估追加一行快照(version 单调递增),保留评估历史可追溯。
|
||||
//! 对齐 [`KnowledgeEventsRepo`](crate::crud::KnowledgeEventsRepo) 模式:
|
||||
//! `impl_repo!` 宏生成基础 CRUD(insert/get_by_id/list_all/query/update_field/delete/
|
||||
//! update_full)+ 额外专用方法 `list_by_idea`(按 idea_id 过滤,version DESC 排序)。
|
||||
//!
|
||||
//! # ⚠️ 宏 list_all / query 不可用(表无 created_at)
|
||||
//!
|
||||
//! 本表时间列是 `evaluated_at`(非 `created_at`),但 `impl_repo!` 宏生成的
|
||||
//! `list_all` / `query` 硬编码 `ORDER BY created_at DESC`(见 crud/mod.rs 宏内
|
||||
//! `ORDER BY created_at DESC` 字面量)。误调 `state.idea_eval.list_all()` 或
|
||||
//! `state.idea_eval.query(...)` 会触发 SQLite "no such column: created_at",运行时崩。
|
||||
//! **跨灵感按时间倒序浏览评估历史须走专用方法 [`IdeaEvalRepo::list_recent_idea_evals`]**
|
||||
//! (按 evaluated_at DESC + limit 钳制 200),对标 `KnowledgeEventsRepo::list_recent`
|
||||
//! 对 knowledge_events 表的同款兜底处理(那张表亦无 created_at,时间列名是 timestamp)。
|
||||
|
||||
use std::sync::Arc;
|
||||
|
||||
use rusqlite::{params, Connection, OptionalExtension, Row};
|
||||
use tokio::sync::Mutex;
|
||||
|
||||
use df_types::error::Result;
|
||||
|
||||
use crate::db::Database;
|
||||
use crate::models::IdeaEvaluationRecord;
|
||||
|
||||
use super::impl_repo;
|
||||
use super::{now_millis_str, storage_err, validate_column_name};
|
||||
|
||||
// ============================================================
|
||||
// from_row 辅助函数
|
||||
// ============================================================
|
||||
|
||||
/// 按 name 取 idea_evaluations 表 8 列 → IdeaEvaluationRecord。
|
||||
fn idea_eval_from_row(row: &Row<'_>) -> std::result::Result<IdeaEvaluationRecord, rusqlite::Error> {
|
||||
Ok(IdeaEvaluationRecord {
|
||||
id: row.get("id")?,
|
||||
idea_id: row.get("idea_id")?,
|
||||
version: row.get("version")?,
|
||||
ai_analysis: row.get("ai_analysis")?,
|
||||
scores: row.get("scores")?,
|
||||
score: row.get("score")?,
|
||||
evaluated_by: row.get("evaluated_by")?,
|
||||
evaluated_at: row.get("evaluated_at")?,
|
||||
})
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
// Repo 实现
|
||||
// ============================================================
|
||||
|
||||
impl_repo!(
|
||||
/// 灵感评估历史表 CRUD(追加型审计表,只增不改)
|
||||
IdeaEvalRepo,
|
||||
IdeaEvaluationRecord,
|
||||
"idea_evaluations",
|
||||
from_row => |row| idea_eval_from_row(row),
|
||||
insert => |conn, rec| {
|
||||
conn.execute(
|
||||
"INSERT INTO idea_evaluations (id, idea_id, version, ai_analysis, scores, score, evaluated_by, evaluated_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8)",
|
||||
params![
|
||||
rec.id, rec.idea_id, rec.version, rec.ai_analysis,
|
||||
rec.scores, rec.score, rec.evaluated_by, rec.evaluated_at
|
||||
],
|
||||
)
|
||||
},
|
||||
update => |_conn, _rec| {
|
||||
// 加固(审计不可篡改):追加型审计表语义禁止 UPDATE(历史快照只增不改,版本单调递增)。
|
||||
// 宏(impl_repo!)要求 update 闭包生成 update_full,但本表设计上仅 insert 追加新版本。
|
||||
// 若有调用方误调 update_full → 返回 Err(SQLITE_ERROR)而非真执行 UPDATE,杜绝历史被
|
||||
// 静默篡改(否则审计追溯失真,版本单调性被破坏)。Err 经 storage_err 映射为
|
||||
// Error::Storage,调用方拿到 Err 可发现误用。注:update_field 走宏通用 SQL 路径,
|
||||
// 不经此闭包,故本表 update_field 同样不应被调用(调用方约束,非编译期保证)。
|
||||
//
|
||||
// 显式标注 rusqlite::Result<usize>(宏吃闭包 expr 不能写 -> 返回类型,改 let 绑定标注):
|
||||
// 闭包返回 Err 经 update_full 的 .map_err(storage_err)? → Err(Error::Storage),
|
||||
// affected=usize 的类型锚点确保宏内 `affected > 0` 编译。
|
||||
let res: rusqlite::Result<usize> = Err(rusqlite::Error::SqliteFailure(
|
||||
rusqlite::ffi::Error::new(rusqlite::ffi::SQLITE_ERROR),
|
||||
Some("idea_evaluations 为追加型审计表,禁止 UPDATE(仅 insert 追加新版本)".to_string()),
|
||||
));
|
||||
res
|
||||
}
|
||||
);
|
||||
|
||||
impl IdeaEvalRepo {
|
||||
/// 按 idea_id 查询全部评估历史(version DESC,最新版本在前)。
|
||||
///
|
||||
/// 对标 [`KnowledgeEventsRepo::list_by_knowledge`](crate::crud::KnowledgeEventsRepo::list_by_knowledge)
|
||||
/// 的 spawn_blocking + prepare + query_map 模式。前端取最新评估直接取返回 Vec 首元素。
|
||||
pub async fn list_by_idea(&self, idea_id: &str) -> Result<Vec<IdeaEvaluationRecord>> {
|
||||
let conn = self.conn.clone();
|
||||
let idea_id = idea_id.to_owned();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare(
|
||||
"SELECT id,idea_id,version,ai_analysis,scores,score,evaluated_by,evaluated_at \
|
||||
FROM idea_evaluations WHERE idea_id = ?1 ORDER BY version DESC",
|
||||
)
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map(params![idea_id], |row| idea_eval_from_row(row))
|
||||
.map_err(storage_err)?;
|
||||
let mut results = Vec::new();
|
||||
for r in rows {
|
||||
results.push(r.map_err(storage_err)?);
|
||||
}
|
||||
Ok(results)
|
||||
})
|
||||
.await
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
|
||||
/// 跨灵感列最近 N 条评估历史(全表 evaluated_at DESC,top-N)。
|
||||
///
|
||||
/// **专用兜底方法**:本表时间列名是 `evaluated_at` 而非 `created_at`,但
|
||||
/// `impl_repo!` 宏生成的 `query()` / `list_all()` 硬编码 `ORDER BY created_at`
|
||||
/// (见 crud/mod.rs 宏内 `ORDER BY created_at DESC` 字面量),误调
|
||||
/// `state.idea_eval.list_all()` 或 `state.idea_eval.query(...)` 会触发 SQLite
|
||||
/// "no such column: created_at"。本方法走专用 SELECT 绕过宏硬编码,供需要跨灵感
|
||||
/// 按评估时间倒序浏览历史的调用方使用(对标 [`KnowledgeEventsRepo::list_recent`]
|
||||
/// 对 knowledge_events 表的同款兜底处理——那张表同样无 created_at,时间列名是 timestamp)。
|
||||
/// limit 上限钳制 200,防前端恶意/失误传超大值。
|
||||
pub async fn list_recent_idea_evals(&self, limit: u32) -> Result<Vec<IdeaEvaluationRecord>> {
|
||||
let conn = self.conn.clone();
|
||||
// 钳制 limit 防滥用(最大 200)
|
||||
let safe_limit = limit.min(200) as i64;
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare(
|
||||
"SELECT id,idea_id,version,ai_analysis,scores,score,evaluated_by,evaluated_at \
|
||||
FROM idea_evaluations ORDER BY evaluated_at DESC LIMIT ?1",
|
||||
)
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map(params![safe_limit], |row| idea_eval_from_row(row))
|
||||
.map_err(storage_err)?;
|
||||
let mut results = Vec::new();
|
||||
for r in rows {
|
||||
results.push(r.map_err(storage_err)?);
|
||||
}
|
||||
Ok(results)
|
||||
})
|
||||
.await
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
// 单元测试 — IdeaEvalRepo 内存 DB(insert + list_by_idea 排序)
|
||||
// ============================================================
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::db::Database;
|
||||
|
||||
fn erec(id: &str, idea_id: &str, version: i64, score: Option<f64>) -> IdeaEvaluationRecord {
|
||||
IdeaEvaluationRecord {
|
||||
id: id.to_string(),
|
||||
idea_id: idea_id.to_string(),
|
||||
version,
|
||||
ai_analysis: Some("{\"ok\":true}".to_string()),
|
||||
scores: Some("{\"overall\":1}".to_string()),
|
||||
score,
|
||||
evaluated_by: Some("glm-4".to_string()),
|
||||
evaluated_at: "1700000000000".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
async fn setup_repo() -> IdeaEvalRepo {
|
||||
let db = Database::open_in_memory().await.expect("open_in_memory");
|
||||
IdeaEvalRepo::new(&db)
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_by_idea_orders_version_desc() {
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(erec("e1", "idea_a", 1, Some(0.5))).await.unwrap();
|
||||
repo.insert(erec("e2", "idea_a", 3, Some(0.9))).await.unwrap();
|
||||
repo.insert(erec("e3", "idea_a", 2, Some(0.7))).await.unwrap();
|
||||
repo.insert(erec("e4", "idea_b", 1, Some(0.1))).await.unwrap();
|
||||
|
||||
let list = repo.list_by_idea("idea_a").await.unwrap();
|
||||
assert_eq!(list.len(), 3, "仅 idea_a 的 3 条评估");
|
||||
let versions: Vec<i64> = list.iter().map(|r| r.version).collect();
|
||||
assert_eq!(versions, vec![3, 2, 1], "version DESC: 最新在前");
|
||||
|
||||
let list_b = repo.list_by_idea("idea_b").await.unwrap();
|
||||
assert_eq!(list_b.len(), 1);
|
||||
assert_eq!(list_b[0].version, 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_by_idea_empty_for_unknown_idea() {
|
||||
let repo = setup_repo().await;
|
||||
let list = repo.list_by_idea("nope").await.unwrap();
|
||||
assert!(list.is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_recent_idea_evals_orders_evaluated_at_desc_and_clamps_limit() {
|
||||
// 兜底方法:跨灵感按 evaluated_at DESC + limit 钳制 200(规避宏硬编码
|
||||
// ORDER BY created_at 致本表崩)。evaluated_at 为毫秒字符串,字典序与时间序一致。
|
||||
let repo = setup_repo().await;
|
||||
// erec 默认 evaluated_at 同值,这里覆盖以验证排序
|
||||
let mut a = erec("e1", "idea_a", 1, Some(0.5));
|
||||
a.evaluated_at = "1700000000001".to_string();
|
||||
let mut b = erec("e2", "idea_b", 1, Some(0.6));
|
||||
b.evaluated_at = "1700000000003".to_string();
|
||||
let mut c = erec("e3", "idea_a", 2, Some(0.7));
|
||||
c.evaluated_at = "1700000000002".to_string();
|
||||
repo.insert(a).await.unwrap();
|
||||
repo.insert(b).await.unwrap();
|
||||
repo.insert(c).await.unwrap();
|
||||
|
||||
// limit=10 取全部,按 evaluated_at DESC(跨灵感)
|
||||
let list = repo.list_recent_idea_evals(10).await.unwrap();
|
||||
assert_eq!(list.len(), 3);
|
||||
let times: Vec<&str> = list.iter().map(|r| r.evaluated_at.as_str()).collect();
|
||||
assert_eq!(times, vec!["1700000000003", "1700000000002", "1700000000001"]);
|
||||
|
||||
// limit=2 截断到前 2
|
||||
let list2 = repo.list_recent_idea_evals(2).await.unwrap();
|
||||
assert_eq!(list2.len(), 2);
|
||||
assert_eq!(list2[0].id, "e2");
|
||||
assert_eq!(list2[1].id, "e3");
|
||||
|
||||
// limit 钳制:超大值被压到 200(不会崩,仅返回实际行数)
|
||||
let list_big = repo.list_recent_idea_evals(u32::MAX).await.unwrap();
|
||||
assert_eq!(list_big.len(), 3, "u32::MAX 钳制到 200,但表仅 3 行");
|
||||
|
||||
// 空表
|
||||
let repo_empty = setup_repo().await;
|
||||
assert!(repo_empty.list_recent_idea_evals(10).await.unwrap().is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn insert_then_get_by_id_roundtrip() {
|
||||
let repo = setup_repo().await;
|
||||
let rec = erec("e1", "idea_a", 1, Some(0.42));
|
||||
let id = repo.insert(rec.clone()).await.unwrap();
|
||||
assert_eq!(id, "e1");
|
||||
let got = repo.get_by_id("e1").await.unwrap().expect("记录存在");
|
||||
assert_eq!(got.idea_id, "idea_a");
|
||||
assert_eq!(got.version, 1);
|
||||
assert!((got.score.unwrap() - 0.42).abs() < 1e-9);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_fields_persist_none() {
|
||||
let repo = setup_repo().await;
|
||||
let rec = IdeaEvaluationRecord {
|
||||
id: "e1".to_string(),
|
||||
idea_id: "idea_a".to_string(),
|
||||
version: 1,
|
||||
ai_analysis: None,
|
||||
scores: None,
|
||||
score: None,
|
||||
evaluated_by: None,
|
||||
evaluated_at: "1700000000000".to_string(),
|
||||
};
|
||||
repo.insert(rec).await.unwrap();
|
||||
let got = repo.get_by_id("e1").await.unwrap().expect("记录存在");
|
||||
assert!(got.ai_analysis.is_none());
|
||||
assert!(got.scores.is_none());
|
||||
assert!(got.score.is_none());
|
||||
assert!(got.evaluated_by.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn update_full_rejected_for_append_only_audit() {
|
||||
// 加固(审计不可篡改):追加型审计表禁止 UPDATE。update_full 应返 Err 而非真执行
|
||||
// (防误用静默篡改历史快照,破坏版本单调性 + 审计追溯)。调用方应走 insert 追加新版本。
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(erec("e1", "idea_a", 1, Some(0.5))).await.unwrap();
|
||||
// 试 update_full(篡改 version 1 → 2 + 改 score)
|
||||
let mut rec = repo.get_by_id("e1").await.unwrap().expect("记录存在");
|
||||
rec.version = 2;
|
||||
rec.score = Some(0.99);
|
||||
let res = repo.update_full(&rec).await;
|
||||
assert!(
|
||||
res.is_err(),
|
||||
"update_full 应被拒(追加型审计表禁止 UPDATE),实际: {:?}",
|
||||
res
|
||||
);
|
||||
// 原记录未被篡改(version 仍 1,score 仍 0.5)
|
||||
let got = repo.get_by_id("e1").await.unwrap().expect("记录存在");
|
||||
assert_eq!(got.version, 1, "原记录 version 不应被 update_full 篡改");
|
||||
assert!(
|
||||
(got.score.unwrap() - 0.5).abs() < 1e-9,
|
||||
"原记录 score 不应被 update_full 篡改"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -17,22 +17,38 @@ use super::{now_millis_str, storage_err, validate_column_name};
|
||||
// 知识库 SELECT 列清单(防 COLS 漂移)
|
||||
// ============================================================
|
||||
|
||||
/// `knowledges` 表对应 `KnowledgeRecord` 14 个字段的列名(顺序与结构体一致)。
|
||||
/// `knowledges` 表对应 `KnowledgeRecord` 15 个字段的列名(顺序与结构体一致)。
|
||||
///
|
||||
/// 多处 `search`/`search_vector` 内联 COLS 串的 DRY 收口(CR-260615-03):集中一处定义,
|
||||
/// 配合下方 `KNOWLEDGE_COL_COUNT` 断言,任一处加列漏改会被测试 `test_knowledge_cols_matches_record`
|
||||
/// 立即捕获(`knowledge_from_row` 按 name 取列,SELECT 漏列会运行时 rusqlite 报错,故提前断言)。
|
||||
const KNOWLEDGE_COLS: &str = "id,kind,title,content,tags,status,confidence,reuse_count,verified,source_project,source_ref,reasoning,created_at,updated_at";
|
||||
///
|
||||
/// V23 新增 embedding_status 列(嵌入失败可补偿重试),已纳入列清单 + 计数。
|
||||
const KNOWLEDGE_COLS: &str = "id,kind,title,content,tags,status,confidence,reuse_count,verified,source_project,source_ref,reasoning,embedding_status,created_at,updated_at";
|
||||
/// `KnowledgeRecord` 字段数(与上面列清单的逗号分隔项数一致,被测试断言)。
|
||||
/// 仅测试期消费(列漂移断言);保留为非 `cfg(test)` 以便测试外的阅读者一眼看到字段数。
|
||||
#[cfg_attr(not(test), allow(dead_code))]
|
||||
const KNOWLEDGE_COL_COUNT: usize = 14;
|
||||
const KNOWLEDGE_COL_COUNT: usize = 15;
|
||||
/// `search_vector` 用的列清单:KNOWLEDGE_COLS + embedding(余弦计算用,不入 KnowledgeRecord)。
|
||||
const KNOWLEDGE_COLS_WITH_EMBEDDING: &str = concat!(
|
||||
"id,kind,title,content,tags,status,confidence,reuse_count,verified,",
|
||||
"source_project,source_ref,reasoning,created_at,updated_at,embedding"
|
||||
"source_project,source_ref,reasoning,embedding_status,created_at,updated_at,embedding"
|
||||
);
|
||||
|
||||
/// `ideas` 表对应 `IdeaRecord` 14 个字段的列名(顺序与结构体一致)。
|
||||
///
|
||||
/// 同 KNOWLEDGE_COLS 的列漂移防护(CR-260615-03):idea 表 INSERT/UPDATE/from_row 三处
|
||||
/// 各写一份列名串,加列须三处同步(如 V24 加 related_ids 即三处齐改),漏一处
|
||||
/// 只在运行时 rusqlite 报错(INSERT 列数与参数数不匹配 / from_row 取不到列)。集中一处
|
||||
/// 定义 + 配合 `IDEA_COL_COUNT` 断言 + 测试 `test_idea_cols_matches_record`,加列漏改即捕获。
|
||||
///
|
||||
/// V24 新增 related_ids 列(灵感间关联关系持久化打底),已纳入列清单 + 计数。
|
||||
#[cfg_attr(not(test), allow(dead_code))]
|
||||
const IDEA_COLS: &str = "id,title,description,status,priority,score,tags,source,promoted_to,ai_analysis,scores,related_ids,created_at,updated_at";
|
||||
/// `IdeaRecord` 字段数(与上面列清单的逗号分隔项数一致,被测试断言)。
|
||||
#[cfg_attr(not(test), allow(dead_code))]
|
||||
const IDEA_COL_COUNT: usize = 14;
|
||||
|
||||
// ============================================================
|
||||
// 向量工具 — embedding BLOB 序列化 + 余弦相似度
|
||||
// ============================================================
|
||||
@@ -74,6 +90,7 @@ fn idea_from_row(row: &Row<'_>) -> std::result::Result<IdeaRecord, rusqlite::Err
|
||||
promoted_to: row.get("promoted_to")?,
|
||||
ai_analysis: row.get("ai_analysis")?,
|
||||
scores: row.get("scores")?,
|
||||
related_ids: row.get("related_ids")?,
|
||||
created_at: row.get("created_at")?,
|
||||
updated_at: row.get("updated_at")?,
|
||||
})
|
||||
@@ -93,6 +110,7 @@ fn knowledge_from_row(row: &Row<'_>) -> std::result::Result<KnowledgeRecord, rus
|
||||
source_project: row.get("source_project")?,
|
||||
source_ref: row.get("source_ref")?,
|
||||
reasoning: row.get("reasoning")?,
|
||||
embedding_status: row.get("embedding_status")?,
|
||||
created_at: row.get("created_at")?,
|
||||
updated_at: row.get("updated_at")?,
|
||||
})
|
||||
@@ -121,22 +139,22 @@ impl_repo!(
|
||||
from_row => |row| idea_from_row(row),
|
||||
insert => |conn, rec| {
|
||||
conn.execute(
|
||||
"INSERT INTO ideas (id, title, description, status, priority, score, tags, source, promoted_to, ai_analysis, scores, created_at, updated_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13)",
|
||||
"INSERT INTO ideas (id, title, description, status, priority, score, tags, source, promoted_to, ai_analysis, scores, related_ids, created_at, updated_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14)",
|
||||
params![
|
||||
rec.id, rec.title, rec.description, rec.status, rec.priority,
|
||||
rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis,
|
||||
rec.scores, rec.created_at, rec.updated_at
|
||||
rec.scores, rec.related_ids, rec.created_at, rec.updated_at
|
||||
],
|
||||
)
|
||||
},
|
||||
update => |conn, rec| {
|
||||
conn.execute(
|
||||
"UPDATE ideas SET title = ?1, description = ?2, status = ?3, priority = ?4, score = ?5, tags = ?6, source = ?7, promoted_to = ?8, ai_analysis = ?9, scores = ?10, updated_at = ?11 WHERE id = ?12",
|
||||
"UPDATE ideas SET title = ?1, description = ?2, status = ?3, priority = ?4, score = ?5, tags = ?6, source = ?7, promoted_to = ?8, ai_analysis = ?9, scores = ?10, related_ids = ?11, updated_at = ?12 WHERE id = ?13",
|
||||
params![
|
||||
rec.title, rec.description, rec.status, rec.priority,
|
||||
rec.score, rec.tags, rec.source, rec.promoted_to, rec.ai_analysis,
|
||||
rec.scores, rec.updated_at, rec.id
|
||||
rec.scores, rec.related_ids, rec.updated_at, rec.id
|
||||
],
|
||||
)
|
||||
}
|
||||
@@ -151,23 +169,23 @@ impl_repo!(
|
||||
insert => |conn, rec| {
|
||||
let verified = if rec.verified { 1i32 } else { 0i32 };
|
||||
conn.execute(
|
||||
"INSERT INTO knowledges (id, kind, title, content, tags, status, confidence, reuse_count, verified, source_project, source_ref, reasoning, created_at, updated_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14)",
|
||||
"INSERT INTO knowledges (id, kind, title, content, tags, status, confidence, reuse_count, verified, source_project, source_ref, reasoning, embedding_status, created_at, updated_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15)",
|
||||
params![
|
||||
rec.id, rec.kind, rec.title, rec.content, rec.tags, rec.status, rec.confidence,
|
||||
rec.reuse_count, verified, rec.source_project, rec.source_ref, rec.reasoning,
|
||||
rec.created_at, rec.updated_at
|
||||
rec.embedding_status, rec.created_at, rec.updated_at
|
||||
],
|
||||
)
|
||||
},
|
||||
update => |conn, rec| {
|
||||
let verified = if rec.verified { 1i32 } else { 0i32 };
|
||||
conn.execute(
|
||||
"UPDATE knowledges SET kind = ?1, title = ?2, content = ?3, tags = ?4, status = ?5, confidence = ?6, reuse_count = ?7, verified = ?8, source_project = ?9, source_ref = ?10, reasoning = ?11, updated_at = ?12 WHERE id = ?13",
|
||||
"UPDATE knowledges SET kind = ?1, title = ?2, content = ?3, tags = ?4, status = ?5, confidence = ?6, reuse_count = ?7, verified = ?8, source_project = ?9, source_ref = ?10, reasoning = ?11, embedding_status = ?12, updated_at = ?13 WHERE id = ?14",
|
||||
params![
|
||||
rec.kind, rec.title, rec.content, rec.tags, rec.status, rec.confidence,
|
||||
rec.reuse_count, verified, rec.source_project, rec.source_ref, rec.reasoning,
|
||||
rec.updated_at, rec.id
|
||||
rec.embedding_status, rec.updated_at, rec.id
|
||||
],
|
||||
)
|
||||
}
|
||||
@@ -223,17 +241,15 @@ impl KnowledgeRepo {
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare(
|
||||
"SELECT id,kind,title,content,tags,status,confidence,reuse_count,verified,\
|
||||
source_project,source_ref,reasoning,created_at,updated_at \
|
||||
FROM knowledges WHERE status = ?1
|
||||
.prepare(&format!(
|
||||
"SELECT {KNOWLEDGE_COLS} FROM knowledges WHERE status = ?1
|
||||
ORDER BY CASE confidence
|
||||
WHEN 'high' THEN 3
|
||||
WHEN 'medium' THEN 2
|
||||
WHEN 'low' THEN 1
|
||||
ELSE 0
|
||||
END DESC, created_at DESC",
|
||||
)
|
||||
))
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map(params![status], |row| knowledge_from_row(row))
|
||||
@@ -270,6 +286,7 @@ impl KnowledgeRepo {
|
||||
/// 写入向量嵌入(BLOB = Vec<f32> 小端字节序列化)
|
||||
///
|
||||
/// embedding 列不进 KnowledgeRecord(IPC 不需要传向量给前端),专用方法读写。
|
||||
/// V23:同时把 embedding_status 置 'done'(成功标记),供补偿重试逻辑判别。
|
||||
pub async fn set_embedding(&self, id: &str, embedding: &[f32]) -> Result<bool> {
|
||||
let conn = self.conn.clone();
|
||||
let id = id.to_owned();
|
||||
@@ -278,7 +295,7 @@ impl KnowledgeRepo {
|
||||
let guard = conn.blocking_lock();
|
||||
let affected = guard
|
||||
.execute(
|
||||
"UPDATE knowledges SET embedding = ?1 WHERE id = ?2",
|
||||
"UPDATE knowledges SET embedding = ?1, embedding_status = 'done' WHERE id = ?2",
|
||||
params![blob, id],
|
||||
)
|
||||
.map_err(storage_err)?;
|
||||
@@ -288,6 +305,54 @@ impl KnowledgeRepo {
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
|
||||
/// 标记嵌入生成失败(embedding_status = 'failed'),供补偿重试逻辑定位。
|
||||
///
|
||||
/// 失败时不写 embedding(保持 NULL,检索侧 `embedding IS NOT NULL` 自然跳过该条走 LIKE)。
|
||||
/// 幂等:重复标记 failed 无副作用(同值覆写)。
|
||||
pub async fn mark_embedding_failed(&self, id: &str) -> Result<bool> {
|
||||
let conn = self.conn.clone();
|
||||
let id = id.to_owned();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let affected = guard
|
||||
.execute(
|
||||
"UPDATE knowledges SET embedding_status = 'failed' WHERE id = ?1",
|
||||
params![id],
|
||||
)
|
||||
.map_err(storage_err)?;
|
||||
Ok(affected > 0)
|
||||
})
|
||||
.await
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
|
||||
/// 列出 embedding_status = 'failed' 的已发布知识(补偿重试入口用)。
|
||||
///
|
||||
/// 仅返回 published(候选/归档不参与检索,重试无意义),按 created_at 升序(老条目优先补)。
|
||||
pub async fn list_failed_embeddings(&self) -> Result<Vec<KnowledgeRecord>> {
|
||||
let conn = self.conn.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare(&format!(
|
||||
"SELECT {KNOWLEDGE_COLS} FROM knowledges \
|
||||
WHERE status = 'published' AND embedding_status = 'failed' \
|
||||
ORDER BY created_at ASC"
|
||||
))
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map([], |row| knowledge_from_row(row))
|
||||
.map_err(storage_err)?;
|
||||
let mut results = Vec::new();
|
||||
for r in rows {
|
||||
results.push(r.map_err(storage_err)?);
|
||||
}
|
||||
Ok(results)
|
||||
})
|
||||
.await
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
|
||||
/// 向量检索: 加载全部 published 且有 embedding 的记录,纯 Rust 余弦相似度取 top-N
|
||||
///
|
||||
/// 返回 (记录, 相似度分数)。数据量 <50k 时暴力遍历 <50ms,够用;
|
||||
@@ -308,7 +373,7 @@ impl KnowledgeRepo {
|
||||
let query_vec = query_vec.to_vec();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
// 显式列: 14 个 KnowledgeRecord 字段 + embedding(余弦计算用,不入 KnowledgeRecord)
|
||||
// 显式列: 15 个 KnowledgeRecord 字段(含 embedding_status)+ embedding(余弦计算用,不入 KnowledgeRecord)
|
||||
let mut stmt = guard
|
||||
.prepare(&format!(
|
||||
"SELECT {KNOWLEDGE_COLS_WITH_EMBEDDING} FROM knowledges WHERE status = 'published' AND embedding IS NOT NULL"
|
||||
@@ -347,17 +412,15 @@ impl KnowledgeRepo {
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare(
|
||||
"SELECT id,kind,title,content,tags,status,confidence,reuse_count,verified,\
|
||||
source_project,source_ref,reasoning,created_at,updated_at \
|
||||
FROM knowledges WHERE status != 'archived'
|
||||
.prepare(&format!(
|
||||
"SELECT {KNOWLEDGE_COLS} FROM knowledges WHERE status != 'archived'
|
||||
ORDER BY CASE confidence
|
||||
WHEN 'high' THEN 3
|
||||
WHEN 'medium' THEN 2
|
||||
WHEN 'low' THEN 1
|
||||
ELSE 0
|
||||
END DESC, created_at DESC",
|
||||
)
|
||||
))
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map([], |row| knowledge_from_row(row))
|
||||
@@ -379,7 +442,9 @@ impl KnowledgeRepo {
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare("SELECT id,kind,title,content,tags,status,confidence,reuse_count,verified,source_project,source_ref,reasoning,created_at,updated_at FROM knowledges WHERE status = 'published' ORDER BY reuse_count DESC LIMIT ?1")
|
||||
.prepare(&format!(
|
||||
"SELECT {KNOWLEDGE_COLS} FROM knowledges WHERE status = 'published' ORDER BY reuse_count DESC LIMIT ?1"
|
||||
))
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map(params![limit_i], |row| knowledge_from_row(row))
|
||||
@@ -539,6 +604,23 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// IDEA_COLS 列数须等于 IDEA_COL_COUNT(任一处漂移:加列漏改 INSERT/UPDATE/from_row
|
||||
/// 三处之一 → 立即失败)。`idea_from_row` 按 name 取列,SELECT/INSERT 漏列会在运行时被
|
||||
/// rusqlite 报错;此断言提前到测试期捕获(本次 V24 加 related_ids 已改 3 处的回归保险)。
|
||||
#[test]
|
||||
fn test_idea_cols_matches_record() {
|
||||
let count = IDEA_COLS.split(',').count();
|
||||
assert_eq!(
|
||||
count, IDEA_COL_COUNT,
|
||||
"IDEA_COLS({count}列) ≠ IDEA_COL_COUNT({IDEA_COL_COUNT}); \
|
||||
修改一处须同步另一处(INSERT/UPDATE/from_row/IDEA_COLS/IDEA_COL_COUNT 五处)"
|
||||
);
|
||||
// 每个列名须能被 split 出来(防末尾多逗号 / 空段)
|
||||
for col in IDEA_COLS.split(',') {
|
||||
assert!(!col.is_empty(), "IDEA_COLS 含空列名段");
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- 向量纯函数 ----------
|
||||
|
||||
#[test]
|
||||
@@ -631,6 +713,7 @@ mod tests {
|
||||
source_project: Some("proj-1".to_string()),
|
||||
source_ref: Some("conv:c1".to_string()),
|
||||
reasoning: None,
|
||||
embedding_status: None,
|
||||
created_at: "1700000000000".to_string(),
|
||||
updated_at: "1700000000000".to_string(),
|
||||
}
|
||||
@@ -868,4 +951,105 @@ mod tests {
|
||||
let results = repo.search_vector(&[1.0, 0.0], 10).await.unwrap();
|
||||
assert!(results.is_empty());
|
||||
}
|
||||
|
||||
// ---------- V23 embedding_status 补偿重试 ----------
|
||||
|
||||
#[tokio::test]
|
||||
async fn set_embedding_marks_status_done() {
|
||||
// V23:set_embedding 成功写入时应同步置 embedding_status='done'
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(krec("k1", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.set_embedding("k1", &[1.0, 0.0]).await.unwrap();
|
||||
let rec = repo.get_by_id("k1").await.unwrap().expect("记录存在");
|
||||
assert_eq!(rec.embedding_status.as_deref(), Some("done"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn mark_embedding_failed_sets_status() {
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(krec("k1", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(repo.mark_embedding_failed("k1").await.unwrap());
|
||||
let rec = repo.get_by_id("k1").await.unwrap().expect("记录存在");
|
||||
assert_eq!(rec.embedding_status.as_deref(), Some("failed"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn mark_embedding_failed_idempotent() {
|
||||
// 重复标记 failed 无副作用
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(krec("k1", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.mark_embedding_failed("k1").await.unwrap();
|
||||
assert!(repo.mark_embedding_failed("k1").await.unwrap());
|
||||
let rec = repo.get_by_id("k1").await.unwrap().unwrap();
|
||||
assert_eq!(rec.embedding_status.as_deref(), Some("failed"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn mark_embedding_failed_missing_row_returns_false() {
|
||||
let repo = setup_repo().await;
|
||||
// 不存在的 id → affected=0
|
||||
let ok = repo.mark_embedding_failed("ghost").await.unwrap();
|
||||
assert!(!ok);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_failed_embeddings_only_returns_published_failed() {
|
||||
let repo = setup_repo().await;
|
||||
// k1:published + failed → 应列出
|
||||
repo.insert(krec("k1", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.mark_embedding_failed("k1").await.unwrap();
|
||||
// k2:candidate + failed → 不应列出(候选不参与检索,重试无意义)
|
||||
repo.insert(krec("k2", "t", "c", "lesson", "candidate", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.mark_embedding_failed("k2").await.unwrap();
|
||||
// k3:published + done → 不应列出
|
||||
repo.insert(krec("k3", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.set_embedding("k3", &[1.0, 0.0]).await.unwrap();
|
||||
// k4:published + 未标记(NULL)→ 不应列出
|
||||
repo.insert(krec("k4", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let failed = repo.list_failed_embeddings().await.unwrap();
|
||||
let ids: Vec<_> = failed.iter().map(|r| r.id.as_str()).collect();
|
||||
assert_eq!(ids, vec!["k1"], "只应返回 published + failed 的条目");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn list_failed_embeddings_empty_when_none_failed() {
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(krec("k1", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.set_embedding("k1", &[1.0, 0.0]).await.unwrap();
|
||||
let failed = repo.list_failed_embeddings().await.unwrap();
|
||||
assert!(failed.is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn failed_retry_flow_done_after_set_embedding() {
|
||||
// 补偿重试完整流程:failed → set_embedding 成功 → done(不再出现在 list_failed)
|
||||
let repo = setup_repo().await;
|
||||
repo.insert(krec("k1", "t", "c", "lesson", "published", None, 0))
|
||||
.await
|
||||
.unwrap();
|
||||
repo.mark_embedding_failed("k1").await.unwrap();
|
||||
assert_eq!(repo.list_failed_embeddings().await.unwrap().len(), 1);
|
||||
// 重试成功
|
||||
repo.set_embedding("k1", &[0.5, 0.5]).await.unwrap();
|
||||
assert!(repo.list_failed_embeddings().await.unwrap().is_empty());
|
||||
let rec = repo.get_by_id("k1").await.unwrap().unwrap();
|
||||
assert_eq!(rec.embedding_status.as_deref(), Some("done"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,12 +8,14 @@
|
||||
//! - [`mod@task_repo`]:TaskRepo(含 advance_status_atomic 状态机收口)
|
||||
//! - [`mod@conversation_repo`]:AiProviderRepo/AiConversationRepo/AiToolExecutionRepo
|
||||
//! - [`mod@idea_repo`]:IdeaRepo/KnowledgeRepo/KnowledgeEventsRepo + 向量工具
|
||||
//! - [`mod@idea_eval_repo`]:IdeaEvalRepo(灵感评估历史追加型审计表 idea_evaluations,V22)
|
||||
//! - [`mod@message_repo`]:AiMessageRepo(F-260619-03 消息拆分存储,全专用方法不走宏)
|
||||
//!
|
||||
//! re-export(`pub use ...::*`)保持 `df_storage::crud::XxxRepo` /
|
||||
//! `df_storage::crud::is_allowed_column` 路径不变,**调用方零改动**。
|
||||
|
||||
mod conversation_repo;
|
||||
mod idea_eval_repo;
|
||||
mod idea_repo;
|
||||
mod message_repo;
|
||||
mod project_repo;
|
||||
@@ -21,6 +23,7 @@ mod settings;
|
||||
mod task_repo;
|
||||
|
||||
pub use conversation_repo::*;
|
||||
pub use idea_eval_repo::*;
|
||||
pub use idea_repo::*;
|
||||
pub use message_repo::*;
|
||||
pub use project_repo::*;
|
||||
@@ -37,8 +40,9 @@ pub use task_repo::*;
|
||||
///
|
||||
/// 约束:`$record` 须实现 `Clone`(`update_full` 内部 clone 后丢给 spawn_blocking)。
|
||||
///
|
||||
/// 宏通过 `#[macro_use]`(见本文件末 `mod` 声明上方的文本注入)对各子模块可见,
|
||||
/// 子模块 `impl_repo!(...)` 直接调用。`from_row`/`insert`/`update` 体内引用的 helper
|
||||
/// 宏通过文件末 `pub(crate) use impl_repo;`(Rust 2e textual scope 重导出,非
|
||||
/// `#[macro_use]`)对各子模块可见,子模块 `use super::impl_repo;` 后直接调用。
|
||||
/// `from_row`/`insert`/`update` 体内引用的 helper
|
||||
/// (storage_err/now_millis_str/validate_column_name 及各 from_row)须在调用处可见。
|
||||
macro_rules! impl_repo {
|
||||
(
|
||||
@@ -203,6 +207,21 @@ pub(crate) fn storage_err<E: std::string::ToString>(e: E) -> df_types::error::Er
|
||||
df_types::error::Error::Storage(e.to_string())
|
||||
}
|
||||
|
||||
/// 判定扁平化后的存储错误是否为 SQLite 唯一约束冲突(SQLite extended code 2067 /
|
||||
/// SQLITE_CONSTRAINT_UNIQUE)。
|
||||
///
|
||||
/// 设计取舍:存储层在 [`storage_err`] 已将原始 `rusqlite::Error` 扁平化为 `String`
|
||||
/// (Error::Storage(String)),调用方拿不到结构化 `rusqlite::Error::SqliteFailure` 的
|
||||
/// `ErrorCode`,无法直接按 code 判定。故在此 flatten 边界集中收口检测逻辑,避免每个
|
||||
/// 调用方各自 `e.to_string().contains("UNIQUE constraint failed")` 散落脆弱匹配。
|
||||
///
|
||||
/// `"UNIQUE constraint failed"` 是 SQLite C 库对 2067 固定输出的文案(大写为 C 库常量,
|
||||
/// 不随 SQLite 版本/locale 变化),大小写不敏感匹配兜底未来可能的微小差异。
|
||||
pub fn is_unique_constraint_err(err: &df_types::error::Error) -> bool {
|
||||
let df_types::error::Error::Storage(msg) = err else { return false };
|
||||
msg.to_ascii_lowercase().contains("unique constraint failed")
|
||||
}
|
||||
|
||||
/// 规范化路径用于比较:canonicalize 解析绝对规范路径(失败降级),
|
||||
/// 统一正斜杠 + 小写。与 `df_project::scan::normalize_path` **同算法镜像**(df-storage
|
||||
/// 不依赖 df-project,故独立实现;改动须同步)。防 `C:\a\b` vs `C:/a/b/` 绕过。
|
||||
@@ -284,5 +303,6 @@ mod baseline_tests {
|
||||
let _ = KnowledgeRepo::new(&db);
|
||||
let _ = KnowledgeEventsRepo::new(&db);
|
||||
let _ = AiMessageRepo::new(&db);
|
||||
let _ = IdeaEvalRepo::new(&db);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -120,7 +120,7 @@ pub fn allowed_columns_for(table: &str) -> Option<&'static [&'static str]> {
|
||||
Some(match table {
|
||||
"ideas" => &[
|
||||
"id", "title", "description", "status", "priority", "score", "tags", "source",
|
||||
"promoted_to", "ai_analysis", "scores", "created_at", "updated_at",
|
||||
"promoted_to", "ai_analysis", "scores", "related_ids", "created_at", "updated_at",
|
||||
],
|
||||
"projects" => &[
|
||||
"id", "name", "description", "status", "idea_id", "path", "stack", "created_at",
|
||||
@@ -138,7 +138,12 @@ pub fn allowed_columns_for(table: &str) -> Option<&'static [&'static str]> {
|
||||
// update_task/update_field 白名单均不含)。后人勿把 review_rounds 补进白名单,
|
||||
// 否则破坏「review_rounds 唯一写入路径」收口、引入旁路写导致计数错乱。
|
||||
"project_id", "title", "description", "priority", "branch_name",
|
||||
"assignee", "workflow_def_id", "base_branch",
|
||||
"assignee",
|
||||
// workflow_def_id / base_branch: 预留字段,阶段4 Git/workflow_defs 集成前无写入路径。
|
||||
// 当前推进链用硬编码三模板(task_workflow_templates.rs,不建 workflow_defs 表,
|
||||
// tasks.workflow_def_id 留 None),无任何代码写这两列。白名单列入仅为阶段4 预留 +
|
||||
// 允许手动/未来填充,勿判死代码删除。base_branch 同理(code kind 闸门接 git 前预留)。
|
||||
"workflow_def_id", "base_branch",
|
||||
// output_json:ai_execute 写产出 / ai_self_review 读产出自审 / human_review 展示对象
|
||||
// (决策 a:task 中心,产出跟 task 走)。非状态机收口字段,合法可写。
|
||||
"output_json",
|
||||
|
||||
@@ -73,7 +73,7 @@ impl_repo!(
|
||||
impl TaskRepo {
|
||||
/// 列出未删除任务(deleted_at IS NULL)— 对标 ProjectRepo::list_active
|
||||
///
|
||||
/// 显式列出 14 个 TaskRecord 列名(同 ProjectRepo::list_active 写法),
|
||||
/// 显式列出全部 15 个 TaskRecord 列名(同 ProjectRepo::list_active 写法),
|
||||
/// 不 SELECT deleted_at:TaskRecord 不带该字段,取了 from_row 会因未知列报错。
|
||||
pub async fn list_active(&self) -> Result<Vec<TaskRecord>> {
|
||||
let conn = self.conn.clone();
|
||||
@@ -193,10 +193,35 @@ impl TaskRepo {
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
|
||||
/// 按项目列出未删除任务(deleted_at IS NULL AND project_id = ?),按创建时间降序。
|
||||
///
|
||||
/// SQL 下推 project_id 过滤:替代旧 list_active + 内存 retain 全表扫(任务量增长后
|
||||
/// N×M 热点,每页都拉全表进内存再丢)。list_tasks 在有 project_id 时优先走本方法。
|
||||
pub async fn list_active_by_project(&self, project_id: &str) -> Result<Vec<TaskRecord>> {
|
||||
let conn = self.conn.clone();
|
||||
let pid = project_id.to_owned();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
let guard = conn.blocking_lock();
|
||||
let mut stmt = guard
|
||||
.prepare("SELECT id, project_id, title, description, status, priority, branch_name, assignee, workflow_def_id, base_branch, review_rounds, output_json, idea_id, created_at, updated_at FROM tasks WHERE deleted_at IS NULL AND project_id = ?1 ORDER BY created_at DESC")
|
||||
.map_err(storage_err)?;
|
||||
let rows = stmt
|
||||
.query_map(params![pid], |row| task_from_row(row))
|
||||
.map_err(storage_err)?;
|
||||
let mut results = Vec::new();
|
||||
for r in rows {
|
||||
results.push(r.map_err(storage_err)?);
|
||||
}
|
||||
Ok(results)
|
||||
})
|
||||
.await
|
||||
.map_err(storage_err)?
|
||||
}
|
||||
|
||||
/// 列出回收站(deleted_at IS NOT NULL),按更新时间(≈删除时间)降序。对标 ProjectRepo::list_deleted。
|
||||
///
|
||||
/// 注:任务表无专用 list_active_by_project 方法,按项目列活跃任务由 commands/task.rs
|
||||
/// 的 list_tasks 用 list_active 后内存过滤 project_id 实现(任务量小,无需 SQL 下推)。
|
||||
/// 注:按项目列活跃任务走 list_active_by_project(SQL 下推 project_id),
|
||||
/// 无 pid 时 fallback list_active。
|
||||
pub async fn list_deleted(&self) -> Result<Vec<TaskRecord>> {
|
||||
let conn = self.conn.clone();
|
||||
tokio::task::spawn_blocking(move || {
|
||||
|
||||
@@ -23,8 +23,12 @@ pub fn run(conn: &Connection) -> Result<()> {
|
||||
|
||||
// 迁移步骤链: 顺序执行,跳过已应用的版本(current_version < N 才跑)。
|
||||
// 新增版本时,在此数组追加一项 (N, migrate_vN) 即可,无需改逻辑。
|
||||
// V20 = F-260619-01(任务关联灵感 idea_id);V21 = 消息拆分存储 + audit message_id。
|
||||
let steps: [(i32, fn(&Connection) -> Result<()>); 21] = [
|
||||
// V20 = F-260619-01(任务关联灵感 idea_id);V21 = 消息拆分存储 + audit message_id;
|
||||
// V22 = 灵感评估历史持久化(idea_evaluations 追加型审计表);
|
||||
// V23 = knowledges.embedding_status 列(嵌入失败可补偿重试);
|
||||
// V24 = ideas.related_ids 列(灵感间关联关系持久化打底);
|
||||
// V25 = idea_evaluations (idea_id, version) 唯一约束(评估版本并发重复兜底)。
|
||||
let steps: [(i32, fn(&Connection) -> Result<()>); 25] = [
|
||||
(1, migrate_v1),
|
||||
(2, migrate_v2),
|
||||
(3, migrate_v3),
|
||||
@@ -46,6 +50,10 @@ pub fn run(conn: &Connection) -> Result<()> {
|
||||
(19, migrate_v19),
|
||||
(20, migrate_v20),
|
||||
(21, migrate_v21),
|
||||
(22, migrate_v22),
|
||||
(23, migrate_v23),
|
||||
(24, migrate_v24),
|
||||
(25, migrate_v25),
|
||||
];
|
||||
|
||||
for (version, migrate_fn) in steps {
|
||||
@@ -512,6 +520,118 @@ fn migrate_v21(conn: &Connection) -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// V22:灵感评估历史持久化 — idea_evaluations 追加型审计表
|
||||
///
|
||||
/// 把灵感每次 AI 评估快照(ai_analysis / scores / score)按版本追加存表,
|
||||
/// 替代覆写 ideas.ai_analysis / ideas.scores 列。一条灵感多次评估产生多条记录,
|
||||
/// version 单调递增,前端按 (idea_id, version DESC) 取最新 + 翻历史。
|
||||
///
|
||||
/// 设计要点(对齐 knowledge_events 追加型审计表模式 V10):
|
||||
/// - **追加型**:只 INSERT 不 UPDATE,审计语义(评估快照不可篡改,历史可追溯)
|
||||
/// - **IF NOT EXISTS 幂等**:对新库建表 / 老库已有表跳过,均安全
|
||||
/// - **索引**:`(idea_id, version DESC)` 覆盖「取某灵感最新评估」最高频查询
|
||||
/// - **evaluated_by**:评估发起者(model 名 / human / system,可空)
|
||||
/// - **evaluated_at**:评估时间(毫秒字符串,同既有 model 约定)
|
||||
///
|
||||
/// 不登记通用列白名单(allowed_columns_for):本表走专用 list_by_idea,
|
||||
/// 宏生成的 query/update_field 未登记表会被 validate_column_name 保守拒绝
|
||||
/// (FR-S6),与追加型审计语义一致(历史不改),不开放通用写路径。
|
||||
fn migrate_v22(conn: &Connection) -> Result<()> {
|
||||
conn.execute_batch(V22_SQL)?;
|
||||
conn.execute("INSERT INTO schema_version (version) VALUES (?)", [22])?;
|
||||
tracing::info!("迁移 v22 完成");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// V23:幂等补 knowledges.embedding_status 列(嵌入生成失败可补偿重试)
|
||||
///
|
||||
/// P1 修复(嵌入失败无标记):spawn_embedding_for_knowledge 此前 fire-and-forget,
|
||||
/// provider 临时不可用 → 仅 warn,该条永久无向量索引但无人感知(下次也不会重试)。
|
||||
/// 新增 embedding_status 列跟踪嵌入生命周期:
|
||||
/// - NULL:未生成(老库行迁移后默认 NULL;代码从无显式写 NULL/pending 的路径,
|
||||
/// 仅此两种取值实际出现:done / failed)
|
||||
/// - done:成功(已有有效 embedding,由 KnowledgeRepo::set_embedding 写入)
|
||||
/// - failed:失败可重试(下次发布 / 手动 retry 时补偿,由 KnowledgeRepo::mark_embedding_failed 写入)
|
||||
///
|
||||
/// 语义:embedding 列(BLOB)与 embedding_status 解耦 —— embedding 仅在 done 时有值;
|
||||
/// 失败时 status=failed + embedding 仍 NULL,检索侧 `embedding IS NOT NULL` 自然跳过。
|
||||
/// 老库行(已成功嵌入的)embedding 有值但 status=NULL:这类条目检索正常(embedding IS NOT NULL),
|
||||
/// 不影响功能;若需精确状态,可在后台补偿脚本回填 done,但非必需(检索不依赖 status)。
|
||||
///
|
||||
/// TEXT NULL 向后兼容;不进通用 update_field 白名单(写入走 KnowledgeRepo::set_embedding /
|
||||
/// mark_embedding_failed 两个专用方法,而非独立的 set_embedding_status)。用 PRAGMA 探测列存在性,
|
||||
/// 缺失才 ALTER(同既有幂等模式),对新库/老库均安全。
|
||||
fn migrate_v23(conn: &Connection) -> Result<()> {
|
||||
if !column_exists(conn, "knowledges", "embedding_status") {
|
||||
conn.execute("ALTER TABLE knowledges ADD COLUMN embedding_status TEXT", [])?;
|
||||
tracing::info!("v23: 补建 knowledges.embedding_status 列(嵌入失败可补偿重试)");
|
||||
}
|
||||
conn.execute("INSERT INTO schema_version (version) VALUES (?)", [23])?;
|
||||
tracing::info!("迁移 v23 完成");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// V24:幂等补 ideas.related_ids 列(灵感间关联关系持久化打底)
|
||||
///
|
||||
/// 为灵感关联关系 UI 打底:related_ids 存「关联灵感 id JSON 数组」字符串
|
||||
/// (同 tags 的 JSON-in-TEXT 模式)。TEXT NULL 向后兼容:老库行默认 NULL,
|
||||
/// IdeaRecord 字段为 Option<String>(未设关联的灵感为 None)。
|
||||
///
|
||||
/// 进通用 update_field 白名单(Ideas.vue 关联关系 UI 走 update_idea →
|
||||
/// update_field('related_ids', ...),同 tags 一样白名单登记该列;另有整行
|
||||
/// update(update_full)路径,二者均可写入)。
|
||||
///
|
||||
/// 用 PRAGMA 探测列存在性,缺失才 ALTER(同 v4/v5/v6/v8/v10/v11/v14/v15/v16/v17
|
||||
/// /v18/v19/v20/v23 模式),对新库/老库均安全。
|
||||
fn migrate_v24(conn: &Connection) -> Result<()> {
|
||||
if !column_exists(conn, "ideas", "related_ids") {
|
||||
conn.execute("ALTER TABLE ideas ADD COLUMN related_ids TEXT", [])?;
|
||||
tracing::info!("v24: 补建 ideas.related_ids 列(灵感关联关系持久化打底)");
|
||||
}
|
||||
conn.execute("INSERT INTO schema_version (version) VALUES (?)", [24])?;
|
||||
tracing::info!("迁移 v24 完成");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// V25:幂等补 idea_evaluations (idea_id, version) 唯一约束(评估版本并发重复兜底)
|
||||
///
|
||||
/// 评估历史 version 此前由 evaluate_idea 算 `list_by_idea().first().version + 1`
|
||||
/// 得到,读-改-写非原子。并发两次评估同一灵感可能读到相同最新 version,各自 +1 后
|
||||
/// 写入相同 version(重复),破坏「version 单调递增 + 唯一」语义。单用户桌面应用
|
||||
/// 概率低,但唯一约束是数据完整性兜底,值得加。
|
||||
///
|
||||
/// 实现选 CREATE UNIQUE INDEX IF NOT EXISTS 而非 ALTER TABLE ADD CONSTRAINT:
|
||||
/// SQLite 不支持 ALTER TABLE 加约束 / 也不支持 ALTER ... IF NOT EXISTS,而
|
||||
/// `CREATE UNIQUE INDEX IF NOT EXISTS` 原生幂等(新库建 / 老库已有则跳过),满足
|
||||
/// 迁移「对新库与老库均安全」要求。索引语义等价于表级 UNIQUE(idea_id, version),
|
||||
/// 同样在 INSERT 冲突时抛 SQLITE_CONSTRAINT_UNIQUE。
|
||||
///
|
||||
/// 注:既有重复数据(若老库已有重复 version 行)会导致建索引失败。单用户桌面应用
|
||||
/// 几乎不会有重复,若真发生此处**降级跳过**(建索引失败 → warn + 继续迁移),而非 `?` 上抛
|
||||
/// 致整个应用启动崩溃、用户无感。理由:唯一索引只是并发重复的兜底防御网,缺失它不影响历史
|
||||
/// 数据读取(list_by_idea / list_recent_idea_evals 照常工作),应用仍可用,远胜启动失败黑屏。
|
||||
/// 建索引失败时日志带原始错误,用户/开发者可据此清理重复后手动重跑迁移补索引。
|
||||
fn migrate_v25(conn: &Connection) -> Result<()> {
|
||||
let build_result = conn.execute_batch(
|
||||
"CREATE UNIQUE INDEX IF NOT EXISTS uq_idea_evaluations_idea_version \
|
||||
ON idea_evaluations(idea_id, version)",
|
||||
);
|
||||
if let Err(e) = build_result {
|
||||
// 降级:建唯一索引失败(典型根因——老库已存在重复 (idea_id, version) 行)不阻断迁移。
|
||||
// 索引缺失仅削弱并发重复防御,不破坏既有数据可读性;跳过继续记录 schema_version=25。
|
||||
tracing::warn!(
|
||||
error = %e,
|
||||
"v25: 建 idea_evaluations(idea_id, version) 唯一索引失败(老库可能有重复 version 行),\
|
||||
降级跳过索引创建不阻断启动。清理重复后可手动重跑迁移补建索引"
|
||||
);
|
||||
} else {
|
||||
tracing::info!("v25: 建 idea_evaluations(idea_id, version) 唯一索引完成");
|
||||
}
|
||||
conn.execute("INSERT INTO schema_version (version) VALUES (?)", [25])?;
|
||||
tracing::info!("迁移 v25 完成");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// V21 建表 SQL — 消息拆分存储 ai_messages 表
|
||||
///
|
||||
/// 与 V9_SQL 中的 ai_messages 镜像(V9 给新库,此 const 给老库 V21 迁移用 IF NOT EXISTS)。
|
||||
@@ -537,6 +657,27 @@ CREATE TABLE IF NOT EXISTS ai_messages (
|
||||
CREATE INDEX IF NOT EXISTS idx_ai_messages_conv ON ai_messages(conversation_id, seq);
|
||||
";
|
||||
|
||||
/// V22 建表 SQL — 灵感评估历史(追加型审计表)
|
||||
///
|
||||
/// 8 列:id(主键)/ idea_id(关联灵感)/ version(评估版本号,单调递增)/
|
||||
/// ai_analysis(AI 分析结果 JSON,可空)/ scores(多维评分 JSON,可空)/
|
||||
/// score(综合评分 REAL,可空)/ evaluated_by(评估者,可空)/ evaluated_at(毫秒字符串)。
|
||||
/// 索引:(idea_id, version DESC) 覆盖「取某灵感最新评估」最高频查询。
|
||||
const V22_SQL: &str = "
|
||||
CREATE TABLE IF NOT EXISTS idea_evaluations (
|
||||
id TEXT PRIMARY KEY,
|
||||
idea_id TEXT NOT NULL,
|
||||
version INTEGER NOT NULL,
|
||||
ai_analysis TEXT,
|
||||
scores TEXT,
|
||||
score REAL,
|
||||
evaluated_by TEXT,
|
||||
evaluated_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_idea_evaluations_idea ON idea_evaluations(idea_id, version DESC);
|
||||
";
|
||||
|
||||
/// V1 建表 SQL
|
||||
const V1_SQL: &str = "
|
||||
-- 想法表
|
||||
@@ -698,6 +839,9 @@ CREATE TABLE IF NOT EXISTS knowledges (
|
||||
verified INTEGER NOT NULL DEFAULT 0,
|
||||
source_project TEXT,
|
||||
source_ref TEXT,
|
||||
-- V23 补列(嵌入失败可补偿重试):新库直接带列,老库由 migrate_v23 ALTER 补;
|
||||
-- 两边列定义须一致(老库迁移注释 V23 已注明)。
|
||||
embedding_status TEXT,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
@@ -21,10 +21,28 @@ pub struct IdeaRecord {
|
||||
pub promoted_to: Option<String>, // 晋升后的 project_id
|
||||
pub ai_analysis: Option<String>, // AI 分析结果 JSON
|
||||
pub scores: Option<String>, // 多维评分 JSON (feasibility/impact/urgency/overall)
|
||||
pub related_ids: Option<String>, // 关联灵感 id JSON 数组(同 tags 模式,为关联关系 UI 打底)
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
||||
/// 灵感评估历史记录(追加型审计表 idea_evaluations,V22)
|
||||
///
|
||||
/// 每次灵感 AI 评估产生一行快照,version 单调递增。
|
||||
/// 替代覆写 ideas.ai_analysis / ideas.scores:保留评估历史可追溯。
|
||||
/// ai_analysis/scores 为 JSON 字符串,score 为综合评分,evaluated_by 标评估发起者。
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct IdeaEvaluationRecord {
|
||||
pub id: String,
|
||||
pub idea_id: String,
|
||||
pub version: i64,
|
||||
pub ai_analysis: Option<String>,
|
||||
pub scores: Option<String>,
|
||||
pub score: Option<f64>,
|
||||
pub evaluated_by: Option<String>,
|
||||
pub evaluated_at: String,
|
||||
}
|
||||
|
||||
// ============================================================
|
||||
// 项目相关模型
|
||||
// ============================================================
|
||||
@@ -280,6 +298,14 @@ pub struct KnowledgeRecord {
|
||||
pub source_project: Option<String>, // 来源项目(仅溯源不过滤)
|
||||
pub source_ref: Option<String>, // 来源实体引用(如 conv:{id})
|
||||
pub reasoning: Option<String>, // AI 提炼判断依据("为何值得沉淀"),手动录入为 None
|
||||
/// 嵌入生成状态(V23):None/pending=未生成,done=成功,failed=失败可重试。
|
||||
///
|
||||
/// P1 修复(嵌入失败无标记):spawn_embedding_for_knowledge 此前 fire-and-forget,
|
||||
/// provider 临时不可用 → 仅 warn,该条永久无向量索引无人感知。
|
||||
/// 现写入 done/failed,failed 可由 knowledge_retry_embedding 触发补偿重试。
|
||||
/// `#[serde(default)]` 兼容老前端/老 JSON(未带该字段反序列化为 None)。
|
||||
#[serde(default)]
|
||||
pub embedding_status: Option<String>,
|
||||
pub created_at: String,
|
||||
pub updated_at: String,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user