RAG索引新鲜度：相同语料相同编辑，为何账单翻倍

A retrieval index is a cache, and every other cache in your stack has an invalidation protocol. HTTP has ETag; your CDN has a purge API. An index has a cron job and a hope, and how wrong it is comes down to two numbers: how often its answers change, and how often you look.

检索索引本质上是一个缓存，而技术栈中任何其他缓存都有一套失效协议。HTTP 有 ETag，CDN 有 purge API。索引只有一个 cron 任务和一份期待，而它会错得多离谱，取决于两个数字：答案多久改变一次，以及你多久查询一次。

// a document scanned every 1/r hours, answers changing Poisson at rate mu
S(x) = 1 - (1 - Math.exp(-x)) / x,   x = mu / r
// NOT "half the refresh period" - that answers a different question

S agrees with a from-scratch numeric integral of its own definition to 2e-11 and with the simulation to 0.0028 across seven budgets. Sum it over a corpus and notice there are two ways to average. Your dashboard averages per document; your users average per query, and queries are Zipf. At the defaults the same index is 98.8% fresh and ships 3.04% stale answers, separated by the query distribution's inverse participation ratio - 12.5 at Zipf 1.0 over 300 documents, one line of SQL you have never run: https://dev48.infy.uk/ai/days/day70-rag-index-freshness.html

S 与从头数值积分自身定义的结果在 2e-11 精度上一致，在七个 budget 下与模拟结果在 0.0028 精度上一致。对语料库求和后会发现有两种平均方式。你的 dashboard 按文档平均，你的用户按查询平均，而查询服从 Zipf 分布。在默认参数下，同一个索引有 98.8% 是新鲜的，但送出了 3.04% 的过期答案，二者之间的差距来自查询分布的逆参与率——在 Zipf 1.0、300 篇文档的场景下是 12.5，这行 SQL 你从未跑过：https://dev48.infy.uk/ai/days/day70-rag-index-freshness.html

The second averaging trap is worth the whole page. Hold the corpus, the edits per hour, the budget and the bill fixed, and change only which documents get edited.

第二种均值陷阱值得用整页篇幅来说明。保持语料库不变、每小时编辑次数不变、budget 不变、账单不变，只改变哪些文档被编辑。

Six times worse, and the number you watch moves the wrong way while it happens. Popular documents are popular because they are about live things, and live things are what people edit. That correlation appears in no vector-store setting and no eval harness.

结果会糟糕六倍，而你所监控的指标还在往错误的方向变动。热门文档之所以热门，是因为它们讲的是热门内容，而热门内容正是人们会去编辑的东西。这种相关性在任何向量库的配置中都不会出现，在任何评测框架里也找不到。

The policy ranking inverts on the same axis. Proportional-to-change-rate is the worst option at zero correlation, 12.7 against 9.1 for a flat cadence - Cho and Garcia-Molina's 2000 crawler result reproduced on a retrieval index - and the best at full correlation, 16.7 against that same cadence's 57.3. A corpus property decides the ordering of your options.

策略排名在同一条件下会反转。在零相关时，按变更率分配是最差选项，12.7 对比均匀频率的 9.1——Cho 和 Garcia-Molina 2000 年爬虫结果在检索索引上复现了；而在完全相关时它变成最好选项，16.7 对比同一均匀频率的 57.3。语料库的一个内在属性决定了你的选项排序。

The recommendation I set out to make lost twice

我原本打算给出的建议输掉了两次

Solve the allocation properly: a per-document Lagrangian over four modes, water-filled until the bill hits the budget. It wins its own objective, 8.0 against the heuristic's 9.9 predicted stale copies, and ships 17.7 stale answers per thousand against a one-line sqrt(w*lambda) rule's 10.5. Sixty-nine per cent worse, with a p99 error age of 199.6 hours against 5.6. The objective is separable per document; a fact with a copy in a second document is not, and 66.2% of what the optimum ships comes from duplicates.

正确地求解分配：对每个文档在四种模式上做拉格朗日优化，水漫过直到账单触及预算。它在自己的目标函数上获胜：预测过期副本数 8.0 对比启发式的 9.9；每千次查询送出 17.7 个过期答案，对比一行 sqrt(w*lambda) 规则的 10.5。差距是 69%，p99 错误年龄 199.6 小时对比 5.6 小时。目标函数对每个文档是可分的，但一份事实如果同时存在于第二个文档中则不然——最优解送出的内容中，66.2% 来自重复。

The obvious repair lost too. Put a floor sweep under every webhook and it predicts the best number on the page, 5.8, and ships 32.7 - worse than the thing it was fixing, because the floor comes out of the same budget and starves the sweeps doing the work. What works is deleting the mode: sweeps only ships 8.3.

显而易见的修复也失败了。给每个 webhook 加上底限扫描，它会报出页面上最好看的数字 5.8，却送出 32.7——比它要修复的东西还差，因为底限来自同一个 budget，饿死了真正在干活的扫描。真正有效的是删除这个模式：纯扫描只送出 8.3。

And a perfect webhook still ships 35.5 stale answers per thousand, because the event fires for the document that changed and the stale copy sits in a document that did not. Break ties on chunk recency in the reranker and the same configuration ships 0.0. It is the only lever here that costs nothing per hour; 94% of the embedding bill is fixed by your writers anyway.

而一个完美的 webhook 仍然每千次查询送出 35.5 个过期答案，因为事件只触发在发生变化的文档上，而过期副本躺在另一个没有变化的文档里。在重排时按 chunk 新近度打破平局，同一套配置送出了 0.0。这是这里唯一不产生额外小时费用的杠杆；94% 的 embedding 账单反正已经由你的编辑们承担了。

Caught a real NaN bug mid-build, now pinned by an assertion. 111 verifier assertions, 33 in the page, five worlds of 3,000 queries, not one asset fetched off the page: https://dev48.infy.uk/aifromzero.php

在构建中途抓到了一个真实的 NaN bug，现在已被断言固定。111 个验证器断言，页面内有 33 个，五组各 3000 条查询的世界，没有一个资源是从页面上获取的：https://dev48.infy.uk/aifromzero.php

For further actions, you may consider blocking this person and/or reporting abuse