cache-identity

cache-identity

A cache is a claim about identity. If the key is wrong, every hit is a bug.

Contract

• Key fields: runtime version, git SHA (dirty marker), model revision, effective parameters, prompt or skill hash, load bucket.
• Length-prefix each field before hashing so concatenation cannot collide.
• Store numbers and references only. Never deserialize executable content from a cache path.
• Enforce a TTL and a row or byte cap; evict oldest first.
• Publish atomically: temp file, flush, fsync, rename, marker. A losing concurrent publish is success.
• A cache read failure returns a miss, never an exception to the caller.
• Success and failure latency are separate fields.

Procedure

1. Compute the key from resolved values, not raw user input.
2. Look up; return a miss when the row is absent, expired, malformed, or version-mismatched.
3. On write, update running means and the timestamp in one statement.
4. Prune on a schedule or at write time; never let the store grow unbounded.
5. On release, record the format version so a shape change cannot mix generations.

Worked example

A key built from resolved values, with a version-mismatch miss.

key = f"summary:v3:{tenant_id}:{doc_id}:{content_hash}"
hit  -> {"key": key, "value": "...", "version": 3}
miss -> {"key": key, "value": null, "reason": "version_mismatch"}

Anti-patterns

• Keying on model name alone while parameters change.
• A cache directory trusted as a code or binary source.
• Unbounded grammar or prompt caches; they are denial-of-service vectors.
• Deleting a file a reader may still have open; expire first, delete after a slack window.

References

• key

Verification Checklist

• [ ] Changing any key field produces a miss. (step 1)
• [ ] Expired rows are invisible. (step 2)
• [ ] Concurrent publish loses safely. (step 3)
• [ ] Corrupt entries are ignored. (step 2)
• [ ] The store stays under its cap. (step 4)
• [ ] Success and failure latencies stay separate. (step 5)

Composes With

• measure-first — benchmark baselines are a separate artifact
• execution-observability — cache identity appears in request metadata
• context-optimization — artifact spill uses the same publish rules