Scoped semantic answer cache.
Return a stored answer for a similar enough question, but only inside its tenant, language and prompt version, never for personal answers, and purge it when a source changes.
A semantic cache embeds the question, finds the nearest past question, and returns its stored answer if it's similar enough, skipping the model entirely. Day 5 (section 3.2) lists how that goes wrong: "opened" and "unopened" earphones sit close in embedding space but flip the answer; a personal answer leaks to the next customer; a policy change leaves the cache citing the old rule. This lab builds the cache with the lesson's four rules:
- Scope the key by tenant, language and prompt version, through one key-building function that refuses to run without them (section 9.5's prevention).
- Cache only non-personal answers.
- Invalidate from the sources: every answer built from a changed page is purged.
- Measure: count hits and misses, and set the threshold from data (the tests use the lesson's flip).
Implement:
| Name | Does |
|---|---|
make_scope(tenant, language, prompt_version) |
returns the tuple (tenant, language, prompt_version). Any part missing or empty: ValueError |
SemanticCache(threshold, ttl_s, clock) |
|
store(scope, question, vector, answer, sources, personal=False) |
stores an entry and returns True; a personal answer is not stored (False). A scope that isn't a valid 3-part tuple: ValueError |
lookup(scope, vector) |
the stored answer of the most similar live entry in the same scope if its cosine similarity is >= threshold, else None. Ties: the newest entry |
invalidate_source(source_id) |
removes every entry whose sources include it; returns how many |
hit_rate() |
hits / lookups so far; 0.0 before any lookup |
Rules the tests check:
- An entry expires when
clock.now() - stored_at >= ttl_s; expired entries never hit. - Similarity is
cosine(a, b)(given). Compare with>=: the threshold is the lowest similarity you accept. lookupreturns only the answer string, and every call counts as a hit or a miss.
Write your code, then run 3 example tests and 8 hidden tests. Each run is limited to 8 seconds: an infinite loop gets stopped and Python restarts, your code stays.