Evaluation and guardrails: measure it, fence it in

One illustrative week at MegaMart Assist (a prompt shipped on a vibe check, a refund no policy allows, an injection hidden in a seller's return policy) and the three layers that answer it: offline evals on a versioned golden set with code graders, calibrated LLM judges and per-slice gates (and what a run costs); the judge's position, verbosity and self-preference biases; retrieval recall versus faithfulness, and why faithful isn't correct; online metrics, canaries and A/B tests by customer; direct and indirect prompt injection, image exfiltration, EchoLeak and the lethal trifecta; defence in depth with a tool gate, taint tracking and confirmation; PII tokenisation; hallucination controls after Moffatt v. Air Canada; and a degraded-mode ladder that fails closed for actions.