Cutting LLM inference cost and latency without making the feature worse

Cut LLM API cost and latency in TypeScript: prompt caching with a stable prefix, a model per route, streaming for time to first token, batches and cost logs.