One Hung API Call Used to Kill My 1,000-Run Benchmark. Here's the Fix.

Retry LLM API calls on 429, 5xx, and timeouts with exponential backoff plus jitter, honor the Retry-After header when the provider sends one, cap...

I built TaskFlow, a project management app with an AI feature called Quick Add. You type "assign the...

This is a submission for the Kaggle Benchmarking Challenge. What I Benchmarked When an...

Most cost tracking keeps the attempt that worked and drops the ones that failed. Here is what that hides, and how to put every attempt on the bill.

A practical comparison of GitHub Copilot, Cursor, and Codeium based on coding workflows, context understanding, IDE integration, pricing, and real-world use cases.

Most classification and routing calls use a chat model, then parse prose and hope the JSON holds. The new /v1/systemone LLM decision API on LLM Gateway returns typed yes/no, choice, and score answers with calibrated probabilities, billed o…

A worked model of LLM cost optimization for a team spending $100k a month: what difficulty-based model routing saves at three traffic mixes, what the classifier costs, and the break-even point. Real catalogue prices, explicit assumptions, …

Automatic model selection now reads the request before it routes it. Choose the models `smart` may resolve to, and a classifier rates each prompt so a one-line fix goes to a cheap model and a hard problem goes to a frontier one. Available …

A live Smart Routing benchmark compares the same prompts across routed and fixed-model requests, including classifier charges, answer checks, and latency. See where routing saved money and where it missed.

Today's blog is mostly about the common ways cache misses have happened in our system, and how we...