One LLM API for Anthropic, OpenAI and Google Models
Joe Estephan
An internal LLM service that gives every app in the company one API for Anthropic, OpenAI and Google models, so switching a model is a config change, not a rewrite.
It handles prompt caching, structured output and metrics for every call, and it's sized for traffic that lands in one daily burst. With Langfuse tracing and evals on top, each task goes to the model with the best quality for its cost.
Like this project
Posted Oct 20, 2026
An internal LLM service that gives every app one API for Anthropic, OpenAI and Google models, with prompt caching, structured output and metrics on every call.
