AI that worksin productionAnd you can prove it

For most teams, with the 1-week Eval Audit on one AI feature. If you already have evals you no longer trust, the audit starts by checking them. If you already know your top failure modes, go straight to the Implementation Sprint.

Eval Audit: from USD 8,000, 1 week. Implementation Sprint: from USD 24,000, 3 to 4 weeks. Evals Retainer: from USD 6,000/month, capped at 16 hours a month. Each has a fixed scope, written down before we start.

Because the dataset is the product. Someone who knows what a correct answer looks like has to label real cases and sign off on the results. A dataset written without them only encodes a guess, and a gate built on it will be argued with or switched off. The audit is a trust check run with your expert, not error analysis done for you, so your team can continue afterwards.

Not now if nobody can say what a correct answer looks like, if there are no real traces you can share (for example a vendor owns the AI), if the system is being rewritten so fast that a dataset would be out of date within days, if the output is deterministic and a unit test already covers it, or if you want a dashboard or a generic benchmark score. It is also not ML research or fine-tuning.

Past client work is under NDA, so I don't publish case studies, client names or numbers. What I can do is a short walkthrough on a few of your own traces or workflows, so you see the approach on your problem rather than someone else's.

Yes. I sign your NDA before I see any traces or documents. I work inside your environment or on a redacted sample you choose, and nothing leaves the access you give me.

On the gold set built from your own cases, it compares the models you are considering on quality and on cost per case, so a cheaper model is chosen only where the scores show quality holds. It is part of the Eval Audit and the Implementation Sprint, and the Retainer includes one run a month.

Yes, as part of the same offers. An AI automation can keep running while it gets things wrong: a misread document, a ticket sent to the wrong queue, a report with bad numbers. The Implementation Sprint puts a test set and checks that fail loudly around an automation or agent, and the Retainer keeps them true. If you also need the automation or an agent setup built, we scope that on a call and quote it separately.

Yes. If you already use Braintrust, Langfuse, LangSmith or a similar tool, I build on top of it instead of replacing it. If you have nothing yet, I set up a light harness in your repo. It works with any model provider, any agent framework, Python or TypeScript, and the CI you already run.

The Eval Audit takes 1 week from the day I have access. The Implementation Sprint takes 3 to 4 weeks. The Evals Retainer is monthly, with a 3-month minimum and then month to month.

Read access to a sample of real traces or workflow data (redacted is fine), one named domain expert who can give a few hours across the week to label cases and sign off on results, and repo and CI access where needed for the Sprint.

Up to 16 hours a month on keeping your evals true: a gold-set refresh from new traces, judge re-calibration against fresh human labels, regression triage, one model-swap run and a one-page quality card. It is not on-call support or feature development, and it needs an existing harness, so it follows a Sprint.

No. I don't train or fine-tune models. If a fix needs a different model, the evals will show you which one holds up.

Yes, end to end, when an engagement needs one, scoped and quoted separately. It ships with evals from day one.

You keep the ranked failure modes, the gold set and the fix plan whatever you decide next. Your team can fix things on its own, or we move to the 3 to 4 week Implementation Sprint, and later the Evals Retainer.