好文集ChatGPT Plus¥119

How to get AI Agents from Prototype to Production using Evals

文章目录

In high stakes environments like regulated industries, teams need to build agents evals first. We can learn a lot from those teams. This is a walkthrough.

What makes regulated industries different

AI development teams in regulated industries usually face three requirements for their projects: they need evidence before launch, oversight after launch, and control over data. All of this should be true for any AI agent being built.

文章配图

The approach: start the loop offline

The AI engineering loop describes how teams improve AI applications: trace, monitor, build datasets, run experiments, and evaluate, and then repeat.

文章配图

Many teams enter the loop in production, with tracing and monitoring live traffic. Regulated teams run most of it before going to production. They trace offline runs, build datasets, run experiments, and evaluate until the agent meets the quality bar. Only then do they monitor live traffic. The eight steps below show how.

文章配图

1. Set up tracing

Add tracing before anyone outside the team uses the agent. Traces provide you insight into what happened during a run and provide you with a way of showing evidence of what the agent did. Record the prompt version, model, tool calls, retrieved context, and any human approval on every run.

文章配图

Decide early where the traces will live. Many banks and insurers self-host Langfuse so that trace data never leaves their infrastructure. Where self-hosting is not required, teams use Langfuse Cloud, including a HIPAA-ready region for healthcare.

2. Run example inputs through the agent

Next, run a set of examples through the agent that roughly matches what you expect to hit your system in production. Each run produces a trace you can learn from. At this point, you do not need expected answers yet. Good sources for cases are:

  • Synthetic examples. Generate inputs that cover typical requests and their variations. Domain experts can add cases they know from experience.

  • Historical data. Run the agent on cases your team has already decided, such as resolved support tickets or reviewed claims.

  • Shadow runs. Let the agent process live cases next to the current workflow without acting on anything. This gives you traces from live data without bringing the agent into customer contact.

3. Find failure modes with error analysis

Now have your domain experts read those traces. For each one, they note the first thing that went wrong, then group the notes into failure modes. This process is called error analysis.

Some failures show that the agent needs more specification, such as a missing rule in the prompt, and you can fix them right away. Others keep coming back and need systematic evaluation.

4. Define what to evaluate, including compliance

What you evaluate comes from two places:

  • Failure modes. The recurring failures from error analysis become quality evaluators. Generic metrics such as helpfulness rarely match how your agent fails, and they can skew your view on quality.

  • Compliance requirements. Agree with compliance on what the agent must never do. For a bank, that could be giving investment advice, quoting a rate that is not on the price list, or sending a customer message without approval. Turn them into evaluators.

A code check, such as a regex for a missing disclaimer, is the preferred choice. Use LLM-as-a-judge for anything that needs judgment, such as a response that sounds like investment advice.

The Langfuse Academy covers both steps in detail. Error analysis walks through reading traces and building a failure taxonomy, and choosing what to evaluate shows how to turn those failures and your compliance rules into a small set of metrics.

5. Build a ground truth dataset

With the evaluation criteria in place, build a ground truth dataset together with your domain experts. It should contain your typical inputs, the cases that could go wrong, and cases that should trigger each compliance limit, along with how the agent should handle each one.

Want to go deeper on scoping a dataset? The Langfuse Academy guide on designing datasets covers how to set the goal, choose the input distribution, and decide how each item will be evaluated.

Experts disagree more often than teams expect. To get to a reliable ground truth dataset, let your experts review each other's work. Ideally, bring the experts into one room to talk through those cases, agree on the right handling, and write the missing rule into your guidelines.

The more humans disagree on a case, the harder it is for an agent to handle it. Research has a name for this: inter-rater reliability, the extent to which different raters assign the same score to the same item. It comes from qualitative and clinical research, and applies equally in the context of AI agents (McHugh, 2012).

In Langfuse, annotation queues let several experts label the same items. Score analytics then calculates percent agreement and Cohen's kappa between them, so you can see which cases need a discussion.

6. Iterate until the agent meets the quality bar

Run the dataset as an experiment and record the scores as your baseline. Then review failed cases and improve the agent by changing its context, tool calls, or prompts, rerun the experiment, and compare it with the previous run.

文章配图

Mature setups often keep a held-out test dataset to make sure the system is not being overfitted during improvement efforts.

7. Set up production monitors

Once you are happy with the results, decide which evaluators should also run in production with online evaluation. Compliance evaluators belong on every trace. Add the quality evaluators you want to keep watching live, for example whether answers are phrased the way your guidelines require.

For chat agents, also track explicit feedback and signs of trouble, such as a user asking for a person or correcting the agent.

8. Go live and run the AI engineering loop

Now go live, depending on the risk profile with a partial or complete human in the loop setup. From here on, you are in the regular AI engineering loop: trace production, watch the monitors, turn interesting traces into new dataset items, and rerun the dataset before every release.

Keep humans on a subset, or on everything at first.

At launch, let the agent escalate a little too often to a human. A reply like "I can't help with that, I'll connect you to a person" costs a few minutes of staff time, and a wrong answer in a regulated setting can cost much more. The amount of human review shrinks as the evidence grows. A common pattern, described by Coinbase at AI Engineer NYC, has three steps:

  1. Supervised. Reviewers check every output.

  2. Sampled. Once measured quality passes a threshold set by policy, reviewers check a sample.

  3. Exceptions. Reviewers mainly handle flagged cases and keep improving the datasets.

Match the review rate to the risk. Guidance from the Joint Commission and CHAI recommends checking tools that inform clinical decisions more often than tools that help with administrative tasks. The same logic applies in a bank: a credit decision deserves more review than a branch-hours question.

Automate the slices that work.

Asking a person to approve everything does not scale, and reviewers who approve hundreds of routine cases a day stop looking closely. Classify incoming requests by category, let the agent handle categories that score well in your dataset and in production, such as a password reset, and keep sending the rest to a person.

Conclusion

Regulated teams can bring AI agents to production, so the rest of us should be able to as well. Most of the work happens before launch: trace from the start, learn how the agent fails, agree with compliance on what to evaluate, and test the agent against a ground truth dataset your experts trust. The same traces, datasets, and scores then become the evidence for your review board. After launch, keep people on the cases that need them and hand more categories to the agent as the evidence grows. Start with one workflow that has clear rules and a team that already reviews the work by hand, and expand from there.

Further reading

全文完
查看原文