How to Use Evals in Microsoft Foundry: Build an Agent and Test It

Today we will look at how to use Evals in Microsoft Foundry. We will build a simple agent, create a small evaluation dataset, run several evaluators, and review the results.

Evals are essentially automated tests for AI systems. In a previous post, we talked about Copilot Studio Evals and built a small agent that utilized them. Microsoft Foundry also lets us evaluate models, datasets, and agents from the portal or SDK. We can use built-in evaluators, custom evaluators, and agent-specific evaluators such as Task Adherence, Tool Selection, Tool Input Accuracy, Tool Output Utilization, and Tool Call Accuracy.

Let’s explore this together.

What Are Evals in Microsoft Foundry?

An eval is a repeatable test of an AI system.

For example, suppose our agent is supposed to answer questions about a fictional employee travel policy.

We might ask:

What is the maximum reimbursable hotel rate?

Our agent answers:

The maximum hotel rate is $250 per night before taxes.

We can then evaluate that answer.

Instead of checking only for an exact string, we might ask:

• Is the answer relevant?
• Is it grounded in the source material?
• Did the agent complete the task?
• Did it follow its instructions?
• Did it use the correct tool?
• Was the response safe?

Microsoft Foundry provides built-in evaluators covering general quality, textual similarity, risk and safety, and agent behavior. It also supports custom evaluators when the built-in metrics do not match our business requirements.

Build a Simple Agent in Microsoft Foundry

For our evaluation demo, we will build a very simple Contoso Travel Policy Agent directly in the Microsoft Foundry portal. We don’t need to write any code for this part.

Open Microsoft Foundry

Open the Microsoft Foundry portal at https://ai.azure.com/ and sign in with your Azure account. If you are already inside another project, select Foundry in the upper-left corner to return to the home page.

I will create a new Project:

From the home page, select:

Build an Agent

Give the agent a name and click Create

Choose the Model and add the Agent Instructions

Now we can select the Model. If you want to change it, you can do that here. Also find the Instructions field and enter:

You are the Contoso Travel Policy Agent. Answer questions about Contoso employee business travel. Be concise and helpful. Only answer using the travel policy information provided to you. If the policy does not contain enough information to answer the question, say that you do not have enough information rather than inventing an answer.

These instructions give us a behavior that will later be useful for evals.

We are telling the agent to do two important things:

1. Answer from the supplied travel policy.
2. Avoid making up information when the policy does not contain the answer.

Add the Travel Policy

For the first version of this demo, we can keep the policy very simple.

Create a text file from this:

Contoso Employee Travel Policy

Flights

Employees may book economy-class flights for domestic travel.

Business class is permitted for international flights longer than eight hours and requires manager approval.

Hotels

The maximum reimbursable hotel rate is $250 per night before taxes.

Meals

Employees may claim up to $75 per day for meals.

Alcohol is not reimbursable.

Ground Transportation

Taxi, rideshare, rental cars, and public transportation may be reimbursed when used for business purposes.

Approval

International travel requires manager approval before booking.

Add the text file as a Tool by clicking Upload Files:

Also remove the Web Search.

Foundry Prompt Agents can be extended with tools and other resources, but for this tutorial we want the agent to stay simple so we can clearly understand what the eval is testing.

Save the agent.

Test the Agent in the Playground

Now use the chat/playground area to test the agent manually.

Ask:

What is the maximum hotel rate?

We should get an answer mentioning:

$250 per night before taxes

Then try:

Can I fly business class internationally?

The answer should explain both conditions:

• the flight must be longer than eight hours,
• and manager approval is required.

Finally, ask something that is not in the policy:

Can I expense Wi-Fi on the plane?

Because our policy doesn’t mention Wi-Fi, the agent should tell us that it does not have enough information to answer.

This last question is especially useful because we can later build an eval that verifies the agent does not hallucinate a policy that doesn’t exist.

Confirm the Agent Is Ready for Evaluation

At this point, we have everything we need:

Agent: Contoso Travel Policy Agent
Model: General-purpose deployed model
Instructions: Travel-policy-specific behavior
Knowledge: Small Contoso travel policy
Manual test: Working

Now we can move into the evaluation section and create a dataset containing questions such as:

What is the hotel limit?

Are meals reimbursable?

Can I fly business class internationally?

Are Wi-Fi charges reimbursable?

Then we can run Foundry evaluators against the agent and see how consistently it behaves.

Run an Evaluation from the Foundry Portal

On the left, click on Evaluations, then Create:

Open the evaluation experience and choose the target we want to test. We will evaluate an agent:

Next, let’s evaluate Individual Turns:

And for this test, one-time evaluations:

Here, we will choose Synthetic Generation and click Generate. Foundry can generate synthetic test queries, send them to the agent, and then score the responses:

Review the options and click Confirm:

Note the quality and safety. Microsoft provides several evaluator categories.

These evaluate characteristics such as:

• coherence,
• fluency,
• relevance,
• groundedness,
• similarity,
• completeness.

For example, Coherence checks whether the response is logically structured and understandable, while similarity evaluators compare a response against expected content.

Microsoft Foundry includes agent evaluators such as:

• Intent Resolution
• Task Adherence
• Task Completion
• Tool Call Success
• Tool Selection
• Tool Input Accuracy
• Tool Output Utilization
• Tool Call Accuracy

These help answer a different question:

Did the agent take the right actions, not just produce a plausible answer?

For example, imagine our travel agent has a tool called:

Check Travel Approval

A user asks:

Has my trip been approved?

The response might sound correct, but we may also want to verify that the agent actually called the correct approval tool.

That is exactly what agent-specific evaluators help us test:Provide an eval name, and submit:

You should now see the eval running, and finally complete:

We see the run completed with an 85% overall score, 94% Tool Selection, and 91% Tool Output Utilization. That means the next step is not to rerun blindly; it is to inspect the failed cases and understand what drove the missing 15%. Microsoft Foundry lets you drill into row-level results for each query, including the agent response, evaluator score, and explanation.

• Tool Selection 94% means the agent generally chose the correct tool, but not always. The agent selected the correct and necessary tools in 94% of the evaluated cases.
• Tool Output Utilization 91% means the agent usually used the tool result correctly, but some responses likely ignored, misread, or underused returned data. Microsoft defines Tool Output Utilization as evaluating whether the agent correctly interprets and uses tool outputs in its reasoning and final response.
• Task Completion means 20 out of 90 Task Completion evaluations passed, which gives a 22% pass rate. 20 = number of Task Completion passes, 90 = total cases evaluated for Task Completion. Task Completion measures whether the agent actually completed the requested task, not simply whether it understood the request or took reasonable steps toward completing it.

Click the run name Contoso-Travel-Policy-Agent. This opens the detailed results for that run. We can then see if there were any failed tests:

Scroll to the right to see more results:

You can also click on the column to sort by Fail, then see each of the failure causes:

We can then modify our instructions, tools, knowledge, or agent behavior and run the same evaluation again.

Final Thoughts

Microsoft Foundry gives us a much deeper way to test AI agents than simply chatting with them manually.

We can evaluate whether the agent understood the request, followed its instructions, completed the task, selected the correct tools, passed the right parameters, and correctly used tool results. Just as importantly, we can run these evaluations repeatedly as the agent changes. That means our eval suite can eventually become something very similar to a regression-test suite in traditional software development.

In future articles, we will look at Rubric Evaluators, custom evaluators, production evaluations, and using Foundry Evals in CI/CD pipelines.

Keep learning. Keep building.

Explore AI, agents & Microsoft technology.

I share practical ideas, tutorials, and videos about AI, AI agents, Microsoft technologies, and the Power Platform.

Subscribe on YouTube →

Carl de Souza Enterprise Architect at Microsoft · AI Technology Expert