From a274a3c526f7a378defc66793ac4eed4b3ee7ece Mon Sep 17 00:00:00 2001 From: westey <164392973+westey-m@users.noreply.github.com> Date: Fri, 15 May 2026 14:17:41 +0000 Subject: [PATCH 1/4] Adding default providers and tools to HarnessAgent --- .../Harness_Step01_Research/Program.cs | 95 +-- .../Program.cs | 26 +- .../Harness_Step03_DataProcessing.csproj | 4 - .../Harness_Step03_DataProcessing/Program.cs | 21 +- .../Harness_Step03_DataProcessing/README.md | 8 +- .../{data => working}/sales.csv | 0 .../HarnessAgent.cs | 142 +++- .../HarnessAgentOptions.cs | 151 +++- .../Harness/AgentMode/AgentModeProvider.cs | 51 +- .../Harness/FileMemory/FileMemoryProvider.cs | 2 +- .../Harness/Todo/TodoProvider.cs | 4 +- .../HarnessAgentOptionsTests.cs | 48 ++ .../HarnessAgentTests.cs | 783 ++++++++++++++++-- 13 files changed, 1147 insertions(+), 188 deletions(-) rename dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/{data => working}/sales.csv (100%) diff --git a/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs index 1c9e93588c3..d0ac5c56cf2 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs @@ -1,8 +1,9 @@ // Copyright (c) Microsoft. All rights reserved. -// This sample demonstrates how to use a HarnessAgent with the Harness AIContextProviders -// (TodoProvider and AgentModeProvider) for interactive research tasks with web search -// capabilities powered by Azure AI Foundry. +// This sample demonstrates how to use a HarnessAgent for interactive research tasks. +// The HarnessAgent comes pre-configured with TodoProvider, AgentModeProvider, FileMemoryProvider, +// ToolApproval, WebSearch, and OpenTelemetry — so this sample only needs custom instructions +// and a WebBrowsingTool. // The agent plans research tasks, creates a todo list, gets user approval, // and then executes each step — all within an interactive conversation loop. // @@ -34,86 +35,32 @@ // and research-focused instructions including the mandatory planning workflow. var instructions = """ + ## Research Assistant Instructions + You are a research assistant. When given a research topic, research it thoroughly using web search and web browsing. Use your knowledge to form good search queries and hypotheses, but always verify claims with the tools available to you rather than relying on memory alone. - ## Mandatory planning workflow - - For every new substantive user request, including short factual questions, your behavior is determined by the mode you are in. - If you are in plan mode, start with the *Plan Mode* steps, and if you are in execute mode, skip directly to the *Execute Mode* steps below. - - *Plan Mode* - - 1. Analyze the request with the purpose of building a research plan. - 2. Create a list of todo items. - 3. If needed, use the provided tools to do some exploratory checks to help build a plan and determine what clarifying questions you may need from the user. - 4. Ask for clarifications from the user where needed. - 1. Ask each clarification one by one. - 2. When asking for clarification and you have specific options in mind, present them to the user, so they can choose the option instead of having to retype the entire response. - 3. Do not proceed until you have received all the needed clarifications. - 4. Do short exploratory research if it helps with being able to ask sensible clarifications from the user. - 5. Write the plan to a memory file, so that it is retained even if compaction happens. Make sure to update the plan file if the user requests changes. - 6. Present the plan to the user and ask for approval to switch to execute mode and process the plan. - 7. When approval is granted, always switch to execute mode (using the `AgentMode_Set` tool), and follow the steps for *Execute mode*. - - *Execute Mode* - - 1. If you don't have a plan or tasks yet, analyse the user request and create tasks and a plan. (**Skip this step if you came from plan mode**) - 2. Work autonomously — use your best judgement to make decisions and keep progressing without asking the user questions. The goal is to have a complete, useful result ready when the user returns. - 3. If you encounter ambiguity or an unexpected situation during execution, choose the most reasonable option, note your choice, and keep going. - 4. Mark tasks as completed as you finish them. - 5. Continue working, thinking and calling tools until you have the research result for the user. - - ## General Instructions - - - You must check the current mode after any user input, since the user may have changed the mode themselves, - e.g. the user may have switched to 'plan' mode after a previous research task finished in 'execute' mode, meaning they want to review a plan first before execution. - - Explain your reasoning and thought process as you work through tasks. - - Explain what you learned and what you are going to do next between tool calls, so the user can follow along with your thought process. - - Avoid making more than 4 tool calls in a row without explaining what you are doing. - - Do not answer the underlying question before the plan has been presented and approved. - - This rule applies even when the answer seems obvious or the task seems small. - - For short requests, use a brief micro-plan rather than skipping planning. The only exceptions are: - - greetings, - - pure acknowledgments, - - clarification questions needed to form the plan, - - follow-up questions about results you have already presented, - - meta-discussion about the workflow itself. - - **Todo management** - - Mark each todo complete as you finish it so the list stays current. - If a todo turns out to be unnecessary or is blocked, remove it and briefly explain why. - Once the user finishes with a topic and moves onto a new one, clean up old completed todos by deleting them. - - **Research quality** + ### Research quality Consult multiple sources when possible and cross-reference key claims. When sources disagree, note the discrepancy and explain which source you consider more reliable and why. If a web page fails to load or a search returns irrelevant results, try alternative search queries or sources before moving on. Track your sources — you will need them when presenting results. - **Presenting results** + ### Presenting results When presenting your final findings: + - Use Markdown formatting for clarity. - Use clear sections with headings for each major topic or sub-question. - Cite your sources inline (e.g., "According to [source name](URL), ..."). - End with a brief summary of key takeaways. - - Save the final research report to file memory so it survives compaction and can be referenced later. - - **File memory** - - Use the FileMemory_* tools to: - - Store downloaded search results or web pages. - - Store plans. - - Read the current plan to make sure tasks were done according to plan. - - Store findings. - - Check for relevant previously downloaded data / findings before starting new research. + - In addition to returning the results to the user, save the final research report to file memory so it survives compaction and can be referenced later. """; // Create the agent using AsHarnessAgent, which pre-configures function invocation, -// per-service-call chat history persistence, and in-loop compaction. -// Then wrap with UseToolApproval to allow auto-approving tools once confirmed. +// per-service-call chat history persistence, in-loop compaction, TodoProvider, AgentModeProvider, +// FileMemoryProvider, ToolApproval, WebSearch, AgentSkillsProvider, and OpenTelemetry. +// Only custom instructions, a WebBrowsingTool, and FileAccess opt-out are needed. AIAgent agent = // Create an OpenAIClient that communicates with the Foundry responses service. new OpenAIClient( @@ -132,30 +79,20 @@ Track your sources — you will need them when presenting results. { Name = "ResearchAgent", Description = "A research assistant that plans and executes research tasks.", - AIContextProviders = - [ - new TodoProvider(), // Add an AIContextProvider to allow the agent to create a TODO list, which is stored in the session. - new AgentModeProvider(), // Add an AIContextProvider that tracks the agent mode and allows switching mode. Current mode is stored in the session. - new FileMemoryProvider( // Add an AIContextProvider that can store memories in files under a session specific working folder. - new FileSystemAgentFileStore(Path.Combine(AppContext.BaseDirectory, "agent-files")), - (_) => new FileMemoryState() { WorkingFolder = DateTime.UtcNow.ToString("yyyyMMdd_HHmmss") + "_" + Guid.NewGuid().ToString() }) - ], + DisableFileAccess = true, + FileMemoryStore = new FileSystemAgentFileStore(Path.Combine(AppContext.BaseDirectory, "agent-files")), ChatOptions = new ChatOptions { Instructions = instructions, Tools = [ - ResponseTool.CreateWebSearchTool().AsAITool(), // Add the foundry hosted web search tool that runs in the service. new WebBrowsingTool( // Add a local web browsing tool that converts html to markdown. new WebBrowsingToolOptions { AllowPublicNetworks = true }), ], MaxOutputTokens = MaxOutputTokens, // Set a high token limit for long research tasks with many tool calls and long outputs. Reasoning = new() { Effort = ReasoningEffort.Medium }, }, - }) - .AsBuilder() - .UseToolApproval() // Add the ability to auto approve tools once a user has said they don't want to be asked again. Approval rules are tied to the session. - .Build(); + }); // Run the interactive console session using the shared HarnessConsole helper. await HarnessConsole.RunAgentAsync( diff --git a/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs index 721da3339ce..489478d8ee6 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs @@ -2,8 +2,9 @@ // This sample demonstrates how to use the SubAgentsProvider to delegate work to sub-agents. // A parent agent is given a list of stock tickers and instructed to find the closing price -// for each ticker on December 31, 2025. It delegates the web searches to a sub-agent -// equipped with Foundry's hosted web search tool. +// for each ticker on December 31, 2025. It delegates the web searches to a sub-agent. +// The HarnessAgent provides built-in WebSearch (HostedWebSearchTool) so no manual web search +// tool configuration is needed on the sub-agent. // // Special commands: // /exit — End the session. @@ -26,7 +27,8 @@ const int MaxOutputTokens = 128_000; // --- Sub-agent: Web Search Agent --- -// This agent can search the web and is used by the parent agent to look up stock prices. +// This agent uses the HarnessAgent's built-in HostedWebSearchTool to search the web. +// Features not needed by this sub-agent are disabled. AIAgent webSearchAgent = new OpenAIClient( new BearerTokenPolicy(new DefaultAzureCredential(), "https://ai.azure.com/.default"), @@ -41,13 +43,14 @@ { Name = "WebSearchAgent", Description = "An agent that can search the web to find information.", + DisableTodoProvider = true, + DisableAgentModeProvider = true, + DisableFileMemory = true, + DisableFileAccess = true, + DisableToolApproval = true, ChatOptions = new ChatOptions { Instructions = "You are a web search assistant. When asked to find information, use the web search tool to look it up and return a concise, factual answer.", - Tools = - [ - ResponseTool.CreateWebSearchTool().AsAITool(), - ], }, }); @@ -75,6 +78,9 @@ 5. Clear all completed tasks to free memory. - Present results in a clean markdown table format. """; +// --- Parent agent: Stock Price Researcher --- +// This agent orchestrates the sub-agent to look up stock prices in parallel. +// Most features are disabled since the parent only needs SubAgentsProvider. AIAgent parentAgent = new OpenAIClient( new BearerTokenPolicy(new DefaultAzureCredential(), "https://ai.azure.com/.default"), @@ -89,6 +95,12 @@ 5. Clear all completed tasks to free memory. { Name = "StockPriceResearcher", Description = "An agent that researches stock prices using sub-agents.", + DisableTodoProvider = true, + DisableAgentModeProvider = true, + DisableFileMemory = true, + DisableFileAccess = true, + DisableToolApproval = true, + DisableWebSearch = true, AIContextProviders = [ new SubAgentsProvider([webSearchAgent]), diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj index 2d2a47d6bef..21922126503 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj +++ b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj @@ -18,8 +18,4 @@ - - - - diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs index b1b5bc5f2d4..220423f0b9b 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs @@ -1,10 +1,12 @@ // Copyright (c) Microsoft. All rights reserved. -// This sample demonstrates how to use a HarnessAgent with the FileAccessProvider +// This sample demonstrates how to use a HarnessAgent with the default FileAccessProvider // to give an agent access to a folder of CSV data files. The agent can read, analyze, // and extract information from the data, then write results back as new files. // -// The sample includes a pre-populated `data/` folder with sales transaction data. +// The sample includes a pre-populated `working/` folder with sales transaction data. +// The HarnessAgent's default FileAccessProvider uses `{cwd}/working` as its working directory, +// which matches this sample's folder layout. // Ask the agent to analyze the data, produce summaries, or create new output files. // // Special commands: @@ -27,10 +29,6 @@ const int MaxContextWindowTokens = 1_050_000; const int MaxOutputTokens = 128_000; -// Point the file store at the data/ folder that ships with the sample. -var dataFolder = Path.Combine(AppContext.BaseDirectory, "data"); -var fileStore = new FileSystemAgentFileStore(dataFolder); - var instructions = """ You are a data analyst assistant. You have access to a folder of data files via the FileAccess_* tools. @@ -56,7 +54,8 @@ You are a data analyst assistant. You have access to a folder of data files via - Always explain what you learned and what you are going to do next between tool calls, so the user can follow along with your thought process. """; -// Create the chat client from the OpenAI provider. +// Create the agent using AsHarnessAgent. The default FileAccessProvider uses {cwd}/working, +// which matches the sample's working/ folder. Unused features are disabled. AIAgent agent = new OpenAIClient( new BearerTokenPolicy(new DefaultAzureCredential(), "https://ai.azure.com/.default"), @@ -71,10 +70,10 @@ You are a data analyst assistant. You have access to a folder of data files via { Name = "DataAnalyst", Description = "A data analyst assistant that reads, analyzes, and processes data files.", - AIContextProviders = - [ - new FileAccessProvider(fileStore), - ], + DisableTodoProvider = true, + DisableAgentModeProvider = true, + DisableFileMemory = true, + DisableWebSearch = true, ChatOptions = new ChatOptions { Instructions = instructions, diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/README.md b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/README.md index eb61ba96548..a9d6cba3848 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/README.md +++ b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/README.md @@ -1,11 +1,11 @@ # What this sample demonstrates -This sample demonstrates how to use a `HarnessAgent` with the `FileAccessProvider` to give an agent access to a folder of data files for reading, analyzing, and writing results. The `HarnessAgent` pre-configures function invocation, per-service-call chat history persistence, and in-loop compaction — so the sample only needs to supply the chat client, token limits, and application-specific options. +This sample demonstrates how to use a `HarnessAgent` with the default `FileAccessProvider` to give an agent access to a folder of data files for reading, analyzing, and writing results. The `HarnessAgent` pre-configures function invocation, per-service-call chat history persistence, in-loop compaction, tool approval, and OpenTelemetry — so the sample only needs to supply the chat client, token limits, custom instructions, and opt out of unused features. Key features showcased: - **HarnessAgent** — a pre-configured agent that wraps a `ChatClientAgent` with function invocation, per-service-call persistence, and context-window compaction -- **FileAccessProvider** — gives the agent tools to read, write, list, search, and delete files in a shared data folder +- **FileAccessProvider** — the HarnessAgent's default file access provider uses `{cwd}/working` as its working directory, matching this sample's `working/` folder - **CSV data processing** — the agent reads sales transaction data and performs analysis on demand - **Output file creation** — the agent can write summaries, filtered data, or reports back to the data folder - **Streaming output** — responses are streamed token-by-token for a natural experience @@ -39,7 +39,7 @@ dotnet run --project samples/02-agents/Harness/Harness_Step03_DataProcessing ## What to Expect -The sample starts an interactive conversation with a data analyst agent. The `data/` folder contains a `sales.csv` file with ~50 rows of sales transaction data (date, product, category, quantity, unit price, region, salesperson). +The sample starts an interactive conversation with a data analyst agent. The `working/` folder contains a `sales.csv` file with ~50 rows of sales transaction data (date, product, category, quantity, unit price, region, salesperson). You can ask the agent to: @@ -53,7 +53,7 @@ E.g. try the following prompt `Please process the sales.csv file by first filter ## Sample Data -The included `data/sales.csv` contains sales transactions from January to March 2025 with the following columns: +The included `working/sales.csv` contains sales transactions from January to March 2025 with the following columns: | Column | Description | | --- | --- | diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/data/sales.csv b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/working/sales.csv similarity index 100% rename from dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/data/sales.csv rename to dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/working/sales.csv diff --git a/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgent.cs b/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgent.cs index c22adca0906..a08511e882e 100644 --- a/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgent.cs +++ b/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgent.cs @@ -1,6 +1,9 @@ // Copyright (c) Microsoft. All rights reserved. +using System; +using System.Collections.Generic; using System.Diagnostics.CodeAnalysis; +using System.IO; using Microsoft.Agents.AI.Compaction; using Microsoft.Extensions.AI; using Microsoft.Shared.DiagnosticIds; @@ -10,7 +13,8 @@ namespace Microsoft.Agents.AI; /// /// A pre-configured that wraps a with -/// function invocation, per-service-call chat history persistence, and in-loop compaction. +/// function invocation, per-service-call chat history persistence, in-loop compaction, and a rich set +/// of default context providers and agent decorators. /// /// /// @@ -23,6 +27,27 @@ namespace Microsoft.Agents.AI; /// /// /// +/// By default, the following context providers are included (each can be disabled via ): +/// +/// — todo list management. +/// — agent mode tracking (plan/execute). +/// — file-based session memory. +/// — shared file access. +/// — skill discovery and loading. +/// +/// +/// +/// The agent is also wrapped with the following decorators by default (each can be disabled): +/// +/// — "don't ask again" tool approval rules. +/// — OpenTelemetry instrumentation. +/// +/// +/// +/// A is added to the chat options by default (can be disabled via +/// ). +/// +/// /// The underlying is configured with /// and /// set to @@ -48,7 +73,9 @@ You are a helpful AI assistant that uses tools to complete tasks. - Think through the task before acting. Break complex work into clear steps. - Use the tools available to you to gather information, perform actions, and verify results. - - Explain your reasoning between tool calls so the user can follow your progress. + - Explain your reasoning and thought process as you work through tasks. + - Explain what you learned and what you are going to do next between tool calls, so the user can follow along with your thought process. + - Avoid making more than 4 tool calls in a row without explaining what you are doing. - If a tool call fails or returns unexpected results, adapt your approach rather than repeating the same call. - When you have completed the task, present a clear and concise summary of what you did and what you found. """; @@ -74,15 +101,15 @@ You are a helpful AI assistant that uses tools to complete tasks. /// additional context providers, and chat history provider. /// When , the agent uses built-in default settings. /// - /// + /// /// is . /// - /// + /// /// is not positive, or /// is negative or greater than or equal to . /// public HarnessAgent(IChatClient chatClient, int maxContextWindowTokens, int maxOutputTokens, HarnessAgentOptions? options = null) - : base(BuildInnerAgent( + : base(BuildAgent( Throw.IfNull(chatClient), maxContextWindowTokens, maxOutputTokens, @@ -90,6 +117,25 @@ public HarnessAgent(IChatClient chatClient, int maxContextWindowTokens, int maxO { } + private static AIAgent BuildAgent(IChatClient chatClient, int maxContextWindowTokens, int maxOutputTokens, HarnessAgentOptions? options) + { + ChatClientAgent innerAgent = BuildInnerAgent(chatClient, maxContextWindowTokens, maxOutputTokens, options); + + AIAgentBuilder builder = innerAgent.AsBuilder(); + + if (options?.DisableToolApproval is not true) + { + builder.UseToolApproval(); + } + + if (options?.DisableOpenTelemetry is not true) + { + builder.UseOpenTelemetry(); + } + + return builder.Build(); + } + private static ChatClientAgent BuildInnerAgent(IChatClient chatClient, int maxContextWindowTokens, int maxOutputTokens, HarnessAgentOptions? options) { var compactionStrategy = new ContextWindowCompactionStrategy( @@ -102,15 +148,28 @@ private static ChatClientAgent BuildInnerAgent(IChatClient chatClient, int maxCo ChatReducer = compactionStrategy.AsChatReducer(), }); - string instructions = options?.ChatOptions?.Instructions ?? DefaultInstructions; + string harnessInstructions = options?.HarnessInstructions ?? DefaultInstructions; + string? agentInstructions = options?.ChatOptions?.Instructions; + + string instructions = (string.IsNullOrWhiteSpace(harnessInstructions), string.IsNullOrWhiteSpace(agentInstructions)) switch + { + (true, true) => harnessInstructions, + (true, false) => agentInstructions!, + (false, true) => harnessInstructions, + (false, false) => $"{harnessInstructions}\n\n{agentInstructions}", + }; - ChatOptions chatOptions = BuildChatOptions(options?.ChatOptions, instructions, maxOutputTokens); + ChatOptions chatOptions = BuildChatOptions(options, instructions, maxOutputTokens); var compactionProvider = new CompactionProvider(compactionStrategy); + IEnumerable contextProviders = BuildContextProviders(options); + return chatClient .AsBuilder() - .UseFunctionInvocation() + .UseFunctionInvocation(configure: options?.MaximumIterationsPerRequest is int maxIterations + ? ficc => ficc.MaximumIterationsPerRequest = maxIterations + : null) .UseMessageInjection() .UsePerServiceCallChatHistoryPersistence() .UseAIContextProviders(compactionProvider) @@ -121,17 +180,78 @@ private static ChatClientAgent BuildInnerAgent(IChatClient chatClient, int maxCo Description = options?.Description, ChatOptions = chatOptions, ChatHistoryProvider = chatHistoryProvider, - AIContextProviders = options?.AIContextProviders, + AIContextProviders = contextProviders, UseProvidedChatClientAsIs = true, RequirePerServiceCallChatHistoryPersistence = true, }); } - private static ChatOptions BuildChatOptions(ChatOptions? source, string instructions, int maxOutputTokens) + private static ChatOptions BuildChatOptions(HarnessAgentOptions? options, string instructions, int maxOutputTokens) { - ChatOptions result = source?.Clone() ?? new ChatOptions(); + ChatOptions result = options?.ChatOptions?.Clone() ?? new ChatOptions(); result.Instructions = instructions; result.MaxOutputTokens ??= maxOutputTokens; + + if (options?.DisableWebSearch is not true) + { + result.Tools ??= []; + result.Tools.Add(new HostedWebSearchTool()); + } + return result; } + + private static List BuildContextProviders(HarnessAgentOptions? options) + { + var providers = new List(); + + if (options?.DisableTodoProvider is not true) + { + providers.Add(new TodoProvider()); + } + + if (options?.DisableAgentModeProvider is not true) + { + providers.Add(new AgentModeProvider(options?.AgentModeProviderOptions)); + } + + if (options?.DisableFileMemory is not true) + { + AgentFileStore fileMemoryStore = options?.FileMemoryStore + ?? new FileSystemAgentFileStore( + Path.Combine(Directory.GetCurrentDirectory(), "agent-file-memory")); + + providers.Add(new FileMemoryProvider( + fileMemoryStore, + _ => new FileMemoryState + { + WorkingFolder = DateTime.UtcNow.ToString("yyyyMMdd_HHmmss") + "_" + Guid.NewGuid().ToString(), + })); + } + + if (options?.DisableFileAccess is not true) + { + AgentFileStore fileAccessStore = options?.FileAccessStore + ?? new FileSystemAgentFileStore( + Path.Combine(Directory.GetCurrentDirectory(), "working")); + + providers.Add(new FileAccessProvider(fileAccessStore)); + } + + if (options?.DisableAgentSkillsProvider is not true) + { + AgentSkillsProvider skillsProvider = options?.AgentSkillsSource is AgentSkillsSource source + ? new AgentSkillsProvider(source) + : new AgentSkillsProvider(Directory.GetCurrentDirectory()); + + providers.Add(skillsProvider); + } + + if (options?.AIContextProviders is IEnumerable userProviders) + { + providers.AddRange(userProviders); + } + + return providers; + } } diff --git a/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgentOptions.cs b/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgentOptions.cs index 38856484c32..117f7f380e1 100644 --- a/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgentOptions.cs +++ b/dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgentOptions.cs @@ -36,13 +36,31 @@ public sealed class HarnessAgentOptions /// Use to supply additional tools the agent can invoke. /// /// - /// Use to override the 's built-in - /// default instructions. When is or not set, - /// the default instructions are used. + /// Use to provide agent-specific instructions (e.g., research methodology, + /// data analysis workflow). These are combined with to form the final instructions + /// sent to the model: harness instructions appear first, followed by agent-specific instructions. + /// When is , only + /// (or the default) is used. /// /// public ChatOptions? ChatOptions { get; set; } + /// + /// Gets or sets the harness-level instructions that control general tool usage and behavior patterns. + /// + /// + /// + /// Harness instructions provide guidance on how to use tools, explain reasoning, and structure work. + /// They are combined with . (agent-specific instructions) + /// to produce the final instructions sent to the model: harness instructions first, then agent-specific instructions. + /// + /// + /// When (the default), is used. + /// Set to to omit harness instructions entirely. + /// + /// + public string? HarnessInstructions { get; set; } + /// /// Gets or sets the to use for storing chat history. /// @@ -61,4 +79,131 @@ public sealed class HarnessAgentOptions /// . /// public IEnumerable? AIContextProviders { get; set; } + + /// + /// Gets or sets the maximum number of function-invocation loop iterations per request. + /// + /// + /// When set, this value is passed to . + /// When , the default is used. + /// + public int? MaximumIterationsPerRequest { get; set; } + + /// + /// Gets or sets a value indicating whether the wrapper is disabled. + /// + /// + /// When (the default), the agent is wrapped with tool approval middleware + /// that supports "don't ask again" auto-approval rules. + /// + public bool DisableToolApproval { get; set; } + + /// + /// Gets or sets a value indicating whether the is disabled. + /// + /// + /// When (the default), a is included in the + /// agent's context providers, using either or a default + /// rooted at {cwd}/agent-file-memory/{timestamp}_{guid}. + /// + public bool DisableFileMemory { get; set; } + + /// + /// Gets or sets a custom for the . + /// + /// + /// When and is , + /// a default is created. + /// This property is ignored when is . + /// + public AgentFileStore? FileMemoryStore { get; set; } + + /// + /// Gets or sets a value indicating whether the is disabled. + /// + /// + /// When (the default), a is included in the + /// agent's context providers, using either or a default + /// rooted at {cwd}/working. + /// + public bool DisableFileAccess { get; set; } + + /// + /// Gets or sets a custom for the . + /// + /// + /// When and is , + /// a default is created. + /// This property is ignored when is . + /// + public AgentFileStore? FileAccessStore { get; set; } + + /// + /// Gets or sets a value indicating whether the is disabled. + /// + /// + /// When (the default), a is added + /// to .. + /// + public bool DisableWebSearch { get; set; } + + /// + /// Gets or sets a value indicating whether the is disabled. + /// + /// + /// When (the default), a is included + /// in the agent's context providers for tracking work items. + /// + public bool DisableTodoProvider { get; set; } + + /// + /// Gets or sets a value indicating whether the is disabled. + /// + /// + /// When (the default), an is included + /// in the agent's context providers. Use to configure + /// custom modes. + /// + public bool DisableAgentModeProvider { get; set; } + + /// + /// Gets or sets custom options for the . + /// + /// + /// When , the uses its built-in default + /// modes ("plan" and "execute"). This property is ignored when + /// is . + /// + public AgentModeProviderOptions? AgentModeProviderOptions { get; set; } + + /// + /// Gets or sets a value indicating whether the is disabled. + /// + /// + /// When (the default), an is included + /// in the agent's context providers. Use to provide a custom + /// skills source; otherwise, the provider defaults to file-based skill discovery from the current + /// working directory. + /// + public bool DisableAgentSkillsProvider { get; set; } + + /// + /// Gets or sets a custom for the . + /// + /// + /// When and is , + /// the provider defaults to file-based skill discovery from the current working directory. + /// This property is ignored when is . + /// + public AgentSkillsSource? AgentSkillsSource { get; set; } + + /// + /// Gets or sets a value indicating whether the wrapper is disabled. + /// + /// + /// When (the default), the agent is wrapped with an + /// that provides OpenTelemetry instrumentation + /// following the Semantic Conventions for Generative AI systems. + /// + public bool DisableOpenTelemetry { get; set; } } diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs b/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs index 714f52c577e..bfeca78e549 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs @@ -45,20 +45,54 @@ public sealed class AgentModeProvider : AIContextProvider """ ## Agent Mode - You can operate in different modes. Depending on the mode you are in, you will be required to follow different processes. + - You can operate in different modes. Depending on the mode you are in, you will be required to follow different processes. + - You must check the current mode after any user input, since the user may have changed the mode themselves, + e.g. the user may have switched to 'plan' mode after a previous research task finished in 'execute' mode, meaning they want to review a plan first before execution. Use the AgentMode_Get tool to check your current operating mode. Use the AgentMode_Set tool to switch between modes as your work progresses. Only use AgentMode_Set if the user explicitly instructs/allows you to change modes. - {available_modes} - You are currently operating in the {current_mode} mode. + + ### Mandatory Mode based Workflow + + For every new substantive user request, including short factual questions, your behavior is determined by the mode you are in. + + {available_modes} """; private static readonly IReadOnlyList s_defaultModes = [ - new("plan", "Use this mode when analyzing requirements, breaking down tasks, and creating plans. This is the interactive mode — ask clarifying questions, discuss options, and get user approval before proceeding."), - new("execute", "Use this mode when carrying out approved plans. Work autonomously using your best judgement — do not ask the user questions or wait for feedback. Make reasonable decisions on your own so that there is a complete, useful result when the user returns. If you encounter ambiguity, choose the most reasonable option and note your choice."), + new( + "plan", + """ + Use this mode when analyzing requirements, breaking down tasks, and creating plans. This is the interactive mode — ask clarifying questions, discuss options, and get user approval before proceeding. + + Process to follow when in plan mode: + 1. Analyze the request with the purpose of building a research plan. + 2. Create a list of todo items. + 3. If needed, use the provided tools to do some exploratory checks to help build a plan and determine what clarifying questions you may need from the user. + 4. Ask for clarifications from the user where needed. + 1. Ask each clarification one by one. + 2. When asking for clarification and you have specific options in mind, present them to the user, so they can choose the option instead of having to retype the entire response. + 3. Do not proceed until you have received all the needed clarifications. + 4. Do short exploratory research if it helps with being able to ask sensible clarifications from the user. + 5. Write the plan to a memory file, so that it is retained even if compaction happens. Make sure to update the plan file if the user requests changes. + 6. Present the plan to the user and ask for approval to switch to execute mode and process the plan. + 7. When approval is granted, always switch to execute mode (using the `AgentMode_Set` tool), and follow the steps for *Execute mode*. + """), + new( + "execute", + """ + Use this mode when carrying out approved plans. Work autonomously using your best judgement — do not ask the user questions or wait for feedback. + + Process to follow when in execute mode: + 1. If you don't have a plan or tasks yet, analyse the user request and create tasks and a plan. (**Skip this step if you came from plan mode**) + 2. Work autonomously — use your best judgement to make decisions and keep progressing without asking the user questions. The goal is to have a complete, useful result ready when the user returns. + 3. If you encounter ambiguity or an unexpected situation during execution, choose the most reasonable option, note your choice, and keep going. + 4. Mark tasks as completed as you finish them. + 5. Continue working, thinking and calling tools until you have the research result for the user. + """), ]; private readonly ProviderSessionState _sessionState; @@ -187,12 +221,15 @@ protected override ValueTask ProvideAIContextAsync(InvokingContext co private string BuildInstructions(string currentMode) { - // Build list of modes text: var modesListBuilder = new StringBuilder(); foreach (var mode in this._modes) { - modesListBuilder.AppendLine($"- \"{mode.Name}\": {mode.Description}"); + modesListBuilder.AppendLine($"#### {mode.Name}"); + modesListBuilder.AppendLine(); + modesListBuilder.AppendLine(mode.Description.TrimEnd()); + modesListBuilder.AppendLine(); } + var modesListText = modesListBuilder.ToString(); return new StringBuilder(this._instructions) diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/FileMemory/FileMemoryProvider.cs b/dotnet/src/Microsoft.Agents.AI/Harness/FileMemory/FileMemoryProvider.cs index 7052cd0b421..8394cf5ef87 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/FileMemory/FileMemoryProvider.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/FileMemory/FileMemoryProvider.cs @@ -55,7 +55,7 @@ public sealed class FileMemoryProvider : AIContextProvider, IDisposable - Use descriptive file names (e.g., "projectarchitecture.md", "userpreferences.md"). - Include a description when saving a file to help with future discovery. - - Before starting new tasks, use FileMemory_ListFiles and FileMemory_SearchFiles to check for relevant existing memories. + - Before starting new tasks, use FileMemory_ListFiles and FileMemory_SearchFiles to check for relevant existing memories to avoid duplicate work. - Keep memories up-to-date by overwriting files when information changes. - When you receive large amounts of data (e.g., downloaded web pages, API responses, research results), save them to files if they will be required later, so that they are not lost when older context is compacted or truncated. diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/Todo/TodoProvider.cs b/dotnet/src/Microsoft.Agents.AI/Harness/Todo/TodoProvider.cs index f5f222e28ba..bb39e71c201 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/Todo/TodoProvider.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/Todo/TodoProvider.cs @@ -48,9 +48,9 @@ public sealed class TodoProvider : AIContextProvider, IDisposable You have access to a todo list for tracking work items. While planning, make sure that you break down complex tasks into manageable todo items and add them to the list. Ask questions from the user where clarification is needed to create effective todos. - If the user provides feedback on your plan, adjust your todos accordingly by adding new items or removing irrelevant ones. + If the user provides feedback on your plan, adjust your todos accordingly by adding new items or removing irrelevant/old ones. During execution, use the todo list to keep track of what needs to be done, mark items as complete when finished, and remove any items that are no longer needed. - When a user changes the topic or changes their mind, ensure that you update the todo list accordingly by removing irrelevant items or adding new ones as needed. + When a user changes the topic or changes their mind, ensure that you update the todo list accordingly by removing irrelevant/old items or adding new ones as needed. Use these tools to manage your tasks: - Use TodoList_Add to break down complex work into trackable items (supports adding one or many at once). diff --git a/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentOptionsTests.cs b/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentOptionsTests.cs index f07a08046c3..8e74853f718 100644 --- a/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentOptionsTests.cs +++ b/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentOptionsTests.cs @@ -1,5 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. +using Moq; + namespace Microsoft.Agents.AI.UnitTests; public class HarnessAgentOptionsTests @@ -18,8 +20,22 @@ public void DefaultPropertyValues() Assert.Null(options.Name); Assert.Null(options.Description); Assert.Null(options.ChatOptions); + Assert.Null(options.HarnessInstructions); Assert.Null(options.ChatHistoryProvider); Assert.Null(options.AIContextProviders); + Assert.False(options.DisableToolApproval); + Assert.False(options.DisableFileMemory); + Assert.False(options.DisableFileAccess); + Assert.False(options.DisableWebSearch); + Assert.False(options.DisableTodoProvider); + Assert.False(options.DisableAgentModeProvider); + Assert.False(options.DisableAgentSkillsProvider); + Assert.False(options.DisableOpenTelemetry); + Assert.Null(options.MaximumIterationsPerRequest); + Assert.Null(options.FileMemoryStore); + Assert.Null(options.FileAccessStore); + Assert.Null(options.AgentModeProviderOptions); + Assert.Null(options.AgentSkillsSource); } /// @@ -31,6 +47,10 @@ public void PropertiesCanBeSetAndRetrieved() // Arrange var chatHistoryProvider = new InMemoryChatHistoryProvider(); var contextProviders = new AIContextProvider[] { new TodoProvider() }; + var fileMemoryStore = new Mock().Object; + var fileAccessStore = new Mock().Object; + var agentModeOptions = new AgentModeProviderOptions(); + var skillsSource = new Mock().Object; // Act var options = new HarnessAgentOptions @@ -39,8 +59,22 @@ public void PropertiesCanBeSetAndRetrieved() Name = "test-name", Description = "test-description", ChatOptions = new() { Temperature = 0.5f, Instructions = "custom instructions" }, + HarnessInstructions = "custom harness instructions", ChatHistoryProvider = chatHistoryProvider, AIContextProviders = contextProviders, + MaximumIterationsPerRequest = 42, + DisableToolApproval = true, + DisableFileMemory = true, + FileMemoryStore = fileMemoryStore, + DisableFileAccess = true, + FileAccessStore = fileAccessStore, + DisableWebSearch = true, + DisableTodoProvider = true, + DisableAgentModeProvider = true, + AgentModeProviderOptions = agentModeOptions, + DisableAgentSkillsProvider = true, + AgentSkillsSource = skillsSource, + DisableOpenTelemetry = true, }; // Assert @@ -50,7 +84,21 @@ public void PropertiesCanBeSetAndRetrieved() Assert.NotNull(options.ChatOptions); Assert.Equal(0.5f, options.ChatOptions!.Temperature); Assert.Equal("custom instructions", options.ChatOptions.Instructions); + Assert.Equal("custom harness instructions", options.HarnessInstructions); Assert.Same(chatHistoryProvider, options.ChatHistoryProvider); Assert.Same(contextProviders, options.AIContextProviders); + Assert.Equal(42, options.MaximumIterationsPerRequest); + Assert.True(options.DisableToolApproval); + Assert.True(options.DisableFileMemory); + Assert.Same(fileMemoryStore, options.FileMemoryStore); + Assert.True(options.DisableFileAccess); + Assert.Same(fileAccessStore, options.FileAccessStore); + Assert.True(options.DisableWebSearch); + Assert.True(options.DisableTodoProvider); + Assert.True(options.DisableAgentModeProvider); + Assert.Same(agentModeOptions, options.AgentModeProviderOptions); + Assert.True(options.DisableAgentSkillsProvider); + Assert.Same(skillsSource, options.AgentSkillsSource); + Assert.True(options.DisableOpenTelemetry); } } diff --git a/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentTests.cs b/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentTests.cs index 29464431c3c..0d963a1061a 100644 --- a/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentTests.cs +++ b/dotnet/tests/Microsoft.Agents.AI.Harness.UnitTests/HarnessAgentTests.cs @@ -15,6 +15,21 @@ public class HarnessAgentTests private const int TestMaxContextWindowTokens = 100_000; private const int TestMaxOutputTokens = 10_000; + /// + /// Creates a HarnessAgent with all default features disabled to isolate tests for specific behaviors. + /// + private static HarnessAgentOptions CreateAllDisabledOptions() => new() + { + DisableToolApproval = true, + DisableOpenTelemetry = true, + DisableFileMemory = true, + DisableFileAccess = true, + DisableWebSearch = true, + DisableTodoProvider = true, + DisableAgentModeProvider = true, + DisableAgentSkillsProvider = true, + }; + #region Constructor Validation /// @@ -81,13 +96,12 @@ public void NameAndDescription_ArePassedThrough() { // Arrange var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.Name = "TestAgent"; + options.Description = "A test agent"; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - Name = "TestAgent", - Description = "A test agent", - }); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); // Assert Assert.Equal("TestAgent", agent.Name); @@ -102,12 +116,11 @@ public void Id_IsPassedThrough() { // Arrange var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.Id = "my-agent-id"; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - Id = "my-agent-id", - }); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); // Assert Assert.Equal("my-agent-id", agent.Id); @@ -127,7 +140,7 @@ public void Instructions_DefaultsToBuiltInInstructions() var chatClient = new Mock().Object; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); var innerAgent = agent.GetService(); // Assert @@ -136,19 +149,18 @@ public void Instructions_DefaultsToBuiltInInstructions() } /// - /// Verify that default instructions are used when options is provided but ChatOptions.Instructions is null. + /// Verify that default instructions are used when options is provided but neither HarnessInstructions nor ChatOptions.Instructions is set. /// [Fact] - public void Instructions_DefaultsWhenChatOptionsInstructionsIsNull() + public void Instructions_DefaultsWhenBothNull() { // Arrange var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.ChatOptions = new ChatOptions { Temperature = 0.5f }; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - ChatOptions = new ChatOptions { Temperature = 0.5f }, - }); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); var innerAgent = agent.GetService(); // Assert @@ -157,24 +169,106 @@ public void Instructions_DefaultsWhenChatOptionsInstructionsIsNull() } /// - /// Verify that ChatOptions.Instructions overrides the defaults. + /// Verify that ChatOptions.Instructions is appended to the default HarnessInstructions. /// [Fact] - public void Instructions_CanBeOverriddenViaChatOptions() + public void Instructions_CombinesDefaultHarnessWithAgentInstructions() { // Arrange var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.ChatOptions = new ChatOptions { Instructions = "You are a custom assistant." }; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - ChatOptions = new ChatOptions { Instructions = "You are a custom assistant." }, - }); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + var expected = $"{HarnessAgent.DefaultInstructions}\n\nYou are a custom assistant."; + Assert.Equal(expected, innerAgent!.Instructions); + } + + /// + /// Verify that custom HarnessInstructions replaces the default. + /// + [Fact] + public void Instructions_CustomHarnessInstructionsReplacesDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.HarnessInstructions = "Custom harness rules."; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); var innerAgent = agent.GetService(); // Assert Assert.NotNull(innerAgent); - Assert.Equal("You are a custom assistant.", innerAgent!.Instructions); + Assert.Equal("Custom harness rules.", innerAgent!.Instructions); + } + + /// + /// Verify that custom HarnessInstructions and ChatOptions.Instructions are combined. + /// + [Fact] + public void Instructions_CombinesCustomHarnessWithAgentInstructions() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.HarnessInstructions = "Custom harness rules."; + options.ChatOptions = new ChatOptions { Instructions = "You are a research agent." }; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + Assert.Equal("Custom harness rules.\n\nYou are a research agent.", innerAgent!.Instructions); + } + + /// + /// Verify that empty HarnessInstructions omits harness portion, using only agent instructions. + /// + [Fact] + public void Instructions_EmptyHarnessInstructionsUsesOnlyAgentInstructions() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.HarnessInstructions = string.Empty; + options.ChatOptions = new ChatOptions { Instructions = "Agent only instructions." }; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + Assert.Equal("Agent only instructions.", innerAgent!.Instructions); + } + + /// + /// Verify that empty HarnessInstructions with no agent instructions results in empty string. + /// + [Fact] + public void Instructions_EmptyHarnessInstructionsWithNoAgentInstructions() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.HarnessInstructions = string.Empty; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + Assert.Equal(string.Empty, innerAgent!.Instructions); } #endregion @@ -191,7 +285,7 @@ public void ChatHistoryProvider_DefaultsToInMemory() var chatClient = new Mock().Object; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); var innerAgent = agent.GetService(); // Assert @@ -208,12 +302,11 @@ public void ChatHistoryProvider_UsesCustomProviderWhenSpecified() // Arrange var chatClient = new Mock().Object; var customProvider = new InMemoryChatHistoryProvider(); + var options = CreateAllDisabledOptions(); + options.ChatHistoryProvider = customProvider; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - ChatHistoryProvider = customProvider, - }); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); var innerAgent = agent.GetService(); // Assert @@ -235,7 +328,7 @@ public void Pipeline_IncludesFunctionInvokingChatClient() var chatClient = new Mock().Object; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); var innerAgent = agent.GetService(); // Assert @@ -256,7 +349,7 @@ public void Pipeline_HasDecoratedChatClient() var rawClient = mockClient.Object; // Act - var agent = new HarnessAgent(rawClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(rawClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); var innerAgent = agent.GetService(); // Assert — the pipeline wraps the raw client, so the outer client is not the same object. @@ -269,45 +362,45 @@ public void Pipeline_HasDecoratedChatClient() #region AIContextProviders /// - /// Verify that additional AIContextProviders from options are passed to the inner ChatClientAgent, - /// not merged into the chat client builder pipeline. + /// Verify that additional AIContextProviders from options are passed to the inner ChatClientAgent. /// [Fact] public void AIContextProviders_ArePassedToInnerAgent() { // Arrange var chatClient = new Mock().Object; - var todoProvider = new TodoProvider(); + var customProvider = new TodoProvider(); + var options = CreateAllDisabledOptions(); + options.AIContextProviders = [customProvider]; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - AIContextProviders = [todoProvider], - }); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); var innerAgent = agent.GetService(); - // Assert — the TodoProvider should appear in the inner agent's AIContextProviders. + // Assert — the custom provider should appear in the inner agent's AIContextProviders. Assert.NotNull(innerAgent); Assert.NotNull(innerAgent!.AIContextProviders); - Assert.Contains(todoProvider, innerAgent.AIContextProviders!); + Assert.Contains(customProvider, innerAgent.AIContextProviders!); } /// - /// Verify that when no AIContextProviders are specified, the inner agent has no additional providers. + /// Verify that when all default providers are disabled and no user AIContextProviders are specified, + /// the inner agent has an empty providers list. /// [Fact] - public void AIContextProviders_IsNullWhenNoneSpecified() + public void AIContextProviders_IsEmptyWhenAllDisabledAndNoneSpecified() { // Arrange var chatClient = new Mock().Object; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); var innerAgent = agent.GetService(); // Assert Assert.NotNull(innerAgent); - Assert.Null(innerAgent!.AIContextProviders); + Assert.NotNull(innerAgent!.AIContextProviders); + Assert.Empty(innerAgent.AIContextProviders!); } #endregion @@ -332,13 +425,10 @@ public async Task ChatOptions_ToolsArePreservedAsync() .Callback, ChatOptions?, CancellationToken>((_, opts, _) => capturedOptions = opts) .ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, "Done"))); - var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - ChatOptions = new ChatOptions - { - Tools = [tool], - }, - }); + var options = CreateAllDisabledOptions(); + options.ChatOptions = new ChatOptions { Tools = [tool] }; + + var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens, options); var session = await agent.CreateSessionAsync(); // Act @@ -389,7 +479,7 @@ public void GetService_ReturnsSelfForHarnessAgentType() var chatClient = new Mock().Object; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); // Assert Assert.Same(agent, agent.GetService()); @@ -405,7 +495,7 @@ public void GetService_ReturnsInnerChatClientAgent() var chatClient = new Mock().Object; // Act - var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); // Assert Assert.NotNull(agent.GetService()); @@ -430,7 +520,7 @@ public async Task RunAsync_DelegatesToInnerAgentAsync() It.IsAny())) .ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, "Hello!"))); - var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens); + var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); var session = await agent.CreateSessionAsync(); // Act @@ -487,19 +577,19 @@ public void AsHarnessAgent_PassesOptionsThrough() { // Arrange var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.Name = "ExtensionAgent"; + options.ChatOptions = new ChatOptions { Instructions = "Custom instructions" }; // Act - var agent = chatClient.AsHarnessAgent(TestMaxContextWindowTokens, TestMaxOutputTokens, new HarnessAgentOptions - { - Name = "ExtensionAgent", - ChatOptions = new ChatOptions { Instructions = "Custom instructions" }, - }); + var agent = chatClient.AsHarnessAgent(TestMaxContextWindowTokens, TestMaxOutputTokens, options); var innerAgent = agent.GetService(); // Assert Assert.Equal("ExtensionAgent", agent.Name); Assert.NotNull(innerAgent); - Assert.Equal("Custom instructions", innerAgent!.Instructions); + var expected = $"{HarnessAgent.DefaultInstructions}\n\nCustom instructions"; + Assert.Equal(expected, innerAgent!.Instructions); } /// @@ -513,4 +603,579 @@ public void AsHarnessAgent_ThrowsWhenChatClientIsNull() } #endregion + + #region Feature: ToolApproval + + /// + /// Verify that ToolApprovalAgent is included in the pipeline by default. + /// + [Fact] + public void ToolApproval_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableToolApproval = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + + // Assert + Assert.NotNull(agent.GetService()); + } + + /// + /// Verify that ToolApprovalAgent is excluded when disabled. + /// + [Fact] + public void ToolApproval_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + + // Assert + Assert.Null(agent.GetService()); + } + + #endregion + + #region Feature: OpenTelemetry + + /// + /// Verify that OpenTelemetryAgent is included in the pipeline by default. + /// + [Fact] + public void OpenTelemetry_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableOpenTelemetry = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + + // Assert + Assert.NotNull(agent.GetService()); + } + + /// + /// Verify that OpenTelemetryAgent is excluded when disabled. + /// + [Fact] + public void OpenTelemetry_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + + // Assert + Assert.Null(agent.GetService()); + } + + #endregion + + #region Feature: WebSearch + + /// + /// Verify that HostedWebSearchTool is added to ChatOptions.Tools by default. + /// + [Fact] + public async Task WebSearch_IncludedByDefaultAsync() + { + // Arrange + var mockClient = new Mock(); + ChatOptions? capturedOptions = null; + mockClient + .Setup(c => c.GetResponseAsync(It.IsAny>(), It.IsAny(), It.IsAny())) + .Callback, ChatOptions?, CancellationToken>((_, opts, _) => capturedOptions = opts) + .ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, "Done"))); + + var options = CreateAllDisabledOptions(); + options.DisableWebSearch = false; + + var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var session = await agent.CreateSessionAsync(); + + // Act + await agent.RunAsync([new ChatMessage(ChatRole.User, "Hi")], session); + + // Assert + Assert.NotNull(capturedOptions?.Tools); + Assert.Contains(capturedOptions!.Tools!, t => t is HostedWebSearchTool); + } + + /// + /// Verify that HostedWebSearchTool is not added when disabled. + /// + [Fact] + public async Task WebSearch_ExcludedWhenDisabledAsync() + { + // Arrange + var mockClient = new Mock(); + ChatOptions? capturedOptions = null; + mockClient + .Setup(c => c.GetResponseAsync(It.IsAny>(), It.IsAny(), It.IsAny())) + .Callback, ChatOptions?, CancellationToken>((_, opts, _) => capturedOptions = opts) + .ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, "Done"))); + + var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var session = await agent.CreateSessionAsync(); + + // Act + await agent.RunAsync([new ChatMessage(ChatRole.User, "Hi")], session); + + // Assert + Assert.NotNull(capturedOptions); + if (capturedOptions!.Tools != null) + { + Assert.DoesNotContain(capturedOptions.Tools, t => t is HostedWebSearchTool); + } + } + + /// + /// Verify that user-provided tools are preserved alongside the default HostedWebSearchTool. + /// + [Fact] + public async Task WebSearch_CoexistsWithUserToolsAsync() + { + // Arrange + var mockClient = new Mock(); + ChatOptions? capturedOptions = null; + mockClient + .Setup(c => c.GetResponseAsync(It.IsAny>(), It.IsAny(), It.IsAny())) + .Callback, ChatOptions?, CancellationToken>((_, opts, _) => capturedOptions = opts) + .ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, "Done"))); + + var userTool = AIFunctionFactory.Create(() => "test", "UserTool"); + var options = CreateAllDisabledOptions(); + options.DisableWebSearch = false; + options.ChatOptions = new ChatOptions { Tools = [userTool] }; + + var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var session = await agent.CreateSessionAsync(); + + // Act + await agent.RunAsync([new ChatMessage(ChatRole.User, "Hi")], session); + + // Assert + Assert.NotNull(capturedOptions?.Tools); + Assert.Contains(capturedOptions!.Tools!, t => t is HostedWebSearchTool); + Assert.Contains(capturedOptions.Tools!, t => t == userTool); + } + + #endregion + + #region Feature: TodoProvider + + /// + /// Verify that TodoProvider is included in AIContextProviders by default. + /// + [Fact] + public void TodoProvider_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableTodoProvider = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is TodoProvider); + } + + /// + /// Verify that TodoProvider is excluded when disabled. + /// + [Fact] + public void TodoProvider_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + if (innerAgent!.AIContextProviders != null) + { + Assert.DoesNotContain(innerAgent.AIContextProviders, p => p is TodoProvider); + } + } + + #endregion + + #region Feature: AgentModeProvider + + /// + /// Verify that AgentModeProvider is included in AIContextProviders by default. + /// + [Fact] + public void AgentModeProvider_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableAgentModeProvider = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is AgentModeProvider); + } + + /// + /// Verify that AgentModeProvider is excluded when disabled. + /// + [Fact] + public void AgentModeProvider_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + if (innerAgent!.AIContextProviders != null) + { + Assert.DoesNotContain(innerAgent.AIContextProviders, p => p is AgentModeProvider); + } + } + + /// + /// Verify that custom AgentModeProviderOptions are passed through. + /// + [Fact] + public void AgentModeProvider_UsesCustomOptions() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableAgentModeProvider = false; + options.AgentModeProviderOptions = new AgentModeProviderOptions + { + Modes = + [ + new AgentModeProviderOptions.AgentMode("custom-mode", "A custom mode for testing"), + ], + DefaultMode = "custom-mode", + }; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert — AgentModeProvider should be present (we can't easily inspect its internal options, + // but we verify it is created and present). + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is AgentModeProvider); + } + + #endregion + + #region Feature: FileMemoryProvider + + /// + /// Verify that FileMemoryProvider is included in AIContextProviders by default. + /// + [Fact] + public void FileMemoryProvider_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableFileMemory = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is FileMemoryProvider); + } + + /// + /// Verify that FileMemoryProvider is excluded when disabled. + /// + [Fact] + public void FileMemoryProvider_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + if (innerAgent!.AIContextProviders != null) + { + Assert.DoesNotContain(innerAgent.AIContextProviders, p => p is FileMemoryProvider); + } + } + + /// + /// Verify that a custom FileMemoryStore is used when provided. + /// + [Fact] + public void FileMemoryProvider_UsesCustomStore() + { + // Arrange + var chatClient = new Mock().Object; + var customStore = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableFileMemory = false; + options.FileMemoryStore = customStore; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert — FileMemoryProvider should be present with the custom store. + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is FileMemoryProvider); + } + + #endregion + + #region Feature: FileAccessProvider + + /// + /// Verify that FileAccessProvider is included in AIContextProviders by default. + /// + [Fact] + public void FileAccessProvider_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableFileAccess = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is FileAccessProvider); + } + + /// + /// Verify that FileAccessProvider is excluded when disabled. + /// + [Fact] + public void FileAccessProvider_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + if (innerAgent!.AIContextProviders != null) + { + Assert.DoesNotContain(innerAgent.AIContextProviders, p => p is FileAccessProvider); + } + } + + /// + /// Verify that a custom FileAccessStore is used when provided. + /// + [Fact] + public void FileAccessProvider_UsesCustomStore() + { + // Arrange + var chatClient = new Mock().Object; + var customStore = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableFileAccess = false; + options.FileAccessStore = customStore; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert — FileAccessProvider should be present with the custom store. + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is FileAccessProvider); + } + + #endregion + + #region Feature: AgentSkillsProvider + + /// + /// Verify that AgentSkillsProvider is included in AIContextProviders by default. + /// + [Fact] + public void AgentSkillsProvider_IncludedByDefault() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableAgentSkillsProvider = false; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is AgentSkillsProvider); + } + + /// + /// Verify that AgentSkillsProvider is excluded when disabled. + /// + [Fact] + public void AgentSkillsProvider_ExcludedWhenDisabled() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var innerAgent = agent.GetService(); + + // Assert + Assert.NotNull(innerAgent); + if (innerAgent!.AIContextProviders != null) + { + Assert.DoesNotContain(innerAgent.AIContextProviders, p => p is AgentSkillsProvider); + } + } + + /// + /// Verify that a custom AgentSkillsSource is used when provided. + /// + [Fact] + public void AgentSkillsProvider_UsesCustomSource() + { + // Arrange + var chatClient = new Mock().Object; + var customSource = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.DisableAgentSkillsProvider = false; + options.AgentSkillsSource = customSource; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + + // Assert — AgentSkillsProvider should be present. + Assert.NotNull(innerAgent?.AIContextProviders); + Assert.Contains(innerAgent!.AIContextProviders!, p => p is AgentSkillsProvider); + } + + #endregion + + #region Feature: MaximumIterationsPerRequest + + /// + /// Verify that MaximumIterationsPerRequest configures the FunctionInvokingChatClient. + /// + [Fact] + public void MaximumIterationsPerRequest_ConfiguresFunctionInvokingChatClient() + { + // Arrange + var chatClient = new Mock().Object; + var options = CreateAllDisabledOptions(); + options.MaximumIterationsPerRequest = 42; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, options); + var innerAgent = agent.GetService(); + var ficc = innerAgent!.ChatClient.GetService(); + + // Assert + Assert.NotNull(ficc); + Assert.Equal(42, ficc!.MaximumIterationsPerRequest); + } + + /// + /// Verify that the default MaximumIterationsPerRequest is used when not set. + /// + [Fact] + public void MaximumIterationsPerRequest_UsesDefaultWhenNotSet() + { + // Arrange + var chatClient = new Mock().Object; + + // Act + var agent = new HarnessAgent(chatClient, TestMaxContextWindowTokens, TestMaxOutputTokens, CreateAllDisabledOptions()); + var innerAgent = agent.GetService(); + var ficc = innerAgent!.ChatClient.GetService(); + + // Assert — default is not 0 and not our custom value. + Assert.NotNull(ficc); + Assert.NotEqual(0, ficc!.MaximumIterationsPerRequest); + } + + #endregion + + #region Feature: All Defaults Enabled + + /// + /// Verify that when no options are provided, all default features are enabled. + /// + [Fact] + public async Task AllDefaults_AllFeaturesEnabledAsync() + { + // Arrange + var mockClient = new Mock(); + ChatOptions? capturedOptions = null; + mockClient + .Setup(c => c.GetResponseAsync(It.IsAny>(), It.IsAny(), It.IsAny())) + .Callback, ChatOptions?, CancellationToken>((_, opts, _) => capturedOptions = opts) + .ReturnsAsync(new ChatResponse(new ChatMessage(ChatRole.Assistant, "Done"))); + + // Act + var agent = new HarnessAgent(mockClient.Object, TestMaxContextWindowTokens, TestMaxOutputTokens); + var innerAgent = agent.GetService(); + + // Assert — agent wrappers + Assert.NotNull(agent.GetService()); + Assert.NotNull(agent.GetService()); + + // Assert — default context providers + Assert.NotNull(innerAgent); + Assert.NotNull(innerAgent!.AIContextProviders); + + var providers = innerAgent.AIContextProviders!.ToList(); + Assert.Contains(providers, p => p is TodoProvider); + Assert.Contains(providers, p => p is AgentModeProvider); + Assert.Contains(providers, p => p is FileMemoryProvider); + Assert.Contains(providers, p => p is FileAccessProvider); + Assert.Contains(providers, p => p is AgentSkillsProvider); + + // Assert — HostedWebSearchTool is present in the tools sent to the model + var session = await agent.CreateSessionAsync(); + await agent.RunAsync([new ChatMessage(ChatRole.User, "Hi")], session); + Assert.NotNull(capturedOptions?.Tools); + Assert.Contains(capturedOptions!.Tools!, t => t is HostedWebSearchTool); + } + + #endregion } From 4a6d06a642d8d999bb8f9ddd27375412ae0234d9 Mon Sep 17 00:00:00 2001 From: westey <164392973+westey-m@users.noreply.github.com> Date: Fri, 15 May 2026 14:42:35 +0000 Subject: [PATCH 2/4] Address PR comments --- .../Harness_Step03_DataProcessing.csproj | 4 ++++ .../Harness/Harness_Step03_DataProcessing/Program.cs | 6 ++++-- .../Harness/AgentMode/AgentModeProvider.cs | 6 +++--- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj index 21922126503..c65a7552a64 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj +++ b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Harness_Step03_DataProcessing.csproj @@ -18,4 +18,8 @@ + + + + diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs index 220423f0b9b..f4b27b21fc6 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs @@ -54,8 +54,9 @@ You are a data analyst assistant. You have access to a folder of data files via - Always explain what you learned and what you are going to do next between tool calls, so the user can follow along with your thought process. """; -// Create the agent using AsHarnessAgent. The default FileAccessProvider uses {cwd}/working, -// which matches the sample's working/ folder. Unused features are disabled. +// Create the agent using AsHarnessAgent. The FileAccessStore is explicitly set to the +// sample's working/ folder (copied to the output directory) so it works regardless of cwd. +// Unused features are disabled. AIAgent agent = new OpenAIClient( new BearerTokenPolicy(new DefaultAzureCredential(), "https://ai.azure.com/.default"), @@ -70,6 +71,7 @@ You are a data analyst assistant. You have access to a folder of data files via { Name = "DataAnalyst", Description = "A data analyst assistant that reads, analyzes, and processes data files.", + FileAccessStore = new FileSystemAgentFileStore(Path.Combine(AppContext.BaseDirectory, "working")), DisableTodoProvider = true, DisableAgentModeProvider = true, DisableFileMemory = true, diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs b/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs index bfeca78e549..36fe069af3d 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs @@ -84,11 +84,11 @@ 3. Do not proceed until you have received all the needed clarifications. new( "execute", """ - Use this mode when carrying out approved plans. Work autonomously using your best judgement — do not ask the user questions or wait for feedback. + Use this mode when carrying out approved plans. Work autonomously using your best judgment — do not ask the user questions or wait for feedback. Process to follow when in execute mode: - 1. If you don't have a plan or tasks yet, analyse the user request and create tasks and a plan. (**Skip this step if you came from plan mode**) - 2. Work autonomously — use your best judgement to make decisions and keep progressing without asking the user questions. The goal is to have a complete, useful result ready when the user returns. + 1. If you don't have a plan or tasks yet, analyze the user request and create tasks and a plan. (**Skip this step if you came from plan mode**) + 2. Work autonomously — use your best judgment to make decisions and keep progressing without asking the user questions. The goal is to have a complete, useful result ready when the user returns. 3. If you encounter ambiguity or an unexpected situation during execution, choose the most reasonable option, note your choice, and keep going. 4. Mark tasks as completed as you finish them. 5. Continue working, thinking and calling tools until you have the research result for the user. From 87330ef5d41abb2b7549a4cdde463e2017875c84 Mon Sep 17 00:00:00 2001 From: westey <164392973+westey-m@users.noreply.github.com> Date: Fri, 15 May 2026 18:06:16 +0000 Subject: [PATCH 3/4] Add further comments to clarify certain setings. --- .../Harness/Harness_Step01_Research/Program.cs | 11 ++++++----- .../Harness_Step02_Research_WithSubAgents/Program.cs | 12 ++++++------ .../Harness/Harness_Step03_DataProcessing/Program.cs | 2 +- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs index d0ac5c56cf2..7ea54d0e6c6 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step01_Research/Program.cs @@ -74,22 +74,23 @@ Track your sources — you will need them when presenting results. RetryPolicy = new ClientRetryPolicy(3) // Enable retries to improve resiliency. }) .GetResponsesClient() - .AsIChatClientWithStoredOutputDisabled(deploymentName) // We want to manage chat history locally (not stored in the responses service), so that we can manage compaction ourselves. + .AsIChatClientWithStoredOutputDisabled(deploymentName) // We want to manage chat history locally (not stored in the responses service), so that we can manage compaction ourselves. .AsHarnessAgent(MaxContextWindowTokens, MaxOutputTokens, new HarnessAgentOptions { Name = "ResearchAgent", Description = "A research assistant that plans and executes research tasks.", - DisableFileAccess = true, - FileMemoryStore = new FileSystemAgentFileStore(Path.Combine(AppContext.BaseDirectory, "agent-files")), + DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session + FileMemoryStore = new FileSystemAgentFileStore( // Configure the file memory provider to store files in a local folder called "agent-files". + Path.Combine(AppContext.BaseDirectory, "agent-files")), ChatOptions = new ChatOptions { Instructions = instructions, Tools = [ - new WebBrowsingTool( // Add a local web browsing tool that converts html to markdown. + new WebBrowsingTool( // Add a local web browsing tool that converts html to markdown. new WebBrowsingToolOptions { AllowPublicNetworks = true }), ], - MaxOutputTokens = MaxOutputTokens, // Set a high token limit for long research tasks with many tool calls and long outputs. + MaxOutputTokens = MaxOutputTokens, // Set a high token limit for long research tasks with many tool calls and long outputs. Reasoning = new() { Effort = ReasoningEffort.Medium }, }, }); diff --git a/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs index 489478d8ee6..bb4c50e0d7a 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step02_Research_WithSubAgents/Program.cs @@ -45,9 +45,9 @@ Description = "An agent that can search the web to find information.", DisableTodoProvider = true, DisableAgentModeProvider = true, - DisableFileMemory = true, - DisableFileAccess = true, - DisableToolApproval = true, + DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session + DisableFileAccess = true, // If enabled, this would allow the agent to read/write files in a working directory + DisableToolApproval = true, // If enabled, this allows don't-ask-again approval functionality. ChatOptions = new ChatOptions { Instructions = "You are a web search assistant. When asked to find information, use the web search tool to look it up and return a concise, factual answer.", @@ -97,9 +97,9 @@ 5. Clear all completed tasks to free memory. Description = "An agent that researches stock prices using sub-agents.", DisableTodoProvider = true, DisableAgentModeProvider = true, - DisableFileMemory = true, - DisableFileAccess = true, - DisableToolApproval = true, + DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session + DisableFileAccess = true, // If enabled, this would allow the agent to read/write files in a working directory + DisableToolApproval = true, // If enabled, this allows don't-ask-again approval functionality. DisableWebSearch = true, AIContextProviders = [ diff --git a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs index f4b27b21fc6..6b77e31f15c 100644 --- a/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs +++ b/dotnet/samples/02-agents/Harness/Harness_Step03_DataProcessing/Program.cs @@ -74,7 +74,7 @@ You are a data analyst assistant. You have access to a folder of data files via FileAccessStore = new FileSystemAgentFileStore(Path.Combine(AppContext.BaseDirectory, "working")), DisableTodoProvider = true, DisableAgentModeProvider = true, - DisableFileMemory = true, + DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session DisableWebSearch = true, ChatOptions = new ChatOptions { From 1de8c79a1fcaabb7de28053698ef7611fdf94788 Mon Sep 17 00:00:00 2001 From: westey <164392973+westey-m@users.noreply.github.com> Date: Mon, 18 May 2026 10:33:43 +0100 Subject: [PATCH 4/4] Apply suggestion from @SergeyMenshykh Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com> --- .../Harness/AgentMode/AgentModeProvider.cs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs b/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs index 36fe069af3d..a7f4aca2866 100644 --- a/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs +++ b/dotnet/src/Microsoft.Agents.AI/Harness/AgentMode/AgentModeProvider.cs @@ -73,10 +73,10 @@ 1. Analyze the request with the purpose of building a research plan. 2. Create a list of todo items. 3. If needed, use the provided tools to do some exploratory checks to help build a plan and determine what clarifying questions you may need from the user. 4. Ask for clarifications from the user where needed. - 1. Ask each clarification one by one. - 2. When asking for clarification and you have specific options in mind, present them to the user, so they can choose the option instead of having to retype the entire response. - 3. Do not proceed until you have received all the needed clarifications. - 4. Do short exploratory research if it helps with being able to ask sensible clarifications from the user. + 1. Ask each clarification one by one. + 2. When asking for clarification and you have specific options in mind, present them to the user, so they can choose the option instead of having to retype the entire response. + 3. Do not proceed until you have received all the needed clarifications. + 4. Do short exploratory research if it helps with being able to ask sensible clarifications from the user. 5. Write the plan to a memory file, so that it is retained even if compaction happens. Make sure to update the plan file if the user requests changes. 6. Present the plan to the user and ask for approval to switch to execute mode and process the plan. 7. When approval is granted, always switch to execute mode (using the `AgentMode_Set` tool), and follow the steps for *Execute mode*.