Agent does not execute functions

[jgato]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS: Fedora
- Continue version: 1.1.35
- IDE version: VS Code
- Model: granite
- config:
models:
  - name: My Model (litellm)
    provider: openai
    model: gpt-4o
    apiKey: 192....8edcde
    apiBase: http://127.0.0.1:4000/v1/
    systemMessage: Answer the user's request using the relevant tool(s), if they are
      available. Check that all the required parameters for each tool call are
      provided or can reasonably be inferred from context. IF there are no
      relevant tools or there are missing values for required parameters, ask
      the user to supply these values; otherwise proceed with the tool calls. If
      the user provides a specific value for a parameter (for example provided
      in quotes), make sure to use that value EXACTLY. DO NOT make up values for
      or ask about optional parameters. Carefully analyze descriptive terms in
      the request as they may indicate required parameter values that should be
      included even if not explicitly quoted  When using a tool, follow the json
      schema very carefully and make sure to include ALL required
      properties.Always output valid JSON when using a tool.If a tool exists to
      do a task, use the tool instead of asking the user to manually take an
      action.If you say that you will take an action, then go ahead and use the
      tool to do it. No need to ask permission.Never use that does not exist.
      Use tools using the proper procedure, DO NOT write out a json codeblock
      with the tool inputs.Never say the name of a tool to a user.If you think
      running multiple tools can answer the user's question, prefer calling them
      in parallel whenever possible, but do not call semantic_search in
      parallel.If semantic_search returns the full contents of the text files in
      the workspace, you have all the workspace context.Don't call the
      run_in_terminal tool multiple times in parallel. Instead, run one command
      and wait for the output before running the next command.After you have
      performed the user's task, if the user corrected something you did,
      expressed a coding preference, or communicated a fact that you need to
      remember, use the update_user_preferences tool to save their preferences.

In the middle I am using litellm, with a model that is OpenAPI compatible.

Description

When using the AGent mode, it does not execute the function, but... it just tells me how these work:

I try also with the functions of an MCP, but:

Using the VS Copilot chat/agent it executes the functions:

any hint what could be happening?

To reproduce

No response

Log output

[chezsmithy]

Does it work if you remove the system message override?

[jgato]

Does it work if you remove the system message override?

sorry, where could I configure this?

[ahmdgawad]

I have the same problem, agent mode list the queries to be done, but doesn't executing functions, e.g atlassian mcp

[Patrick-Erichsen]

@jgato can you confirm if you are in Agent mode?

[jgato]

yes:

[Patrick-Erichsen]

Got it - this is likely an issue with Granite, it often is hesitant to call tools. Could you try prompting it a bit more directly and seeing if that helps?

[jgato]

I dont see a big difference :(

trying a mistral model:

the same :(

[Andy-Voigt]

Same for qwen2.5-coder

[gaby]

Same with Mistral/Devstral via vLLM.

• Sometimes tools are called
• Most of the time tools are not called, it just prints the tool as the comments above
• Sometimes it will say it can't see any tools
• Sometimes it works, then randomly stops working for the next prompt.

[johny-mnemonic]

• Sometimes tools are called
• Most of the time tools are not called, it just prints the tool as the comments above
• Sometimes it will say it can't see any tools
• Sometimes it works, then randomly stops working for the next prompt.

Same issue here with local Ollama running directly on the same system as the VScode.

[Patrick-Erichsen]

In general this is due to the capability of the model to perform tool calling - we have some work planned to try to make it more clear which models are/are not great at tool calling, but outside of those warnings there isn't really anything we can do. I'd recommend trying to prompt the model more directly to invoke tools but that will only help a bit.

[gaby]

@Patrick-Erichsen in the case of devstral it's an agentic model. I'm not sure if the issue is continue.dev or vLLM.

Tools work, just 50% of the time. Even if you tell the model to use the tools, it randomly doesnt.