toolChoice: 'required' and { toolName: 'x', type: 'tool' } send ai vercel SDK into endless loop when using streamText

[punkpeye]

Description

toolChoice: 'required' and { toolName: 'x', type: 'tool' } send ai vercel SDK into endless loop when using streamText

Code example

const { fullStream } = await streamText({
  abortSignal: signal,
  experimental_toolCallStreaming: false,
  maxSteps: 20,
  messages: relevantMessages.slice(),
  model
  toolChoice: { toolName: 'webSearch', type: 'tool' },
  tools: {
    webSearch: tool({
      description:
        'Search the web for information. Useful for when you need to answer questions about current events or a specific topic. The answer includes sources, which must be referenced in the answer using markdown footnotes.',
      execute: async ({ query }) => {
        return {
          answer: 'Paris',
        };
      },
      parameters: z.object({
        query: z
          .string()
          .describe(
            'The query to search for. Phrased in natural language as a question.',
          ),
      }),
    }),
  },
});

for await (const chunk of fullStream) {
  console.log('>>> chunk', chunk);
}

AI provider

@ai-sdk/openai

Additional context

Logs:

{
  type: 'tool-call',
  toolCallId: 'call_b0DkOmitAwd93Mt4xXd6u1gH',
  toolName: 'searchWeb',
  args: { query: 'What is the capital of France?' }
}
{
  type: 'tool-result',
  toolCallId: 'call_b0DkOmitAwd93Mt4xXd6u1gH',
  toolName: 'searchWeb',
  args: { query: 'What is the capital of France?' },
  result: {
    answer: 'The capital of France is Paris.',
    sources: [
      {
        name: 'What is the Capital of France? - WorldAtlas',
        snippet: "Learn about the history, geography, economy, tourism, and administration of Paris, the capital city of France and the country's largest city.",
        url: 'https://www.worldatlas.com/articles/what-is-the-capital-of-france.html'
      },
      {
        name: 'France | History, Maps, Flag, Population, Cities, Capital, & Facts ...',
        snippet: 'The capital and by far the most important city of France is Paris, one of the world’s preeminent cultural and commercial centres.',
        url: 'https://www.britannica.com/place/France'
      },
      {
        name: 'Paris - Wikipedia',
        snippet: 'Paris is the capital and largest city of France.',
        url: 'https://en.wikipedia.org/wiki/Paris'
      }
    ]
  }
}
{
  type: 'step-finish',
  finishReason: 'tool-calls',
  usage: { promptTokens: 456, completionTokens: 20, totalTokens: 476 },
  experimental_providerMetadata: { openai: { reasoningTokens: 0, cachedPromptTokens: 0 } },
  logprobs: undefined,
  response: {
    id: 'chatcmpl-AZpStkvY5DoWFRuCTA0L5hk1hOMOI',
    timestamp: 2024-12-02T01:24:35.000Z,
    modelId: 'gpt-4o-2024-11-20'
  },
  isContinued: false
}
{
  type: 'tool-call',
  toolCallId: 'call_4jKKKQAlZwguD1glEhCPAbDI',
  toolName: 'searchWeb',
  args: { query: 'capital of France' }
}

Notice how it starts over with a new tool-call for the same question.

The same pattern repeats indefinitely.

[lgrammel]

Tool choice required means that you force tool calls every time the LLM is called. Together with maxSteps this enters such a loop. This is working as expected.

Can you describe what you want to achieve? there may be other approaches that could work here.

[punkpeye]

Suppose user sent a message: "What's the capital of France?"

User also ticked "Search", so I know their intent is to search.

I want to tell LLM that for this request it has to use webSearch tool.

Just leaving it as auto does not achieve the same result – it sometimes uses it, sometimes does not.

My understanding was that by setting tool to required or specific tool, I am telling LLM that to answer this question it must use a/the tool, but once the answer is resolved, it should not continue looping.