Unable to catch error from toDataStreamResponse

[namukang]

Description

In my application, I allow users to pass in their own Anthropic API key. When a request is made using streamText and toDataStreamResponse with an Anthropic API key that has no credit balance, the following error occurs:

 ⨯ Error: failed to pipe response
    at pipeToNodeResponse (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/pipe-readable.js:126:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async sendResponse (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/send-response.js:40:13)
    at async doRender (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:1375:25)
    at async cacheEntry.responseCache.get.routeKind (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:1567:28)
    at async DevServer.renderToResponseWithComponentsImpl (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:1475:28)
    at async DevServer.renderPageComponent (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:1901:24)
    at async DevServer.renderToResponseImpl (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:1939:32)
    at async DevServer.pipeImpl (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:914:25)
    at async NextNodeServer.handleCatchallRenderRequest (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/next-server.js:272:17)
    at async DevServer.handleRequestImpl (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/base-server.js:810:17)
    at async /project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/dev/next-dev-server.js:339:20
    at async Span.traceAsyncFn (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/trace/trace.js:154:20)
    at async DevServer.handleRequest (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/dev/next-dev-server.js:336:24)
    at async invokeRender (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/lib/router-server.js:173:21)
    at async handleRequest (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/lib/router-server.js:350:24)
    at async requestHandlerImpl (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/lib/router-server.js:374:13)
    at async Server.requestListener (/project/node_modules/.pnpm/[email protected]_@[email protected][email protected][email protected][email protected]/node_modules/next/dist/server/lib/start-server.js:141:13) {
  [cause]: APICallError [AI_APICallError]: Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits.
      at eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/@[email protected][email protected]/node_modules/@ai-sdk/provider-utils/dist/index.mjs:472:14)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async postToApi (webpack-internal:///(rsc)/../../node_modules/.pnpm/@[email protected][email protected]/node_modules/@ai-sdk/provider-utils/dist/index.mjs:379:28)
      at async AnthropicMessagesLanguageModel.doStream (webpack-internal:///(rsc)/../../node_modules/.pnpm/@[email protected][email protected]/node_modules/@ai-sdk/anthropic/dist/index.mjs:574:50)
      at async fn (webpack-internal:///(rsc)/../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/ai/dist/index.mjs:4101:25)
      at async eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/ai/dist/index.mjs:290:22)
      at async _retryWithExponentialBackoff (webpack-internal:///(rsc)/../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/ai/dist/index.mjs:118:12)
      at async streamStep (webpack-internal:///(rsc)/../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/ai/dist/index.mjs:4056:15)
      at async fn (webpack-internal:///(rsc)/../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/ai/dist/index.mjs:4444:9)
      at async eval (webpack-internal:///(rsc)/../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/ai/dist/index.mjs:290:22) {
    cause: undefined,
    url: 'https://api.anthropic.com/v1/messages',
    requestBodyValues: {
      model: 'claude-3-5-sonnet-20241022',
      max_tokens: 4096,
      temperature: 0,
      top_k: undefined,
      top_p: undefined,
      stop_sequences: undefined,
      system: [Array],
      messages: [Array],
      tools: [Array],
      tool_choice: [Object],
      stream: true
    },
    statusCode: 400,
    responseHeaders: {
      'cf-cache-status': 'DYNAMIC',
      'cf-ray': '8e87d682492f436d-EWR',
      connection: 'keep-alive',
      'content-length': '190',
      'content-type': 'application/json',
      date: 'Tue, 26 Nov 2024 06:31:55 GMT',
      'request-id': 'req_013aQeUV5XRegcLsiC5foppy',
      server: 'cloudflare',
      via: '1.1 google',
      'x-robots-tag': 'none',
      'x-should-retry': 'false'
    },
    responseBody: '{"type":"error","error":{"type":"invalid_request_error","message":"Your credit balance is too low to access the Anthropic API. Please go to Plans & Billing to upgrade or purchase credits."}}',
    isRetryable: false,
    data: { type: 'error', error: [Object] },
    [Symbol(vercel.ai.error)]: true,
    [Symbol(vercel.ai.error.AI_APICallError)]: true
  }
}

I'd like to catch this error to show a useful error message to the user, but the error occurs without being caught in a try/catch or showing up in onFinish in streamText or getErrorMessage in toDataStreamResponse (in other words, the error does not show up anywhere it can be handled programmatically):

  try {
    const result = streamText({
      model: createAnthropic({ apiKey } )("claude-3-5-sonnet-20241022"),
      messages,
      onFinish: (result) => {
        console.log("onFinish", result);
      },
    });
    return result.toDataStreamResponse({
      getErrorMessage: (error) => {
        console.log("getErrorMessage", error);
        return "";
      },
    });
  } catch (e) {
    console.log("Error", e);
  }

The server fails with a 500 error message due to the uncaught exception.

Related issue due to the same cause: #3655

I'd like to be able to handle this error on the server, or alternatively, have this error streamed back to the client so that it can be handled in onError in useChat.

If there is a way to handle this error, I'd appreciate pointers. Thanks!

Code example

No response

AI provider

No response

Additional context

No response

[arnab710]

After removal of await from streamText in AI sdk v4, we've started facing this issue. If any AI provider returns an error (e.g. Invalid API key), we're now unable to catch it on the server side and handle it (e.g. retry with alternative provider).

@lgrammel any workaround?

[arnab710]

@namukang, have you found a fix yet?

[Praneeth-Pike]

I'm facing the same issue and created an issue here : #3875