Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

base64Encode causes RangeError: Maximum call stack size exceeded #41

Open
mwohlan opened this issue Sep 30, 2024 · 0 comments
Open

base64Encode causes RangeError: Maximum call stack size exceeded #41

mwohlan opened this issue Sep 30, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@mwohlan
Copy link

mwohlan commented Sep 30, 2024

Environment

Digital Ocean App Platform running Nuxt3 App

Reproduction

It runs fine locally, but crashes when deployed on more limited hardware

Describe the bug

Since I have not provided a reproduction, here is a detailed description of the issue:

I am using undio to transform uint8 Array to a Base64 encoded String ( first page of a pdf to pass it into an llm API).

 const pdf: ReadableStream = await graphClient.api(`/users/xxxxxx/drive/items/${pdfID}/content`).get()
    const firstPage = await extractFirstPageFromPdf(pdf)
    
    const base64 = uint8ArrayToBase64(firstPage, { dataURL: false })

This works fine locally but when deployed to DigitalOcean App Platform ( smallest resource size) the following error occurs when undio is calling the internal function _base64Encode:

function _base64Encode(data, opts) {
  let encoded = btoa(String.fromCodePoint(...data));
  if (opts?.urlSafe) {
    encoded = encoded.replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
  }
  return opts?.dataURL === false ? encoded : `data:${opts?.type || ""};base64,${encoded}`;
}

RangeError: Maximum call stack size exceeded

This is explanation the o1-preview model provided:

The error you're encountering—"RangeError: Maximum call stack size exceeded"—is due to the way you're using the spread operator (...) with String.fromCodePoint() on a potentially large array of data. When data is large, spreading it as arguments to String.fromCodePoint() can exceed the maximum number of arguments that a function can accept, leading to a stack overflow.

I can't really judge if this is a valid explanation but the provided fix did work for my environment. But it is of course a Node only solution:

import { Buffer } from 'node:buffer'

export function uInt8ToBase64(data: Uint8Array, opts?: { urlSafe?: boolean, dataURL?: boolean, type?: string }) {
  let encoded = Buffer.from(data).toString('base64')
  if (opts?.urlSafe) {
    encoded = encoded.replace(/\+/g, '-').replace(/\//g, '').replace(/=+$/, '')
  }
  return opts?.dataURL === false ? encoded : `data:${opts?.type || ''};base64,${encoded}`
}

Additional context

No response

Logs

No response

@mwohlan mwohlan added the bug Something isn't working label Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant