Expo: Streamed response buffers first 512 characters

[matthiasbayer]

Description

Problem
The streamed response on iOS feels delayed and choppy.

Context
When streaming the LLM response from the API endpoint back to the iOS app, the first 512 characters will be buffered by the iOS native layer. This behavior apparently can't be configured, leaving it up to the callsite to work around this issue by supplying a content-type header that causes iOS to skip the buffering.

More technical context can be found this issue: expo/expo#32950

I'm not sure if a fix should be included in the code examples in the docs, or if a warning/note somewhere in the docs is enough. The header isn't strictly necessary to get the examples working, but greatly improves the UX and this behavior is really hard to debug. Another possibility to address this is to make a change to the toDataStreamResponse function and always set that Content-Type header, but I have not enough context to judge the impact and possible side-effects of such a change.

Solution
Explicitly add an HTTP header that doesn't trigger content buffering, e.g. Content-Type: application/octet-stream

Pull Request
Happy to create a pull request with either documentation or code changes depending on if and how this should be fixed (see additional context above).

Code example

https://github.com/matthiasbayer/expo-ai-sdk

return result.toDataStreamResponse({
  headers: {
    'Content-Type': 'application/octet-stream',
  }
});

AI provider

4.0.9

Additional context

No response

[lgrammel]

CC @nicoalbanese wdyt about adding this to the expo guide?

[nicoalbanese]

Thanks @matthiasbayer! Will test and add to the guide.

[matthiasbayer]

Awesome! Let me know if I can help with anything regarding the reproduction.

[nicoalbanese]

@lgrammel updated the guide here: #3950