Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Byte stream support for Socket readableWebStream #56004

Open
seriousme opened this issue Nov 26, 2024 · 0 comments
Open

Byte stream support for Socket readableWebStream #56004

seriousme opened this issue Nov 26, 2024 · 0 comments
Labels
feature request Issues that request new features to be added to Node.js.

Comments

@seriousme
Copy link

seriousme commented Nov 26, 2024

What is the problem this feature will solve?

As mentioned by the winterCG it is reasonable to assume that most TCP cases are bytes oriented. And my case (MQTT server/client) is one of them.

However:

import { createConnection } from "node:net";
import { Readable } from "node:stream";

const socket = createConnection({ port:1883, host: "localhost" }, () => {
    const readable= Readable.toWeb(socket);
    const reader = readable.getReader({ mode: "byob" });
});

Gives me:

 node:internal/webstreams/readablestream:2280
    throw new ERR_INVALID_ARG_VALUE('stream', stream, 'must be a byte stream');
    ^

TypeError [ERR_INVALID_ARG_VALUE]: The argument 'stream' must be a byte stream. Received ReadableStream { locked: false, state: 'readable', supportsBYOB: false }
    at setupReadableStreamBYOBReader (node:internal/webstreams/readablestream:2280:11)
    at new ReadableStreamBYOBReader (node:internal/webstreams/readablestream:935:5)
    at ReadableStream.getReader (node:internal/webstreams/readablestream:346:12)
    at Socket.<anonymous> (file:///home/hansklunder/github/opifex/tmp.js:6:29)
    at Object.onceWrapper (node:events:632:28)
    at Socket.emit (node:events:518:28)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1595:10) {
  code: 'ERR_INVALID_ARG_VALUE'
}

Which makes sense in its current implementation.

Solving this would bring better compatibility with the WinterCG standards and would make it easier to implement TCP bytestream protocols in NodeJS.

What is the feature you are proposing to solve the problem?

It would be nice if I could signal either at the creation of the socket or the Readable.toWeb() that I want a ReadableByteStream instead of ReadableDefaultStream.

Even more brilliant would be if I could skip the Readable.toWeb() call also somehow, e.g. by importing node:sockets or something similar.

What alternatives have you considered?

I've currently hacked my own layer on top of the socket event API:

import { Writable } from "node:stream";
function closer(sock) {
    if (!sock.closed) {
        sock.end();
    }
}
export function wrapNodeSocket(socket) {
    const readable = new ReadableStream({
        type: "bytes",
        start(controller) {
            socket.on("data", (data) => {
                controller.enqueue(data);
                const desiredSize = controller.desiredSize ?? 0;
                if (desiredSize <= 0) {
                    // The internal queue is full, so propagate
                    // the backpressure signal to the underlying source.
                    socket.pause();
                }
            });
            socket.on("error", (err) => controller.error(err));
            socket.on("end", () => {
                // close the controller
                controller.close();
                // and unlock the last BYOB read request
                controller.byobRequest?.respond(0);
            });
        },
        pull: () => {
            socket.resume();
        },
        cancel: () => {
            socket.end();
        },
    });
    const writable = Writable.toWeb(socket);
    const remoteAddr = {
        hostname: socket.remoteAddress || "",
        port: socket.remotePort || 0,
    };
    const conn = {
        readable: readable,
        writable: writable,
        close: () => closer(socket),
        remoteAddr,
    };
    return conn;
}

This works, but it would be nice if it was part of standard NodeJS functionality.

@seriousme seriousme added the feature request Issues that request new features to be added to Node.js. label Nov 26, 2024
@github-project-automation github-project-automation bot moved this to Awaiting Triage in Node.js feature requests Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Issues that request new features to be added to Node.js.
Projects
Status: Awaiting Triage
Development

No branches or pull requests

1 participant