keeping node alive while workers are working

[bakkot]

Version of emscripten/emsdk:

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.73 (ac676d5e437525d15df5fd46bc2c208ec6d376a3)
clang version 20.0.0git (https:/github.com/llvm/llvm-project 1d810ece2b2c8fab77720493864257f0ea3336a9)

Failing command line in full:

emcc  --pre-js pre.js -pthread -s MODULARIZE=1 -s EXPORTED_FUNCTIONS="_spawn" main.cc && node run.mjs

---

I have a library which is intended to be driven from JS (i.e., no main loop in C, and using -s MODULARIZE=1).

I'm trying to spawn a thread, do some work, and then get the result back in the main thread. I have the following three bits of code, with the first two being the inputs to emcc and the third being the driver:

// pre.js
let spawnTimeout;
let resolve;

Module.async_sum = () => {
  _spawn();
  return new Promise((res) => {
    resolve = res;
  });
};

// main.cc
#include <thread>
#include <stdio.h>
#include <emscripten.h>

extern "C" void spawn() {
  printf("spawning\n");
  std::thread t([] {
    MAIN_THREAD_ASYNC_EM_ASM({
      clearTimeout(spawnTimeout);
    });

    printf("spawned\n");

    double v = 1.0;
    for (int i = 1; i < 1e8; ++i) {
      v += 1.0 / i;
    }
    printf("sum: %f\n", v);
    MAIN_THREAD_ASYNC_EM_ASM({
      resolve($0);
    }, v);
  });
  t.detach();
  EM_ASM({
    spawnTimeout = setTimeout(() => {}, 5000);
  });
}

// run.mjs
import { default as init } from './a.out.js';

let x = await init();

console.log(await x.async_sum());

Running node run.mjs prints

spawning
spawned
Warning: Detected unsettled top-level await at file:///Users/kevin/code/emscripten-thread-test/run.mjs:5
console.log(await x.async_sum());

and then exits, without printing the sum or giving me the result.

I was really hoping it would instead do the work and give me the result. That worked prior to #19073 (or to be more precise, it worked in version 3.1.15; I'm assuming that PR is the relevant change). I assume this is because node exits when the main thread has no tasks queued and the workers are all unref'd.

If I comment out the "clearTimeout" line it works, but then of course I need to wait for the timeout. I can move the clearTimeout later, but then if the worker crashes or is killed (as sometimes happens) it will again need to wait for the timeout.

Is there some way I can get it to stay alive? Maybe keep the workers .ref'd while they're actually doing work?

[kripken]

ccing people from that PR: @kleisauke @sbc100

[kleisauke]

Ah, this is indeed due to PR #19073. Calling the internal _emscripten_thread_set_strongref() function would prevent Node.js from exiting.

How about exposing this function in Emscripten's public API? That way, you could do:

--- a/main.cc
+++ b/main.cc
@@ -2,14 +2,11 @@
 #include <thread>
 #include <stdio.h>
 #include <emscripten.h>
+#include <emscripten/threading.h>
 
 extern "C" void spawn() {
     printf("spawning\n");
     std::thread t([] {
-      MAIN_THREAD_ASYNC_EM_ASM({
-        clearTimeout(spawnTimeout);
-      });
-
       printf("spawned\n");
 
       double v = 1.0;
@@ -21,8 +18,6 @@ extern "C" void spawn() {
         resolve($0);
       }, v);
     });
+    emscripten_thread_set_strongref(t.native_handle());
     t.detach();
-    EM_ASM({
-      spawnTimeout = setTimeout(() => {}, 5000);
-    });
 }

I just opened PR #23152 for this.