Code generation for modules/groups (inlining)

[maximecb]

The code in compiler.js currently doesn't handle modules, so we can't really use them in projects. To implement this, we should implement inlining, basically transforming the incoming graph until it contains no modules anymore and everything is flattened. This has to work recursively (modules inside of modules).

[mindplay-dk]

Should probably only inline smaller and/or few-instance modules?

For larger modules, a function call is probably worth while - since, otherwise, you're paying with potentially long compile times.

Very cool project btw! 😄👍

[maximecb]

Thanks :)

I'm hoping to increase performance of the audio thread by inlining things. The issue is that JS JITs don't optimize multiple return values AFAIK, so some other trick would have to be used such as storing the outputs on an object field, which is awkward.

[mindplay-dk]

Have you thought about generating WebAssembly bytecode instead?

I don't have any experience doing that myself, but would expect this is a whole lot faster than JS - you would get native 64-bit floating point operations, and you can use "arena" style memory management, avoiding allocations/deallocations at run-time entirely.

You might even consider adding a module with WASM source code input as well - for things that modules are less helpful for, such as FFTs, which could then be created in userland.

It's been a dream of mine for years, learning enough WASM to be able to build basically what you're building, but fast enough that it becomes a replacement for desktop products like Reaktor or PureData - but online, enabling users to build and share modules via community features etc.

(I wish I had the time or energy to get invested in this. 😅 I built a shoddy prototype of a tool of this type in Pascal, probably 30 years ago while I was in school, and actually used it to build a reverb effect that still exists in a few products today. I worked in music software for a few years, but not since the 2000s. I just love that it's even possible to build something like this on the web now, and it's so cool that you're actually building it!)

[maximecb]

NoiseCraft already gets native float64 arithmetic. Modern JS JITs are good enough to unbox the values, especially in cases like this because the code that's generated is very flat and dumb. This isn't the bottleneck. I could also make better use of Float64Array and optimize the code more. That being said, one of the things that's awkward with JS is that we can't realistically have more than one audio thread. AFAIK we also can't have a MIDI thread yet, and it would have been nice to have some custom UI rendering, but I don't think I could make this fast in JS.

I have been thinking about maybe building a more advanced version of NoiseCraft in rust. I think I can make it very portable, but I don't know how easy it would be to make it run in a browser. That would make it more of a desktop/laptop app by default.

[mindplay-dk]

NoiseCraft already gets native float64 arithmetic. Modern JS JITs are good enough to unbox the values, especially in cases like this because the code that's generated is very flat and dumb. This isn't the bottleneck. I could also make better use of Float64Array and optimize the code more.

is it able to make use of SIMD optimizations? for some high level ops (e.g. adding or multiplying n numbers) the SIMD operations can still be up to 3-4 times faster, from what I've seen of JS vs WASM benchmarks. (?)

That being said, one of the things that's awkward with JS is that we can't realistically have more than one audio thread.

and an WASM, you could, right? this would definitely be a step up, I think?

it would also unblock the main thread for UI work, wouldn't it?

AFAIK we also can't have a MIDI thread yet, and it would have been nice to have some custom UI rendering, but I don't think I could make this fast in JS.

from what I know, Figma does UI rendering on canvas with WebGL - offloading to a GPU, if available, so even better than just offloading the main thread... but yeah, that is definitely not simple or easy to do.

I have been thinking about maybe building a more advanced version of NoiseCraft in rust. I think I can make it very portable, but I don't know how easy it would be to make it run in a browser. That would make it more of a desktop/laptop app by default.

while that would be cool, there are definitely pros to being JS - mainly, a much larger audience can read the code. 🙂

also, I don't think you have a module yet with custom source code input, but that could also be a benefit to using JS - e.g. Reaper ships with a bunch of JS effects, but of course has some sort of hyper optimized custom run-time to make this possible, and it has the freedom to launch multiple instances of these in threads, and so on.

another potentially cool thing about the run-time being native JS is you could allow exporting the internal generated source code, and someone could take the generated code and use it in their own webaudio projects.

hmm, there is the AssemblyScript compiler, which can run in the browser - I wonder if this could be used to bridge the run-time generated code to WASM? you'd only need to add types. might be less disruptive to the project than trying to port it to WASM.

another approach might be web workers? you need a certain level of complexity before offloading anything to workers is worth while, but for example, as I recall, SynC Modular (a very old modular synth for PC) used the approach of offloading individual voices (in polyphonic mode) to separate threads - that's a reasonably simple approach, but of course only really useful for polyphonic synthesizers. (I'm not even sure if NC has polyphony?)

just throwing out ideas here. ☺️