build wasm file is very big???

[cyanBone]

build wasm file is very big
golang is 2mb+
rust is 20k+

this is 9.5mb

[vshymanskyy]

After running wasm-strip, it reduces to 7.5 Mb
Next, after running wasm-opt -Os it gets to 3.4 Mb

Still quite big, as with AssemblyScript, Rust, and TinyGo I was able to generate useful wasm files of just 200...3500 bytes. See https://github.com/wasm3/wasm3-arduino/tree/master/examples_pio/Wasm_Advanced/wasm_apps

[kateinoigakukun]

Now, we're focussing on making produced binary work properly. But I think binary size is one of the most important things too.

Fat stdlib

As far as I investigated, Swift stdlib is so fat that weights about 8.8MB because it depends on ICU and libc. Other platform projects are facing on same problem too.

Fortunately, Swift6 will work on supporting more platforms, and core team know this problem. If they decide to implement a subset of stdlib to be more small, it can be helpful.

LTO approach

Most of stdlib parts are unused, so if linker omits unused code, the produced binary will be more small.

Here is a Swift LTO talk at LLVM meeting.
This presentation shows that effective LTO reduces 20% of binary size. I've not tried this method yet and I don't know whether this works as well for wasm target, but in theory it can.

Swift has many dynamic language feature, like dynamic casting, dynamic dispatch or etc.., so my concerns with this approach is that linker can't know what protocol are used.

For example,

1. ModuleA.swift has protocol P and conformance of P for String and String.foo method.

public protocol P {
  func foo()
}

extension String: P {
  func foo() {}
}

public func useP<T>(_ value: T) {
  (value as? P)?.foo()
}

2. main.swift import ModuleA but not use it.

import ModuleA

struct S: P {}
useP(S())

In this case, ModuleA's conformance and String.foo are included in produced binary because linker can't know whether the main.swift uses String: P conformance or not. (conformance metadata are always marked as llvm.used to reference them sequentially)

So, to achieve more reduction, we need to consider new optimization approach, like WMO or something LTO plugin that tells linker which protocols are used and omits unused conformances and functions.

[syrusakbary]

We are integrating SwiftWasm into Wasienv and realized that creating a file for a simple hello world is 42Mb (see attached file at the bottom of this comment).

You can follow the steps to compile easily Swift to Wasm easily here:

https://docs.wasmer.io/ecosystem/wasienv/compile-swift-to-wasm-wasi

How Wasienv works

This is what wasienv is doing under the hood:

• wasienv install-swift:
  1. Installs SwiftEnv (if is not already installed): git clone https://github.com/kylef/swiftenv.git (1)
  2. Installs SwiftWasm for SwitfEnv: swiftenv install https://github.com/swiftwasm/swift/releases/download/swift-wasm-DEVELOPMENT-SNAPSHOT-2020-03-08-a/swift-wasm-DEVELOPMENT-SNAPSHOT-2020-03-08-a-osx.tar.gz (2)
• wasiswiftc example.swift -o example.wasm:
  1. Calls wasienv which swiftc (to get the swiftc) (3)
  2. Calls swift -target wasm32-unknown-wasi -sdk ${WASI_SYSROOT} (4)

Perhaps we are doing something wrong?

Simple Example - 42Mb

The file that wasiswiftc example.swift -o example.wasm generated for the following is 42Mb.

if CommandLine.arguments.count < 2 {
   print("Hello, WASI!\n");
} else {
    print("Hello, \(CommandLine.arguments[1])\n");
}

example.wasm.zip

[kateinoigakukun]

@syrusakbary Thanks for trying Swift for WASM!

Wasienv looks doing right. swift-wasm-DEVELOPMENT-SNAPSHOT-2020-03-08-a toolchain is already old.

Please try the lastest toolchain from here https://github.com/swiftwasm/swift/releases
Recently I worked on reducing binary size, and succeeded to make it around 10MB.
I know it's still very fat, so I'm working on dieting more.

[syrusakbary]

Cool, let me update the release and I'll put my findings after.

Thanks for the quick response!

[syrusakbary]

Nice!

I just got it running!! The file is now much smaller as you pointed (9Mb), and takes much less time to execute.
I also published a new version of Wasienv that uses the latest release:

wasienv install-swift
wasiswiftc example.swift -o example.wasm

🎉

Timings

While trying things out, I've been doing an analysis of the example.wasm execution in different runtimes.

I just compared wasmer and wasmtime for running a generated file, and Wasmer timings are much better (about 10x faster runtime speed once the file is compiled)

Wasmtime: 1s to execute

➜  wasienv git:(master) ✗ time wasmtime example.wasm
Hello, WASI!

wasmtime example.wasm  0.97s user 0.10s system 98% cpu 1.087 total

Wasmer: .2s to execute (5x faster)

➜  wasienv git:(master) ✗ time wasmer example.wasm
Hello, WASI!

wasmer example.wasm  0.12s user 0.08s system 96% cpu 0.215 total

[kateinoigakukun]

I tried to use wasmer instead of wasmtime, it saved 13min for test execution! It's really good news!
#451

[MaxGraey]

wasmtime has opt flags --optimize and -O2 btw which not good described in docs. Did you try it?

[syrusakbary]

Yup, tried them both, but the timings are exactly the same than without those flags (mainly because -O2 is already the default in wasmtime).

You can also try them locally, following instructions here: https://docs.wasmer.io/ecosystem/wasienv/compile-swift-to-wasm-wasi

But the timings are: .2s for Wasmer and 1s for Wasmtime (with opt flags turned on)

[MaxDesiatov]

I can confirm that running wasm-strip from wabt and wasm-opt -Os from binaryen (in this order, wasm-opt seems to choke on raw binaries produced by swift build without having wasm-strip run on them at first) reduced an 11M binary to 4.9M. Interestingly enough the same code built in release mode with swift build -c release was only reduced to 5.1M 🤔

Both became 1.5M when zipped, that's not great, but not terrible either, almost order of magnitude better than the raw binary 😄

[MaxDesiatov]

Whoever is interested, the detailed plan for improvements is available in this Swift Forums thread:

Phase 1

• swift driver changes for providing flags for LTO
• swift driver changes to invoke the clang linker driver properly for enabling LTO
• swift frontend changes to emit LLVM BC instead of object files

Measurements for this phase would be interesting as they would identify the benefits of extra IPO of the IRGen (generic, non-language specific optimizations)

Phase 2

• changes to LLD to add support setting multiple compiler pipelines for LTO
• changes to swift frontend to support multiple pipelines
• changes to swift frontend to emit SIB instead of IRGen
• changes to LLD to setup a swift pipeline if a SIB is encountered, call back into swift to SILGen
• changes to the pipeline to do IPO with language specific considerations - e.g. late monomorphisation of generics, late devirt of calls

Parts of phase 1 are currently in review as swiftlang#31146.

[MaxDesiatov]

The progress report is now posted in this new Swift Forums thread.

[ferologics]

I tried reducing a 7.9mb wasm binary with the following results:

• wasm-strip => 32,91% reduction (5.3mb)
• wasm-opt -Os => 45.57% reduction (4.3mb)
• a combination of strip then opt => 56.96% reduction (3.4mb)
• a zip of the above measures 1.2mb => a 84,81% size reduction
  • this is not the worst, but still not suitable for embedded devices

There's a meta point on building Swift with size optimizations. Swift LTO -lto=llvm-full, which as @kateinoigakukun mentioned could potentially shave off another 20%, but I couldn't make it work (fails on linking). So the best I could do at the moment is compiling any swift with the -Osize flag.

Are there any other ways to reduce the binary size? @MaxDesiatov wrote a nice summary of what improvements could be made in future. With respect to that, I saw that stdlib dropped the ICU dependency. Is this change already in effect? What's the word on WASI custom allocators?

Attaching results for reference - SizeOptimization.zip

[MaxDesiatov]

I've posted an update about ICU here recently: #2411 (comment). Basically, we should phrase this as "removal of Unicode tables" instead of "removal of ICU". Even though ICU was removed as a dependency, Unicode tables were copied from it to stdlib itself, that shouldn't make a big difference in binary size if any.

As for build flags, there's ongoing work on swiftlang#42366, which may be relevant, but I haven't done any measurements with that myself.

WRT custom allocators and WASI-less builds, we could link with uSwift instead of stdlib, but we need to make #4374 fully work, and uSwift itself is super barebones and doesn't even support generics. This option is going to take most time and effort to implement when compared to other options, in my opinion.

[turbolent]

@MaxDesiatov How could one assist in reducing the binary size?

Is it possible to use uSwift with swiftwasm? If so, how does it work? If not, what work would be needed to make it possible?

Would a sponsorship help to implement this?

[MaxDesiatov]

One blocker that I know of for implementing a minimal standard library is swiftlang#63603. There may be more of course to be found when this one is resolved.

[turbolent]

Is it possible to use SwiftWasm with -experimental-hermetic-seal-at-link?

Also, it looks like Apple is working on "Embedded Swift": https://forums.swift.org/t/embedded-swift/67057

[turbolent]

-experimental-hermetic-seal-at-link requires -lto=llvm-full or -lto=llvm-thin.

Trying -lto=llvm-full or -lto=llvm-thin, with or without -experimental-hermetic-seal-at-link, on a Hello World program (swift package init --type executable), results in

$ swiftc -target wasm32-unknown-wasi -lto=llvm-full  -lswiftCore -static Sources/main.swift
wasm-ld: error: /home/bastian/Downloads/swift-wasm-5.8.0-RELEASE/usr/share/wasi-sysroot/lib/wasm32-wasi/libc.a(__main_argc_argv.o): undefined symbol: main
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
<unknown>:0: error: link command failed with exit code 1 (use -v to see invocation)

Any pointers @kateinoigakukun @MaxDesiatov?

[ephemer]

It indeed looks like LTO hermetic seal may be the most promising option here. I see that @kateinoigakukun actually did a talk about this option last year.

Is the work towards supporting this option for SwiftWasm blocked by something in particular or is it a case of it "just" needing more hands/eyes?

[kateinoigakukun]

As far as I know, there is no major blocker though we might need some minor adjustments specific to wasm. This is "just" a priority and human-resource problem. At this moment, upstreaming has a higher priority

[ephemer]

Ok, thanks so much for your continued work on this @kateinoigakukun. You are doing heroic work. どうもありがとうございます。

[turbolent]

@MaxDesiatov wrote up a great post about using the upcoming Embedded feature of Swift to build small binaries: https://forums.swift.org/t/some-feedback-from-my-short-experience-with-swiftwasm/69605/5

[arasan01]

I was curious how the combination of Embedded Swift and wasm would change this large file size issue. I was surprised to find that when I built the one shown in Wasm I/O at hand, it was an astonishingly small 101k bytes! This definitely indicates that swiftwasm can be adopted for some development, including deployment to wasm.

102810 swift-audio.wasm

Apple Example: https://github.com/apple/swift-for-wasm-examples
Related video: https://www.youtube.com/watch?v=6yxPavqB144