binary.decodeCode cause Memory leak ? #1600
Comments
mathetake
added
question
|
What do you mean by “continuous initialization”? Do you want to be more specific when raising an issue like with a reproducible code snipe? |
|
If you are not closing the compiled module, sure it will increase the memory usage and that’s totally expected |
|
“continuous initialization” , I will call runtime.Instantiate when the function loops dead. and i call api.Module.Close to closing the compiled module, but binary.decodeCode continuously growing. Looking forward to your reply very much |
|
@mathetake Looking forward to your reply very much |
|
Can you provide us with a MCVE? Ideally a repo we can checkout and commands to run. This is how we run wazero ourselves, and we don't see the issue, so we need to figure out (not guess) what's different. |
this is MCVE of mock binary.decodeCode cause memory leak package main
import (
"context"
_ "embed"
"errors"
"fmt"
"log"
"net/http"
_ "net/http/pprof"
"os"
"runtime"
"github.com/tetratelabs/wazero"
"github.com/tetratelabs/wazero/api"
)
// greetWasm was compiled using `zig build`
//
//go:embed testdata/greet.wasm
var greetWasm []byte
// main shows how to interact with a WebAssembly function that was compiled from Zig.
//
// See README.md for a full description.
func main() {
runtime.GOMAXPROCS(1)
runtime.SetMutexProfileFraction(1)
runtime.SetBlockProfileRate(1)
go func() {
if err := http.ListenAndServe(":6060", nil); err != nil {
log.Fatal(err)
}
os.Exit(0)
}()
for {
ctx, mod, _ := run()
mod.Close(ctx)
}
}
func run() (context.Context, api.Module, error) {
// Choose the context to use for function calls.
ctx := context.Background()
// Create a new WebAssembly Runtime.
r := wazero.NewRuntime(ctx)
defer r.Close(ctx) // This closes everything this Runtime created.
// Instantiate a Go-defined module named "env" that exports a function to
// log to the console.
_, err := r.NewHostModuleBuilder("env").
NewFunctionBuilder().WithFunc(logString).Export("log").
Instantiate(ctx)
if err != nil {
return nil, nil, err
}
// Instantiate a WebAssembly module that imports the "log" function defined
// in "env" and exports "memory" and functions we'll use in this example.
mod, err := r.InstantiateWithConfig(ctx, greetWasm,
wazero.NewModuleConfig().WithStdout(os.Stdout).WithStderr(os.Stderr))
if err != nil {
return nil, nil, err
}
// Get references to WebAssembly functions we'll use in this example.
greet := mod.ExportedFunction("greet")
greeting := mod.ExportedFunction("greeting")
malloc := mod.ExportedFunction("malloc")
free := mod.ExportedFunction("free")
// Let's use the argument to this main function in Wasm.
name := "test"
nameSize := uint64(len(name))
// Instead of an arbitrary memory offset, use Zig's allocator. Notice
// there is nothing string-specific in this allocation function. The same
// function could be used to pass binary serialized data to Wasm.
results, err := malloc.Call(ctx, nameSize)
if err != nil {
return nil, nil, err
}
namePtr := results[0]
if namePtr == 0 {
return nil, nil, errors.New("malloc failed")
}
// We have to free this pointer when finished.
defer free.Call(ctx, namePtr, nameSize)
// The pointer is a linear memory offset, which is where we write the name.
if !mod.Memory().Write(uint32(namePtr), []byte(name)) {
return nil, nil, fmt.Errorf("Memory.Write(%d, %d) out of range of memory size %d",
namePtr, nameSize, mod.Memory().Size())
}
// Now, we can call "greet", which reads the string we wrote to memory!
_, err = greet.Call(ctx, namePtr, nameSize)
if err != nil {
return nil, nil, err
}
// Finally, we get the greeting message "greet" printed. This shows how to
// read-back something allocated by Zig.
ptrSize, err := greeting.Call(ctx, namePtr, nameSize)
if err != nil {
return nil, nil, err
}
greetingPtr := uint32(ptrSize[0] >> 32)
greetingSize := uint32(ptrSize[0])
// The pointer is a linear memory offset, which is where we write the name.
if bytes, ok := mod.Memory().Read(greetingPtr, greetingSize); !ok {
return nil, nil, fmt.Errorf("Memory.Read(%d, %d) out of range of memory size %d",
greetingPtr, greetingSize, mod.Memory().Size())
} else {
fmt.Println("go >>", string(bytes))
}
return nil, mod, nil
}
func logString(_ context.Context, m api.Module, offset, byteCount uint32) {
buf, ok := m.Memory().Read(offset, byteCount)
if !ok {
log.Panicf("Memory.Read(%d, %d) out of range", offset, byteCount)
}
fmt.Println(string(buf))
}can see at http://localhost:6060/debug/pprof/heap and top github.com/tetratelabs/wazero/internal/wasm/binary.decodeCode‘s memory is constantly growing |
|
I hope to receive your reply. Thank you very much |
|
Now we have supposedly a repro so reopening it |
Thank you very much for your attention |
Fixes #1600. Signed-off-by: Nuno Cruces <[email protected]>
|
Confirmed the leak, actually it's sufficient to change: wazero/examples/allocation/zig/greet.go Lines 23 to 27 in 1f8c908 To (add infinite loop): func main() {
for {
if err := run(); err != nil {
log.Panicln(err)
}
}
}And run the sample. |
ncruces
added
bug
|
Thank you for your contributions. When will a new release be produced @mathetake |
|
@panchen66 there are some work in progress still in the PR queue. Typically, we release once per month, but have more often sometimes. Right now, we have code already landed that has changed enough to be 1.4.0 (vs a patch). I think the following should happen before 1.4.0 (my opinion)
Meanwhile, if you need to use this patch, depend on the main branch. |
|
this is now in v1.4.0 ! |
Describe the bug
A clear and concise description of what the bug is.
I suspect make binary.decodeCode cause Memory leak
To Reproduce
I will reinitialize the instance when I catch wasm execution exceptions, but after continuous initialization, I find that memory continues to grow
I debugged the shutdown code and it seems that there was no recycling of the source wasm object. How should I recycle this object
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the relevant information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: