Rust Performance Benchmark

Benchmark suite for Rust This harness is a command-line tool to automatically bench, collect and process benchmark results of compile-time/runtime benchmarks in JSON format. It can also draw flamegraphs to show the rust compiler/programs built by the rust compiler's runtime call stack.

Dependencies

Before starting the harness, make sure your perf tool and flamegraph is ready to run.

1. The following link may help if you have trouble setting up perf tool:

https://blog.eastonman.com/blog/2021/02/use-perf/

2. Install flamegraph:

> cargo install flamegraph

3. Libraries needed on Linux

> apt install libfontconfig1-dev

4. Relative dependancies python3 kaleido, matplotlib (can be installed with pip...

pip install -U kaleido matplotlib plotly

or with conda...)

conda install -c conda-forge python-kaleido matplotlib plotly

Attention: The harness & flamegraph will fail to collect some of the specific data we need if you are going to run it in a virtual environment.

Build up

> cargo build --release

Commands

USAGE:
    collector <SUBCOMMAND>

OPTIONS:
    -h, --help       Print help information
    -V, --version    Print version information

SUBCOMMANDS:
    bench_local            
    bench_runtime_local    Benchmarks the performance of programs generated by a local rustc
    help                   Print this message or the help of the given subcommand(s)

Running compile-time benchmark will tell how much time and space it costs for the given rustc compiler to compile the benchmarks' source code with Check, Debug and Release profiles in a non-incremental scenario.

Command for running compile-time benchmark:

USAGE:
    collector bench_local [OPTIONS] <RUSTC>

ARGS:
    <RUSTC>    The path to the local rustc to measure

OPTIONS:
        --cargo <CARGO>              The path to the local Cargo to use
    -h, --help                       Print help information
        --id <ID>                    Identifier to associate benchmark results with
        --iterations <ITERATIONS>    The number of iterations to do for each benchmark [default: 2]
        --out-dir <OUT_DIR>          The path of output file [default: results/]

Running runtime benchmark will tell how much time and space it costs for the benchmark programs to run with some specific input, which are compiled by the given rustc compiler with Release profile.

Command for running runtime benchmark:

USAGE:
    collector bench_runtime_local [OPTIONS] <RUSTC>

ARGS:
    <RUSTC>    The path to the local rustc to measure

OPTIONS:
        --cargo <CARGO>              The path to the local Cargo to use
    -h, --help                       Print help information
        --id <ID>                    Identifier to associate benchmark results with
        --iterations <ITERATIONS>    How many iterations of each benchmark should be executed
                                     [default: 5]
        --out-dir <OUT_DIR>          The path of output file [default: results/]

License

Rust Performance Benchmark is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.