SimNet is a tool to simulate a decentralized application using the cloud to host the simulation nodes. It provides multiple configurations to affect the topology so that specific link between two nodes can have a delay or a loss of packets for instance.
The tool is designed to work with three phases:
- Deployment: everything the simulation needs is deployed to the cloud.
- Execution: the actual simulation is performed and statistics are written to a JSON file.
- Cleaning: it wipes everything in scope to the simulation.
Thoses phases are designed to be independant so that they can be run separatly which means that simulations can be executed multiple times without deploying.
The environment the simulation is running on depends on the chosen strategy. Two are currently available: Kubernetes and Docker.
It is completly interchangeable so that you can test your simulation locally with Docker and a small number of nodes. Then the simulation can be deployed on Kubernetes with a bigger amount of nodes.
When a simulation is deployed to a Kubernetes cluster, each instance of the application uses a POD that will contain the application Docker container alongside with a monitor container that will gather data for the statistics.
One POD will also be used to deploy a router that will simply run OpenVPN so
that the simulation can open a tunnel to the cluster network, on the simnet-router
pod and thus make requests to the simnet nodes. Simnet uses a NodePort to
communicate with the Kubernetes node running the simnet-router Pod. If you
are using a public cloud provider, please ensure the NodeIP is publically
accessible and UDP traffic is allowed on the cluster.
Simulations with the Docker strategy will interact with the local Docker setup to create one container per Node. Each of those containers will need another temporary container to configure the network emulation (latencies, etc...) but they will terminate before the simulation begins and they are booted one after the other.
Some strategies like Kubernetes might need additionnal containers to work. These are built independently using the automated build of DockerHub.
| Router | Router Init | Monitor |
|---|---|---|
 |
 |
 |
Docker images for the daemons have their own version that will trigger a build for each new version. The master branch will also build its own tag for each new commit.
Dockerhub is configured to automatically create the following tags:
latestfor the master branchx.y.zfor eachdaemon-vx.y.ztag on the repository
To run simulations with Kubernetees, OpenVPN needs to be installed.
By default, simnet looks for OpenVPN in /usr/local/opt/openvpn/sbin/openvpn (that should be the case for example if you used Homebrew on MacOS to install OpenVPN), but you can specify a different path with the -vpn CLI option.
For MacOS users, OpenVPN is also required to run Docker simulation (this is due to MacOS runing docker via a VM).
A basic template with the Docker strategy is shown below:
type simRound struct {}
func (s simRound) Execute(ctx context.Context) error {
return nil
}
func main() {
engine, err := docker.NewStrategy(options...)
if err != nil {
panic(err)
}
sim := simnet.NewSimulation(simRound{}, engine)
err = sim.Run(os.Args)
if err != nil {
panic(err)
}
}A few number of options is necessary to run an application.
options := []sim.Option{
sim.WithTopology(
net.NewSimpleTopology(3, 25),
),
sim.WithImage(
"nginx", // DockerHub image
nil, // CMD if needed
nil, // ARGS if needed
sim.NewTCP(8080),
),
}Finally, the simulation tool provides a bunch of command-line arguments so that
it is easier to run the different phases. Assuming a main.go, use the
following:
# Runs the simulation from A to Z.
go run main.go
# ... or do it step by step
go run main.go -do-deploy
# This can be done multiple times but be aware that statistics will be
# overwritten.
go run main.go -do-execute
# Important to reduce the cost
go run main.go -do-cleanFirst you need to install the plot tool
go get -u go.dedis.ch/simnet/metrics/simplotThen to draw a plot
simplot graph -output plot-tx.png tx
simplot graph -output plot-rx.png rx
simplot graph -output plot-cpu.png cpu
simplot graph -output plot-mem.png memYou can always use the -h option (for example simplot -h, or simplot graph -h) to see the full list of options and commands available, such as computing the max or average.
The context passed to the round execution contains some pieces of information that can be useful when running a simulation.
nodes := ctx.Value(sim.NodeInfoKey{}).([]sim.NodeInfo)
addr := nodes[0].AddressA file mapper can be specifiec in the strategy options. This file will be read on each application node and stored in the execution context.
files := ctx.Value(sim.FilesKey("my.conf")).(sim.Files)
for ident, file := range files {
...
}See the skipchain example for more details.
This section describes a setup using Minikube to run a local Kubernetes cluster where the simulation can be deployed.
Caveat: Minikube VM does not have the sch_netem module which means Chaos testing does not work out of the box and requires a few steps more.
Follow the instructions here then
# Start the cluster.
minikube start --docker-opt bip=172.18.0.1/16
# In case of error, you can first try to delete the previous cluster.
minikube delete
# After the cluster has started, you can monitor with the dashboard.
minikube dashboardA local change to the daemon images can be used by building the images inside minikube and forcing it to use the local ones. You will need to add "Never" for the image pull policy in the kubernetes deployments.
# Set the docker environment variables for this session.
eval $(minikube docker-env)
cd $SIMNET_DIR
make build_monitor
make build_routerMininet will now have the required images in its local images store. The simulation image still need to be deployed on DockerHub.
The simulation is now ready to be run.
cd $SIMNET_DIR
make EXAMPLE="skipchain" run # be patient, time for a coffee !
# or make run to run the default simulation.
# In case the simulation fails and cannot clean the cluster correctly, you
# can force the deletion.
make cleanThis section describes the steps to get a local cluster that supports Chaos by enabling the right kernel module in the Minikube ISO.
cd $HOME/workspace # ... or a different folder
git clone [email protected]:kubernetes/minikube.git
cd minikube
make buildroot-image
make out/minikube.iso
make linux-menuconfig
# Inside the Kconfig menu, you will to navigate to
# --> Networking support
# --> Networking options
# --> QoS and/or fair queuing
# --> Network Emulator (NETEM) + HTB
# Press space until you have <*> for NETEM
# Finally, save and close the menu
make out/minikube.iso
./out/minikube delete
./out/minikube start --iso-url=file://$(pwd)/out/minikube.isoYou can check that you are good to go by checking that the Minikube VM has the module enabled.
./out/minikube ssh
$ modprobe sch_netem # should not display an error
This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825377.