-
Notifications
You must be signed in to change notification settings - Fork 66
Documents
NUSE can choose variety of underlying network interface, called virtual interface (vif), to communicate with the other nodes/processes.
By default, a raw socket (AF_PACKET) is used for vif. You need a root privilege with this vif.
sudo NUSECONF=nuse.conf ./nuse ping www.google.com
You need to build dpdk sdk first.
make dpdk-sdk ARCH=lib DPDK=yes
then build with the SDK for DPDK channel of NUSE.
make defconfig ARCH=lib
make library ARCH=lib OPT=no DPDK=yes
Additional DPDK configuration is needed, hugepage setup, make an interface DPDK mode.
./dpdk/tools/setup.sh
sudo insmod dpdk/build/kmod/igb_uio.ko
sudo ./dpdk/tools/dpdk_nic_bind.py -b igb_uio 0000:02:01.0
If you want to use nuse with dpdk, interface names of dpdk on nuse.conf must follow the dpdk%d format. The digit X of dpdkX indicates the dpdk port number. An example is shown below.
nuse1:net-next-nuse % cat nuse-dpdk.conf
interface dpdk0
address 172.16.0.1
netmask 255.255.255.0
macaddr 00:01:01:01:01:01
viftype DPDK
interface dpdk1
address 172.16.1.1
netmask 255.255.255.0
macaddr 00:01:01:01:01:02
viftype DPDK
nuse1:net-next-nuse % ./dpdk/tools/dpdk_nic_bind.py --status
Network devices using DPDK-compatible driver
============================================
0000:0a:00.0 'Ethernet 10G 2P X520 Adapter' drv=igb_uio unused=
0000:0a:00.1 'Ethernet 10G 2P X520 Adapter' drv=igb_uio unused=
~ snip ~
This set up means "use PCI 0000:0a:00.0 as dpdk0 and PCI 0000:0a:00.1 as dpdk1".
for a build with netmap,
make library ARCH=lib OPT=no NETMAP=yes
when viftype is PIPE, pipepath attribute must be specified. PIPE vif creates named pipe, and host stack applications can communicate with nuse processes through the FIFO pipe. When a nuse process sends a packet via a PIPE vif, a host stack application can receive the packet from the pipe using read system call, and the application can send packets using write system call.
# viftype PIPE example
interface pipe0
address 192.168.0.11
netmask 255.255.255.0
macaddr 00:01:01:01:01:11
viftype PIPE
pipepath /tmp/nuse-1-pipe0
(to be added)
NUSE relies on the system call proxy implemented by rumpkernel, which are built via submodule of git.
You need to enable and pull the repository, as following commands.
git submodule init
git submodule update arch/lib/rump
Then, build NUSE by make library ARCH=lib, as usual.
In order to an IP address of the process running on NUSE, you can use ip command (iproute2) in an external process.
% sudo NUSECONF=./work.orig/nuse.conf.local ./nuse sleep 100
<5>Linux version 3.18.0+ (tazaki@zakvm-f21) (gcc version 4.9.2 20141101 (Red Hat 4.9.2-1) (GCC) ) #3 Mon Feb 2 15:18:51 JST 2015
<6>NET: Registered protocol family 16
<6>NET: Registered protocol family 2
<6>TCP established hash table entries: 512 (order: 0, 4096 bytes)
<6>TCP bind hash table entries: 512 (order: 0, 4096 bytes)
(snip)
nuse syscall proxy start at unix:///tmp/rump-server-nuse.57772
At the last line, you will see the PATH of rumpserver, /tmp/rump-server-nuse.57772 in this case, which is the entrance for external processes of inter/remote process communication (IPC/RPC).
An external process can look at the status of NUSE process, as shown in the following example.
% export LD_PRELOAD=./arch/lib/libnuse-hijack.so
% export RUMPHIJACK=socket=all
% export RUMP_SERVER=unix:///tmp/rump-server-nuse.57772
% ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default
link/ipip 0.0.0.0 brd 0.0.0.0
3: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN group default
link/gre 0.0.0.0 brd 0.0.0.0
4: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1462 qdisc noop state DOWN group default qlen 1000
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
5: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default
link/sit 0.0.0.0 brd 0.0.0.0
6: ip6tnl0@NONE: <NOARP> mtu 1452 qdisc noop state DOWN group default
link/tunnel6 :: brd ::
7: eno16777736: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
link/ether 00:01:01:01:01:02 brd ff:ff:ff:ff:ff:ff
inet 192.168.209.39/24 brd 192.168.209.255 scope global eno16777736
valid_lft forever preferred_lft forever
inet6 fe80::201:1ff:fe01:102/64 scope link
valid_lft forever preferred_lft forever
All the IP addresses are independent from host environment, but configured by nuse.conf, exposed by rumpserver listened at /tmp/rump-server-nuse.57772.
Note that, the above example is helped by hijack library, which was specified by LD_PRELOAD and RUMPHIJACK environmental variables. This is not perfect at this moment, some of calls like sysctl(2), if_nametoindex(3) are not intercepted. Need more effort on that.