Installation

The operating system shipped with the switch is either a Fedora Remix with a Stable kernel, which tracks the stable tree or an out-of-the-box Ubuntu. Refer to the Installing a New Kernel page for more information regarding kernel versioning. It is possible to use other Linux distributions, including unmodified Fedora, which is available here.

By default, the switch comes pre-installed with ONIE, which "defines an open 'install environment' for bare metal network switches." However, PXE installation is also possible. This document describes both methods.

Note: Installing with PXE uninstalls ONIE, which might be required by other operating systems. Re-installing ONIE is not supported.

Table of Contents

1. Getting the Installation Files
2. Default Credentials
3. Installing with ONIE
   1. Preparing the Installation
   2. Performing the Installation
   3. Uninstalling
4. Installing with PXE
   1. Preparing the Installation
   2. Performing the Installation
5. Post Installation Steps (Ubuntu only)
6. Customizing the Installation
7. Troubleshooting
   1. Automatic Module Loading

Getting the Installation Files

Both the Fedora Remix and Ubuntu installation tarballs are available here. The following files are included in the provided Fedora Remix / Ubuntu tarballs:

• The operating system ISO file
• The operating system ONIE installer, if using ONIE
• An auto-install file (kickstart or preseed for Fedora Remix and Ubuntu, respectively).
• A matching firmware file

Default Credentials

User: root Password: switchdevftw

The root password can be easily changed by editing the rootpw command in the kickstart file or by editing the passwd/root-password command in the preseed file. It is highly recommended to do so!

Installing with ONIE

The ONIE project describes a variety of installation options. This section describes installation using an HTTP server accessible through the management port on the switch.

Preparing the Installation

The extracted contents of the ISO file, the ONIE installer and the kickstart file (e.g. install-onie.ks), should all be accessible through an HTTP server.

To extract the contents from the ISO file, you must first mount it. Run:

$ mount -o loop,ro -t iso9660 /path/file.iso /path/iso/directory

Then copy the contents over to the HTTP server directory:

$ cp -r /path/iso/directory /path/http/server

Finally, edit the auto-install file to point to the HTTP server:

• In the kickstart file, edit the url command to point to the extracted ISO directory.
• In the preseed file, edit the mirror/http/hostname and mirror/http/directory values to point to the HTTP server and the directory within the server, respectively.

Note: The ONIE installer assumes some things about the location and name of the auto-install. In the provided Fedora Remix ONIE package, it is assumed that the kickstart file is named install-onie.ks and that it is located in a directory called ks, placed next to the ONIE installer. For example:

├── ks
│   ├── install-onie.ks
├── mlxsw-4.6.0-0.rc7.git1.1-ONIE-installer.bin

And in the provided Ubuntu ONIE package, it is assumed that the file is named preseed.cfg and is located in directory called preseed placed next to the ONIE installer. For example:

├── preseed
│   ├── preseed.cfg
├── mlxsw-UBUNTU16-ONIE-installer.bin

Performing the Installation

Reboot the switch and select ONIE: Resuce in GRUB menu as the discovery phase is unnecessary:

 +----------------------------------------------------------------------------+
 | ONIE: Install OS                                                           |
 |*ONIE: Rescue                                                               |
 | ONIE: Uninstall OS                                                         |
 | ONIE: Update ONIE                                                          |
 | ONIE: Embed ONIE                                                           |
 |                                                                            |

In ONIE prompt type:

ONIE:/ # onie-nos-install http://server/provided-ONIE-installer.bin

From this point on, the installation is completely automatic. Once it is complete, the switch reboots into the newly installed system.

Uninstalling

Reboot the switch and select ONIE in the GRUB menu:

      Generic (4.6.0-0.rc7.git1.1.fc25.x86_64) 23 (Generic)
      Generic (0-rescue-1c9ab09884c946649fa3e1688040abde) 23 (Generic)
      ONIE

Then select ONIE: Uninstall OS. This removes the partitions used by the OS, but keeps ONIE installed, so that a new OS can be installed using it.

Installing with PXE

The instructions below assume that a configured PXE server is already available on the network and that the installation files are available over HTTP. However, other methods are possible.

Preparing the Installation

As with the ONIE installation, the extracted contents of the ISO file and the kickstart file (e.g. install-pxe.ks) should all be accessible via an HTTP server.

To extract the contents from the ISO file, first mount it. Run:

$ mount -o loop,ro -t iso9660 /path/file.iso /path/iso/directory

And copy the contents over to the HTTP server directory. Run:

$ cp -r /path/iso/directory /path/http/server

In addition, the kernel executable and the initrd image should be copied over to the PXE server. Run:

$ cp /path/iso/directory/images/pxeboot/{initrd.img,vmlinuz} /tftpboot/directory/

Next, edit the auto-install file to point to the HTTP server:

• In the kickstart file, edit the url command to point to the extracted ISO directory.
• In the preseed file, edit the mirror/http/hostname and mirror/http/directory values to point to the HTTP server and the directory within the server, respectively.

Finally, add the following lines to the /tftpboot/pxelinux.cfg/default file on the PXE server:

LABEL MY_LABEL
        kernel directory/vmlinuz
        append initrd=directory/initrd.img inst.ks=http://server/path/install-pxe.ks console=ttyS0,115200n8 modprobe.blacklist=mlxsw_pci

In case using Fedora Remix, or for Ubuntu:

LABEL MY_LABEL
        kernel directory/vmlinuz
        append initrd=directory/initrd.gz console=ttyS0,115200n8 auto=true priority=critical preseed/url=http://server/path/preseed.cfg netcfg/choose_interface=auto modprobe.blacklist=mlxsw_pci

Performing the Installation

Make sure network boot is enabled in the BIOS (Ctrl+b) and reboot the switch. Once up, select the recently configured PXE label (e.g. MY_LABEL) and the automated installation will then begin.

Once done, the switch will reboot into the newly installed system.

Post Installation Steps (Ubuntu only)

After the Ubuntu installation is complete, it's recommended to add official Ubuntu repositories from which packages can be obtained. See this page for more details.

The Ubuntu provided kernel and iproute2 packages might be older than the latest stable upstream counterparts. Therefore, if new features are required, it's recommended to update these packages. Please refer to Installing a New Kernel page for more information.

Customizing the Installation

As explained above, the installation is automated using auto-install files. These files can be easily edited to provide different answers during the installation, and to select different packages to install by default. Please refer to the official RedHat kickstart documentation and Ubuntu preseed documentation for more info.

For Fedora, it is possible to generate different installation ISOs with a different package selection by using mock and pungi as explained in Fedora's wiki.

Creating an ONIE-compatible installer from an ISO image is explained in ONIE's Github repository.

Troubleshooting

Automatic Module Loading

If the driver is not loaded following a reboot, then it's possible the driver isn't included in the initramfs. To verify that, run:

$ lsinitrd /boot/initramfs-$(uname -r).img | grep mlxsw
drwxr-xr-x   2 root     root            0 Feb  3 22:28 usr/lib/modules/4.7.0-0.rc4.git2.2.fc25.x86_64/kernel/drivers/net/ethernet/mellanox/mlxsw
-rw-r--r--   1 root     root        20132 Feb  3 22:28 usr/lib/modules/4.7.0-0.rc4.git2.2.fc25.x86_64/kernel/drivers/net/ethernet/mellanox/mlxsw/mlxsw_core.ko.xz
-rw-r--r--   1 root     root        55976 Feb  3 22:28 usr/lib/modules/4.7.0-0.rc4.git2.2.fc25.x86_64/kernel/drivers/net/ethernet/mellanox/mlxsw/mlxsw_spectrum.ko.xz
-rw-r--r--   1 root     root        18560 Feb  3 22:28 usr/lib/modules/4.7.0-0.rc4.git2.2.fc25.x86_64/kernel/drivers/net/ethernet/mellanox/mlxsw/mlxsw_switchx2.ko.xz

If the output isn't similar to the above, then add the missing drivers to the initramfs. Assuming mlxsw_spectrum is missing:

$ dracut -f --add-drivers mlxsw_spectrum /boot/initramfs-$(uname -r).img