containerbuild.sh taking forever in vagrant

[mikemadden42]

I'm using the following vagrant box to build a centos 6 docker container.

http://cloud.centos.org/centos/7/vagrant/x86_64/images/CentOS-7-x86_64-Vagrant-1611_01.VirtualBox.box

The script isn't outputing any errors, but it's still running 24+ hours later. What is the normal build time? Is there any way I can debug the process?

[vagrant@localhost docker]$ sudo ./containerbuild.sh centos-6.ks
The Build root, /var/tmp/containers/20170110/centos-6, already exists.  Would you like to remove it? [y/N]
y
2017-01-10 19:09:17,742: disk_size = 3GB
2017-01-10 19:09:17,742: disk_img = /var/tmp/centos-6-docker.tar.xz
2017-01-10 19:09:17,743: install_log = /tmp/virt-install.log
mount: /dev/loop0 is write-protected, mounting read-only
WARNING  KVM acceleration not available, using 'qemu'
WARNING  No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.

Starting install...
Retrieving file vmlinuz...                                  | 4.1 MB  00:00
Retrieving file initrd.img...                               |  39 MB  00:00
Allocating 'diskpzc4DO.img'                                 | 3.0 GB  00:00
Creating domain...                                          |    0 B  00:01
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
...............

[jperrin]

In this instance you're doing nested virt using qemu/kvm inside virtualbox, which will slow it down some. Additionally it would likely matter how much memory you've allocated to the vbox vagrant image as you may be swapping.

[mikemadden42]

Thanks. The vagrant box has 512 MB of memory & 1 virtual CPU. Do you think I should increase the resources for the vm?

[jperrin]

Almost certainly. I believe the 'default' value kvm uses is 1 or 2G of ram allocated to the vm it will use to build the container tarball.

[mikemadden42]

I'm retrying the build process with 4 GB of memory for the vm. So far I'm not seeing high memory usage or swapping.

[vagrant@localhost ~]$ free -mt
              total        used        free      shared  buff/cache   available
Mem:           3791         207        2642           9         940        3317
Swap:          1535           0        1535
Total:         5327         207        4178

[vagrant@localhost docker]$ time sudo ./containerbuild.sh centos-6.ks 2>&1 | tee build.log
The Build root, /var/tmp/containers/20170111/centos-6, already exists.  Would you like to remove it? [y/N]
y
2017-01-11 19:36:16,308: disk_size = 3GB
2017-01-11 19:36:16,309: disk_img = /var/tmp/centos-6-docker.tar.xz
2017-01-11 19:36:16,309: install_log = /tmp/virt-install.log
mount: /dev/loop0 is write-protected, mounting read-only
WARNING  KVM acceleration not available, using 'qemu'

Starting install...
Retrieving file vmlinuz...                                  | 4.1 MB  00:00
WARNING  No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.
Retrieving file initrd.img...                               |  39 MB  00:00
Allocating 'diskZrtiAW.img'                                 | 3.0 GB  00:00
Creating domain...                                          |    0 B  00:00
Domain installation still in progress. You can reconnect to
the console to complete the installation process.
...........................

[jperrin]

So you're slowed down at least because nested virt. qemu by itself will be quite slow. You could optionally use another terminal/console (as root) to virsh list to get the virt machine id-number, and then virsh console <id-number> to see what it's doing and what's taking up time.

[mikemadden42]

I'm not seeing any output on the console.

[vagrant@localhost ~]$ sudo virsh list
 Id    Name                           State
----------------------------------------------------
 1     LiveOS-f32bb45b-9049-4b78-8d56-12610a374446 running

[vagrant@localhost ~]$ sudo virsh console 1
Connected to domain LiveOS-f32bb45b-9049-4b78-8d56-12610a374446
Escape character is ^]

[jperrin]

if it's as slow as you're describing, you'll likely have to be patient. That script wasn't really written with the idea of doing it inside virt.

[mikemadden42]

I switched to a physical CentOS 7 server, and containerbuild.sh completed in around 5 minutes.

[jperrin]

Thanks for the feedback. That's about the normal expected time (within reason) that I'd expect.

If you find a way to make it work better via vagrant or other virt, I'd certainly be open to pulling in changes.

[beddari]

@jperrin Interesting. Been wanting to ask, what's the reasoning behind using a full kickstart/anaconda run to build versus a process based on rpm --root / yum --installroot like the scripts in docker contrib?

[jperrin]

@beddari similarity to generating the other distro media types (so you don't need to really know 8 different tools). I find it easier/simpler to troubleshoot the kickstart vs potential chroot issues, shell escapes, etc.

Kickstart + vm should work the same on any distro, so I don't have to deal with rpm 5 in ubuntu/debian doing something different than rpm on centos or fedora. It's also a clean, verifiable build every time, with no possibility for host pollution from either direction.

Most importantly (for me) it was easy (for me) to throw together and crank out tarballs quickly.