Adventures with Containerization #2: Fedora, httpd and virt-sandbox

In my previous post I used Docker to create a container running httpd.   With a Docker container, I could say I am half way to having a virtual machine: I get an isolated and self-contained OS installation in a filesystem which is separate from my host filesystem.  But the container itself is running natively on my host OS; we are not executing code under a virtual machine as happens using KVM or Xen.

In this post I want to compare Docker with another container tool: virt-sandbox, a set of tools created by Dan Walsh and Daniel Berrangé. With virt-sandbox we get an even lighter-weight container in which run httpd.

Dan and Daniel have done great writeups (and presentations!) covering these tools in much more details, e.g. here and here.   Here is a simple walkthrough, starting from a Fedora 20 install.

# yum install -q -y libvirt-sandbox httpd
# systemctl start libvirtd.service 
# virt-sandbox-service create -C -u httpd.service httpd-test2
Created sandbox container dir /var/lib/libvirt/filesystems/httpd-test2
Created unit file /etc/systemd/system/httpd-test2_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/httpd-test2/config/sandbox.cfg

I happen to already have a bridge set up on this machine, so passing the argument --network dhcp,source=default is sufficient to get networking up in the container.  Otherwise networking is more complicated to set up – the libvirt site has more details on configuration of networking.

Using Docker, a set of loopback filesystems got created behind the scenes to store the container images – Alex Larsson explains how this was implemented.  With virt-sandbox the container is going to be created using files stored directly in the host filesystem.  So I have a directory which is going to store container’s filesystem, which looks like any chroot environment might:

# tree -A -L 2 /var/lib/libvirt/filesystems/httpd-test2
├── etc
│   ├── fstab
│   ├── hostname
│   ├── httpd
│   ├── machine-id
│   ├── rc.d
│   ├── sysconfig
│   └── systemd
├── home
├── root
├── usr
│   └── lib
└── var
    ├── cache
    ├── lib
    ├── log
    ├── run -> /run
    ├── spool
    ├── tmp
    └── www

17 directories, 3 files

I’m going to do my “Hello World” in Lua this time.   virt-sandbox-service has already created a systemd service in my host, so I can skip that step.

# cat > /var/lib/libvirt/filesystems/httpd-test2/var/www/html/hello.lua <<EOF
function handle(r)
     r.content_type = "text/plain"
     r:puts("Hello Lua World!\n")
    return apache2.OK
# systemctl start httpd-test2_sandbox.service
# virt-sandbox-service execute httpd-test2 dhclient
# virt-sandbox-service execute httpd-test2 ip addr show dev eth0 | grep 'inet '
    inet brd scope global dynamic eth0
# curl
Hello Lua World!

I discovered I had to run dhclient manually in the container to get this working, which seems like a bug.  Otherwise, it worked!

What’s interesting here is that the httpd running inside the container is the actual httpd installation from my Fedora host OS – overlayed onto that /var/lib/libvirt/filesystems/httpd-test2 chroot-like directory mentioned above.   So, if I install php in the host, will it show up automatically in the container without any additional configuration?

# systemctl stop httpd-test2_sandbox.service
# yum install -q -y php
# systemctl start httpd-test2_sandbox.service
# virt-sandbox-service execute httpd-test2 dhclient
# echo '<?php echo "<h1>Hello World</h1>\n"; ?>' > /var/lib/libvirt/filesystems/httpd-test2/var/www/html/hello.php
# curl
<?php echo "<h1>Hello World</h1>\n"; ?>

Annoyingly, not; PHP is not activated and httpd served the source code.  This is because the container has a private copy of the /etc/httpd directory from my host, but that part of the filesystem doesn’t inherit any changes.  So to activate PHP in the container is a little more work:

# cp /etc/httpd/conf.modules.d/10-php.conf /var/lib/libvirt/filesystems/httpd-test2/etc/httpd/conf.modules.d/
# cp /etc/httpd/conf.d/php.conf /var/lib/libvirt/filesystems/httpd-test2/etc/httpd/conf.d/
# virt-sandbox-service execute httpd-test2 -- httpd -k restart
# curl
<h1>Hello World</h1>

Success!   Note that I only had to copy in the configuration, and the container has inherited the rest of the php package (the loadable module for Apache, etc) from the host.

In either this configuration or with Docker containers, to expose httpd containers to the world, there are two ways to start:

  1. Set up the containers to directly access a bridged Ethernet device.  Since each container requires its own IP address, this is only practical if you have as many IP addresses as you require containers.
  2. Configure the host as an HTTP proxy to the containers.

The httpd configuration in case (2) will look like any standard reverse proxy, except that you happen to be proxying to “backends” which are containers on the same physical machine; this can be as simple as a ProxyPass:

ProxyPass /myapp
ProxyPassReverse /myapp

… or as complicated as mod_rewrite RewriteRule using the [P] flag.


1 thought on “Adventures with Containerization #2: Fedora, httpd and virt-sandbox

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s