Category Archives: Containers

Adventures with Containerization #2: Fedora, httpd and virt-sandbox

In my previous post I used Docker to create a container running httpd.   With a Docker container, I could say I am half way to having a virtual machine: I get an isolated and self-contained OS installation in a filesystem which is separate from my host filesystem.  But the container itself is running natively on my host OS; we are not executing code under a virtual machine as happens using KVM or Xen.

In this post I want to compare Docker with another container tool: virt-sandbox, a set of tools created by Dan Walsh and Daniel Berrangé. With virt-sandbox we get an even lighter-weight container in which run httpd.

Dan and Daniel have done great writeups (and presentations!) covering these tools in much more details, e.g. here and here.   Here is a simple walkthrough, starting from a Fedora 20 install.

# yum install -q -y libvirt-sandbox httpd
# systemctl start libvirtd.service 
# virt-sandbox-service create -C -u httpd.service httpd-test2
Created sandbox container dir /var/lib/libvirt/filesystems/httpd-test2
Created unit file /etc/systemd/system/httpd-test2_sandbox.service
Created sandbox config /etc/libvirt-sandbox/services/httpd-test2/config/sandbox.cfg

I happen to already have a bridge set up on this machine, so passing the argument --network dhcp,source=default is sufficient to get networking up in the container.  Otherwise networking is more complicated to set up – the libvirt site has more details on configuration of networking.

Using Docker, a set of loopback filesystems got created behind the scenes to store the container images – Alex Larsson explains how this was implemented.  With virt-sandbox the container is going to be created using files stored directly in the host filesystem.  So I have a directory which is going to store container’s filesystem, which looks like any chroot environment might:

# tree -A -L 2 /var/lib/libvirt/filesystems/httpd-test2
├── etc
│   ├── fstab
│   ├── hostname
│   ├── httpd
│   ├── machine-id
│   ├── rc.d
│   ├── sysconfig
│   └── systemd
├── home
├── root
├── usr
│   └── lib
└── var
    ├── cache
    ├── lib
    ├── log
    ├── run -> /run
    ├── spool
    ├── tmp
    └── www

17 directories, 3 files

I’m going to do my “Hello World” in Lua this time.   virt-sandbox-service has already created a systemd service in my host, so I can skip that step.

# cat > /var/lib/libvirt/filesystems/httpd-test2/var/www/html/hello.lua <<EOF
function handle(r)
     r.content_type = "text/plain"
     r:puts("Hello Lua World!\n")
    return apache2.OK
# systemctl start httpd-test2_sandbox.service
# virt-sandbox-service execute httpd-test2 dhclient
# virt-sandbox-service execute httpd-test2 ip addr show dev eth0 | grep 'inet '
    inet brd scope global dynamic eth0
# curl
Hello Lua World!

I discovered I had to run dhclient manually in the container to get this working, which seems like a bug.  Otherwise, it worked!

What’s interesting here is that the httpd running inside the container is the actual httpd installation from my Fedora host OS – overlayed onto that /var/lib/libvirt/filesystems/httpd-test2 chroot-like directory mentioned above.   So, if I install php in the host, will it show up automatically in the container without any additional configuration?

# systemctl stop httpd-test2_sandbox.service
# yum install -q -y php
# systemctl start httpd-test2_sandbox.service
# virt-sandbox-service execute httpd-test2 dhclient
# echo '<?php echo "<h1>Hello World</h1>\n"; ?>' > /var/lib/libvirt/filesystems/httpd-test2/var/www/html/hello.php
# curl
<?php echo "<h1>Hello World</h1>\n"; ?>

Annoyingly, not; PHP is not activated and httpd served the source code.  This is because the container has a private copy of the /etc/httpd directory from my host, but that part of the filesystem doesn’t inherit any changes.  So to activate PHP in the container is a little more work:

# cp /etc/httpd/conf.modules.d/10-php.conf /var/lib/libvirt/filesystems/httpd-test2/etc/httpd/conf.modules.d/
# cp /etc/httpd/conf.d/php.conf /var/lib/libvirt/filesystems/httpd-test2/etc/httpd/conf.d/
# virt-sandbox-service execute httpd-test2 -- httpd -k restart
# curl
<h1>Hello World</h1>

Success!   Note that I only had to copy in the configuration, and the container has inherited the rest of the php package (the loadable module for Apache, etc) from the host.

In either this configuration or with Docker containers, to expose httpd containers to the world, there are two ways to start:

  1. Set up the containers to directly access a bridged Ethernet device.  Since each container requires its own IP address, this is only practical if you have as many IP addresses as you require containers.
  2. Configure the host as an HTTP proxy to the containers.

The httpd configuration in case (2) will look like any standard reverse proxy, except that you happen to be proxying to “backends” which are containers on the same physical machine; this can be as simple as a ProxyPass:

ProxyPass /myapp
ProxyPassReverse /myapp

… or as complicated as mod_rewrite RewriteRule using the [P] flag.

Adventures with Containerization: Fedora, Docker and httpd

I have finally got around to experimenting with Docker this week.   Thanks to the hard work of many wonderful hackers, since late last year we’ve had Docker packages working in Fedora (under the package name “docker-io”).  Matt Miller has also been pushing Fedora images to the Docker index.

How can we use Apache httpd with Docker in Fedora?    One simple use-case might be serving a mostly static web site which you want to isolate inside a Docker container.  This is what you might aim for if you have a server hosting a bunch of different web sites for unrelated customers, but you want isolation between those servers.  If one httpd gets compromised, you don’t want them all to go down.

Most of the hard work is already done here.  We need two things:

  1. A Docker image which can launch httpd, serving our web site.
  2. A way to control that Docker container from the host side.

A Dockerfile is a script which describes how to create a docker image.  Here’s my Dockerfile:

# Clone from the Fedora 20 image
FROM fedora:20
# Install httpd
RUN yum install -y httpd
# Change the default docroot
RUN sed -i 's|^DocumentRoot.*|DocumentRoot "/srv"|' /etc/httpd/conf/httpd.conf
# Add in our custom httpd configuration.
ADD extra.conf /etc/httpd/conf.d/root.conf
# Start up httpd.

The first line means this image begins as a clone of the standard Fedora 20 installation.  We only make a few modifications: we install the httpd package, change the default docroot to /srv, and add in a custom configuration file.  The “ENTRYPOINT” line means that httpd is the process invoked by default when containers are created from this image.  The custom configuration file, “extra.conf” I use is this, which relaxes httpd’s access control for the /srv directory:

<Directory /srv>
   Require all granted

Combining that and the change to the DocumentRoot, this image is set up to serve whatever is mounted at /srv.  Placing the two files described above in an empty directory I create a new Docker image as follows – note the argument passed tags the image with the name “httpd-test1”:

# docker build -t httpd-test1 .
Uploading context 10.24 kB
Uploading context 
Step 1 : FROM fedora:20
 ---> 6572f78e5fa5
Step 2 : RUN yum install -y httpd
 ---> Using cache
 ---> 44b5498e707a
Step 3 : RUN echo > /etc/httpd/conf.d/welcome.conf
 ---> Running in c91f82b99bce
 ---> c5a629660e53
Step 4 : RUN sed -i 's|^DocumentRoot.*|DocumentRoot "/srv"|' /etc/httpd/conf/httpd.conf
 ---> Running in 67fe1b96bf51
 ---> 26ad2ac0ebf8
Step 5 : ADD extra.conf /etc/httpd/conf.d/root.conf
 ---> cfe43ce7417a
Step 6 : ENTRYPOINT /usr/sbin/httpd -DFOREGROUND
 ---> Running in 05a4a1154fb7
 ---> f7d8712877b8
Successfully built f7d8712877b8

The remaining piece of the puzzle is integration on the host side.   We could rely on standard Docker tools for this, but let’s do it properly.  I’m going to place my example site, imaginatively named “”, in /srv/, as follows:

# mkdir /srv/
# echo '<h1>Hello,  World</h1>' > /srv/

Now I can set up a systemd service file which launches a Docker container using my image, and launch the container:

# cat > /etc/systemd/system/ <<EOF
[Unit] Container

ExecStart=/usr/bin/docker run -v /srv/ httpd-test1

# systemctl daemon-reload 

And that’s it!  The “docker run” command will create a container using that “httpd-test1” image built earlier, and  mounts the host’s /srv/ as /srv in the container.  I can control this new service like any other systemd service:

# systemctl start
# docker ps
    CONTAINER ID        IMAGE                COMMAND                CREATED             STATUS                  PORTS               NAMES
91e770b4d61f        httpd-test1:latest   /bin/sh -c /usr/sbin   1 seconds ago       Up Less than a second                       kickass_franklin6   
# docker inspect 91e770b4d61f | grep IPAddr
        "IPAddress": "",
# curl
<h1>Hello,  World</h1>

Neat stuff.