Virtual OpenWrt on LXD (redux)

Traffic

Virtual Network in the Palm of your hand

With the latest release of OpenWrt 21.02.0, running a Virtual Router (VR) on Linux Containers (LXD) is much easier, than when I wrote about it back in 2019.

The biggest improvement is that Ubuntu has started to build OpenWrt images on their LXD image server. This allows one to skip all the build-your-own-image steps. Ubuntu is supporting three architectures, x86_64, ARM64, and ARMhf (32 bit). The last is supported on my Raspberry Pi 3b+.

It is now possible to launch an OpenWrt Container with one line (almost). However the Container needs a few fixes after it is launched to work properly.

Motivation, why run OpenWrt in a Container?

Of course, I can run OpenWrt on one of hundreds of real consumer routers, and I do. OpenWrt has excellent IPv6 support, including DHCPv6-PD (prefix delegation), and a really nice web-based firewall configuration.

Why wouldn’t I want that for my virtual machines, as I have for my real ones? Well… I would.

Bridging, the better way to setup LXD

I have been using Linux Containers for a couple of years, and watched people set up LXD in a variety of network configurations, including the default. Unfortunately, even the default network config is not IPv6 friendly.

Setting up a front bridge on the host takes a bit of pre-work, but it is the most transparent way to support IPv6 on your Linux Containers, and also supports running a Virtual OpenWrt router without any additional work.

A front bridge

Bridging is the act of forwarding packets at the ethernet layer. Setting up a front bridge (br0) requires that the host is ethernet attached to the rest of your network. Wifi can not be used, as bridging between Wifi and ethernet requires more than a simple cable plug-in.

Virtual router Network

In the diagram above, br0 is what I am calling a front bridge, everything other than the physical ethernet jack is connected to br0, including the host. Depending on your Linux Distro, this can be daunting to some. Systemd doesn’t help, as it hasn’t really simplified linux networking.

Configure a front bridge

If you haven’t set up a front bridge, see Configuring systemd for a LXD Front Bridge for the 6 easy steps.

Install LXD (if you haven’t already)

If you haven’t already installed LXD on your Raspberry Pi or other Linux machine, please look at Linux Containers on the Pi blog post.

Creating LXD profiles

In order for a Linux Container machine to connect to the network, it needs a profile. The default profile connects the Container to lxdbr0 which is not, by default, connected to anything.

I create a profile to connect my Containers to br0 by default, a profile I call extbridge, which looks like:

$ lxc profile show extbridge
config: {}
description: bridged networking LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: extbridge
used_by: []

After I am happy with that profile, I usually just copy it over to the default profile, as most of my Containers are only attached to br0 and get their addressing from the upstream router (both IPv4 & IPv6).

lxc profile copy extbridge default

Creating a profile for OpenWrt

OpenWrt requires two interfaces in order to route. As the earlier diagram shows, the OpenWrt will be routing between br0 and lxdbr0.

Interestingly, when I first used OpenWrt 21.02.0 as a container, the interfaces were reversed (from my previous articles). So I created another profile with eth0 as the WAN, and eth1 as the LAN. (which I call two intf rev, for reversed)

Create twointfrev profile

lxc profile create twointfrev
lxc profile edit twointfrev
    config: {}
    description: 2 interfaces
    devices:
      eth0:
        name: eth0
        nictype: bridged
        parent: br0
        type: nic
      eth1:
        name: eth1
        nictype: bridged
        parent: lxdbr0
        type: nic
      root:
        path: /
        pool: default
        type: disk
    name: twointfrev
    used_by: []

Launch the OpenWrt container

Finally, we get to the easy part. After all the prep of setting up br0, and the twointfrev profile, launching the container is anti-climatic.

lxc launch -p twointfrev images:openwrt/21.02 router21

LXD will automagically pull down the image from the image server, and create a Container named router21.

Fixing the OpenWrt Container

Unfortunately, you will notice that the WAN (eth0) interface will have an IPv4 and IPv6 address, the LAN address will not.

$ lxc ls router21
+----------+---------+-------------------+-----------------------------------------------+-----------+-----------+
|   NAME   |  STATE  |       IPV4        |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------+---------+-------------------+-----------------------------------------------+-----------+-----------+
| router21 | RUNNING | 10.1.1.108 (eth0) |  2001:db8:8011:fd00::599 (eth0)               | CONTAINER | 0         |
|          |         |                   |  2001:db8:8011:fd00:216:3eff:fef1:25d8 (eth0) |           |           |
+----------+---------+-------------------+-----------------------------------------------+-----------+-----------+

For whatever reason, there are parts missing in this image, most notably the br-lan bridge. Hopefully this will be addressed in future OpenWrt images. But for now, we need to connect to the OpenWrt CLI and do some fixing.

We’ll use LXD’s console access:

lxc exec router21 sh

BusyBox v1.33.1 (2021-08-31 22:20:08 UTC) built-in shell (ash)

~ # 
  1. The following commands will all be done on the OpenWrt CLI. First, create a bridge & br-lan interface. Edit /etc/config/network, add/edit:
config interface 'wan6'
    option reqprefix 'auto'

config device
        option type 'bridge'
        option name 'br-lan'
        list ports 'eth1'
        option bridge_empty '1'

config interface 'lan'
        option proto 'static'
        option device 'br-lan'
        option ipaddr '192.168.88.1'
        option netmask '255.255.255.0'
        option ip6assign '64' 

Note, assign an IPv4 address that works for your network, I chose 192.168.88.1 for my network.

  1. Allow external web management. Edit /etc/config/firewall, add at the bottom of the file:
config rule               
    option target 'ACCEPT'
    option src 'wan'      
    option proto 'tcp'    
    option dest_port '80' 
    option name 'ext_web' 
  1. Restart networking & firewall for changes to take effect
/etc/init.d/network restart
/etc/init.d/firewall restart

OPTIONAL set ULA

  1. Global ULA configuration is not available in Luci – and must be configured manually
uci set network.globals=globals
uci set network.globals.ula_prefix='fdb5:df0c:2121::/64'
uci commit

You will now see the global ULA on the LAN interface now

Exit the LXD console to return to your LXD host

exit
$

Managing the OpenWrt Virtual Router (VR) from a Web Browser

Now you should be able to point your web browser to the WAN address (see output of lxc ls router21 eth0 address). and login, password is blank.

http://[2001:db8:ebbd:2080::599]/

Follow the instructions to set a password, and configure the firewall as you like.

Virtual router Network

Managing your shiny new VR

The OpenWrt router should work just like a real one. This includes the warning message you receive, the first time you click on Network->Interfaces

Virtual router Network

This happens on real routers as well, just click on Continue and all will be well.

You should see that the router now has received Prefix Delegation (PD) from the upstream router, and has applied that to the LAN interface.

$ lxc ls router21
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
|   NAME   |  STATE  |         IPV4          |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| router21 | RUNNING | 192.168.88.1 (br-lan) |  2001:db8:8011:fd04::1 (br-lan)               | CONTAINER | 0         |
|          |         | 10.1.1.35 (eth0)      |  2001:db8:8011:fd00::11b (eth0)               |           |           |
|          |         |                       |  2001:db8:8011:fd00:216:3eff:feb7:c2be (eth0) |           |           |
+----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+

Address Stability of OpenWrt on LXD

Because all of this is running on LXD, there is address stability. No matter how many times you reboot the Raspberry Pi/Linux host, or restart containers in different order, the addresses remain the same. This means the addresses above can be entered into your DNS server with out churn.

Excellent IPv6 support

LXD is the best at container customization, and virtual networking (IPv4 and IPv6). With LXDs flexibility, it is easy to create templates to scale up multiple applications (e.g. a webserver farm running in the palm of your hand). OpenWrt is one of the best Open source router projects, and now it can be run virtually as well. Now you have a server farm in the palm of your hand, with excellent IPv6 support and a great firewall!


Notes:

  • some of the screen shots are from a Pi host, and others from an AMD host.
  • IPv6 addresses have been changed to conform with Documentation addresses RFC 3849

Palm Photo by Alie Koshes

Author: Craig Miller

IPv6 Advocate since 1998