Application Hosting

Introduction

WeOS takes great pride in its ease-of-use, coherent configuration, and quality assurance. Obtaining those objectives relies on making tough decisions on how components are constrained to make sure that they interoperate correctly. As a result, WeOS can not support every configuration under the sun. Applications offer an escape hatch for users to tailor the system to their specific needs by running their own software alongside WeOS, unleashing the full power of the underlying Linux kernel.

.------------------------.------.------.
|          WeOS          | App1 | App2 |
:------------------------'------'------:
|             Linux kernel             |
:--------------------------------------:
|               Hardware               |
'--------------------------------------'

Under the hood, the container facilities in the Linux kernel (cgroups and namespaces) are used to create virtual operating system instances. LXC provides the container environment in which the applications run. This approach has a number of advantages:

• Very low performance overhead compared to hardware virtualization techniques.
• Users can develop against the familiar Linux API.
• The breadth of existing open source software can easily be integrated to user’s applications.

For a quick-start guide to working with Applications, have a look at the HowTo:

• Firewalling with NetBox

Images and Instances

Following the nomenclature popularized by existing Linux container software like LXC and Docker, the application subsystem fundamentally manages two types of resources, application images and application instances.

An application image is a SquashFS file holding a complete root filesystem. Conceptually this can be thought of as a read-only archive with very efficient random access. The layout of this filesystem is by and large left up to the developer, the only exemption being that /sbin/init must be an executable file which is run by LXC when an application starts.

From an image, one or more application instances can be created. Two instances can share the same image, but can have different configuration parameters and runtime generated data. I.e. an image is to an instance as an executable is to a process.

Environment

Application containers run as isolated islands, unable to directly affect any resources managed by WeOS. E.g. it can only see the processes that it itself has created, it can only see files from its own filesystem etc. Starting from this solid barrier, certain types of membranes can be created through an instance’s configuration to allow the application to access parts of the filesystem and/or network. Concurrent accesses to shared resources, filesystem and/or network, can lead to unexpected or erroneous behavior.

Filesystem

     Application Instance
     ^ ^ ^ ^ ^ ^  ^ ^ ^ ^
     | | | | | |  | | | |        Filesystem Operations (open/read/write)
     v v v v v v  v v v v
.---------.
|  Share  |                      Host Operating System Path (ro/rw)
:---------'---------.
|      Overlay      |            Non-Persistent Storage (rw)
:-------------------'---------.
|      Application Image      |  SquashFS File (ro)
'-----------------------------'

In order to provide persistent storage to applications, even though the image is read-only, writable storage such as USB media can be shared and made available for applications to read from and write to. A bind mount is used to import the path to the application, as such the path can be mounted under a different path inside the application. E.g. a file on a USB stick, /usb/instance1/message-of-the-day.txt, can be mounted as /etc/motd. in the application. Files not backed by persistent storage are writable for applications, but wont persist over reboots since they are only backed by non-persistent RAM.

It is important to note that although all files are effectively writable, the application should take care to:

• Limit the number of writes. The backing storage for persistent layer is typically FLASH memory which tolerates a limited number of write cycles before it wears out.
• Limit the set of files being written to. In order to support upgrading an instance’s image, it is important that the application does not write to any file that is expected to be upgradable as those would then be shadowed by the persistent layer.

Network

Applications may be connected to different layers of the WeOS networking stack. In a system with no applications running, a typical configuration may look something like this:

     .-------.  .-------.
     | vlan1 |  | vlan2 |       L3
     '---+---'  '---+---'
         '-.      .-'
       .---+------+---.
       |      br0     |         L2
       '-+---+--+---+-'
   .-----' .-'  '-. '-----.
.--+-.  .--+-.  .-+--.  .-+--.
|eth1|  |eth2|  |eth3|  |eth4|  Port
'----'  '----'  '----'  '----'

At the bottom we see the physical and virtual ports, loosely this is any single port that can transport Ethernet, e.g. a 1000-baseT port or an OpenVPN tunnel in L2 mode. These are then typically connected to the bridge (br0) to enable L2 forwarding (switching). On top of the bridge sit the VLAN interfaces. This is usually where we enter the IP stack and thus it is between these interfaces that L3 forwarding (routing) takes place. Applications can be configured to integrate with the operating system on all three layers.

L3 Interface

WeOS                                     # Application Instance
     .-------.  .-------.    .-------.   #   .------.
     | vlan1 |  | vlan2 |    | myapp |   #   | host |
     '---+---'  '---+---'    '---+---'   #   '--+---'
         '-.      .-'            |       #      |
       .---+------+---.          |       #      |
       |      br0     |          |       #      |
       '-+---+--+---+-'          |       #      |
   .-----' .-'  '-. '-----.      |       #      |
.--+-.  .--+-.  .-+--.  .-+--.   |       #      |
|eth1|  |eth2|  |eth3|  |eth4|   |       #      |
'----'  '----'  '----'  '----'   |       #      |
                                 '--------------'

In this configuration, the application is connected to the host by creating a virtual Ethernet pair (veth pair) where one end (myapp) is attached to WeOS’s network namespace and the other one (host) is connected to the application’s ditto.

WeOS can distribute the network configuration to the application via DHCP. Using the host side as its default gateway and nameserver, the application can communicate with the outside world. myapp is indistinguishable from any other interface in WeOS, as such it can be referenced anywhere where an interface can be specified; in a firewall rule, as a management interface etc.

Depending on the WeOS configuration, access can be limited via the firewall, traffic can be masqueraded to appear to originate from the WeOS itself (NAPT), the internal subnet can be exported using a dynamic routing protocol like OSPF.

L2 Interface

WeOS                                       # Application Instance
     .-------.  .-------.                  #
     | vlan1 |  | vlan2 |                  #
     '---+---'  '---+---'                  #
         '-.      .-'                      #
       .---+------+---.                    #   .------.
       |      br0     +-------------.      #   | host |
       '-+---+--+---+-'             |      #   '--+---'
   .-----' .-'  '-. '-----.         |      #      |
.--+-.  .--+-.  .-+--.  .-+--.  .---+---.  #      |
|eth1|  |eth2|  |eth3|  |eth4|  | myapp |  #      |
'----'  '----'  '----'  '----'  '---+---'  #      |
                                    '-------------'

Just like with an L3 Interface, communication between the host operating system and the guest application is through a veth pair. The difference being that instead of using myapp as an L3 interface, it is attached to the bridge.

WeOS still has control over which networks the application can reach through the VLAN configuration. As the interface now works as a port, it can be a member of one or more VLANs.

In this setup, the application can be attached to the same broadcast domain as devices connected to the physical ports (eth1..eth4). I.e. it can resolve neighbors using ARP, send IP broadcasts etc. From WeOS’s perspective it is perceived as an additional port.

Exclusive Port Access

WeOS                       # Application Instance
     .-------.  .-------.  #
     | vlan1 |  | vlan2 |  #
     '---+---'  '---+---'  #
         '-.      .-'      #
       .---+------+---.    #
       |      br0     |    #
       '-+---+--+-----'    #
   .-----' .-'  '-.        #
.--+-.  .--+-.  .-+--.     #  .----.
|eth1|  |eth2|  |eth3|     #  |eth4|
'----'  '----'  '----'     #  '----'

In this mode, an existing physical port is transferred to the application’s namespace. WeOS will not be able to use the port for any services. The application is free to send and receive any packets, setup netfilter and tc rules etc., just as with any standard NIC.

This setup can be used to create a “virtual industrial PC” where the application can be physically connected to the network by looping a cable back to one of the ports controlled by WeOS.

Much more commmon though, this mode is used in combination with either an L3 Interface or an L2 Interface. Imagine a firewall and routing application which is used to connect to, and filter traffic from, an ISP. It uses a physical port on one side to connect to the ISP (eth4). Once traffic has been filtered, routed, and possibly address translated, it is sent to WeOS via an L2 Interface (fw0/host):

WeOS                                  # Firewall Application
     .-------.  .-------.             # .-------.  .-------.
     | vlan1 |  | vlan2 |             # | vlan1 |  | vlan2 |
     '---+---'  '---+---'             # '---+---'  '---+---'
         '-.      .-'                 #     '---.  .---'
       .---+------+---.               #       .-+--+-.
       |      br0     +-------.       #       | host |
       '-+---+--+-----'       |       #       '--+---'
   .-----' .-'  '-.           |       #          |
.--+-.  .--+-.  .-+--.    .---+---.   #          |             .----.
|eth1|  |eth2|  |eth3|    |  fw0  |   #          |             |eth4|
'----'  '----'  '----'    '---+---'   #          |             '----'
                              '------------------'

The application has full control over which traffic flows that are allowed to ingress the system, but WeOS can still be used to provide local services such as DHCP.

VLAN Restrictions

When creating VLANs in an application it is important to give the VLANs unique names if they contain physical ports. No other VLANs in the operating system nor in other applications can have a VLAN with the same name if they also contains physical ports. This would leed to erroneous behavior when switchcore has more than one VLANs with the same name.

Configuration

Applications are a top-level configuration context in the CLI.

example:/#> configure

[no] app [NAME] [IMAGE]

Create and manage container applications.

When creating a new application, the name is optional. When omitted the container image name is used. Application naming must follow interface naming rules since a default veth pair is created from the application name. To see available container images from inside configuration context, use the do prefix:

do show app image

example:/config/#> app foo netbox
example:/config/app-foo/#> show
Status       : Enabled
Name         : foo
Description  :
Image        : netbox
Init cmd     : /sbin/init
                                                   
Shared Resources                                   
TYPE  HOST              GUEST             OPTS     
veth  bar               eth0

Syntax

[no] cmd [PATH]

Absolute path to init program to use inside container.

Useful for debugging the bootstrap of a container application. E.g., set to /bin/sh to skip all setup and get a plain shell prompt.

Default: /sbin/init

no

Resets to default

[no] description [DESCRIPTION]

Free-form string to describe the container, not used anywhere except in the startup-config.cfg file. Useful as a marker for inspecting a system with application containers.

[no] enable

Enable, or disable application. Use this option to create an app that is not started automatically. The configuration remains in the startup-config.cfg file.

To completely deactivate or remove, an application, use the form no app NAME from the top-level configuration context in the CLI.

no

Disable application instance

[no] share veth [IFNAME [as IFNAME]]

Manage veth pairs between host and guest.

This variant of the share command manages veth pairs. One end is located in the host and can be added as a port to a VLAN, or be used as an interface and assign an IP address to. The other end shows up in the guest application as a regular Linux interface.

no

Remove one or all veth pair(s)

IFNAME

Host end of veth pair, required argument when creating. Note, several namespaces are reserved for the host os:

• dslN
• ethN
• greN
• ifaceN
• ipsecN
• lo
• modemN
• portN
• pppN
• pppoeN
• sslN
• vlanN

as

Optional, if omitted a default name is created (for the first one)

IFNAME

Guest end of veth pair, must be unique inside the container. Note, shares the same namespace as regular port shares.

[no] share port [IFNAME [as IFNAME]]

Manage physical ports, assign to or remove from guest.

This variant of the share command manages physical ports. Beware that when assigning a port to a container application, it is removed from the host while the application is active! Hence, it is important to remove the port from all applications, aggregates and other configurations.

no

Remove one or all physical port(s) from an application

IFNAME

Host name of physical port

as

Optional, if omitted the host name is used

IFNAME

Alternative name for port inside container. Useful if remapping of ports is required, e.g., due to name clashes with veth pairs

[no] share path [PATH [as PATH]]

Manage file shares between host and guest.

This variant of the share command manages file and directory shares between the host and the application container. E.g., reading a file from a USB stick mounted on /usb from a container.

no

Remove one or all file shares from an application

PATH

Absolute path to file or directory on host to share with guest

as

Optional, if omitted the host name is used

PATH

Absolute path to location in the container’s root filesystem. This is required when the host PATH mount point does not exist in the guest

[no] environment [NAME] [VALUE]

Set custom environment variables for the init program

no

Remove one or all of the custom environment variables

NAME

Name of the variable

VALUE

Optional value of the variable. If omitted the variable will be set but empty

CPU and Memory Restrictions

To ensure WeOS has enought cpu and memory to run smooth the apps are restricted to not allocate more than 25% of the cpu periods and not more than 50% of the total RAM memory together. Memory and cpu is divided equally among the container apps.

Status

Interaction with application instances and images can be done in admin-exec, the top-level context of the CLI.

To see what applications are configured and active in the system:

example:/#> show app
NAME STATE   AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED
bar  RUNNING 0         -      -    -    false
foo  STOPPED 0         -      -    -    false

To attach to a running application container:

example:/#> app attach bar
Type  to exit the console,  to enter Ctrl+p itself

NetBox - The Networking Toolbox
netbox login: root
#
^P^Q
example:/#>