Customizing Ubiquiti USG configuration with JSON just got easier

Ubiquiti USG (Unified Service Gateway) is a router and firewall appliance that is closely related to the EdgeMax product line, even though it's marketed as a part of the UniFi product family and focused on a different market segment.

It is meant to be managed by a Unifi controller, which overwrites the settings so configuring it in the traditional way will not make the changes permanent. Since it's hardware is closely related to the EdgeMax routers and its software is derived from EdgeOS, using only the limited configuration options available from the Unifi controller GUI would leave its capabilities seriously underutilized.

Fortunately, the controller allows customizing the config with JSON files, and the USG OS includes a tool for exporting the config from JSON. When a customer wanted to configure QoS for VoIP traffic in a way that is not supported by the controller, we had to look deeper into it, and found that the process is not nearly as easy as we hoped it will be.

The workflow is the following:

1. Add the required configuration directly on the USG (with set commands as in the usual EdgeOS)
   
2. Export it to JSON with mca-ctrl -t dump-cfg
3. Extract the relevant sections from that JSON and put them on the controller

The last part is the offender here. Unlike the "show" command, or the cli-shell-api tool, the tool for exporting the config to JSON (mca-ctrl) does not allow exporting only a part of the configuration. Moreover, the controller is capable of merging configuration files, but not loading a partial configuration, so the JSON dict representing the configuration should include all levels (for example, if you add just one firewall rule, the config should have "firewall", "name" etc. levels in it to work nonetheless).

The official guideline suggests picking the relevant parts from the JSON config by hand. Obviously, this is a tedious and error-prone process, so I started looking for a way to automate it.

Luckily, the USG OS has Python installed. It's Python 2.7 while I would prefer to see Python 3 there, but it still has a JSON parser and formatter in its standard library, so it was good enough for the job.

So, wrote a script that takes a
list of configuration paths as arguments and automatically export them
from the JSON generated by mca-ctrl into a single JSON object that
should be ready for deployment on the controller.

A simple example of how to run it and what the output is like:

ubnt@ubnt# ./usg-config-export.py "service ssh" "system offload"

{

    "system": {

        "offload": {

            "ipv6": {

                "forwarding": "enable",

                "vlan": "enable"

            },

            "ipv4": {

                "forwarding": "enable",

                "pppoe": "enable",

                "vlan": "enable"

            },

            "ipsec": "enable"

        }

    },

    "service": {

        "ssh": {

            "port": "22",

            "protocol-version": "v2"

        }

    }

}

It supports unlimited number of configuration paths, not just two.

The script can be trivially extended to support taking JSON from a file or stdin in addition to running mca-ctrl on its own, if anyone wants it, let me know and I'll add it. Let me know if you have any problems with the script as well.

Installation

You can get the script from my github repo. Just copy it to your USG (with scp or otherwise), chmod +x it, and it's ready to run. Hope it saves you quite a bit of time if you also need to customize your USG configuration.