Unifi-poller Setup and Configuration

Background

The unifi-profiler depends on either InfluxDb or Prometheus for the database and Grafana for the data display. This documents creating three disperate docker containers and configuring them to work with the unifi-profiler as the data source to an influx database that Grafana uses to display our data.

The Information in this guide was inspired and based on the unifi-poller project:

Perhaps my ideal would be similar to this docker compose guide at: https://nerdygeek.uk/2020/06/18/unifi-poller-an-easy-step-by-step-guide/ which lends itself to more a a single applicance but I have other uses for Grafana and InfluxDb so this works for my current needs.

Sections

  1. Pre-requisites
  2. InfluxDb
  3. Grafana
  4. Unifi-poller
  5. Enjoy the fruits of your labors
  6. How to get the docker container id of a running docker container
  7. How to get the IP address of a running docker container
  8. How to get a shell inside a running docker container
    • How to exit a shell inside a docker container
  9. How to restart a docker container
  10. How to create a named docker volume
  11. How to remove a docker container

Pre-Requisites

  1. Create a read-only adminstrator account on the Unifi controller for use by the unifi-poller

    • Add a user to the UniFi Controller. After logging into your controller:
    1. Go to Settings -> Admins
    2. Add a read-only user (unifipoller) with a nice long password.
    3. The new user needs access to each site.
      • For each UniFi Site you want to poll, add admin via the ‘Invite existing admin’ option.
    4. Take note of this info, you need to put it into the unifi-poller config file in a moment.
  2. Install Docker

    • Make sure the Pi is up to date:

      sudo apt-get update && sudo apt-get upgrade
      
    • Agree to any updates, and wait for them to install. Then the one line command to install Docker is. It is worth noting that this downloads a script from the internet and runs it in your bash shell:

      curl -sSL https://get.docker.com | sh
      
    • Now add the user pi to the docker group, so as to avoid permissions issues later. Please note, this guide may not contain a unified use of sudo for the shell commands.

      sudo usermod -aG docker pi
      
    • Reboot the PI

      sudo reboot
      
    • Make sure everything is working by running

      docker run hello-world
      
    • The output should contain the magic lines:

      Hello from Docker!

  3. Create locations for the local version on the docker host of the configuration files and data files the docker containers will have mapped into the container for use.

    • Create the following directories:

      • A host location to hold files and directories one would normally use /etc/ for within the container.
        • /etc/docker_hosts
      • The influxdb directory, will be used to hold the influxdb.conf file.
        • /etc/docker_hosts/influxdb
      • The unifi-poller directory, will be used to hold the unifi-poller.conf
        • /etc/docker_hosts/unifi-poller
    • Optional steps

      These steps are only needed for mapping a host’s local directory as a volume to a docker container. This guide uses a docker named volumes for influxdb and grafana which allows docker to manage it rather than the end-user.

      How to use this method is described in the How to create a named docker volume section, should you choose to use that instead.

      • /var/lib/docker_hosts/grafana
      • /var/lib/docker_hosts/influxdb

InfluxDb

  1. Create config file

    Change to the docker_hosts/influxdb directory. You will want to be sudo/root for this.

    cd /etc/docker_hosts/influxdb
    

    Run the following to extract the influxdb.conf to the directory.

    docker run --rm influxdb influxd config > influxdb.conf
    
  2. Create the docker named volume for the influxdb data:

    docker volume create influxdb_data
    
  3. Run the container as a daemon (the -d argument specified daemon.)

    docker run --name influxdb -d -p 8086:8086 -v influxdb_data:/var/lib/influxdb -v /etc/docker_hosts/influxdb/influxdb.conf:/etc/influxdb/influxdb.conf:ro influxdb -config /etc/influxdb/influxdb.conf
    

    To break down the above docker command

    Parameter Value Meaning
    –name influxdb sets the container id to influxdb
    -d runs the the container as a service/daemon
    -p 8086:8086 maps the port on the left to the host’s port while the port on the right is mapped to the container’s port and IP address. The value here maps host port 8086 to container port 8086.
    -v influxdb_data:/var/lib/influxdb maps a docker volume named influxdb_data on the host to /var/lib/influxdb inside the container for the data on the host to overlay into the container.
    -v /etc/docker_hosts/influxdb/influxdb.conf:/etc/influxdb/influxdb.conf:ro maps the host file /etc/docker_hosts/influxdb/influxdb.conf to the file in the container at /etc/influxdb/influxdb.conf as a Read-Only (:ro) file in the container
    influxdb the image to run , docker resolves this.
    -config /etc/influxdb/influxdb.conf use this configuration file at /etc/influxdb/influxdb.conf in the container
  4. Create a database

    • Documentation as posted (https://hub.docker.com/_/influxdb/) has an issue. The command:

      curl -G http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb"
      
    • Using Get to transfer the data is identified as a deprecated call:

      {"results":[{"statement_id":0,"messages":[{"level":"warning","text":"deprecated use of 'CREATE DATABASE mydb' in a read only context,      please use a POST request instead"}]}]}
      
    • Instead use Post:

      curl -POST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE mydb"
      

    Addtl. Info: https://github.com/influxdata/docs.influxdata.com-ARCHIVE/issues/493

  5. Insert a Row

    curl -i -XPOST 'http://localhost:8086/write?db=mydb' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
    

Grafana

  1. Pull down the docker Grafana image

    docker pull grafana/grafana:latest
    
  2. Create the docker named volume for the grafana data:

    docker volume create grafana_data
    
  3. Run the container as a daemon (the -d argument specified daemon.)

    docker run --name grafana -d -p 3000:3000 -v grafana_data:/var/lib/grafana grafana/grafana:latest
    
  4. Navigate to the Grafana web ui: http://localhost:3000

    • default username: admin
    • default password: admin
  5. Add an InfluxDb data source

    • Click the settings Gear Icon and choose the Data Sources option

    • Click the Add Data Source button

    • Find the InfluxDb data source and choose Select

    • Set the HTTP Url setting to the InfluxDb container’s IP address and port

      1. Find the IP of the influxdb container (see section How to get the IP address of a running docker container)

      2. Add the url including the port number used (8086 was used in this guide.)

      http://172.17.0.2:8086

    • Set the Database value to the name of the database you created during the InfluxDb steps. The guide uses mydb

    • Click the Save and Test button to verify Grafana can connect to InfluxDb.

  6. Go on to add and configure the Grafana dashboards for unifi-poller.

  7. Install the additional Grafana Plug-ins that the unifi-poller dashboards use.

Grafana dashboards

Source: https://grafana.com/grafana/dashboards?search=unifi-poller

  1. Navigate to the Grafana web ui: http://localhost:3000

  2. Click on the + Create icon and choose the Import option

  3. In the Import via grafana.com textbox put the Import Code of the plug-in below to install.

    • Ensure you choose the InfluxDb Data Source in the drop-down at the bottom labeled: Select an InfluxDB data source
  4. Import the following Dashboards

Grafana Plug-ins

Additional Grafana plug-ins used by the Unifi-poller dashboards:

  1. grafana-piechart-panel
  2. grafana-clock-panel
  3. natel-discrete-panel

Note: Grafana requires a restart after installing new plug-ins.

Plug-in Installation

Perform the following from within the running Docker container (see section How to get a shell inside a running docker container):

grafana-cli plugins install PLUGIN-NAME

Unifi-poller

Based on the instructions here: https://github.com/unifi-poller/unifi-poller

  1. Pull down the docker unifi-poller image

    docker pull golift/unifi-poller
    
  2. Create a copy of the unifi-poller.conf in the /etc/docker_hosts/unifi-poller directory on the host your created earlier.

    • edit the /etc/docker_hosts/unifi-poller.conf as needed

    • I disabled Prometheus support as I am using Influx

      [prometheus]
      disable = true
      
    • Configure InfluxDb, I am running default no username or password so only the url and disable = false needed to be updated.

      Do note: The IP address is the InfluxDb container’s IP address in the example below.

      [influxdb]
      disable = false
      # InfluxDB does not require auth by default, so the user/password are probably unimportant.
      url  = "http://172.17.0.3:8086"
      db   = "mydb"
      
    • Configure the unifi.defaults section.

      InfluxDb supports more items than Prometheus so those are enabled below as well as saving the Deep Packet Inspection data (save_dpi)

      # The following section contains the default credentials/configuration for any
      # dynamic controller (see above section), or the primary controller if you do not
      # provide one and dynamic is disabled. In other words, you can just add your
      # controller here and delete the following section. The internal defaults are
      # shown below. Any missing values will assume these displayed defaults.
      [unifi.defaults]
      url            = "https://192.168.2.1:8443"
      user           = "unifi-admin-unifipoller-username"
      pass           = "unifi-admin-unifiprofiler-password"
      sites          = ["all"]
      save_ids       = true
      save_events    = true
      save_alarms    = true
      save_anomalies = true
      save_dpi       = true
      save_sites     = true
      hash_pii       = false
      verify_ssl     = false
      
  3. Run unifi-poller container as a daemon (the -d argument specified daemon.)

    docker run --name unifi-poller -d -v /etc/docker_hosts/unifi-poller/unifi-poller.conf:/config/unifi-poller.conf golift/unifi-poller
    

Enjoy the fruits of your labors

  1. Navigate back to your Grafana instance: http://localhost:3000
  2. Click the Dashboard icon and choose the Manage option.
  3. Click one of the imported dashboards to view the beautiful data.

How to get the docker container id of a running docker container

docker ps

Results:

CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                            NAMES
3183bbb971ed        grafana/grafana:latest   "/run.sh"                3 hours ago         Up 3 hours          0.0.0.0:3000->3000/tcp                           grafana2
dfbce9a7c751        golift/unifi-poller      "/image"                 4 hours ago         Up 3 hours                                                           unifi-poller
a6e4f76a2677        influxdb                 "/entrypoint.sh -con…"   6 hours ago         Up 6 hours          0.0.0.0:8086->8086/tcp                           influxdb

How to get the IP address of a running docker container

Sources:

  1. Get the container id of the container you’d like the IP address from

  2. Execute the ip a command within the container

    docker exec influxdb ip -4 -o address
    

    Results

    1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
    21: eth0    inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0\       valid_lft forever preferred_lft forever
    

How to get a shell inside a running docker container

  1. Get the container id of the container you want to start the shell in

  2. Use docker exec to open a shell

    docker exec -it 3183bbb971ed sh
    

How to exit a shell inside a docker container

  • exit command at the shell
  • CTRL+P followed by CTRL+Q docker shell escape sequence

How to restart a docker container

Source: https://docs.docker.com/engine/reference/commandline/container_restart/

  1. Get the container id of the container you want to restart

  2. Use docker container restart to restart it. The below will restart it immediately.

    docker container restart 3183bbb971ed
    

How to create a named docker volume

A "named docker volume" is simply a location on the host that docker is aware of. Volumes can be used by multiple containers and are available on the host as well. This is a convenient way to save application data and configurations to persist across container runs or the containers that reference them.

These volumes are not required and can be done manually using the full path to a resource with the docker run -v command:

Volume Type Command
Named Volume docker run -v influxdb_data:/var/lib/influxdb
Mapped Directory docker run -v /var/docker_hosts/influxdb/influxdb_data:/var/lib/influxdb

To create a named volume

docker volume create [volume_name]

Example:

docker volume create influxdb_data

which can then later be attached with the -v command line argument of docker run:

docker run –name influxdb -d -p 8086:8086 -v influxdb_data:/var/lib/influxdb […]

How to remove a docker container

If you need to remove a docker container and re-run it with different starting arguments, you can remove an existing docker container using the docker container rm command:

  1. Get the container id of the container you want to start the shell in

  2. Use docker exec to open a shell

    docker container rm 3183bbb971ed
    

    or

    docker container rm grafana