Dec 9th, 2018: [EN][Elastic Stack] Ubiquiti metrics in the Elastic Stack


(Mark Walkom) #1

The Ubiquiti Unifi kit is pretty amazing for its price point. I use it extensively at home to replace a bunch of Google Wifi, power line and wifi extenders and other devices that just never really worked for me. Other than the value for money they provide, it's super simple to setup and manage.

But their logging is currently less than ideal, with only the ability to easily push data to an rsyslog endpoint for their Cloud Key (the config management tool), but not the USG (security gateway). There are other options, but they're a fair bit more advanced.

Let's go beats!

Luckily you can cross compile golang into nearly any architecture, including ARM, which is what the Key runs. And the amazing folk at Archlinux have done just that, hosting up to date packages for Auditbeat, Filebeat, Heartbeat, Journalbeat, Metricbeat and Packetbeat. We'll be using Filebeat and Metricbeat for this topic, just make sure you grab the armv7h distribution so it's compatible.

Now we can ssh to the device, download the package(s) you want, then extract them somewhere you're happy with. I created a /root/beats directory and they all live in there.

Enable and Configure Modules

The configuration process is the same as the Getting Started documentation for each Beat. I have enabled the system, mongodb, nginx modules in both Filebeat and Metricbeat, and imported the dashboards, index templates and ML jobs. I also have Elasticsearch and Kibana running elsewhere and the configs for the beats point to those servers.

We do need a few custom changes for the modules. For Metricbeat, the Key runs MongoDB on port 27117, so make sure you change that in the etc/metricbeat/modules.d/mongodb.yml config file. There's no username or password to enter.

You will also need to add var.paths: /usr/lib/unifi/logs/ to etc/filebeat/modules.d/mongodb.yml, as it is a non-standard location.

Check it all works

You can do a quick validation of your config and setup with;

root@UniFi-CloudKey:~/beats# ./usr/bin/metricbeat test config --path.config `pwd`/etc/metricbeat/ --path.logs `pwd`/var/logs/

And because this isn't a standard platform, we can run this to make sure it starts and there's no other issues outside out config file;

root@UniFi-CloudKey:~/beats# ./usr/bin/metricbeat --path.config `pwd`/etc/metricbeat/ --path.logs `pwd`/var/logs/

If there's a problem it won't start, and will provide the error in your shell session. Otherwise, put the command into the background and then check the logs directory for further info.

The Results :firecracker:

Improvements

The one real downside with this is that we need to make and manage our own service files, or some other management process, for each beat. If you'd like to see official ARM builds for beats, that would include service files, then please feel free to add a thumbs up to this GitHub enhancement issue.

We can also quickly deploy Packetbeat with the above process, capturing HTTP (via nginx), MongoDB, DNS and even DHCP traffic.


(Mark Walkom) #2

Quick update, you can use /etc/rc.local to manage starting things automatically. eg add this to the file;

# Elastic Beats
/root/beats/usr/bin/metricbeat --path.config /root/beats/etc/metricbeat/ --path.logs /root/beats/var/logs/
/root/beats/usr/bin/filebeat --path.config /root/beats/etc/filebeat/ --path.logs /root/beats/var/logs/