File/Packet/Metric beat on Raspi Centos 7 (armhf/armv7l)

@m.pers

There aren't official builds for Beats on ARM. But you can make them and they will work.

Recently I built Beats 7.1.0 on Raspberry Pi 3 B+ Ubuntu 18.04 64-bit ARMv8. Specifically I built Filebeat, Metricbeat, Auditbeat, and Heartbeat. Of those, I tested (and continue to run) Metricbeat and Auditbeat.

I'll walk through the build process that worked for me. It was a different OS (Ubuntu), a different version of ARM (v8), and I didn't try to build Packetbeat or run Filebeat. I won't try to adapt my steps to a platform that I haven't tested. But I expect this walkthrough will be similar enough that you can adjust the steps as needed, such as using yum instead of apt-get.

I look forward to hearing if these instructions help you build Beats on CentOS and ARMv7, and if Packetbeat and Filebeat work as expected on that platform.

Building Beats 7.1.0 on Ubuntu 18.04 (64-bit) ARMv8

Assume you have signed into the terminal as the ubuntu user.

Step 1: Install dependencies

sudo apt-get update
sudo apt-get install -y gcc git make wget

Step 2: Get Golang for ARM (64-bit)

Download Golang.

wget https://dl.google.com/go/go1.12.5.linux-arm64.tar.gz
tar -xzf go1.12.5.linux-arm64.tar.gz
mkdir go_projects

Export the new environment variables.

export GOROOT=$HOME/go
export PATH=$PATH:$GOROOT/bin
export GOPATH=$HOME/go_projects
export PATH=$GOPATH/bin:$PATH

Persist the new environment variables.

echo -e "export GOROOT=$HOME/go" | tee -a ~/.profile
echo -e "export PATH=$PATH:$GOROOT/bin" | tee -a ~/.profile
echo -e "export GOPATH=$HOME/go_projects" | tee -a ~/.profile
echo -e "export PATH=$PATH:$GOPATH/bin" | tee -a ~/.profile

Step 3: Get Beats

Get the source code for Beats from the official git repo, and then checkout the commit on which Beats 7.1.0 was built. If you want to build a specific version of Beats, then download the .tar.gz file for one of the Beats, extract it, and view the contents of the .build_hash file which shows the commit hash of that build. This build process was tested with the commit hash from metricbeat-7.1.0-linux-x86_64.tar.gz (03b3db2).

go get github.com/elastic/beats
cd $GOPATH/src/github.com/elastic/beats
git fetch
git checkout 03b3db2

Step 4: Build Beats

Temporarily enable swap space. The build process may require more memory than the capacity of the Raspberry Pi.

sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo -e "/swapfile swap swap defaults 0 0" | sudo tee -a /etc/fstab
sudo swapon --show

Make Metricbeat. An executable file named metricbeat will be created in $GOPATH/src/github.com/elastic/beats.

cd $GOPATH/src/github.com/elastic/beats/metricbeat
GOOS=linux GOARCH=arm64 go get
make

Make Auditbeat. An executable file named auditbeat will be created in $GOPATH/src/github.com/elastic/beats.

cd $GOPATH/src/github.com/elastic/beats/auditbeat
GOOS=linux GOARCH=arm64 go get
make

Make Filebeat. An executable file named filebeat will be created in $GOPATH/src/github.com/elastic/beats.

cd $GOPATH/src/github.com/elastic/beats/filebeat
GOOS=linux GOARCH=arm64 go get
make

Make Heartbeat. An executable file named heartbeat will be created in $GOPATH/src/github.com/elastic/beats.

cd $GOPATH/src/github.com/elastic/beats/heartbeat
GOOS=linux GOARCH=arm64 go get
make

Disable swap when done.

sudo swapoff -v /swapfile
# Remove the line below from /etc/fstab:
# /swapfile swap swap defaults 0 0
sudo rm /swapfile

Step 5: Organize Beats

Move each Beat to its own subdirectory under a single ~/beats directory.

cd ~
mkdir beats
cp -R $GOPATH/src/github.com/elastic/beats/auditbeat ~/beats/auditbeat
cp -R $GOPATH/src/github.com/elastic/beats/filebeat ~/beats/filebeat
cp -R $GOPATH/src/github.com/elastic/beats/heartbeat ~/beats/heartbeat
cp -R $GOPATH/src/github.com/elastic/beats/metricbeat ~/beats/metricbeat
cp $GOPATH/bin/auditbeat ~/beats/auditbeat
cp $GOPATH/bin/filebeat ~/beats/filebeat
cp $GOPATH/bin/heartbeat ~/beats/heartbeat
cp $GOPATH/bin/metricbeat ~/beats/metricbeat

Adjust the permissions and ownership for each Beat.

chmod go-w ~/beats/auditbeat/auditbeat.yml
chmod go-w ~/beats/filebeat/filebeat.yml
chmod go-w -R ~/beats/filebeat/modules.d
chmod go-w ~/beats/heartbeat/heartbeat.yml
chmod go-w ~/beats/metricbeat/metricbeat.yml
chmod go-w -R ~/beats/metricbeat/modules.d
sudo chown root:root ~/beats/auditbeat/auditbeat.yml

Step 6: Configure Beats

Configure each .yml file as you normally would.

Step 7: Setup Index Templates

Before running Beats for the first time, you should run the setup script to create the index templates and dashboards.

Our build process didn't package those index templates or dashboards. If you want to use this build to run the setup script, then download the official .tar.gz of each Beat and then copy its kibana directory into the right subdirectory of your ~/beats directory. After you do that, you can run commands such as ./metricbeat setup.

Step 8. Run Beats

We didn't build a convenient service script. But we can run Beats in the background using nohup.

For example, here's how to run Metricbeat in the background:

cd $HOME/beats/metricbeat && nohup ./metricbeat -e

Step 9. Start Beats on boot

You can schedule the command from Step 8 to run on boot using the crontab for the ubuntu user.

For example, here's how to schedule Metricbeat to run on boot:

(sudo crontab -u ubuntu -l; echo "@reboot . $HOME/.profile && cd $HOME/beats/metricbeat && nohup ./metricbeat -e &" ) | sudo crontab -u ubuntu -

Any Beats that must run as root, such as Auditbeat, need to be added to the crontab of the root user.

Proof

Here's an Elasticsearch cluster on six nodes of Raspberry Pi 3 B+ with each node running Metricbeat.

Cluster running over two availability zones:

Metricbeat system overview dashboard:

Beats monitoring overview:

Beats monitoring for one Beat:

3 Likes