Package a beat built from source for deploying to other servers running on same platform

Elastic Stack Beats
1

I built MetricBeat from source on an ARMv7 device running Debian Stretch. To do this, I installed Go, then downloaded the Beats source code from GitHub, switched to the matching version for my ElasticSearch and Kibana installations, and successfully ran 'make' under metricbeat, then 'make update'. All this was successful, and I am able to run the metricbeat executable with default configuration.

Question is - How can I create a tarball to deploy this to several other servers running on the same platform (and include all the metricbeat modules and any kibana-related stuff as well)?

2

The make package target produces .tar.gz, .rpm, and .deb. It requires an x86_64 machine because it uses Docker to cross-compile for all targets. To get it to produce armv7 artifacts use.

cd metricbeat
git checkout v6.6.1 # for example
PLATFORMS='+all linux/armv7' make package
ls -la build/distributions

FYI: I have snapshots built from master that include the "armhf" artifacts for ARMv7 at https://beats-ci.elastic.co/job/elastic+beats+master+multijob-package-linux/lastSuccessfulBuild/gcsObjects/.

3

Hi Andrew,

Thanks so much for your quick response. I'll give it a try and post the outcome here.

Will I be able to use your snapshots even though my Kibana and ElasticSearch version is behind (6.5.1)? Snapshots are v6.7.0 or later only.

4

Hi @andrewkroh - Is the "make package" command to be run after a plain "make"? Right now when I run it I get a "make: Nothing to be done for 'package'." message.

I also tried to do a plain "make" on an x86_64 machine and am getting out of memory errors with the Go linker.

Is there a manual way to package the build into a .tar? I could simply do that for now if I knew what would need to be packaged.

Thanks!

5

No, that's not necessary. make package should do everything that is required to produce the packages. Doing a make clean before hand is generally a good practice though.

They should work, but I would only use them for testing purposes (non-production). The latest 6.x release tag is what I would use.

The package target takes a lot resources. It's running multiple compiles inside of Docker. I'm curious how much memory you had available on this host?

I started a package build from v6.6.1 for armv7. When the build finishes this should have some packages (none of them signed since this isn't our official build) https://beats-ci.elastic.co/job/elastic+beats+master+multijob-package-linux/368/gcsObjects/.

6

Thanks @andrewkroh. Seems like my AWS instance might indeed be too small to handle the full-blown build. Since you are doing the v6.6.1 build for armv7, I'll simply use that instead, after upgrading my ELK stack to v6.6.1.

Thanks so much for doing that, btw. Good to have a build that matches the latest GA version of the stack.

7

Hi @andrewkroh ,

Tried to download the armv7 package but I'm getting the following error:

$ gunzip metricbeat-6.6.1-linux-armv7.tar.gz

gzip: metricbeat-6.6.1-linux-armv7.tar.gz: not in gzip format

:anguished:

8

I just downloaded that same file and it unzipped fine for me. We also provide sha512 checksums of the file contents. And it verifies for me:

$ shasum -a 512 -c ./metricbeat-6.6.1-linux-armv7.tar.gz.sha512 
metricbeat-6.6.1-linux-armv7.tar.gz: OK

If you're curl'ing the file URLs try changing the domain to storage.googleapis.com.

9

@andrewkroh - I was able to test this out and I can confirm that it works fine. Something must have gone wrong during my previous download attempt. Thanks again for making packages from the latest stable release available.

Just out of curiosity, I see a modules.d sub-directory but not a separate modules sub-directory. I'm assuming they are all integrated with the metricbeat executable now?

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.