Compiling Filebeat difficulties (Resolved)

Elastic Stack Beats
1

I need to have multiline support, which seems to be implemented in master and backported to the 1.1 branch, however is not yet released.

I was hoping to be able to compile this on mac os x, but I can't seem to find any instructions to do so!

Specifically, cloning the git repository and then doing a GOPATH=~/go make does not appear to work, with the following issue:

docker-compose build
/bin/bash: docker-compose: command not found

So, it seems that the build process requires docker (which is not mentioned anywhere?)

My system runs VMware, is there any way I can avoid having to use docker as I do not want to install two different hypervisors simultaniously? (It is not recommended to use VirtualBox and VMware simultaniously, and docker does not automatically work with VMware).

I will probably have to use log-courier instead, however I am hoping to resolve this so that I can consider creating my own 'beat' package in the near future.

2

Did you clone https://github.com/elastic/beats and run make from the filebeat directory? What if you run make build instead of plain make? Docker should only be required for running integration tests.

Docker isn't a hypervisor and I haven't heard anything about it not working in VMware guests.

3

I think @magnusbaeck is right and you are probably running make inside the beats folder instead of filebeat. Instead of compiling it yourself you can also download the nightly build for 1.1 here: https://beats-nightlies.s3.amazonaws.com/index.html?prefix=filebeat/

4

Hello Magnusbaeck, Thanks for the quick response;

Yes, I did clone the repository:

$ git remote -v show
origin    https://github.com/elastic/beats.git (fetch)
origin    https://github.com/elastic/beats.git (push)

Running 'make' from the filebeat directory instead of the root directory of the repository doesn't work either:

$ pwd && GOPATH=~/go make build
/Users/arahael/dev/logstash_investigation/beats/filebeat
go build
main.go:4:2: cannot find package "github.com/elastic/beats/filebeat/beat" in any of:
    /usr/local/Cellar/go/1.5.2/libexec/src/github.com/elastic/beats/filebeat/beat (from $GOROOT)
    /Users/arahael/go/src/github.com/elastic/beats/filebeat/beat (from $GOPATH)
main.go:5:2: cannot find package "github.com/elastic/beats/libbeat/beat" in any of:
    /usr/local/Cellar/go/1.5.2/libexec/src/github.com/elastic/beats/libbeat/beat (from $GOROOT)
    /Users/arahael/go/src/github.com/elastic/beats/libbeat/beat (from $GOPATH)
make: *** [build] Error 1

Performing 'make build' in the repository root also doesn't work, because there is no rule for it, however there is a default rule but that has the issue with needing docker, as previously explained.

I would prefer to run my own build, so that I can be confident that developing my own "beat" is feasible.

5

Hello Magnusbaeck,

In addition, regarding the docker point, I wasn't referring to using it as a hypervisor.

As docker requires a few features found in Linux to operate, other operating systems require a virtual machine, and docker generally works with VIrtualBox for this reason. In this case, I am doing all the commands on the host system, which is Mac OS X.

On this host system, I have installed VMware, and use it for various unrelated tasks, however, docker does not support VMware (without commercial add-on packages). As a result, it will tend to use VIrtualBox instead.

I have no wish to install VirtualBox, and so can not use Docker on this machine (as docker does not natively support Mac OS X without using a virtual machine)

6

You have the Filebeat source directory (~/dev/logstash_investigation/beats/filebeat) outside of GOPATH (~/go) and I'm not sure that's a good idea. At the very least I'd expect that you'll have to run go get to download the missing packages, but even if that works you'd end up with two copies of the beats git.

While I haven't fully grasped GOPATH, the suggested source code organization, and when go get is run automatically, this should definitely work:

cd ~/go
export GOPATH=$(pwd)
go get github.com/elastic/beats
cd src/github.com/elastic/beats/filebeat
go get
make
2 Likes
7

Hello Magnusbaeck,

That does definitely work, thanks!

I now have:

$ ./filebeat --version
filebeat version 1.2.0-SNAPSHOT (amd64)

So this is a success.