HowTo - Filebeat on OpenBSD 5.9

Elastic Stack Beats
1

Hi,

I am trying to setup filebeat on OpenBSD 5.9.
But I am new to OpenBSD.
I need help on installation and configuration.

I followed this "Filebeat compile on OpenBSD 6.0" procedure but didn't work me I got many errors while "make install"

2

What errors during make install?

3

I am getting dependency errors
while building go lang from src.
Somehow "make install" passed and go-1.5.3 is installed.

Now I got stuck after cloning beats repo,
cd filebeat
go get
github.com/elastic/beats/libbeat/outputs
../libbeat/outputs/tls.go:64: undefined: tls.TLS_RSA_WITH_AES_128_GCM_SHA256
../libbeat/outputs/tls.go:66: undefined: tls.TLS_RSA_WITH_AES_256_GCM_SHA384

gmake
github.com/elastic/beats/libbeat/outputs
../libbeat/outputs/tls.go:64: undefined: tls.TLS_RSA_WITH_AES_128_GCM_SHA256
../libbeat/outputs/tls.go:66: undefined: tls.TLS_RSA_WITH_AES_256_GCM_SHA384
../libbeat/scripts/Makefile:73: recipe for target 'filebeat' failed
gmake: *** [filebeat] Error 2

4

I think you need at least Go 1.6 or 1.7. The Go version requirement is listed in the documentation somewhere.

5

Yes I see it but when I make install from /usr/ports/lang/go
I got 1.5

I have also referred go installation documentation for OpenBSD, there it is mentioned that for OpenBSD 5.9 (Mar 29, 2016) go-1.5.4

Is there any alternative for filebeat!

Can I configure logstash to push OpenBSD logs to elasticsearch!

6

Yes I see it but when I make install from /usr/ports/lang/go
I got 1.5

You can compile a newer Go compiler yourself.

Can I configure logstash to push OpenBSD logs to elasticsearch!

It seems Logstash is available from ports so I suppose Logstash works on OpenBSD. In that case you can configure it to ship logs to ES.

7

The project is using Go 1.7.4 at the moment. See CONTRIBUTING.md.

8

Thanks for the Information.

I am facing issues while compiling go-1.7.
I have installed go-1.5 I want to upgrade it 1.7 on OpenBSD 5.9 .
I am new to go and openbsd. I am following go official documentation but still facing issues.

9

Again, you won't get any help until you tell us what issues you're having. We are not psychic, but some of us are impatient.

10

sounds like you're messing with go development environment setup. You need to set GOPATH and optionally GOROOT in case you have multiple go versions.

Use go version and go env to check your environment. Please, post output, without logs and just prose we have no way to see and confirm what you're doing is right or wrong.

Having go1.5 you will have to download and install go1.7 source yourself.

This document explains how to install go from source (you still need go1.5 compiler to build go1.7). Some people use gimme or gvm.

11

I have installed go in OpenBSD with pkg_add go
I got version go1.5.3
Below are my outputs.

#go version
go version go1.5.3 openbsd/amd64

go env

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="openbsd"
GOOS="openbsd"
GOPATH=""
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/openbsd_amd64"
GO15VENDOREXPERIMENT=""
CC="cc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="c++"
CGO_ENABLED="1"

12

After installing go with pkg_add

I followed the official go document
Step 1: I have set GOPATH to $HOME/gopath
Step 2: I have fetched the go repo
Step 3: checked out with 1.7.5
Step 4: Executed ./all.bash from src directory

At this point I faced issue:
./all.bash

Building Go bootstrap tool.

cmd/dist
ERROR: Cannot find /root/go1.4/bin/go.
Set $GOROOT_BOOTSTRAP to a working Go tree >= Go 1.4.

then exported "GOROOT_BOOTSTRAP=/usr/local/go"

Again Executed " ./all.bash "
it ran for a while smoothly and exited with exit code 1
Below are some output as the output is very huge, couldn't paste everything here

export GOROOT_BOOTSTRAP=/usr/local/go
./all.bash

Building Go bootstrap tool.

cmd/dist

Building Go toolchain using /usr/local/go.

bootstrap/cmd/internal/dwarf
bootstrap/cmd/internal/src
bootstrap/cmd/internal/sys

Building go_bootstrap for host, openbsd/amd64.

runtime/internal/sys
runtime/internal/atomic
runtime
unicode
internal/race
sync/atomic
encoding
unicode/utf8

Building packages and commands for openbsd/amd64.

runtime/internal/sys
runtime/internal/atomic
runtime
errors
internal/race
sync/atomic

Testing packages.

ok archive/tar 0.079s
ok archive/zip 1.576s
ok bufio 0.099s
ok bytes 0.412s
ok compress/bzip2 0.286s
ok compress/flate 2.769s
ok compress/gzip 0.123s
--- FAIL: TestGoroutineParallelism2 (0.34s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x63cedf]

    • -FAIL runtime 5.450s
      2017/02/08 18:48:26 Failed: exit status 1

Did I do anything wrong!

13

make install gives below errors

cd /usr/ports/lang/go
make install
===> Building for go-1.7.4

Building Go bootstrap tool.

cmd/dist
ERROR: Cannot find /usr/local/go/bootstrap/bin/go.
Set $GOROOT_BOOTSTRAP to a working Go tree >= Go 1.4.
*** Error 1 in . (Makefile:63 'do-build': @cd /usr/ports/pobj/go-1.7.4/go/src && CC="cc" CXX="c++" GOROOT=/usr/ports/pobj/go-1.7.4/go GO...)
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2683 '/usr/ports/pobj/go-1.7.4/build-amd64/.build_done')
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:1884 '/usr/ports/packages/amd64/all/go-1.7.4.tgz')
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2409 '_internal-package')
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:2389 'package')
*** Error 1 in . (/usr/ports/infrastructure/mk/bsd.port.mk:1901 '/var/db/pkg/go-1.7.4/+CONTENTS')
*** Error 1 in /home/ports/lang/go (/usr/ports/infrastructure/mk/bsd.port.mk:2389 'install')

14

This says, some of the go runtime unit tests not working. Not sure if it's a bug. As I don't run go on OpenBSD, check out this wiki entry.

I also see some oldish filebeat in the ports. Maybe related. Having had a look at the issue list, some tests on OpenBSD seem to be flaky. Maybe next time you compile it works or try compiling without running tests... Have you checked the golang-nuts mailing list?

15

I tried repeatedly in different ways like with tests and skipping tests.

I didn't check with golang-nuts mailing list.

I need some suggestions regarding other agents.
I have tried deprecated logstash-forwarder 0.4.0 on test OpenBSD 5.9 machine. It is working absolutely fine. Can I install that on my production system what are the issues I may face on long run!

16

logstash-forwarder is deprecated, that is, we don't really support this one anymore. The beats input plugin in logstash 5.x is still backwards compatible supporting both, beats and logstash-forwarder. The protocol used by logstash-forwarder has a problem, potentially overloading your logstash in case of logstash being not 'responsive', by resending a batch of events logstash might choke on (e.g. grok taking much too long). log-courier is a fork of logstash-forwarder, trying to address this problem.

17

What is your suggestions installing ELK agents on OpenBSD5.9

Which one do I need to install on OpenBSD5.9.

I thought of installing logstash to send logs to elastic search but it requires java I don't want to load OpenBSD machine with unnecessary services because that machine is serving for another purpose.
I wanted to process logs with lite application like filebeat.

18

Hi,

I am able to install go1.8 from source code (https://golang.org/doc/install/source)

Built filebeat using go1.8

19

Awesome. I guess you hit some bug in go1.7 on OpenBSD.

20

Is it really a bug in go1.7!