[SOLVED] Build packetbeat on windows

Elastic Stack Beats
1

I am trying to build packetbeat on a windows 10 computer, steps so far:

install:
Go 1.5.1
Git 2.6.3
bzr 2.5.1
gopath=c:\go\bin
goroot=c:\go\

created a directory structure c:\go\src\github.com\elastic
cd c:\go\src\github.com\elastic
git clone https://github.com/elastic/packetbeat.git
cd packetbeat

go install
then i get this error:

github.com/elastic/packetbeat/protos/thrift

protos\thrift\thrift_idl.go:30: field.Id undefined (type *parser.Field has no field or method Id, but does have ID)
protos\thrift\thrift_idl.go:31: field.Id undefined (type *parser.Field has no field or method Id, but does have ID)
protos\thrift\thrift_idl.go:39: field.Id undefined (type *parser.Field has no field or method Id, but does have ID)

github.com/tsg/gopacket/pcap

exec: "gcc": executable file not found in %PATH%

Does anyone know how to fix this?

Cheers

2

packetbeat is using godep for vendoring. The thrift library on your system is newer then the one used by packetbeat. In order to build packetbeat go to $GOPATH/src/github.com/elastic/packetbeat and type 'make'. This gets you the executable packetbeat in said directory.

3

I think you are hitting two issues: one is that you don't compile against the deps from Godep, like Steffen is saying, and the other is that you need a C compiler for the Cgo parts.

For the Godep part, if you don't want/can't use the Makefile, you can setup a custom GOPATH, that includes the Godeps/_worspace folder, e.g. GOPATH=c:\go;c:\go\src\github.com\elastic\packetbeat. Then you can use go build normally.

For the C compiler part, I recommend installing mingw, if you have chocolatey, it's as easy as choco.exe install mingw

Also, any reason for compiling it yourself? We do provide windows binaries.

4

I am really new to Go, and I can't figure out why make does not work:
C:\Go\src\github.com\elastic\packetbeat>make
Must run make.bat from Go src directory.

Running make from the c:\src gives this:
C:\Go\src>make github.com/elastic/packetbeat

Building Go bootstrap tool.

cmd/dist
ERROR: Cannot find C:\Users\admin\Go1.4\bin\go.exe
"Set GOROOT_BOOTSTRAP to a working Go tree >= Go 1.4."

And when I try and add the GOROOT_BOOTSTRAP variable my Go starts giving me compile not found errors and I have to reinstall Go to make it work again.

So I followed Tudor's advice and modified my GOPATH

My paths:
GOPATH=c:\go;c:\go\src\github.com\elastic\packetbeat
GOROOT=c:\go\

That gives me the following error (there is no header file for pcap.h on my system)

C:\Go\src\github.com\elastic\packetbeat>go build

github.com/elastic/packetbeat/protos/thrift

protos\thrift\thrift_idl.go:30: field.Id undefined (type *parser.Field has no field or method Id, but does have ID)
protos\thrift\thrift_idl.go:31: field.Id undefined (type *parser.Field has no field or method Id, but does have ID)
protos\thrift\thrift_idl.go:39: field.Id undefined (type *parser.Field has no field or method Id, but does have ID)

github.com/tsg/gopacket/pcap

....\tsg\gopacket\pcap\pcap.go:18:18: fatal error: pcap.h: No such file or directory
#include <pcap.h>
^
compilation terminated.

The reason why I want to compile myself is that I will try and add support for at second windows driver "Microsoft Network Monitor 3 Driver" so I don't have to install the Riverbed driver, hope that it makes sense :slight_smile:

5

I just tried to compile packetbeat on my mac and it just works, can I compile the windows binary from my mac? Or how do you manage to compile the windows binaries?

6

UPDATE: For packetbeat it is not as easy as described below because of cgo. The easy option is to just download the nightly build in case you want to run the most recent version: https://beats-nightlies.s3.amazonaws.com/index.html?prefix=packetbeat/ Otherwise have a look at https://github.com/elastic/beats-packer where we do the builds ourself. make xgo-image && make packetbeat should to the build.

@bering Yes, you can create the windows binary on your Mac. Use make crosscompile for this. It will run the scripts/crosscompile.sh script and built it for all available platforms.

7

I have just tried to build packetbeat again from my mac, I cloned the new beats with git clone https://github.com/elastic/beats
From src/github.com/elastic/beats/packetbeat I can run make without any errors but running make crosscompile for darwin and windows gives me this error:

go get github.com/mitchellh/gox
mkdir -p build/bin
gox -output="build/bin/{{.Dir}}-{{.OS}}-{{.Arch}}" -os="darwin windows" 
Number of parallel builds: 7

-->   windows/amd64: github.com/elastic/beats/packetbeat
-->    darwin/amd64: github.com/elastic/beats/packetbeat
-->     windows/386: github.com/elastic/beats/packetbeat
-->      darwin/386: github.com/elastic/beats/packetbeat

4 errors occurred:
--> darwin/amd64 error: exit status 2
Stderr: # github.com/elastic/beats/packetbeat/sniffer
sniffer/sniffer.go:26: undefined: pcap.Handle
sniffer/sniffer.go:31: undefined: pcap.Dumper

--> windows/amd64 error: exit status 2
Stderr: # github.com/elastic/beats/packetbeat/sniffer
sniffer/sniffer.go:26: undefined: pcap.Handle
sniffer/sniffer.go:31: undefined: pcap.Dumper

--> darwin/386 error: exit status 2
Stderr: # github.com/elastic/beats/packetbeat/sniffer
sniffer/sniffer.go:26: undefined: pcap.Handle
sniffer/sniffer.go:31: undefined: pcap.Dumper

--> windows/386 error: exit status 2
Stderr: # github.com/elastic/beats/packetbeat/sniffer
sniffer/sniffer.go:26: undefined: pcap.Handle
sniffer/sniffer.go:31: undefined: pcap.Dumper

make: *** [crosscompile] Error 1

Can anyone see what I am missing and how the get past this message?

8

We do not use make crosscompile to build Packetbeat because Packetbeat uses cgo, a golang command that enables go code to depend on C code. Because of this dependency you must have a C compiler for Windows, the required header files, and libraries in order to build.

Like @ruflin said, we use beats-packer to build for Windows. The beats-packer project has a docker container with a C cross-compiler and the required pcap libraries for Windows. Here's the command that does the build in the container. To use beats-packer you would just need to have docker setup and then run make xgo-image && make packetbeat.

To build Packetbeat on Windows you would need to have golang setup, install mingw, install winpcap developer pack to C:/WpdPack/Lib. Our Vagrant box in elastic/beats has all of this setup except for the winpcap developer pack (been meaning to fix this).

9

I downloaded the win2012 vigrant box and was able to build packetbeat on it by adding WpdPack 4.1.2 to c:\wpdpack\lib

I compared the setup from the vigrant box to my windows 10 setup and noticed that there was an additional environment variable set:
GO15VENDOREXPERIMENT=1
With this variable set I am able to build packetbeat on my windows 10 :slight_smile: this enables go to use the go files from the vendor directory.

10

Yeah, over the time of this post we switched from godep go using vendor go experiment. Glad to hear it works now.

11

@bering FYI: The Vagrant box has been updated to include the WinPcap developer pack.