# "make package" broken for community beats?

**URL:** https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511
**Category:** Beats
**Created:** [January 4, 2017, 5:32am UTC](https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511 "2017-01-04T05:32:51Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![fjgal](https://avatars.discourse-cdn.com/v4/letter/f/35a633/32.png) [@fjgal](https://discuss.elastic.co/u/fjgal)
#### Post date: [January 4, 2017, 5:32am UTC](https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511/1 "2017-01-04T05:32:51Z")

</div>

Hi,

I'm trying to build a community beat based on metribeat and it fails. To reproduce the issue I created a dummy beat following these steps:

```
$ cd $GOPATH/src/github.com/someuser
$ cookiecutter $GOPATH/src/github.com/elastic/beats/generate/metricbeat/metricset
project_name [Examplebeat]: somebeat
github_name [your-github-name]: someuser
beat [somebeat]: somebeat
beat_path [github.com/someuser]: github.com/someuser
full_name [Firstname Lastname]: Some User

$ cd somebeat/
$ make setup
$ make update
$ make package

```

I found two issues:

**1) incorrect destination directory in [build.sh](http://build.sh)**

"make package" fails as follows:

```
Working directory=/go/src/github.com
Copying main source folder /source to folder /go/src/github.com
Fetching dependencies...
Execute /scripts/before_build.sh github.com/someuser
/scripts/before_build.sh: 13: cd: can't cd to /go/src/github.com/someuser
make[1]: *** [prepare-package-cgo] Error 2
make: *** [package] Error 2

```

The issue seems to be that the argument passed to [build.sh](https://github.com/elastic/beats/blob/master/dev-tools/packer/docker/xgo-image/base/build.sh) is the value of the BEAT\_DIR variable in the generated Makefile that doesn't contain the beat name.

Extract from generated Makefile

```
BEATNAME=somebeat
BEAT_DIR=github.com/someuser

```

[build.sh](http://build.sh) is extracting the directory name using the dirname command, thus removing the last part of the BEAT\_DIR variable

Extract from [build.sh](http://build.sh)

```
BEAT_PATH=$1
DST_FOLDER=`dirname $GOPATH/src/$BEAT_PATH`
GIT_REPO=$BEAT_PATH

```

In the example BEAT\_PATH is "[github.com/someuser](http://github.com/someuser)", DST\_FOLDER will be "/go/src/github.com". [build.sh](http://build.sh) creates the DST\_FOLDER (/go/src/github.com). Then [before\_build.sh](https://github.com/elastic/beats/blob/master/dev-tools/packer/xgo-scripts/before_build.sh) is called with $BEAT\_PATH ([github.com/someuser](http://github.com/someuser)) as argument and tries to change directory to /go/src/${1} (/go/src/github.com/someuser) and fails.

To workaround the issue I modified [build.sh](http://build.sh) to:

```
BEAT_NAME=$PACK
BEAT_PATH=$1/$BEAT_NAME
DST_FOLDER=`dirname $GOPATH/src/$BEAT_PATH`
GIT_REPO=$BEAT_PATH

```

There are multiple ways in which this could be solved, I tried a few, but this is the only one I found that didn't have side-effects. Not sure if this is the correct way though.

After applying the above workaround, I faced a second issue.

**2) make package tries to modify metricbeat specific files (metricbeat-win.yml and metricbeat-win.full.yml)**

It fails with the following error:

```
[...]
Execute /scripts/before_build.sh github.com/someuser/somebeat
install -d -m 755 /build/homedir/scripts/
install -m 755 ./vendor/github.com/elastic/beats/libbeat/scripts/migrate_beat_config_1_x_to_5_0.py /build/homedir/scripts/
Compiling import_dashboards for windows/amd64
Compiling import_dashboards for windows/386
Compiling import_dashboards for darwin/amd64
Cloning into '/go/src/github.com/tsg/gotpl'...
# disable the system/load metricset on windows
sed -i.bk 's/- load/#- load/' /build/metricbeat-win.yml
sed: can't read /build/metricbeat-win.yml: No such file or directory
make: *** [before-build] Error 2
make[1]: *** [prepare-package-cgo] Error 2
make: *** [package] Error 2

```

I tracked the problem down to [metricbeat/Makefile](https://github.com/elastic/beats/blob/master/metricbeat/Makefile) (line 56-63):

```
# This is called by the beats packer before building starts
.PHONY: before-build
before-build:
	# disable the system/load metricset on windows
	sed -i.bk 's/- load/#- load/' $(PREFIX)/metricbeat-win.yml
	rm $(PREFIX)/metricbeat-win.yml.bk
	sed -i.bk 's/- load/#- load/' $(PREFIX)/metricbeat-win.full.yml
	rm $(PREFIX)/metricbeat-win.full.yml.bk

```

metricbeat/Makefile is included in the generated Makefile for the community beat, and when called it fails since the \*metricbeat-win\*.yml don't exist for the community beat:

```
# disable the system/load metricset on windows
    sed -i.bk 's/- load/#- load/' /build/metricbeat-win.yml
    sed: can't read /build/metricbeat-win.yml: No such file or directory

```

The sed commands should only be executed when packaging metricbeat and not for other beats based on it. A possible solution (probably not the cleanest) is to introduce a condition on the BEATNAME, changing the above lines to:

```
# This is called by the beats packer before building starts
.PHONY: before-build
before-build:
ifeq ($(BEATNAME), metricbeat)
        # disable the system/load metricset on windows
        sed -i.bk 's/- load/#- load/' $(PREFIX)/metricbeat-win.yml
        rm $(PREFIX)/metricbeat-win.yml.bk
        sed -i.bk 's/- load/#- load/' $(PREFIX)/metricbeat-win.full.yml
        rm $(PREFIX)/metricbeat-win.full.yml.bk
endif

```

This issue seems to be introduced by [dd43dd1](https://github.com/elastic/beats/commit/dd43dd1b1e5aa943dea3fdfec6ab035992db47d8)

I tested with the two changes and the community beat gets properly packaged.

If you confirm these are valid issues I can submit separate github issues and pull requests for each one.

Thank you!

/F

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [January 5, 2017, 12:34pm UTC](https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511/2 "2017-01-05T12:34:44Z")

</div>

Thanks for testing this and also finding some workaround. These are definitively valid issue. It would be great if you could open a PR / Github issue for it.

Especially the BEAT\_PATH part is tricky (and ugly). We already have some tweaks inside to support community beats, but we missed the ones based on metricbeat ☹ Lets discuss / test these changes on a PR, then we see also if it does not break our packaging 😃

---

<div class="post-metadata">

### Author: ![fjgal](https://avatars.discourse-cdn.com/v4/letter/f/35a633/32.png) [@fjgal](https://discuss.elastic.co/u/fjgal)
#### Post date: [January 6, 2017, 2:12am UTC](https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511/3 "2017-01-06T02:12:22Z")

</div>

Thanks @ruflin for the feedback. My pleasure to help improving this already great product.

I created Github issues and PRs:

Issue#1

> <https://github.com/elastic/beats/issues/3294>

  

> <https://github.com/elastic/beats/pull/3295>

Issue#2

> <https://github.com/elastic/beats/issues/3296>

  

> <https://github.com/elastic/beats/pull/3297>

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [January 6, 2017, 3:21pm UTC](https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511/4 "2017-01-06T15:21:30Z")

</div>

Thanks. I continued the discussions there.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 25, 2017, 5:33am UTC](https://discuss.elastic.co/t/make-package-broken-for-community-beats/70511/5 "2017-01-25T05:33:06Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.
