# Export template from custom beat based on metricbeat

**URL:** <https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250>\
**Category:** Beats\
**Tags:** beats-development\
**Created:** [February 9, 2018, 2:03pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250 "2018-02-09T14:03:21Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dcroonen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dcroonen/32/27450_2.png) [@dcroonen](https://discuss.elastic.co/u/dcroonen)\
**Post date:** [February 9, 2018, 2:03pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/1 "2018-02-09T14:03:22Z")

</div>

Hi

I created my own beat based on metricbeat 6.0 with it's own modules and metricsets. I also created the necessary `fields.yml` files for the modules and the metricsets and ran successfully `make update` which gave me a `kibana` directory under the beat `_meta` directory, so far so good.

Now I want to generate an index template to import into Kibana but when I run `beat export template > template.json` the process gets stuck forever resulting in an empty `template.json` file. Is this the correct way to get the index template file? Any suggestions how to get this index template file from a custom beat?

Kind Regards  
Davy

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 9, 2018, 2:13pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/2 "2018-02-09T14:13:51Z")

</div>

The `export template` command exports the template mapping for use with Elasticsearch. I guess this is what you need.

The index pattern for Kibana is generated on `setup` and installed when installing the dashboards via `setup`.

Can you share your fields.yml or even better link to repo for us to investigate? `beat export template` must not hang. I'm curious if this is an error in fields.yml or a bug in the exporter.

---

<div class="post-metadata">

**Author:** ![dcroonen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dcroonen/32/27450_2.png) [@dcroonen](https://discuss.elastic.co/u/dcroonen)\
**Post date:** [February 9, 2018, 2:40pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/3 "2018-02-09T14:40:33Z")

</div>

Hi @steffens

> The export template command exports the template mapping for use with Elasticsearch. I guess this is what you need.

Correct. As I'm pretty new to this, I mess up some terms from time to time 😟.

I uploaded my code and (config) files to [this repo](https://github.com/dcroonen/smartbeat) on Github. I hope it's enough for you to reproduce the issue.

Thanks in advance.

Regards

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 12, 2018, 5:04pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/4 "2018-02-12T17:04:45Z")

</div>

The problem is in your code. You're using `cmd/instance` directly. This creates and starts the beat, but removes the sub-command functionality.

Metricbeat has it's own 'root command': [https://github.com/elastic/beats/blob/master/metricbeat/cmd/root.go#L22](https://github.com/elastic/beats/blob/master/metricbeat/cmd/root.go#L22)

The `libbeat/cmd` package is used by metricbeat, to create a standardized beat with sub-commands + hooks the `export template` sub-command into the root command.

Makes me wonder, how come you're using the cmd/instance package instead of having a root command?

---

<div class="post-metadata">

**Author:** ![dcroonen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dcroonen/32/27450_2.png) [@dcroonen](https://discuss.elastic.co/u/dcroonen)\
**Post date:** [February 13, 2018, 8:56am UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/5 "2018-02-13T08:56:59Z")

</div>

@steffens Thanks for your insights and effort.

> [@steffens](#):
>
> Makes me wonder, how come you're using the cmd/instance package instead of having a root command?

To be honest I have no idea. I had some issues when creating a new beat based on metricbeat. After searching a bit, I found this [thread](https://discuss.elastic.co/t/step-3-init-and-create-the-metricset-failure/112046/11), which has a solution that also worked for me. Maybe this is the root cause of the problem?

Can I fix this manually or would it be better to create a new beat based on metricbeat?

---

<div class="post-metadata">

**Author:** ![dcroonen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dcroonen/32/27450_2.png) [@dcroonen](https://discuss.elastic.co/u/dcroonen)\
**Post date:** [February 13, 2018, 9:36am UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/6 "2018-02-13T09:36:16Z")

</div>

@steffens May I ask in which file you saw that I'm using the `cmd/instance` directly? I suppose it was `main.go` but when I generate a new beat based on metricbeat with the command `python ${GOPATH}/src/github.com/elastic/beats/script/generate.py --type=metricbeat` I'll get a `main.go` file more or less the same as the one I got?

Based on the 6.0 branch of metricbeat

```
package main

import (
	"os"

	"github.com/elastic/beats/libbeat/cmd/instance"
	"github.com/elastic/beats/metricbeat/beater"

	// Comment out the following line to exclude all official metricbeat modules and metricsets
	_ "github.com/elastic/beats/metricbeat/include"

	// Make sure all your modules and metricsets are linked in this file
	_ "github.com/dcroonen/examplebeat/include"
)

var Name = "examplebeat"

func main() {
	if err := instance.Run(Name, "", beater.New); err != nil {
		os.Exit(1)
	}
}

```

and based on the master branch of metricbeat:

```
package main

import (
	"os"

	"github.com/elastic/beats/libbeat/cmd/instance"
	"github.com/elastic/beats/metricbeat/beater"

	// Comment out the following line to exclude all official metricbeat modules and metricsets
	_ "github.com/elastic/beats/metricbeat/include"

	// Make sure all your modules and metricsets are linked in this file
	_ "github.com/dcroonen/masterbeat/include"
)

var Name = "masterbeat"
var IndexPrefix = "masterbeat"

func main() {
	if err := instance.Run(Name, IndexPrefix, "", beater.DefaultCreator()); err != nil {
		os.Exit(1)
	}
}

```

So at the moment I don't have clue where to look to hook in the sub commands again....

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 13, 2018, 12:04pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/7 "2018-02-13T12:04:23Z")

</div>

Checking the code [generators template](https://github.com/elastic/beats/tree/master/generator/metricbeat/%7Bbeat%7D), it indeed uses `instance.Run`. This is somewhat ok, but doesn't give you the sub-command functionality as used/provided by the other beats. Feel free to open a [github issue on missing sub-commands when using the generator](https://github.com/elastic/beats/issues).

Checking metricbeat code-base, I see why sub-commands support is missing. The code-base of local `cmd` packages is not much re-usable for use by custom beats right now. I just created [this ticket](https://github.com/elastic/beats/issues/6369).

If you want sub-command support you need to copy the main.go and the `cmd` package from [metricbeat](https://github.com/elastic/beats/tree/master/metricbeat). Then update `cmd/root.go` file to use your beater instance.

---

<div class="post-metadata">

**Author:** ![dcroonen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dcroonen/32/27450_2.png) [@dcroonen](https://discuss.elastic.co/u/dcroonen)\
**Post date:** [February 13, 2018, 12:37pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/8 "2018-02-13T12:37:27Z")

</div>

OK, that explains a lot. I'll open a ticket on github referencing this threat.

Thanks for your support.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 13, 2018, 12:37pm UTC](https://discuss.elastic.co/t/export-template-from-custom-beat-based-on-metricbeat/119250/9 "2018-03-13T12:37:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.