Multiple metricbeat modules each with different output index

Elastic Stack Beats
1

Is it possible to have different modules within a single metricbeat send to different output index location?

Like this in theory:

-module1
metricset
output index1

-module2
metricset
output index2

2

Do you mean elasticseach index? you can use format strings in the Elasticsearch index setting. But then you are responsible for applying the mapping template yourself.

See index setting docs.

One can do something like:

metricbeat.modules:
- module: ...
  fields.index: "module1"

outut.elasticsearch:
  ...
  index: "%{fields.index}-%{+yyyy.MM.dd}"
3

Thank you... However, I'm not sure that answers it exactly. Can you have one module/metricset going to one index, and another module/metricset to a different index in a single metricbeat?

For example:

-module: docker
metricsets:

  • memory
  • cpu
    hosts: ["unix:///var/run/docker.sock"]
    enabled: true
    period: 15s

output.elasticsearch: hosts: ["localhost:9200"]
index: "docker-%[+yyyy.MM.dd]"

-module: system
metricsets: ["process"]
processes: ['.*']

output.elasticsearch: hosts: ["localhost:9200"]
index: "system-%[+yyyy.MM.dd]"

4

you can have only one elasticsearch output.

The syntax is %{...}, not %[...].

you can reference any fieldname %{[...]}. If want to set the index name to be the module name you can just use:

metricbeat.modules:
- module: system
  metricsets: ["process"]
  ...

- module: docker
  metricsets: ...
  ...

output.elasticsearch:
  index: '%{[metricset.module]}-%{+yyyy.MM.dd}'
1 Like
5

Great, looks like that will create two separate indexes which is my goal. Thank you!

6

Is there documentation on the fieldname variables one can use in the yml output? I am looking to do something similar with filebeats for different prospectors sent to their own index.

7

The beats docs do contain a section with all fields names being published by default. Using the fields settings users can add additional fields.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.