How to add other _types to the /etc/fields.yml in beats

(Blacktop) #1

I have a beat that will have MANY different _types.

beats/libbeat/scripts/ doesn't seem to support that?

Any help greatly appreciated.

It looks like you can add more _types with filebeat modles ect, that looks like what I want to do.


(Andrew Kroh) #2

Packetbeat uses a different type for each protocol. Have a look at its fields.yml.

(Blacktop) #3

So I tried using multiple keys, but after running make update I believe that beats/libbeat/scripts/ did not parse it correctly. I will try again.

(Blacktop) #4

I tried again and again it didn't seem to parse out he addtional _types with the key field

@andrewkroh is it because I am using the /etc/fields.yml and not the _meta folder?

I did a cookiecutter to init the beat, but maybe that was an older version?


(Blacktop) #5

I blew my beat away and started from scratch and make update didn't pick up the extra -key _types :cry:

(Andrew Kroh) #6

The vendored version of the script you have appears to be looking at _meta/fields.yml.

(Blacktop) #7

So I was able to figure out how to add more sub-fields except that the way that it works is it creates nested fields under the _default mapping.

I usually like to use document _types to pull apart.. well document types. Is there a reason you have designed it this way? I can see that you are going to start using the concept of modules. For example an nginx filebeat module that I assume would create fields like this:

_index: filebeat-*
_type: filebeat
nginx.access.request.method: GET

To me it makes sense to have it be like:

_index: filebeat-*
_type: nginx-access
request.method: GET

I just think more deeply nested field names are harder to query?


(ruflin) #8

This blog post should share some insights on types vs index: In our case it didn't bring an advantage.

(Blacktop) #9

thank you @ruflin!

(system) #10

This topic was automatically closed after 21 days. New replies are no longer allowed.