Help: How to filter unwanted metadata and fields from filebeats

Elastic Stack Kibana
1

I am very new to this filebeat shipping thing.
Once I ship the log to kibana I am getting so many meta data field both for kibana and filebeat...now I can filter it in kibana visualization but is there any way to filter out/exclude those field during log ingestion .

Thanks in advance..

2

Believe you are looking to drop fields in your filebeat config.

1 Like
3

Hi Aron.

I was just trying to use remove_field option...

remove_field => [ "message","log.file.path","agent.hostname","agent.id" ]

But unfortunately it is removing the 'message' only not other fields.

Could you please elaborate drop_field with some example.

Thanks in advance.

Reagrds,
Moksha

4

Try this format instead of dot notation.

remove_field => [ "message","[log][file][path]","[agent][hostname]","[agent][id]" ]
1 Like
5

Hi Aron,

Thanks its working now.
Another query ....I guess I can not exclude '_id' from the log as its being a metadata??
If its possible do we have any separate syntax for that??

Regards ,
Moksha

6

_id is metadata and lies outside _source data. You can only remove _source data.

1 Like
7

Hi Aaron,

Thanks a lot for the help...
Can I ask you for another help... with below topic

Thanks Again,
Moksha

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.