How to delete beat.name field?

[John16 (John)]

Hello,

I am using filebeat to store logs in ES. filebeat adds some beat.XXX fields and I want to remove them (in order to save space).

I am using ES ingest pipeline to process my logs.

I tried the following ingest processor:
"remove" : { "field": "beat.name" },

but it does not work (field still present in ES).
The same remove processor removes my custom fields without a problem.

Why beat.XXX fields are immortal?

Thanks.

[ywelsch (Yannick Welsch)]

Have you checked that the newly indexed documents still contain the beat.name field? Existing indexed documents will still carry that field and you cannot remove an existing field from the mappings.

Maybe also related to:

Using the Remove processor for ingest node

Hi, I am trying to remove some fields from the document using the REMOVE processor in the pipeline. I tried: remove { "field"=>"a", "field"=>"b", "field"=>"c", "field"=>"d", "field"=>"e", "ignore_failure" : true } And: remove { "field"=>"a", "ignore_failure" : true }, remove { "field"=>"b", "ignore_failure" : true }, remove { "field"=>"c", "ignore_failure" : true }, remove { "field"=>"d", "ignore_failure" : true }, remove { "field"=>"e", "ignore_failure" : true } B…

[John16 (John)]

@ywelsch

Maybe also related to

Oh, yes! The answer by @mvg in that thread solved my problem.
I was missing extra { ... } around each processor!

Well, it is really sad that I have no parse error during pipeline creating, but rather such an unexpected behaviour later