Remove nested dynamic fields

Elastic Stack Logstash
1

Hey, Was hoping someone can help me. I'm trying to reduce the amount of fields that are being created in elasticsearch. I'm using the kafka input for logstash 2.0 and elasticsearch 2.0.

I would like to remove the [bids][][][id] fields, only I'm not sure how to do this.

Can someone share how I would remove these?

"bids" => {
"header_ad" => [
[0] {
"id" => "33",
"w" => 728,
"h" => 90
}
},
[1] {
"id" => "21",

2

So you only want to keep the w and h subfields? You'll have to use a ruby filter for that. I don't have time to write an example right now.

3

Thank you, I figured that would be the case. If anyone has time to write an example that would be great. I haven't used the ruby filter before.

IN that case yes, I just want to keep the w / h. Basically I want to remove the bids.header_ad.[0].id and bids.header_ad.[1].id and so on. This is much bigger than what I've shared but if I can get an example of how to remove those ones I can use it on the rest.

The problem is I have over 85000 fields coming into elasticsearch which is fine except it makes Kibana useless because I run out of browser memory before it can even load the discover window