How to cut and delete elements in a array with Logstash

Elastic Stack Logstash
1

Hi everybody!!

I have Json logs in a array as this:

e":[{"n":"3/0/1","st":"CONTENT","v":"Sensortag"},
{"n":"3/0/3","st":"CONTENT","v":"Contiki-develop-20150508-409-g2147b9e"},
{"n":"3/0/13","st":"CONTENT","v":"1970-01-09T21:02:18Z"},
{"n":"3301/0/5700","st":"CONTENT","v":"376.64"},
{"n":"3303/0/5700","st":"CONTENT","v":"22.843"},
{"n":"3304/0/5700","st":"CONTENT","v":"63.53"},
{"n":"3315/0/5700","st":"CONTENT","v":"1000.34"}]

I would like deleting first 3 elements some array and keeping 4 last ones with a filter

I have this in my filter:

filter {

if ([type] == "testbed"){

       if [MessageParserJson][e[{}] in [MessageParserJson]{
               mutate {
                       remove_field => ["[MessageparserJson][e[{0}]]" , "[MessageparserJson][e[{1}]]" , "[MessageParserJson][e[{2}]]"]
                       add_field => { "[MessageParserJson][e[{3}]]" => "MessageParser" }
                       add_field => { "[MessageParserJson][e[{4}]]" => "MessageParser" }

                       add_field => { "[MessageParserJson][e[{5}]]" => "MessageParser" }
                       add_field => { "[MessageParserJson][e[{6}]]" => "MessageParser" }
                        }
                       }

               drop {
                       remove_field => ["MessageParserJson"]
                       }

}
}

But Logstash puts itself in error

Thank you for help, good day!!

2 
   if [MessageParserJson][e[{}] in [MessageParserJson]{

I don't know what you're trying to do here.

remove_field => ["[MessageparserJson][e[{0}]]" , "[MessageparserJson][e[{1}]]" , "[MessageParserJson][e[{2}]]"]

I'd expect this to be the correct syntax:

remove_field => ["[MessageparserJson][e][0]" , "[MessageparserJson][e][1]" , "[MessageParserJson][e][2]"]

add_field => { "[MessageParserJson][e[{4}]]" => "MessageParser" }

Do you want to add the string "MessageParser" or the contents of the field with that name? In the latter case you need to say "%{MessageParser}".

It's probably easier to do all this with a ruby filter.

1 Like
3

tks!!

add_field => { "[MessageParserJson][e[{4}]]" => "MessageParser" }

yes,

With this, I want to add the contents Of the position 4 of the array in MessageParser

4

Oh. In that case you have things backwards. What you have means "put the string 'MessageParser' in position four of the array".

5

if [MessageParserJson][e[{}] in [MessageParserJson]

Here, I try to verify if the array "e" is in the field "MessageParserJson". But I do not know if the syntax is good

6

"@version" => "1",
"@timestamp" => "2015-10-15T09:03:03.997Z",
"type" => "testbed",
"host" => "template-14-04",
"path" => "/home/ao/leshan-logs.txt",
"MessageParserJson" => {
"bt" => 1442578245,
"bn" => "urn:dev:mac:00124b0006a00987/",
"e" => [
[0] {
"n" => "3/0/1",
"st" => "CONTENT",
"v" => "Sensortag"
},
[1] {
"n" => "3/0/3",
"st" => "CONTENT",
"v" => "Contiki-develop-20150508-409-g2147b9e"
},
[2] {
"n" => "3/0/13",
"st" => "CONTENT",
"v" => "1970-01-01T03:18:56Z"
},
[3] {
"n" => "3301/0/5700",
"st" => "CONTENT",
"v" => "255.36"
},
[4] {
"n" => "3303/0/5700",
"st" => "CONTENT",
"v" => "22.843"
},
[5] {
"n" => "3304/0/5700",
"st" => "CONTENT",
"v" => "65.88"
},
[6] {
"n" => "3315/0/5700",
"st" => "CONTENT",
"v" => "994.2"
}
]
}
}