Logstash array length check

Elastic Stack Logstash
1

Hi ,
How can i check the length of an array in logstash.
I have a request that is request=/acoounts/debenhams/summary.
I had split it by "/".
I have to check the request.length i.e. the array size of the splitted array request.
I need to basically check if request[].length==4.

What is the syntax to do it.

regards
Dibyananda

2

You'll have to use a ruby filter.

ruby {
  code => "event['length'] = event['request'].length"
}
3

Hi I m trying to get lenght of event but not able to do so can u help me correcting below config..

filter {
ruby{
code => "if event.length > 5 then
event['new_msg']= 'test_message'
end
"
}

4

event.length won't work. You have to indicate which field you're interested in, e.g. with event.get('name-of-field').length. To set the value of a field with Logstash 2.4 or later, see https://www.elastic.co/guide/en/logstash/current/event-api.html.

5

Hi,
I want to get count of Event object. or either convert event object to array will also help me..

6

What does "count of Event object" mean? The number of fields in the event?

7

Yes exactly.. I want to get number of fields in event. As incoming events does not have fixed number of fields and i want to restrict number of fields.

8

In that case event.to_hash.length should work.

9

NOT helping below is my passed json from UDP and I am also attaching output from logstash
input :-
{
"timestamp":"2017-04-18 17:04:30",
"message":"My logger is now ready",
"messageLevel":"FATAL",
"component":6,
"transactionId":0,
"test":"message",
"test1":"message",
"test2":"message",
"test3":"message",
"test4":"message",
"test5":"message",
"test6":"message",
"test7":"message",
"test8":"message",
"test9":"message",
"test10":"message",
"test11":"message",
"test12":"message",
"test13":"message",
"test14":"message",
"test15":"message",
"test16":"message",
"test17":"message",
"test18":"message",
"test19":"message",
"test20":"message",
"test21":"message",
"test22":"message",
"test23":"message",
"test24":"message",
"test25":"message",
"test26":"message"
}

output json shown in kibana :-

{
"_index": "logstash-6-2017.04.18",
"_type": "udplogs",
"_id": "AVuA15Tl_55gfmV0iY7d",
"_score": null,
"_source": {
"message": "My logger is now ready",
"@version": "1",
"@timestamp": "2017-04-18T11:34:23.250Z",
"host": "10.10.3.44",
"port": 39516,
"type": "udplogs",
"count": 6,
"timestamp": "2017-04-18 17:04:22",
"messageLevel": "FATAL",
"component": "Back1-Others",
"transactionId": 0,
"test": "message",
"test1": "message",
"test2": "message",
"test3": "message",
"test4": "message",
"test5": "message",
"test6": "message",
"test7": "message",
"test8": "message",
"test9": "message",
"test10": "message",
"test11": "message",
"test12": "message",
"test13": "message",
"test14": "message",
"test15": "message",
"test16": "message",
"test17": "message",
"test18": "message",
"test19": "message",
"test20": "message",
"test21": "message",
"test22": "message",
"test23": "message",
"test24": "message",
"test25": "message",
"test26": "message",
"componentId": 6
},
"fields": {
"@timestamp": [
1492515263250
]
},
"sort": [
1492515263250
]
}

10

And what does your ruby filter look like?

11

ruby {
code => "
event['count'] = event.to_hash.length
"
}

12

I can't reproduce:

$ cat input.json 
{ "timestamp":"2017-04-18 17:04:30", "message":"My logger is now ready", "messageLevel":"FATAL", "component":6, "transactionId":0, "test":"message", "test1":"message", "test2":"message", "test3":"message", "test4":"message", "test5":"message", "test6":"message", "test7":"message", "test8":"message", "test9":"message", "test10":"message", "test11":"message", "test12":"message", "test13":"message", "test14":"message", "test15":"message", "test16":"message", "test17":"message", "test18":"message", "test19":"message", "test20":"message", "test21":"message", "test22":"message", "test23":"message", "test24":"message", "test25":"message", "test26":"message" }
$ cat test.config 
input { stdin { codec => json_lines } }
output { stdout { codec => rubydebug } }
filter {
  ruby {
    code => "
      event['count'] = event.to_hash.length
    "
  }
}
$ /opt/logstash/bin/logstash -f test.config < input.json
Settings: Default pipeline workers: 8
Pipeline main started
{
        "timestamp" => "2017-04-18 17:04:30",
          "message" => "My logger is now ready",
     "messageLevel" => "FATAL",
        "component" => 6,
    "transactionId" => 0,
             "test" => "message",
            "test1" => "message",
            "test2" => "message",
            "test3" => "message",
            "test4" => "message",
            "test5" => "message",
            "test6" => "message",
            "test7" => "message",
            "test8" => "message",
            "test9" => "message",
           "test10" => "message",
           "test11" => "message",
           "test12" => "message",
           "test13" => "message",
           "test14" => "message",
           "test15" => "message",
           "test16" => "message",
           "test17" => "message",
           "test18" => "message",
           "test19" => "message",
           "test20" => "message",
           "test21" => "message",
           "test22" => "message",
           "test23" => "message",
           "test24" => "message",
           "test25" => "message",
           "test26" => "message",
         "@version" => "1",
       "@timestamp" => "2017-04-18T13:19:47.223Z",
             "host" => "lnxolofon",
            "count" => 35
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}
13

Below is my logstash config.

input {
tcp {
'host' => '10.10.3.44'
'port' => '9564'
'type' => 'udplogs'
}
}

filter {
ruby {
# Cancel 90% of events
code => "
event['count'] = event.to_hash.length
"
}
if [type] == "udplogs" {
json {
source => 'message'
}
mutate {
add_field => {
"componentId" => "%{component}"
}
}
}
}

14

And where are you doing anything with the count field? Your comment says "Cancel 90% of events" but there's no cancelling taking place anywhere.

15

Yes i want to cancel events which had field count with more then 20 but in first i am not able to get count of event .