Logstash filtering

Elastic Stack Logstash
1

How do you filter something that has multiple of the same Keys. in my "message" i have 3 different _id keys.
im currently using kv to split up my data , but it is showing all 3 values into that 1 key

Its not working correctly. Let me see if this makes more sense
The message is `

`"_id": "2345345c-97e3453535", "event_id": 78406744, "event_values": {"processEvent/processCmdLine": "C:\WINDOWS\tasksche.exe /i", "processEvent/parentPid": 1756, "processEvent/processPath": "C:\Windows\tasksche.exe", "url": "dfgdfgfdgd", "hostname": "dfgdgdfgd", "primary_ip_address": "345345353",, "_id": "6756757567"}, "is_false_positive": false, "event_at": "2019-03-27T02:35:08.039Z", "source": "gdfgd", "matched_at": "2019-03-27T02:35:40.000Z", "url": "sfdsdfsdfsf", "_id": 334545,``

I am using kv to split it
kv {
#split by Commas, value defined by equal sign , and exclude fields
field_split => ","
value_split => ":"
}

But how do i keep these 3 _ID seperated because it is outputting like
_id": "2345345c-97e3453535", "6756757567"}, : 334545,`

2

If you have multiple fields with the same key then a kv filter will create an array. What else would you like it to do?

3 
key1:value2,key1:value2,key1:value3

Would would I rename the keys into 3 different keys with these values

I dont want key1 : value1, value2, value3

i want newkey: value1
newkey:value2
newkey:value3

5

Or is there a way to pass each keypair into a new field

6

Well you do not say exactly what you want, but you could add additional fields using a ruby filter.

    kv { }
    ruby {
        code => '
        foo = event.get("foo")
        if foo.kind_of?(Array)
            foo.each_index { |x|
                event.set("foo#{x}", foo[x])
            }
        end
        '
    }

which will turn

   "message" => "foo=a foo=b foo=c",

into

      "foo0" => "a",
      "foo1" => "b",
      "foo2" => "c"
1 Like
7

im assuming foo is the key that is has multiple values in it ?

8

Yes, as you can see in the "message" field that I am parsing.

9

Its not working correctly. Let me see if this makes more sense
The message is `

`"_id": "2345345c-97e3453535", "event_id": 78406744, "event_values": {"processEvent/processCmdLine": "C:\WINDOWS\tasksche.exe /i", "processEvent/parentPid": 1756, "processEvent/processPath": "C:\Windows\tasksche.exe", "url": "dfgdfgfdgd", "hostname": "dfgdgdfgd", "primary_ip_address": "345345353",, "_id": "6756757567"}, "is_false_positive": false, "event_at": "2019-03-27T02:35:08.039Z", "source": "gdfgd", "matched_at": "2019-03-27T02:35:40.000Z", "url": "sfdsdfsdfsf", "_id": 334545,``

I am using kv to split it
kv {
#split by Commas, value defined by equal sign , and exclude fields
field_split => ","
value_split => ":"
}

But how do i keep these 3 _ID seperated because it is outputting like
_id": "2345345c-97e3453535", "6756757567"}, : 334545,`

10

With that message I get two separate fields on the event, one of which has a leading space on the field name.

                      "\"_id\"" => "2345345c-97e3453535",
                     " \"_id\"" => [
    [0] "\"6756757567\"}",
    [1] "334545"
],

If I change field_split to include a space

field_split => " ,"

Then I get the 3 ids in a single array and that ruby filter I posted will work.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.