How to parse "DTG string string ..."?

Elastic Stack Logstash
1

I created a log with:

  1. DTG
  2. All users currently logged on.

Sample log:
Mon, 29 Feb 2016 17:28:01 +0000 bob fred xavier

I can get the DTG with:

filter {
grok {
match => { "message" => "%{DATESTAMP_RFC2822:dtg}" }
}
}

What is a good way to also get the users?

Thanks,
Blake

2

How about

%{DATESTAMP_RFC2822:dtg} %{GREEDYDATA:users}
3

That would work, but I want to be able to count the number of users. Could I do that with GREEDYDATA?

I was searching for a solution and it looks like I need to loop through an array, but I don't know how to tackle that.

Any suggestions?

4

The grok filter doesn't count anything, it only extracts text to fields. You can use the mutate filter's split option to turn a string of space-separated usernames into an array. I don't think there's a stock filter for counting the number of elements in an array, but the ruby filter is always an option (that filter could of course also do the splitting).

mutate {
  split => { "users" => " " }
}
ruby {
  code => "event['user_count'] = event['users'].length"
}
5

That looks good. My thought was to have Kibana do the counting by setting up a visualization metric with "Aggregation" set to "Count". There might be a better way to do this, but I'm thinking that %{GREEDYDATA:users} needs to be split up so that Kibana can do the counting.

Do you have any suggestions on how to make that happen?

6

I don't think aggregations can count the number of elements in an array field. If you want to use that kind of aggregation you'd have to split each event into one event per user, which might be possible with the split filter. But maybe you don't have to use a count aggregation; if you save the number of users into a field (as above) you could use a sum aggregation. Depending on what the data represents that may or may not be what you're looking for.