Aggregation of Regex of Terms


(Bryce Kaspar) #1

I have some textual data (snort alert messages) I would like to do a regex of a field and then an aggregation (count by day) of the results of that regex'd field. For instance:

data:
"01/01/2016","WEB more words here"
"01/01/2016","WEB different words"
"01/01/2016","MS-SQL woooorrrddssss"

I regex out to just the Caps bit at the beginning to get for that day:
"WEB",2
"MS-SQL",1

I can't quite figure out if Kibana can DO that, or how to do it if it can.


(Tanya Bragin) #2

If you want to parse out a specific field and then do aggregations on it, the parsing step needs to happen prior to indexing the data in Elasticsearch. In ES 1.x and 2.x (currently shipping versions), grok filter in Logstash can do it, or in 5.0 (still in development) Ingest Node in Elasticsearch will contain a grok pipeline step.


(system) #3