Extracting values from log body

Elastic Stack Logstash
1

Hi there,

Suppose I am getting a log message through logstash, with some custom format of the form FIELD1=VALUE1 etc. I am still quite a bit in the dark here, but I've been playing a little with visualizations. What I would like to do is obtain a unique count of, say, values of FIELD1. I think this is called an aggregation what I'm trying to do.

My question is: do I need to grok these fields at the logstash stage, and put all values in separate fields? Or can the field be parsed at search-time, Splunk-style? Or is there another recommended solution? I'm looking for "the ELK way" here.

Thanks for your thoughts.

Dan

2

Hi Dan,

I've moved your question to the Logstash forum as it seems to be the more appropriate place for this question.

Daniel

3

Hi Daniel, and thanks. I honestly wouldn't know, hopefully it will get answered there.

4

Hi,
If I understand your use case properly, I would try the kv filter, it seems the most suited to answer your needs here.

https://www.elastic.co/guide/en/logstash/current/plugins-filters-kv.html

  • purbon
5

Hi Pere, and thanks for the answer. It doesn't really address my question though.
I am using kv, of course. The question is, do I need it?

6

The fields need to be extracted at index time in order for them to be available for aggregations in Kibana.

7

Thanks Christian! This answers my question.