I want to tag or categorize these fields by adding a tag or a new field and grouping them as say:
All fields that contain "AAAA*" , tag them as say "Prime"
All fields that contain "BBBB*" tag them as say "Secondary"
All that contain "CCCC*" tag them as say "Tertiary"
and so on. Note that the mapping could go on to ZZZZ.
I could use grok pattern for each of these, but looks like thats a lot of processing and would probablu impact the performance.
Is there an easier way to do this mapping, like for ex have a file of patterns and then look it up before tagging?
My logstash talks to elastic. Is there anything in elastic I could use to achieve this
If not, I use Kibana. Can scripted field help for this scenario?
If you can extract the classification criteria (AAAA, BBBB etc) to a separate field with e.g. a grok filter you can use a translate filter to look up the value in a separate file and avoid having dozens (or hundreds?) of grok or mutate filters.
I did that already. I have a filter that groks just AAAA or BBBB etc into a seperate field. But then, AAAA is not a friendly field as it needs to be represented as say "Prime" or tagged as "Prime". So I am still at the same place.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.