Tagging Patterns of data into a field

Elastic Stack Logstash
1

Hi,

I have a data set which has a field that contain different values like:

FieldA : AAAA-REL
FieldA : AAAA-DEV
FieldA : BBBB-REL
FieldA : DDDD-REL
FieldA : BBBB-DEV

... and so on

I want to tag or categorize these fields by adding a tag or a new field and grouping them as say:

All fields that contain "AAAA*" , tag them as say "Prime"
All fields that contain "BBBB*" tag them as say "Secondary"
All that contain "CCCC*" tag them as say "Tertiary"

and so on. Note that the mapping could go on to ZZZZ.
I could use grok pattern for each of these, but looks like thats a lot of processing and would probablu impact the performance.

  • Is there an easier way to do this mapping, like for ex have a file of patterns and then look it up before tagging?
  • My logstash talks to elastic. Is there anything in elastic I could use to achieve this
  • If not, I use Kibana. Can scripted field help for this scenario?
2

If you can extract the classification criteria (AAAA, BBBB etc) to a separate field with e.g. a grok filter you can use a translate filter to look up the value in a separate file and avoid having dozens (or hundreds?) of grok or mutate filters.

3

Hi Magnus,

I did that already. I have a filter that groks just AAAA or BBBB etc into a seperate field. But then, AAAA is not a friendly field as it needs to be represented as say "Prime" or tagged as "Prime". So I am still at the same place.

4

As I said, use a translate filter to turn "AAAA" into "Prime".

5

Thats a nice one! Thanks Magnus, I will try this out.