How to add tags to the Document , based on the dictionary

Elastic Stack Logstash
1

Hello Team,

I am new to the ELK stack.
I have a use case where I need to tag documents based on the dictionary. Where Dictionary key is String and value is list of Strings.
So basically if the document field matches with dictionary key (regex pattern) then I want to add all the values as tags to that document.

For example.

Input
Doc1 :
{
"Field 1" :"A",
"Field 2":"B",
"Field 3":"abcd"

}

Doc2:
{
"Field 1" :"C",
"Field 2":"D",
"Field 3":"abcd1"

}

Dictionary :
{ "abcd1": <"abcd1","abcd11","abcd111">,
"abcd2":<abcd2","abcd22","abcd222">
}

Here Doc1 matches with both the Dictionary entries so Doc1 should be tagged with all the values ("abcd1","abcd11","abcd111","abcd2","abcd22","abcd222")

But Doc2 matches with only 1st Dictionary entry
So it should be tagged with only entry 1 values (
"abcd1","abcd11","abcd111")

Any pointers would be really helpful.

Thanks in advance.
Aditya

2

You can use

if "value" in "list" {
   # do stuff
}

I think this should work in your use case... It just a bunch of if else..

3

Hello pjanzen,

Thanks for your reply.

Is there any way we can add a loop and keep dictionary terms in a config file,
writing if else may not work as dictionary terms may get changed over the period of time. like new terms would get added or some terms may get deleted.

Thank you,
Aditya

4

No, you cannot keep dictionaries in a seperate file. You could do something with the elasticsearch filter plugin and store your dict data there.

5

Ok, Thank you pjanzen.
Just wanted to check can we use Translate filter plugin.

Thank you,
Aditya

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.