Compare field value with field from csv , database and set a new field

Elastic Stack Logstash
1

My events are having fields like timestamp, version , sitename , url and score
Whenever the data parsed to logstash , I want to compare the score field of the url fields between different version field.

As i have searched and found i can't make dynamic compare on same fields between different events which is my intial approach.

So i want to keep some static data in csv like url, score fields and once the data approached to logstash will try to find the url from csv file and compare the score field based on the difference of the score value i want to set a new field like Positive , negative, neutral ??

Can anyone help above mentioned approaches are feasible or not ? @magnusbaeck @Badger @bardie

2

You could do this using a translate filter. Have a csv containing url,score and do a lookup of the url to a new destination, then compare the destination to the score in the message and mutate+add_field appropriately based on the comparison.

3

@Badger i checked translate filter , can we send the dynamic value to the dictionary_path

mutate{
add_field => {"csv_path" => "/etc/logstash/reports/%{fsite}_%{url}.csv"}
}
translate {
field => "[url_type_uniqref]"
destination => "[new_cwe]"
dictionary_path => "%{csv_path}"
exact => true
regex => true
}

4

No, I do not believe a sprintf reference will work there.

5

Is there any workaround

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.