Substituting Value of a Field to a Mapping from a CSV file


(Jason Christopher Tan) #1

I have used logstash's xpath filter to parse an xml file and get a specified attribute. However, I would like to replace this value to another value based on the mapping of a separated CSV file.

For example, I have used logstash to parse my log's "id3" attribute to get the value "000000" stored in the "Merchant Name" field. I would then want logstash to replace this value by comparing this to a separated csv file with the following lines of code

  ID,MERCHANT NAME
  000000,Store 1
  000001,Store 2
  000002,Store 3

After the comparison, i would like logstash to map the "Merchant Name" field to "Store 1" instead of "000000". If the mapped "id3" value is "000002", then I would like logstash to map the "Merchant Name" field to "Store 2".

Any help would be much appreciated.

Best,
Jason Tan


(Mark Walkom) #2

Have you seen the translate filter, it should do what you want.


(Jason Christopher Tan) #3

Yes I have seen the Translate Filter. However, as far as i know, it only works for YML files. Is there a filter that can work for CSV files?

Thanks.


(Christian Dahlqvist) #4

Why not transform your CSV file into YML and use the translate plugin? I am not aware of any similar filter that works with CSV files.


(system) #5