How to parse csv using logstash


(Saket Kumar) #1

I have CSV files where columns are in following sequence-->

File1: A,B,C,D
File2: A,C,D,B

I have to parse only A and B columns. how to map this and push to ES.


(Saket Kumar) #2

I am not sure about the sequence of columns in the Third file, however the columns A and B be persisting in the file. I have to pick A and B. IS there any way?

With CSV filter you need to define columns. but in case of many files where sequence is not sure how to extract specific columns by just names.


(Mark Walkom) #3

Look at the auto detection options.

But you will probably have an option only keeping two columns and removing all others. Logstash will let you remove things, but you need to know the names.


(Saket Kumar) #4

What is auto detection options. please elaborate.

I know the names which i can remove any time.


(Mark Walkom) #5

Have a look at https://www.elastic.co/guide/en/logstash/6.1/plugins-filters-csv.html


(Saket Kumar) #6

Do you mean "autodetect_column_names"?


(Saket Kumar) #7

Thanks I will give a try. Thank you very much.


(Krunal Kalaria) #8

Hi @Saket_Kumar,

Can you send one log sample ? so i know that exactly what you want.


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.