Logstash Filter CSV - Multiple Header


(Cody) #1

Hello,
I am trying to parse a relatively complex .csv file with mulitple header lines. I am curious if there has been any headway on this issue or if there is a clever work around. I don't have an issue filtering the header lines into Elasticsearch, my issue is associating the following data lines to the headers that have been filtered. An example of the data goes as follows

Header1,Header2,Header3,Header4
1,2,3,4
5,6,7,8
Header11,Header12,Header13,Header14,Header15
1,2,3,4,5
6,7,8,3,1

When I utilized the autodetect_column_names => true feature only the columns are created but the following data does not relate. It would be nice if there was a way to insert these read columns into an object for the columns => filter. Any help is appreciated.. Thank you

Edit:
I understand the CSV filter is stateless from this previous discussion

I would assume there are many people who have run across this issue.