Getting parsing errors for csv filter


(Mohsin Khan Adp) #1

Below is my header and record i am trying to push to elastic search. I am getting getting parsing errors and i could not ship it.

can you help me here ?

HEADER1|HEADER2|HEADER3|HEADER4|HEADER5|HEADER6|HEADER7|HEADER8|HEADER9|HEADER10|HEADER11|HEADER12|HEADER13|HEADER14|HEADER15|HEADER16|HEADER17|HEADER18|HEADER19|HEADER20|HEADER21|HEADER22|HEADER23|HEADER24|HEADER25|HEADER26|HEADER27|HEADER28|HEADER29|HEADER30|HEADER31|HEADER32|HEADER33|HEADER34|HEADER35|HEADER36|HEADER37|HEADER38|HEADER39|HEADER40|HEADER41|HEADER42|HEADER43|HEADER44|HEADER45|HEADER46|HEADER47|HEADER48|HEADER49|HEADER50|HEADER51|HEADER52|HEADER53|HEADER54|HEADER55|HEADER56|HEADER57|HEADER58|HEADER59|HEADER60|HEADER61|HEADER62|HEADER63|HEADER64|HEADER65|HEADER66|HEADER67|HEADER68|HEADER69|HEADER70|HEADER71|HEADER72|HEADER73|HEADER74|HEADER75|HEADER76|HEADER77|HEADER78|HEADER79|HEADER80|HEADER81|HEADER82|HEADER83|HEADER84|HEADER85|HEADER86
121|2|2018-09-19 07:01:19|abcd||0370|Auto|5120370|364003921|Days_prior_Auto|FIE|M|7|eams|k|k|Service|service||95|Drior|||to||1112|||||||||fully.|141||||||||||2018-09-19 07:01:19|1|||141|||||||EVEN|N||||||||"{"firstname":"R","derno":"001","delie":"","ate":"18/09/18","title":"Mrs","toid":"141","flex8":"","fl":"","x6":"","key":"abcde","fl":"","fl":"22/09/18","surname":"GIz","ink":"","abcd":"Abcde","yyyy":"king","fl":"","abcd":\123","driver":"","fl":"","fl":"","art":"","fle":"","va":"","abcd":"true","no":""}"|||0|0|0|0||||||||||||||

###############################
#This is the filter config i am using.
###############################

filter {

    csv{
            separator => "|"
            skip_header => "true"
            columns => [HEADER1|HEADER2|HEADER3|HEADER4|HEADER5|HEADER6|HEADER7|HEADER8|HEADER9|HEADER10|HEADER11|HEADER12|HEADER13|HEADER14|HEADER15|HEADER16|HEADER17|HEADER18|HEADER19|HEADER20|HEADER21|HEADER22|HEADER23|HEADER24|HEADER25|HEADER26|HEADER27|HEADER28|HEADER29|HEADER30|HEADER31|HEADER32|HEADER33|HEADER34|HEADER35|HEADER36|HEADER37|HEADER38|HEADER39|HEADER40|HEADER41|HEADER42|HEADER43|HEADER44|HEADER45|HEADER46|HEADER47|HEADER48|HEADER49|HEADER50|HEADER51|HEADER52|HEADER53|HEADER54|HEADER55|HEADER56|HEADER57|HEADER58|HEADER59|HEADER60|HEADER61|HEADER62|HEADER63|HEADER64|HEADER65|HEADER66|HEADER67|HEADER68|HEADER69|HEADER70|HEADER71|HEADER72|HEADER73|HEADER74|HEADER75|HEADER76|HEADER77|HEADER78|HEADER79|HEADER80|HEADER81|HEADER82|HEADER83|HEADER84|HEADER85|HEADER86]

            }

    mutate{
            remove_field => [ "path"]
            replace => { "message" => "shipped_logs" }
            }

}
####################################


(Magnus B├Ąck) #2

You're setting the columns option wrong. See the example at https://qbox.io/blog/import-csv-elasticsearch-logstash-sincedb.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.