Logtash CSV Parser Fail Muline Value is counting next raw

see in my First raw last coulumn have multiline and second raw haveing signle line

It would help if you could show your current config as well as some sample data illustrating the problem.

@Christian_Dahlqvist

Sure Thanks for Reply

This is my Currunt Config

input {
file {
path => "/home/query.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}

filter {
mutate {
gsub => [
'statement_text', '\"', '',
'statement_text', '"', '',
'statement_text', ',\s+', ','

]

}

csv {
separator => ","
columns => ["db_name","dbid","objectid","creation_time","last_execution_time","execution_count","total_rows","min_rows","max_rows","avg_rows","total_elapsed_time_ms","min_elapsed_time_ms","max_elapsed_time_ms","avg_elapsed_time_ms","total_cpu_time_ms","min_cpu_time_ms","max_cpu_time_ms","avg_cpu_time_ms","statement_text"]
}

mutate {
remove_field => ["desc"]
}
}

output {
elasticsearch {
hosts => "http://xyz:9200"
index => "project"
}
stdout {}
}

Data Output is Proper but some coulumn have multiline so its not taking proper.

Please show an example of the data you are having issues with.

image

See this is my output some of the document is not comming poper mostly if any coulumn have multiline data

Getting Error like CSV Parser Fail

@Christian_Dahlqvist

What does the raw data that is input into Logstash look like?

this is my csv file see in first raw last coulumn and cmpaire with second raw last coulumn my second raw data is properly showing but if the data is in multiline in coulumn its not showing properly
@Christian_Dahlqvist

I have to use mutate gsub after csv parser ?

@Christian_Dahlqvist

Any thing you can help ?

If it breaks across multiple lines in the CSV text file you will need to use a multiline codec with your file input.

Can you show what LS received in the message field?
Use ruby debug in the output. Pls don't post pics.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.